| User Agent | Number of hits | Data Transferred (Kb) |
| rogerbot/1.2 (https://moz.com/help/guides/moz-procedures/what-is-rogerbot, rogerbot-crawler aardwolf-production-crawler-27@moz.com) | 40,060 | 726,122 |
| rogerbot/1.2 (https://moz.com/help/guides/moz-procedures/what-is-rogerbot, rogerbot-crawler aardwolf-production-crawler-25@moz.com) | 20,026 | 362,033 |
| rogerbot/1.2 (https://moz.com/help/guides/moz-procedures/what-is-rogerbot, rogerbot-crawler aardwolf-production-crawler-68@moz.com) | 20,026 | 362,073 |
| netEstate NE Crawler ( http://www.website-datenbank.de/) | 1,037 | 20,456 |
| ltx71 - (http://ltx71.com/) | 821 | 16,716 |
| facebookexternalhit/1.1 ( http://www.facebook.com/externalhit_uatext.php) | 494 | 25,167 |
| 8LEGS | 457 | 7,789 |
| Photon/1.0 | 286 | 6,488 |
| rogerbot/1.1 (http://moz.com/help/pro/what-is-rogerbot-, rogerbot-crawler vanguard@moz.com) | 264 | 7,831 |
| Dispatch/0.11.3 | 195 | 25,847 |
| Go-http-client/1.1 | 114 | 743 |
| Gigabot | 72 | 1,568 |
| bidswitchbot/1.0 | 72 | 38 |
| The Knowledge AI | 54 | 27 |
| MixnodeCache/1.8( https://cache.mixnode.com/) | 52 | 67 |
| CCBot/2.0 (https://commoncrawl.org/faq/) | 51 | 630 |
| YahooMailProxy; https://help.yahoo.com/kb/yahoo-mail-proxy-SLN28749.html | 42 | 662 |
| cortex/1.0 | 25 | 558 |
| rogerbot/1.2 (https://moz.com/help/guides/moz-procedures/what-is-rogerbot, rogerbot-crawler aardwolf-production-api-01@moz.com) | 24 | 163 |
| PayPal IPN ( https://www.paypal.com/ipn ) | 20 | 6 |
| Python-urllib/2.7 | 19 | 774 |
| SuperPagesUrlVerifyBot/1.0 | 16 | 312 |
| SEMrushBot | 16 | 312 |
| Twitterbot/1.0 | 13 | 282 |
| lua-resty-http/0.07 (Lua) ngx_lua/10000 | 12 | 62 |
| lua-resty-http/0.10 (Lua) ngx_lua/10000 | 12 | 5 |
| curl/7.58.0 | 12 | 202 |
| curl/7.47.1 | 10 | 2 |
| Pandalytics/1.0 (https://domainsbot.com/pandalytics/) | 10 | 63 |
| PulsePoint-Ads.txt-Crawler/1.0 | 10 | 4 |
| CheckMarkNetwork/1.0 ( http://www.checkmarknetwork.com/spider.html) | 8 | 122 |
| python-requests/2.22.0 | 7 | 99 |
| adreview/1.0 | 6 | 91 |
| Jersey/2.25.1 (HttpUrlConnection 1.8.0_141) | 6 | 2 |
| python-requests/2.21.0 | 6 | 98 |
| adstxt.com/1.2 | 6 | 2 |
| python-requests/2.18.1 | 6 | 41 |
| YahooCacheSystem; YahooWebServiceClient | 5 | 96 |
| IAS crawler (ias_crawler; http://integralads.com/site-indexing-policy/) | 5 | 101 |
| Python-urllib/3.7 | 5 | 353 |
| Others | 80 | 1,502 |
| Total | 84,462 | 1,569,509 |
