Crawler User Agents

This is a listing of only some of our most popular Crawler user agents. Click on the user agent to view detailed information about it.

To decode any user agent, use our API. To get our entire categorised listing of 2,100,000+ user agents, download our database.

User agent Software OS Layout engine Popularity
Mozilla/5.0 (Windows NT 5.1; rv:11.0) Gecko Firefox/11.0 (via ggpht.com GoogleImageProxy) Google Image Proxy 11 Windows Gecko Very common
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) Googlebot 2.1 -- Very common
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html) Baidu Spider 2.0 -- Very common
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) BingBot 2.0 -- Very common
Mozilla/5.0 (compatible; MJ12bot/v1.4.5; http://www.majestic12.co.uk/bot.php?+) Majestic-12 Distributed Search Bot 1.4 -- Very common
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) Yahoo! Slurp Web Crawler Bot -- Very common
Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler) MegaIndex Crawler 2.0 -- Very common
Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/) Ahrefs Backlink Research Bot 5.2 -- Very common
Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com) OpenSiteExplorer Crawler -- Very common
Mozilla/5.0 (compatible; SemrushBot/2~bl; +http://www.semrush.com/bot.html) SEMRush Crawler 2.0 -- Very common
Mozilla/5.0 (compatible; MJ12bot/v1.4.7; http://mj12bot.com/) Majestic-12 Distributed Search Bot 1.4 -- Very common
Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots) Yandex Search Bot 3 -- Very common
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) Googlebot 2.1 -- Very common
Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/) Majestic-12 Distributed Search Bot 1.4 -- Very common
Mozilla/5.0 (compatible; SemrushBot/1.2~bl; +http://www.semrush.com/bot.html) SEMRush Crawler 1.2 -- Very common
Googlebot-Image/1.0 Googlebot Image Crawler 1.0 -- Very common
Mozilla/5.0 (compatible; YandexImages/3.0; +http://yandex.com/bots) Yandex Image Bot 3 -- Very common
Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp) Yahoo! Slurp Web Crawler Bot 3 -- Very common
msnbot/1.0 (+http://search.msn.com/msnbot.htm) MSN Bot 1.0 -- Very common
msnbot/2.0b (+http://search.msn.com/msnbot.htm) MSN Bot 2.0b -- Very common
Mozilla/5.0 (compatible; AhrefsBot/5.0; +http://ahrefs.com/robot/) Ahrefs Backlink Research Bot 5 -- Very common
Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) BingBot 2.0 -- Very common
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) BingPreview/1.0b BingPreview 1.0b -- Very common
Mozilla/5.0 (compatible; AhrefsBot/5.1; +http://ahrefs.com/robot/) Ahrefs Backlink Research Bot 5.1 -- Very common
Mozilla/5.0 (compatible; SemrushBot/1.1~bl; +http://www.semrush.com/bot.html) SEMRush Crawler 1.1 -- Very common
msnbot/1.1 (+http://search.msn.com/msnbot.htm) MSN Bot 1.1 -- Very common
Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 BingPreview/1.0b BingPreview 1.0b -- Very common
Baiduspider+(+http://www.baidu.com/search/spider.htm) Baidu Spider -- Very common
Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1) VoilaBot BETA 1.2 (support.voilabot@orange-ftgroup.com) VoilaBot Beta 1.2 -- Very common
facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php) Facebook Bot 1.1 -- Very common
Googlebot/2.1 (+http://www.google.com/bot.html) Googlebot 2.1 -- Very common
Mozilla/5.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml) Ask Jeeves Crawler -- Very common
msnbot-media/1.0 (+http://search.msn.com/msnbot.htm) MSN Media Bot 1.0 -- Very common
Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8B117 Safari/6531.22.7 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html) Googlebot Mobile 2.1 -- Very common
msnbot-media/1.1 (+http://search.msn.com/msnbot.htm) MSN Media Bot 1.1 -- Very common
msnbot/0.01 (+http://search.msn.com/msnbot.htm) MSN Bot 0.01 -- Very common
Mozilla/5.0 (compatible; DotBot/1.1; http://www.dotnetdotcom.org/, crawler@dotnetdotcom.org) DotNetDotComDotOrg Crawler 1.1 -- Very common
Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07) Sogou "Search Dog" 4 -- Very common
Mozilla/2.0 (compatible; Ask Jeeves/Teoma) Ask Jeeves Crawler -- Very common
Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot) ExaLead Crawler 3 -- Very common
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.75 Safari/537.36 Google Favicon Google Favicon Crawler -- Very common
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Safari/537.36 Googlebot 2.1 -- Very common
Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://sp.ask.com/docs/about/tech_crawling.html) Ask Jeeves Crawler -- Very common
Mozilla/5.0 (iPhone; CPU iPhone OS 8_3 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12F70 Safari/600.1.4 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) Googlebot 2.1 -- Very common
Jyxobot/1 Jyxo.cz Crawler 1.0 -- Very common
msnbot/0.3 (+http://search.msn.com/msnbot.htm) MSN Bot 0.3 -- Very common
Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1) VoilaBot BETA 1.2 (http://www.voila.com/) VoilaBot Beta 1.2 -- Very common
Mozilla/4.0 (compatible; MSIE 5.0; Windows 95) VoilaBot BETA 1.2 (http://www.voila.com/) VoilaBot Beta 1.2 -- Very common
Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) Googlebot 2.1 -- Very common
Mozilla/5.0 (compatible; MJ12bot/v1.4.3; http://www.majestic12.co.uk/bot.php?+) Majestic-12 Distributed Search Bot 1.4 -- Very common

Download our database of user agents

Do you want to easily download all our 2,100,000+ user agents? It's available with an API subscription. Find out more about our User agent database.