Crawler User Agents

This is a complete listing of Crawler user agents. Click on the user agent to view detailed information about it.

User agent Software OS Layout engine Popularity
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) Googlebot 2.1 -- Very common
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) BingBot 2.0 -- Very common
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html) Baidu Spider 2.0 -- Very common
Pinterest/0.2 (+http://www.pinterest.com/) Pinterest Bot 0.2 -- Very common
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.75 Safari/537.36 Google Favicon Google Favicon Crawler -- Very common
Mozilla/5.0 (compatible; DuckDuckGo-Favicons-Bot/1.0; +http://duckduckgo.com) DuckDuckGo Favicons Bot 1.0 -- Very common
Mozilla/5.0 (compatible; MJ12bot/v1.4.5; http://www.majestic12.co.uk/bot.php?+) Majestic-12 Distributed Search Bot 1.4 -- Very common
Mozilla/5.0 (compatible; Pinterestbot/1.0; +http://www.pinterest.com/bot.html) Pinterest Bot -- Very common
Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/) Majestic-12 Distributed Search Bot 1.4 -- Very common
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) Yahoo! Slurp Web Crawler Bot -- Very common
Mediapartners-Google Google's Media Partners system (AdSense) -- Common
Mozilla/5.0 (compatible; proximic; +http://www.proximic.com/info/spider.php) Proximic Search -- Common
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) Googlebot 2.1 -- Common
Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07) Sogou "Search Dog" 4 -- Common
Pinterest/0.2 (+http://www.pinterest.com/bot.html) Pinterest Bot 0.2 -- Common
Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots) Slackbot Link Checker 1.0 -- Common
Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots) Yandex Search Bot 3 -- Common
Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/) Ahrefs Backlink Research Bot 5.2 -- Common
Twitterbot/1.0 TwitterBot 1.0 -- Common
Mozilla/5.0 (Windows NT 6.1; rv:6.0) Gecko/20110814 Firefox/6.0 Google Favicon Google Favicon Crawler -- Common
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36; 360Spider 360Spider -- Common
ADmantX Platform Semantic Analyzer - ADmantX Inc. - www.admantx.com - support@admantx.com ADmantX Platform Semantic Analyzer -- Common
Mozilla/5.0 (TweetmemeBot/4.0; +http://datasift.com/bot.html) Gecko/20100101 Firefox/31.0 TweetMeme Bot 4 -- Common
Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com) OpenSiteExplorer Crawler -- Common
Mozilla/5.0 (compatible; proximic; +https://www.comscore.com/Web-Crawler) Proximic Search -- Common
Google favicon Google Favicon Crawler -- Common
Mozilla/5.0 (compatible; MJ12bot/v1.4.7; http://mj12bot.com/) Majestic-12 Distributed Search Bot 1.4 -- Common
Mozilla/5.0 (compatible; GrapeshotCrawler/2.0; +http://www.grapeshot.co.uk/crawler.php) Grapeshot Bot 2.0 -- Common
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36 Google (+https://developers.google.com/+/web/snippet/) Google+ Snippet Fetcher -- Common
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) BingPreview/1.0b BingPreview 1.0b -- Common
Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot) ExaLead Crawler 3 -- Common
Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider(compatible; HaosouSpider; http://www.haosou.com/help/help_3_2.html) 360Spider -- Common
Mozilla/5.0 (Windows; U; Windows NT 5.1; en; rv:1.9.0.13) Gecko/2009073022 Firefox/3.5.2 (.NET CLR 3.5.30729) SurveyBot/2.3 (DomainTools) DomainTools SurveyBot 2.3 -- Common
admantx-usastn/2.4 (+http://www.admantx.com/service-fetcher.html) ADmantX Platform Semantic Analyzer -- Common
ADmantX Platform Sync US Semantic Analyzer - ADmantX Inc. - www.admantx.com - support@admantx.com ADmantX Platform Semantic Analyzer -- Common
ADmantX Platform Semantic Analyzer US - Turn - ADmantX Inc. - www.admantx.com - support@admantx.com ADmantX Platform Semantic Analyzer -- Common
Mozilla/5.0 (compatible; meanpathbot/1.0; +http://www.meanpath.com/meanpathbot.html) MeanPath Bot 1.0 -- Common
Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html) Genieo Bot 1.0 -- Common
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36 evaliant Evaliant Impressions Bot 41 Windows Blink Common
Mozilla/5.0 (iPhone; CPU iPhone OS 9_3_2 like Mac OS X) AppleWebKit/601.1 (KHTML, like Gecko) CriOS/51.0.2704.104 Mobile/13F69 Safari/601.1.46 evaliant Evaliant Impressions Bot 51 iOS Webkit Common
ADmantX Platform Semantic Analyzer Appnexus - ADmantX Inc. - www.admantx.com - support@admantx.com ADmantX Platform Semantic Analyzer -- Average
Mozilla/5.0 (compatible; YodaoBot/1.0; http://www.yodao.com/help/webmaster/spider/; ) YodaoBot Search Bot 1.0 -- Average
Mozilla/5.0 (compatible; Googlebot/2.1; startmebot/1.0; +https://start.me/bot) Googlebot 2.1 -- Average
msnbot/2.0b (+http://search.msn.com/msnbot.htm) MSN Bot 2.0b -- Average
Sosospider+(+http://help.soso.com/webspider.htm) Sosospider Search Bot -- Average
admantx-sap/2.4 (+http://www.admantx.com/service-fetcher.html) ADmantX Platform Semantic Analyzer -- Average
Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider 360Spider -- Average
Mozilla/5.0 (iPhone; CPU iPhone OS 8_3 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12F70 Safari/600.1.4 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) Googlebot 2.1 -- Average
facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php) Facebook Bot 1.1 -- Average
Googlebot/2.1 (+http://www.googlebot.com/bot.html) Googlebot 2.1 -- Average

More features coming soon!

This read out of user agents is brand new! Soon we're going to add:

  • Adding refinement: eg in-app browser on Windows
  • A Percentage breakdown: eg: Chrome is x% Desktop, y% Mobile