travel.earth

Authentic Travel Content I Travel Inspiration & Experiences I Travel Earth

Authentic travel experiences, travel inspiration, and great travel content to fuel wanderlust. Find hidden gems and travel the earth differently

Alexa stats for travel.earth

Site Seo for travel.earth

Tag :

H1	H2	H3	H4	H5
5	12	92	0	0

Image :

There are 6 images on this website and 6 images have alt attributes

Frame :

There are 0 embed on this website.

Flash :

There are 0 flash on this website.

Size :

679,929 characters

Meta Description :

Yes

Meta Keyword :

Magestic Backlinks for travel.earth

About travel.earth

Domain	travel.earth
MD5	81a0be6e43b1feae6a3289af14ade2f1
Charset	UTF-8

Page Speed	Check now
Programming Language	PHP/7.4.10
Web server	Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips PHP/7.4.10
Javascript library	jquery
IP Address	13.232.79.48

User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Disallow: /wp-admin/
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /linkout/
Disallow: /recommended/
Disallow: /comments/feed/
Disallow: /trackback/
Disallow: /index.php
Disallow: /xmlrpc.php
Disallow: /tag/
Disallow: /page/
Disallow: /online-game-18s/
Disallow: /best-sex-dating-websitess/
Disallow: /craigslist-minneapolis-datings/
Disallow: /www-craigslist-sacramentos/
Disallow: /manhunt-gay-dating-sites/
Disallow: /craigslist-modesto-datings/
Disallow: /online-game-18s/
Disallow: /random-dating-chatroulettes/
Disallow: /craigslist-pa-women-seeking-mens/
Disallow: /craigslist-ie-datings/
Disallow: /anr-dating-meanings/
Disallow: /craigslist-tulsa-datings/
Disallow: /what-does-bbc-stand-for-sexuallys/
Disallow: /online-free-dating-chat-roomss/
Disallow: /one-night-stand-dating-appss/



User-agent: rogerbot
#Crawl-delay: 30
Disallow: /tag/
Disallow: /quizs*


User-agent: Pinterest
Crawl-delay: 10

User-agent: Pinterestbot
Crawl-delay: 10

User-agent: Pinterestbot/1.0
Crawl-delay: 10

User-agent: Pinterest/0.2 (+https://www.pinterest.com/bot.html)
Crawl-delay: 10

User-agent: Mozilla/5.0 (compatible; Pinterestbot/1.0; +https://www.pinterest.com/bot.html)
Crawl-delay: 10

User-agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Pinterestbot/1.0; +https://www.pinterest.com/bot.html)
Crawl-delay: 10

#Removing crawl-delays and disallow for all the bots below,on Sep13 due to user traffic decline between sep 10-13

#User-agent: Slurp
#Crawl-delay: 10

#User-agent: DuckDuckBot
#Crawl-delay: 10

#User-agent: ia_archiver
#Crawl-delay: 10

#User-agent: amazonbot
#Crawl-delay: 10

User-agent: SemrushBot
Crawl-delay: 10

User-agent: SemrushBot-SA
Crawl-delay: 10

#User-agent: NinjaBot
#Disallow: /

User-agent: Mediapartners-Google*
Allow: /

User-agent: Googlebot-Image
Allow: /

User-agent: Adsbot-Google
Allow: /
 
User-agent: Googlebot-Mobile
Allow: /

#Allow Seekbot with delay - https://www.seekport.co.uk/seekbot/
User-agent: Seekport Crawler
Crawl-delay: 10

#Allow Seekbot with delay - https://www.seekport.co.uk/seekbot/
User-agent: Seekbot
Crawl-delay: 10

User-Agent: bingbot
Crawl-delay: 2

#User-agent: Mozilla/5.0 (compatible; YaK/1.0; https://linkfluence.com/; [email protected])
#Disallow: /

#User-agent: Buck/2.2; (+https://app.hypefactors.com/media-monitoring/about.html)
#Disallow: /

#User-agent: Mozilla/5.0 (compatible; Daum/4.1; +https://cs.daum.net/faq/15/4118.html?faqId=28966)
#Disallow: /

#new bots info added
User-agent: Yandex
Disallow: /admin/*

#User-agent: Yandex
#Crawl-delay: 10

#User-agent: Mozilla/5.0 (compatible; Semanticbot/1.0; +https://sempi.tech/bot.html)
#Crawl-delay: 10

#User-agent: Baiduspider
#Crawl-delay: 10

User-agent: Mozilla/5.0 (compatible; MJ12bot/v1.4.8; https://mj12bot.com/)
Crawl-delay: 10

User-agent: MJ12bot
Crawl-delay: 10

#User-agent: PetalBot
#Disallow: /

#User-agent: Mozilla/5.0 (Linux; Android 7.0;) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; PetalBot;+https://aspiegel.com/petalbot)
#Disallow: /

User-agent: facebookexternalhit/1.1 
Crawl-delay: 10

User-agent: facebookexternalhit/1.1 (+https://www.facebook.com/externalhit_uatext.php)
Crawl-delay: 10

User-agent: BLEXBot
Crawl-delay: 5

User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9) AppleWebKit/537.71 (KHTML, like Gecko) Version/7.0 Safari/537.71 (Rival IQ, rivaliq.com)
Crawl-delay: 10

User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36 (Rival IQ, rivaliq.com)
Crawl-delay: 10

User-agent: Mozilla/5.0(Macintosh; Intel Mac OS X 10_11_6)AppleWebKit/537.36(KHTML, like Gecko)Chrome/52.0.2743.116Safari/537.36(Rival IQ, rivaliq.com)
Crawl-delay: 10

User-agent: mozilla/5.0 (macintosh; intel mac os x 10_11_6) applewebkit/537.36 (khtml, like gecko) chrome/52.0.2743.116 safari/537.36 (rival iq, rivaliq.com)
Crawl-delay: 10

User-agent: mozilla/5.0 (macintosh; intel mac os x 10_9) applewebkit/537.71 (khtml, like gecko) version/7.0 safari/537.71 (rival iq, rivaliq.com)
Crawl-delay: 10


# Source: https://www.rocketrental.co.nz/robots.txt
# Header: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider
User-agent: 360Spider
Crawl-delay: 10

# Header: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0); 360Spider(compatible; HaosouSpider; https://www.haosou.com/help/help_3_2.html)
User-agent: HaosouSpider
Crawl-delay: 10

# Source: https://developers.whatismybrowser.com/useragents/parse/1109735-uptime-robot
User-agent: Mozilla/5.0+(compatible; UptimeRobot/2.0; https://www.uptimerobot.com/)
Crawl-delay: 10

#bing other agents delay goes here. Source: https://www.bing.com/webmaster/help/which-crawlers-does-bing-use-8c184ec0
# and siteground.com/kb/how_to_decrease_the_crawl_rate_of_bings_search_engine_bot/

User-agent: Mozilla/5.0 (compatible; bingbot/2.0; +https://www.bing.com/bingbot.htm)
Crawl-delay: 2

User-agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +https://www.bing.com/bingbot.htm) Chrome/W.X.Y.Z Safari/537.36 Edg/W.X.Y.Z
Crawl-delay: 2

User-agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 Edg/W.X.Y.Z (compatible; bingbot/2.0; +https://www.bing.com/bingbot.htm)
Crawl-delay: 2
 
User-agent: Mozilla/5.0 (compatible; adidxbot/2.0; +https://www.bing.com/bingbot.htm)
Crawl-delay: 2

User-agent: Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; adidxbot/2.0; +https://www.bing.com/bingbot.htm)
Crawl-delay: 2

User-agent: Mozilla/5.0 (Windows Phone 8.1; ARM; Trident/7.0; Touch; rv:11.0; IEMobile/11.0; NOKIA; Lumia 530) like Gecko (compatible; adidxbot/2.0; +https://www.bing.com/bingbot.htm)
Crawl-delay: 2

User-agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) BingPreview/1.0b
Crawl-delay: 2

User-agent: Mozilla/5.0 (Windows Phone 8.1; ARM; Trident/7.0; Touch; rv:11.0; IEMobile/11.0; NOKIA; Lumia 530) like Gecko BingPreview/1.0b
Crawl-delay: 2

User-agent: msnbot
Crawl-delay: 2

User-agent: msnbot-media
Crawl-delay: 2

Sitemap: https://travel.earth/sitemap_index.xml