Specialist SEO Agency with clients in Oxford, across Oxfordshire & the Thames Valley
SEO Video Training Course  |  SEO Knowledgebase

How do you stop crawlers?

SEO Knowledgebase > ,

Using your robots.txt file you can stop search engines like Google from crawling certain pages or the entire site. To stop Google crawling your website altogether set your root domain to be disallowed. In many websites, especially sites like WordPress you can use SEO tools like Yoast to block search engines if your site is temporarily under construction.
get in
touch