Use your robots.txt file to direct search engine Net crawlers on what to crawl. A robots.txt file can prevent your internet site from having crawled and indexed, so check your set up with Google’s totally free robots.txt tester Software. Common white-hat methods of search engine optimisation Search engine optimisation approaches https://rsac.org