How to block URLs with robots.txt

A robots.txt file is a file at the root of your site that indicates those parts of your site you don’t want accessed by search engine crawlers. The “User-agent: *” means this section applies to all robots. The value of this field specifies a partial URL that is not to be visited. This can be a full path, or a partial path; any URL that starts with this value will not be retrieved.

For example : disallows all of backend path , see image.User-agent: * Disallow: /backend/*


Phuket
More
Addresses
Contact Us
Intouch
newsletter
© 2019 Imarketingonly All rights reserved.
  • We will use the data you submit to fulfil your request. Privacy Policy.

Imarketingonly

The Great Oast, Steep Marsh Petersfield Hampshire GU32 2BN
125/5A Srisoonthorn Road. Cherngtalay District. Thalang Phuket 83110
+66 (0) 76 324184 +66(0) 898 740 040 +44 (0) 1730 895648 +44 (0) 208 1234 164 +44 (0) 7798 554540 Start Price 120 GBP