A robots.txt file is a file at the root of your site that indicates those parts of your site you don’t want accessed by search engine crawlers. The “User-agent: *” means this section applies to all robots. The value of this field specifies a partial URL that is not to be visited. This can be a full path, or a partial path; any URL that starts with this value will not be retrieved.
For example : disallows all of backend path , see image.User-agent: * Disallow: /backend/*
