On running my asp.net website which is deployed in local IIS , I got a message User-agent: * Disallow: /. Why did I get this message? How can I overcome this problem and run my website successfully?
Thanks in Advance
This is the entry in your robots.txt file.
Translated, it means this:
The User-agent: * means this section applies to all robots. The Disallow: / tells the robot that it should not visit any pages on the site.
Note: Check in your top-level directory for a robots.txt file and either remove it or change the configuration to allow access.
robots.txtfile. If you're getting that for any request then your webserver is seriously misconfigured.