# - A - B - C - D - E - F - G - H - I - J - K - L - M - N - O - P - Q - R - S - T - U - V - W - X - Y - Z

The file name utilized by the robot exclusion protocol. Web robots download this file from the server’s document root and parse it for instructions on what to index and not to index. It can deny access by robots to certain pages or sub-directories of the site. Only robots which comply with the Robots Exclusion Standard will read and obey the commands in this file. Robots will read this file on each visit, so that pages or areas of sites can be made public or private at any time by changing the content of robots.txt before re-submitting to the search engines.

Previous Web Hosting Term: Robot
Next Web Hosting Term: ROI - Return On Investment