Innovative Web Design and Application Development Glossary of Terms

Main Menu -> Web Design -> robots.txt


A text file stored in the top level directory of a web site which is used to tell the Robots (spiders) what parts of a web site should, and should not be indexed.

Copyright © 1992-2020
web development: