During the website construction process, we will have some content that we do not want to be crawled by search engines or appear on the Internet, so what should we do? How do I tell search engines not to crawl my content? This is where it comes in handy.
It is a text file stored in the root directory of the website and used to specify the scope of crawling of website content by search engines. This sentence can explain what it is, but we need to pay attention to a few points
It must be stored on the website under the root directory.
Its name must be
It is the first page that the search engine visits the website
It must be specified
新闻中心