Internet bots, also known as web robots, WWW robots or simply bots, are software applications that run automated tasks over the Internet. Typically, bots perform tasks that are both simple and structurally repetitive, at a much higher rate than would be possible for a human alone...
A web page or webpage is a document or information resource that is suitable for the World Wide Web and can be accessed through a web browser and displayed on a monitor or mobile device. This information is usually in HTML or XHTML format, and may provide navigation to other web pages via hypertext...
. There are a multitude of reasons why one might want to use this tag. These include preventing robots from trying to index very large database or pages that are very transitory, ones that one wishes to keep slightly more private, or the printer and mobile-friendly versions of webpages. Since the burden of honoring a website's noindex tag lies with the author of the search robot, sometimes these tags are willfully ignored. Also the interpretation of the noindex tag is sometimes slightly different from one search engine company to the next.
Yandex is a Russian IT company which operates the largest search engine in Russia and develops a number of Internet-based services and products. Yandex is ranked as 5-th world largest search engine...
introduce a new tag which only prevents indexing of the content between the tags, not a whole Web page.
Do index this text block.
Bot-specific Directives
There are ways to send the noindex directive only for certain bots, typically by using a different "name" value.
For example, to specifically block Google's bot, specify
or to block Yahoo!'s bot, specify
The source of this article is wikipedia, the free encyclopedia. The text of this article is licensed under the GFDL.
