noindex directives
If you don't want a document from your site to appear in our search results, you can use the noindex directive. You can include this directive in either your HTML or HTTP response headers.
If your document is an HTML page, you can add the following meta tag to the <head> section of your code:
<html>
<head>
<meta name="robots" content="noindex">
...
</head>
<body>
...
</body>
</html>
This meta tag instructs all our robots, as well as other robots supporting the noindex directive, not to include the tagged document in search results. If you only want to apply the noindex directive to certain robots, you can use those robots’ specific agent names like this:
<meta name="coccocbot-web" content="noindex"> <meta name="coccocbot-fast" content="noindex">
If your document is not HTML (e.g. a .pdf, Word .doc, etc.), you can use HTTP response headers:
HTTP/1.1 200 OK ... X-Robots-Tag: noindex ...
If you only want to block specific robots using HTTP response headers, you can use those robots’ specific agent names like this:
X-Robots-Tag: coccocbot-web: noindex X-Robots-Tag: coccocbot-fast: noindex
The noindex directive still allows robots to download and analyze your documents. If you want to disallow the actual downloading of your content, please consider using the nofollow tag or robots.txt.
Another option is the noimageindex directive, which affects how robots process images in your HTML documents. If you add it to your HTML document, our robot won't extract and download image links from that document. As with the noindex directive, you can specify it in either your HTML or HTTP response headers and apply it to either all robots or just specific robots:
<meta name="robots" content="noimageindex"> <meta name="coccocbot-image" content="noimageindex">
X-Robots-Tag: noimageindex X-Robots-Tag: coccocbot-image: noimageindex