Google's NOINDEX Policy Discussion:
The latest post from Matt Cutts comes as a policy discussion about NOINDEX and how Google should treat the NOINDEX meta tag.
Different search engines have been found to handle the “NOINDEX” meta tag differently:
· Google doesn’t show the page in any way
· Ask doesn’t show the page in any way
· MSN shows a reference Cached link and url, but no summary. Clicking the cached link doesn’t return a page.
· Yahoo! shows a reference to the url and Cached link, but no summary. Clicking on the cached link returns the cached page.
The question now arises whether Google should completely drop a NOINDEX’ed page from its search results vs. show a reference to the page, or something in between? The three different scenarios have been deliberated upon:
Completely drop a NOINDEX’ed page
Google has implemented this policy for several years, and webmasters have become adjusted to this policy. The NOINDEX meta tag gives a way to completely remove all traces of a site from Google, however there's also the Google url removal tool. Sometimes it happens that Google sees a link to a page 'A' but doesn’t actually crawl the page, it won’t know that page 'A' has a NOINDEX tag and then it might show the page as an uncrawled url. There’s however a remedy for that: currently, Google allows a NOINDEX directive in robots.txt and it will completely remove all matching site urls from Google. At the same time, this behavior could change based on the policy discussion about it.
Sometimes Webmasters harm themselves by using NOINDEX, but if a site’s traffic from Google is very low, the webmaster might want to diagnose the issue themselves. Furthermore, Google could add a NOINDEX check into the webmaster console to help webmasters self-diagnose if they’ve removed their own site with NOINDEX. Moreover, the NOINDEX meta tag serves a useful role that’s different than robots.txt.
