telophase: (Default)
telophase ([personal profile] telophase) wrote2022-12-05 10:17 am
Entry tags:

The woes of the web designer

Google: You want to stop robots indexing your site? Disallow them in robots.txt!
Also Google: That won't stop us from indexing your those pages, of course, and complaining when they don't meet our standards.

Google: But remove them from robots.txt and put a noindex meta tag in the header and it'll stop us!
Also Google: runs validate fix function Nope, sorry, those pages with the noindex meta tag still don't meet our standards.
selenite0: (Default)

[personal profile] selenite0 2022-12-05 06:44 pm (UTC)(link)
Pity there's not an undocumented Google-f-off tag.