Major Search Engines Team Up to Establish Standards

Google, Yahoo, and Live have all teamed up to establish a standard for robots.txt. Live says it beautifully,

While most search engines already comply with the REP, this is the first time the three major search engines have come together to detail how we actually implement the protocol. This effort makes it easier for webmasters to know how REP directives will be handled by search providers.

You can read the documentation here.

No Comments Yet - You can be the first to comment!