Web Design Ireland

Following a few recent searches on Google, I’ve noticed that they have to stepped up their algorithm on protecting their index from being scraped or queried automatically, incorporating a catpcha asking whether I was human or not.  The frequency of this appearing of late has definitely increased.

Which is a good idea, considering the recent spate of web server worm attacks, which use the Google index to indentify prospective servers to attack.

I just have to wonder whether this over protective strategy is the best one, in the long run. How much more could be done on the web if Google’s index wasn’t closed up so tightly.

How many more amazing apps would make their way onto the web. Google API’s for all but their actual index, are on the whole very good, but this is one area that they refuse to budge on. Perhaps because they think that someone will come along with Google rips off, that actually add extra value to the index. *shrug*.

ReadWriteWeb covered the alternative ways to tweet in April of this year, which shows that 44% of traffic from twitter comes from their API. Google’s traffic at the minute has got to be pushing 90% for browser based traffic. If the main search index was opened up to the API - what impact would that have on their bottom line?  Any rip off that springs up that becomes popular enough could be snatched up and bought by the, or alternatively put something in the terms and conditions of use - similar to the strategy that they are employing with App Engine.

Just my two cents. Anyone any thoughts?