Rand Fishkin on Scraping and Search APIs
Last night, Rand Fishkin wrote an interesting article on the SEOmoz blog regarding the necessity of having proper search data and the inadequacy of the existing search APIs from the major search engines. He then explains the resulting need to scrape to get accurate data, and follows that up with the consequences of scraping, both for the scraper, and for the search engine being scraped.
I think that representatives from the big engines really should read this - it provides compelling reasons to update and ‘fix’ their broken/poorly implemented APIs. The data they can give is extremely valuable to companies, to the point that if necessary, many companies would be more than willing to pay for it.
At any rate, Rand provides a great listing of the important APIs, and great descriptions of their function. In addition, he makes some great suggestions about how such engines should be improved. Definately go read the article!
