I am planning to write a series of entries on screen scraping, automated
Web navigation, deep Web mining, wrapper generation, screen scraping from
Rails with Ajax and possibly more, depending on my time and your feedback.
Since these entries are going to be longer, I will be posting them to
separate pages, and announce them on my blog.
The first article is ready, you can read it here.
It is an introduction to screen scraping/Web extraction with Ruby,
evaluation of the tools along with installation instructions and examples.
Feedback would be appreciated (leave your comment here/on the article page, or
send me a mail at peter@[name of this site].com), I will update/extend the
document and publish new ones based on your feedback.
What’s more, European data-protection laws also demand proportionality in how data is collected and used. The customer’s consent only covers data strictly necessary to the job with which the he or she has tasked the company. In the US, there has been some concern that screen scraping might give financial-service companies ongoing access, allowing them to harvest a broad range of data from customer accounts. In Europe, this just isn’t possible.