Comments on: Data Extraction for Web 2.0: Screen Scraping in Ruby/Rails, Episode 1

by: Tom

Sun, 29 Jun 2008 06:46:29 +0000

This website provide Web2DB Service : www.knowlesys.com, no need programming!

by: Jeff

Fri, 20 Jun 2008 14:06:40 +0000

I am trying to get scrubyt to work with the Ruby on Rails framework. I saw some talk on the forums (that took place last year), but not any clear answers. And I haven’t seen anything about it posted lately. Do you know if anyone has it working with RoR? If so, could you point me in the right direction or in contact with someone that has it working. Thank you.

by: Rusça tercüman

Mon, 25 Feb 2008 14:49:50 +0000

Your comment contains very useful information about all thank you... rusça tercüman

by: peter

Sat, 09 Feb 2008 05:39:44 +0000

Arno,

of course.... that's the whole point! For example:

http://snippets.aktagon.com/snippets/90-Scraping-Google-search-results-with-Scrubyt-and-Ruby

by: Arno Nym

Fri, 08 Feb 2008 14:23:57 +0000

you write "Once you run the extractor and you think the data it scrapes is correct, you can export it. Let’s see how the exported finances.yahoo.com extractor looks like:"

is there a command for exporting this or i have to make this step by hand to find out the xpath?

by: ramp

Thu, 07 Feb 2008 12:10:58 +0000

undefined method `fetch’ for main:Object (NoMethodError)

by: Yitzhak cohen

Thu, 30 Aug 2007 11:21:28 +0000

undefined method `fetch’ for main:Object (NoMethodError)

by: 电子网

Sun, 20 May 2007 03:23:22 +0000

The site looks great ! Thanks for all your help ( past, present and future !)

by: peter

Thu, 17 May 2007 18:58:14 +0000

Thanks alan!

Well, actually scRUBYt! knows all those things you have stated above (as for the ’scripting engine that works with most popular languages’ - scRUBYt! is ‘just’ pure Ruby). I am neither trying to compete with screen-scraper (or other tools) nor to replicate or try to resemble to their feature set, so I do not mean to start a scraper war here…

I know at least one guy (actually a gal) who used screen-scraper and switched to scRUBYt!, and AFAIK she is happy so far. On the other hand, she uses a Rails frontend to present the scraped data, so scRUBYt! was a natural fit for her. I guess if you are using Java tools, screen-scraper may be better.

Bottom line: use the right tool for the right job! scRUBYt! is based on a different philosophy, has different features and is currently kind of immature - on the other hand it has the best community!

by: alan

Thu, 17 May 2007 15:14:22 +0000

I know this is a little late, El, but there’s an app I use to do screen scraping and data extraction called screen-scraper. It handles going through forms, picking out data, downloading files, etc. and has a nice scripting engine that works with most popular languages. Can even be programmed with regular expressions .

Great article, Peter, scRUBYy! looks awesome.