Comments on: Evaluating XPaths with indices in HPricot http://www.rubyrailways.com/evaluating-xpaths-with-indicdes-in-hpricot/ Experiences with Ruby and Rails, Web2.0 and other development technologies Thu, 28 Aug 2008 19:24:49 +0000 http://wordpress.org/?v=2.0.2 by: rhubarb http://www.rubyrailways.com/evaluating-xpaths-with-indicdes-in-hpricot/#comment-1052 Fri, 03 Nov 2006 19:37:45 +0000 http://www.rubyrailways.com/evaluating-xpaths-with-indicdes-in-hpricot/#comment-1052 <p>Funny I was looking at Hpricot, and then looking at alternatives (I want to scrape my online bank statements), when I found your original - nicely detailed - page on scraping written in June. I was researching all of these suggestions you made until I read to the last comment and saw that you'd settled on Hpricot anyway. ;)</p> <p>What I'd like to know is do you still maintain that Mechanize is the best way to do the navigation, and then maybe Hpricot for the scraping once you have the page?</p> <p>Or does Hpricot give you some way to navigate?</p> <p>If the former can you show us some Hpricot, Mechanize combined code?</p> Funny I was looking at Hpricot, and then looking at alternatives (I want to scrape my online bank statements), when I found your original - nicely detailed - page on scraping written in June. I was researching all of these suggestions you made until I read to the last comment and saw that you’d settled on Hpricot anyway. ;)

What I’d like to know is do you still maintain that Mechanize is the best way to do the navigation, and then maybe Hpricot for the scraping once you have the page?

Or does Hpricot give you some way to navigate?

If the former can you show us some Hpricot, Mechanize combined code?

]]> by: juzzin http://www.rubyrailways.com/evaluating-xpaths-with-indicdes-in-hpricot/#comment-974 Sat, 21 Oct 2006 06:03:15 +0000 http://www.rubyrailways.com/evaluating-xpaths-with-indicdes-in-hpricot/#comment-974 <p>Sounds very similar to Ariel: http://ariel.rubyforge.org/</p> Sounds very similar to Ariel: http://ariel.rubyforge.org/

]]> by: Peter Szinek http://www.rubyrailways.com/evaluating-xpaths-with-indicdes-in-hpricot/#comment-936 Sun, 08 Oct 2006 07:46:24 +0000 http://www.rubyrailways.com/evaluating-xpaths-with-indicdes-in-hpricot/#comment-936 <p>Yeah, I know about scrAPI, but I am going to take a different (X)Path, both in the wrapper generation phase - instead of providing concrete XPaths (= CSS selectors in scrAPI), there will be a possibility to define the data you are looking for with examples, so in an ideal case you won't need to use XPath at all - and in the evaluation phase - lots of heuristics, XPath instead of CSS. The primary aim will be to create a wrapper generator which needs minimum input and technical knowledge (of course it will be possible to use XPaths or even Ruby if you wish) yet performs robust extractions and will be usable to quickly scrap sites like amazon, ebay etc. and further integrate the data into something usable in real-life.</p> <p>The goal is to create an easy-to-use wraper generator which <b>works in practice</b> and will make possible to create mashups or use the extracted data further in any RoR or Ruby (or any other) app...</p> Yeah, I know about scrAPI, but I am going to take a different (X)Path, both in the wrapper generation phase - instead of providing concrete XPaths (= CSS selectors in scrAPI), there will be a possibility to define the data you are looking for with examples, so in an ideal case you won’t need to use XPath at all - and in the evaluation phase - lots of heuristics, XPath instead of CSS. The primary aim will be to create a wrapper generator which needs minimum input and technical knowledge (of course it will be possible to use XPaths or even Ruby if you wish) yet performs robust extractions and will be usable to quickly scrap sites like amazon, ebay etc. and further integrate the data into something usable in real-life.

The goal is to create an easy-to-use wraper generator which works in practice and will make possible to create mashups or use the extracted data further in any RoR or Ruby (or any other) app…

]]> by: ruby licious http://www.rubyrailways.com/evaluating-xpaths-with-indicdes-in-hpricot/#comment-935 Sat, 07 Oct 2006 18:18:40 +0000 http://www.rubyrailways.com/evaluating-xpaths-with-indicdes-in-hpricot/#comment-935 <p>webextraction projects are always interesting, what path are you taking If I may ask? =)</p> <p>I guess You've seen http://blog.labnotes.org/2006/07/11/scraping-with-style-scrapi-toolkit-for-ruby/ ? seems pretty decent.</p> webextraction projects are always interesting, what path are you taking If I may ask? =)

I guess You’ve seen http://blog.labnotes.org/2006/07/11/scraping-with-style-scrapi-toolkit-for-ruby/ ? seems pretty decent.

]]>