i am screenscraping using watir and i download a xls file. when i open this file in notepad, i find that its just a bunch of html tables. is there a any function or gem that will convert this page into a bunch of arrays. any ideas is appreciated.
1
-
Show us the code. What you have and what you would like to get from it.Željko Filipin– Željko Filipin2010-10-18 09:58:07 +00:00Commented Oct 18, 2010 at 9:58
Add a comment
|
3 Answers
- Narrow it down to ...
- Clear out the whitespace
- Replace the tabs with "
- Replace tags with ",
- Replace the & & tags with nothing
- Replace the tags with |
- Split the rows with |
- Split the fields with ,
You can simplify it a little bit more, but that's the gist of it.
Comments
In general it is a simple exercise to walk through a HTML file with a table and extract rows and columns as long as they don't use colspan or rowspan attributes. Those mess up the logical flow requiring some sensing of the gaps they cause, and a need to fill in the gaps with the repeated value from the *spans. How do I parse an HTML table with Nokogiri? might help.
From looking at XLS files on my desktop I don't think they're XML or HTML. I'm not sure what you downloaded. I did a quick search and roo (http://roo.rubyforge.org/) appears to be a good starting point.
Comments
XLS is a binary format. If you are seeing HTML tables in the file contents it means you probably did not download the file correctly.
How is the XLS file being downloaded through Watir? Are you having to automate the File Download window, or did you just follow a link to the XLS file and write the contents to a file?
