Can anyone recommend me a java library to allow me XPath Queries over URLs? I've tried JAXP without success.
Thank you.
3
-
See stackoverflow.com/questions/9022140/… - not quite a duplicate as it asks about specific XPath functionality but there are better answers there.Mark Butler– Mark Butler2013-01-07 00:34:58 +00:00Commented Jan 7, 2013 at 0:34
-
@Reonarudo I am in the same situation as you were when you asked this question. There are many possible suggestions/solutions in the answers, but I would like to know which solution(library) you used and did it work out the way you wanted it ?Uther Pendragon– Uther Pendragon2015-06-20 19:08:10 +00:00Commented Jun 20, 2015 at 19:08
-
@UtherPendragon I'm sorry but this was a long time ago and I cannot recall which project was this. Anyway there should be newer/better libraries available nowadays.Leonardo Marques– Leonardo Marques2015-06-23 12:14:52 +00:00Commented Jun 23, 2015 at 12:14
Add a comment
|
5 Answers
There are several different approaches to this documented on the Web:
Using HtmlCleaner
- HtmlCleaner / Java DOM parser - Using XPath Contains against HTML in Java (This is the way I recommend)
- HtmlCleaner itself has a built in utility supporting XPath - See the javadocs http://htmlcleaner.sourceforge.net/doc/org/htmlcleaner/XPather.html or this example http://thinkandroid.wordpress.com/2010/01/05/using-xpath-and-html-cleaner-to-parse-html-xml/
Using Jericho
I have tried a few different variations of these approaches, i.e. HtmlParser plus the Java DOM parser, and JSoup plus Jaxen, but the combination that worked best is HtmlCleaner plus the Java DOM parser. The next best combination was Jericho plus Jaxen.
3 Comments
yanchenko
Note that on Android 4.2.2 HtmlCleaner 2.5 turned out to be 4x slower compared to jSoup 1.7.2.
sibbl
Note that HtmlCleaner only supports XPath 1.0.
Andrew Scott Evans
HTML Cleaner + DOM Serializer + Threading = Really bad memory leak
jsoup, Java HTML Parser Very similar to jQuery syntax way.
5 Comments
Artem Barger
I'm not sure. It does much simpler queries, which xpath based. you can read some documentation and there are a lot of cool examples, explaining how to run that queries.
brabec
jsoup (at least in version 1.7.3) doesn't suppport XPath.
phil
jsoup use css/jQuery syntax way ,which is similar as and better than XPath
Neil McGuigan
CSS Selectors are not better than XPath. There are some things which you can select in XPath but not CSS Selectors
Jonathan Hedley
jsoup now supports xpath, as well as CSS selectors. Since September 2021 in jsoup 1.14.3.
Use Xsoup. According to the docs, it's faster than HtmlCleaner. Example
@Test
public void testSelect() {
String html = "<html><div><a href='https://github.com'>github.com</a></div>" +
"<table><tr><td>a</td><td>b</td></tr></table></html>";
Document document = Jsoup.parse(html);
String result = Xsoup.compile("//a/@href").evaluate(document).get();
Assert.assertEquals("https://github.com", result);
List<String> list = Xsoup.compile("//tr/td/text()").evaluate(document).list();
Assert.assertEquals("a", list.get(0));
Assert.assertEquals("b", list.get(1));
}
Link to Xsoup - https://github.com/code4craft/xsoup
Comments
You could use TagSoup together with Saxon. That way you simply replace any XML SAX parser used with TagSoup and the XPath 2.0 or XSLT 2.0 or XQuery 1.0 implementation works as usual.
Comments
I've used JTidy to make HTML into a proper DOM, then used plain XPath to query the DOM.
If you want to do cross-document/cross-URL queries, better use JTidy with XQuery.
