Added NodeIterator and Stream support (#2051)

Improvement: added the NodeIterator class, to efficiently traverse a node tree using the Iterator interface.

And added Stream Element#stream() and Node#nodeStream() methods, to enable fluent composable stream pipelines of node traversals.

NodeIterator only hits nodes once (vs head and tail for NodeTraversor), is restartable, supports modifications of the node it just emitted (e.g. replace, remove), supports type filtering, and emits in document order.

Refactored most head-only uses of NodeTraversor to use the NodeIterator or a Stream backed by it.
15 files changed
tree: dea2335c4c2a67fe3964417fd8ffdbaeece7e3d7
  1. .github/
  2. src/
  3. .gitattributes
  4. .gitignore
  5. CHANGES
  6. LICENSE
  7. pom.xml
  8. README.md
  9. SECURITY.md
README.md

jsoup: Java HTML Parser

jsoup is a Java library that simplifies working with real-world HTML and XML. It offers an easy-to-use API for URL fetching, data parsing, extraction, and manipulation using DOM API methods, CSS, and xpath selectors.

jsoup implements the WHATWG HTML5 specification, and parses HTML to the same DOM as modern browsers.

  • scrape and parse HTML from a URL, file, or string
  • find and extract data, using DOM traversal or CSS selectors
  • manipulate the HTML elements, attributes, and text
  • clean user-submitted content against a safe-list, to prevent XSS attacks
  • output tidy HTML

jsoup is designed to deal with all varieties of HTML found in the wild; from pristine and validating, to invalid tag-soup; jsoup will create a sensible parse tree.

See jsoup.org for downloads and the full API documentation.

Build Status

Example

Fetch the Wikipedia homepage, parse it to a DOM, and select the headlines from the In the News section into a list of Elements:

Document doc = Jsoup.connect("https://en.wikipedia.org/").get();
log(doc.title());
Elements newsHeadlines = doc.select("#mp-itn b a");
for (Element headline : newsHeadlines) {
  log("%s\n\t%s", 
    headline.attr("title"), headline.absUrl("href"));
}

Online sample, full source.

Open source

jsoup is an open source project distributed under the liberal MIT license. The source code is available on GitHub.

Getting started

  1. Download the latest jsoup jar (or add it to your Maven/Gradle build)
  2. Read the cookbook
  3. Enjoy!

Android support

When used in Android projects, core library desugaring with the NIO specification should be enabled to support Java 8+ features.

Development and support

If you have any questions on how to use jsoup, or have ideas for future development, please get in touch via the mailing list.

If you find any issues, please file a bug after checking for duplicates.

The colophon talks about the history of and tools used to build jsoup.

Status

jsoup is in general, stable release.