11 releases

new 0.0.11 Oct 25, 2024
0.0.10 Aug 29, 2024

#13 in #readability

Download history 575/week @ 2024-08-24 124/week @ 2024-08-31 13/week @ 2024-09-07 35/week @ 2024-09-14 205/week @ 2024-09-21 179/week @ 2024-09-28 362/week @ 2024-10-05 114/week @ 2024-10-12 361/week @ 2024-10-19

1,039 downloads per month
Used in 2 crates (via spider_transformations)

MIT license

27KB
746 lines

llm_readability

The Rust readability library built for performance, AI, and multiple locales. The library is used on Spider Cloud for data cleaning.

Usage

[dependencies]
llm_readability = "0"
use llm_readability::extractor;

fn main() {
  match extractor::extract(&mut "<html>...</html>".as_bytes(), "https://example.com", None) {
      Ok(product) => {
          println!("------- html ------");
          println!("{}", product.content);
          println!("---- plain text ---");
          println!("{}", product.text);
      },
      Err(_) => println!("error occured"),
  }
}

This project is a rewrite of readability-rs for performance and bug fixes.

Dependencies

~9–15MB
~274K SLoC