11 releases

0.0.11 Oct 25, 2024
0.0.10 Aug 29, 2024

#15 in #readability

Download history 487/week @ 2024-08-23 209/week @ 2024-08-30 16/week @ 2024-09-06 31/week @ 2024-09-13 188/week @ 2024-09-20 185/week @ 2024-09-27 370/week @ 2024-10-04 101/week @ 2024-10-11 272/week @ 2024-10-18 290/week @ 2024-10-25 376/week @ 2024-11-01 242/week @ 2024-11-08 153/week @ 2024-11-15

1,109 downloads per month
Used in 2 crates (via spider_transformations)

MIT license

27KB
746 lines

llm_readability

The Rust readability library built for performance, AI, and multiple locales. The library is used on Spider Cloud for data cleaning.

Usage

[dependencies]
llm_readability = "0"
use llm_readability::extractor;

fn main() {
  match extractor::extract(&mut "<html>...</html>".as_bytes(), "https://example.com", None) {
      Ok(product) => {
          println!("------- html ------");
          println!("{}", product.content);
          println!("---- plain text ---");
          println!("{}", product.text);
      },
      Err(_) => println!("error occured"),
  }
}

This project is a rewrite of readability-rs for performance and bug fixes.

Dependencies

~9–17MB
~274K SLoC