#nlp #chinese #segmenation

jieba-rs

The Jieba Chinese Word Segmentation Implemented in Rust

43 releases

0.7.2 Jan 21, 2025
0.7.1 Dec 25, 2024
0.7.0 Apr 16, 2024
0.6.8 Jul 16, 2023
0.2.3 Jul 6, 2018

#63 in Text processing

Download history 8037/week @ 2024-10-27 7843/week @ 2024-11-03 7140/week @ 2024-11-10 7739/week @ 2024-11-17 7203/week @ 2024-11-24 7175/week @ 2024-12-01 7012/week @ 2024-12-08 6806/week @ 2024-12-15 7546/week @ 2024-12-22 8111/week @ 2024-12-29 11995/week @ 2025-01-05 12567/week @ 2025-01-12 9895/week @ 2025-01-19 4558/week @ 2025-01-26 9526/week @ 2025-02-02 10613/week @ 2025-02-09

35,035 downloads per month
Used in 24 crates (17 directly)

MIT license

4MB
2K SLoC

jieba-rs

GitHub Actions codecov Crates.io docs.rs

🚀 Help me to become a full-time open-source developer by sponsoring me on GitHub

The Jieba Chinese Word Segmentation Implemented in Rust

Installation

Add it to your Cargo.toml:

[dependencies]
jieba-rs = "0.7"

then you are good to go. If you are using Rust 2015 you have to extern crate jieba_rs to your crate root as well.

Example

use jieba_rs::Jieba;

fn main() {
    let jieba = Jieba::new();
    let words = jieba.cut("我们中出了一个叛徒", false);
    assert_eq!(words, vec!["我们", "", "", "", "一个", "叛徒"]);
}

Enabling Additional Features

  • default-dict feature enables embedded dictionary, this features is enabled by default
  • tfidf feature enables TF-IDF keywords extractor
  • textrank feature enables TextRank keywords extractor
[dependencies]
jieba-rs = { version = "0.7", features = ["tfidf", "textrank"] }

Run benchmark

cargo bench --all-features

Benchmark: Compare with cppjieba

jieba-rs bindings

License

This work is released under the MIT license. A copy of the license is provided in the LICENSE file.

Dependencies

~11MB
~120K SLoC