Scie

Scie is a research about how to build simple code identify engine for different languages.

goal: build a better code figure engine for code refactoring.

• scie-bingen. generate languages bindata.
• scie-detector. detector for different frameworks & languages.
• scie-grammar. A library that helps tokenize text using Text Mate grammars.
• scie-infra. common infrasturcture support, like fs
• scie-onig. Rust FFI for Oniguruma.
• scie-model. common model of VSCode models & Miao Model.
• scie-scanner. Wrapper Rust Oniguruma FFI api.
• scie-cli. cli part ofr Scie.

Guideline

major issues:

• performance
  • rule in Grammar
  • UTF 8 to UTF 16 in UtfString
  • normal issue
• unstable
  • Random test failure on OnigScanner.
  • GC issues on OnigScanner.
    • GC issue seems resolved with Jemalloc.
    • Signal 6 (SIGABRT) = SIGABRT is commonly used by libc and other libraries to abort the program in case of critical errors. For example, glibc sends an SIGABRT in case of a detected double-free or other heap corruptions.
    • maybe UTF8 encoding issue

Todo:

• replace with fancy-regex for pure Rust impl
• process todo
  • back references
  • multiple languages
• rewrite VSCode-textmate with Rust
  • language for testing
  • support others language
• benchmark
  • fast than VSCode version
• multiple languages one project support
• analyser
  • line counts
  • keywords map

DevSetup

1. git clone

git clone https://github.com/phodal/scie/

2. run

cargo run scie

install just

curl --proto '=https' --tlsv1.2 -sSf https://just.systems/install.sh | bash -s -- --to DEST

run tests

just tests

Documents

refs

• Rust FFI 实践
• Oniguruma Regular Expressions
• https://github.com/atom/node-oniguruma

License

scie-grammar based on vscode-textmate with MIT LICENSE see in scie-grammar/src/scanner/LICENSE

onigvs based on rust-onig

@ 2020 A Phodal Huang's Idea. This code is distributed under the MPL license. See LICENSE in this directory.