6 releases

0.7.2 Aug 19, 2024
0.7.1 Aug 9, 2024
0.6.0 Jul 23, 2024
0.5.3 Jul 10, 2024

#187 in Programming languages

28 downloads per month

MIT/Apache

12MB
3.5K SLoC

A Web Crawling Programming Language

GitHub Actions Workflow Status GitHub Actions Workflow Status GitHub Release Crates.io Version Docker version


ScoutLang is a DSL made for web scraping, focusing on a simple and expressive syntax. A powerful web crawling stack is abstracted away, allowing you to write powerful, easy to read scraping scripts.

Why Scout?

  • Gain access to powerful web scraping technology without needing expertise
  • A focus on developer velocity
  • Builtin debugging tools

example

Iterative script building

ScoutLang comes bundled with a full REPL and a powerful debugging mode, allowing you to visualize your web scraping scripts in real time.

debug

Installation

Eventually Scout installation will come bundled with the necessary pre-reqs. For now, you will need:

The binary can then be installed one of two ways:

  1. Cargo (requires Rust)
cargo install scoutlang
  1. Run the installer (requires Python3):
curl --proto '=https' --tlsv1.2 -LsSf https://raw.githubusercontent.com/maxmindlin/scout-lang/main/scripts/installer.py | python3

Both install the Scout interpreter into your path as scout.

Usage

The scout binary ran with a filename will read and interpret a script file. Without a script will start the REPL.

Available ENV variables:

  • SCOUT_DEBUG: Whether or not to open the debug browser. Defaults to false.
  • SCOUT_PORT: Which port to run Scout on. Defaults to a random open port. Do not set if you intend to run multiple scout instances at once as ports will conflict.
  • SCOUT_PROXY: An optional URL to proxy requests to. Defaults to none.
  • SCOUT_PATH: A path to where Scout installs dependencies, like the standard lib. Defaults to $HOME/scout-lang/.

License

Scout is dual-licensed with MIT & Apache 2.0, at your option.

Dependencies

~22–34MB
~638K SLoC