Lib.rs

This crate just contains a simple token list data structure, which is a list of tokens. Token lists are basically a simple version of symbolic expressions, but can only contain symbols or expressions. This behavior might be extended in the future to also support custom data if it's helpful for some contexts.

It was meant to be used by multiple libraries. For example, there could be multiple parsers, all using this library as backends and generating token lists, and multiple libraries using these lists, for example as markup format to display something or as programming langauge, which they interpret or compile.

This way, developers can separate their langauge representation from their langauge logic, and users can just combine the internal logic and the representation from their favorite languages written this way.

But now you should rather use the token parser, which is similar, but includes utilities for conversion.

If you want to work with token lists directly, you should probably use this. Both libs are even compatible.