ReLearn: A Reinforcement Learning Library

A reinforcement learning library and experiment runner. Uses pytorch as the neural network backend via the tch interface to the C++ API.

At the moment this is designed for personal use. It is in-development and unstable so expect breaking changes with updates.

Read the documentation at https://docs.rs/relearn.

Examples

Chain Environment with Tabular Q Learning

cargo run --release --example chain-tabular-q

This environment has infinitely long episodes.

Cart-Pole with Trust-Region Policy Optimization

cargo run --release --example cartpole-trpo

cargo run --release --example cartpole-trpo data/cartpole-trpo/<time>/actor.cbor

Uses a feed-forward MLP for the policy and a separate MLP for the critic (baseline). The displayed statistics are also saved to data/cartpole-trpo/ and can be viewed with tensorboard --logdir data/cartpole-trpo.

lib.rs:

A reinforcement learning library.

This library defines a set of environments and learning agents and simulates their interaction.

Environments implement the Environment trait, which has associated observation, action, and state types. Agents implement Agent and provide Actors that generate actions in response to environment observations. Agents can learn via the BatchUpdate trait.

Agent traits are generic over the observation (O) and action (A) types of the environment. The EnvStructure trait provides more details about possible values for these types via the Space trait. A Space can be thought of as a runtime-defined type, describing a set of possible values while methods are provided by other traits in spaces.

Environment-actor simulation is performed by Steps and the resulting Step are accessible via an Iterator interface. Training is performed by train_serial and train_parallel.

This library uses PyTorch via [tch].