#directory-structure #path #spacetime-db #represented #hierarchy #server #spacetime-paths

spacetimedb-paths

The spacetimedb directory structure, represented as a type hierarchy

1 release (0 unstable)

new 1.0.0-rc3 Jan 14, 2025

#1185 in Filesystem

Custom license

29KB
487 lines

The spacetimedb directory structure, represented as a type hierarchy.

Directory Structure of the Database.

SpacetimePaths holds the paths to the various directories used by the CLI & database.

  • cli-bin-dir: a directory under which all versions of all SpacetimeDB binaries is be stored. Each binary is stored in a directory named with version number of the binary in this directory. If a binary has any related files required by that binary which are specific to that version, for example, template configuration files, these files will be installed in this folder as well.

  • cli-config-dir: a directory where configuration and state for the CLI, as well as the keyfiles used by the server, are stored.

  • cli-bin-file: the location of the default spacetime CLI executable, which is a symlink to the actual spacetime binary in the cli-bin-dir.

  • data-dir: the directory where all persistent server & database files are stored.

Unix Directory Structure

On Unix-like platforms, such as Linux and macOS, the installation paths follow the XDG conventions by default:

  • cli-config-dir: $XDG_CONFIG_HOME/spacetime/
  • cli-bin-dir: $XDG_DATA_HOME/spacetime/bin/
  • cli-bin-file: $XDG_BIN_HOME/spacetime
  • data-dir: $XDG_DATA_HOME/spacetime/data

As per the XDG base directory specification, those base directories fall back to to the following defaults if the corresponding environment variable is not set:

  • $XDG_CONFIG_HOME: $HOME/.config
  • $XDG_DATA_HOME: $HOME/.local/share
  • $XDG_BIN_HOME: $HOME/.local/bin

For reference, the below is an example installation using the default paths:

 $HOME
 ├── .local
    ├── bin
    │   └── spacetime -> $HOME/.local/share/spacetime/bin/1.10.1/spacetimedb-update # Current, in $PATH
 │   └── share
 │       └── spacetime
 │           ├── bin
 │           |   └── 1.10.1
 │           |       ├── spacetimedb-update # Version manager
 │           |       ├── spacetimedb-cli # CLI
 │           |       ├── spacetimedb-standalone # Server
 │           |       ├── spacetimedb-cloud # Server
 │           |       ├── cli.default.toml # Template CLI configuration file
 │           |       └── config.default.toml # Template server configuration file
 |           └── data/
 |
 └── .config
     └── spacetime
         ├── id_ecdsa # Private key
         ├── id_ecdsa.pub # Public key
         └── cli.toml # CLI configuration

Windows Directory Structure

On Windows the installation paths follow Windows conventions, and is equivalent to a Root Directory (as defined below) at %LocalAppData%\SpacetimeDB\.

Note: the SpacetimeDB directory is in %LocalAppData% and not %AppData%. This is intentional so that different users on Windows can have different configuration and binaries. This also allows you to install SpacetimeDB on Windows even if you are not a privileged user.

Custom Root Directory

Users on all platforms must be allowed to override the default installation paths entirely with a single --root-dir argument passed to the initial installation commands.

If users specify a --root-dir flag, then the installation paths should be defined relative to the root-dir as follows:

  • cli-config-dir: {root-dir}/config/
  • cli-bin-dir: {root-dir}/bin/
  • cli-bin-file: {root-dir}/spacetime[.exe]
  • data-dir: {root-dir}/data/

For reference, the below is an example installation using the --root-dir argument:

{root-dir}
├── spacetime -> {root-dir}/bin/1.10.1/spacetimedb-update # Current, in $PATH
├── config
│   ├── id_ecdsa # Private key
│   ├── id_ecdsa.pub # Public key
│   └── cli.toml # CLI configuration
├── bin
|   └── 1.10.1
|       ├── spacetimedb-update.exe # Version manager
|       ├── spacetimedb-cli.exe # CLI
|       ├── spacetimedb-standalone.exe # Server
|       ├── spacetimedb-cloud.exe # Server
|       ├── cli.default.toml # Template CLI configuration file
|       └── config.default.toml # Template server configuration file
└── data/

Data directory structure

The following is an example of the internal structure of data-dir. Note that this is not a stable hierarchy, and users should not rely on it being stable from version to version.

{data-dir} # {Data}: CLI (--data-dir)
├── spacetime.pid # Lock file to prevent multiple instances, should be set to the pid of the running instance
├── config.toml # Server configuration (Human written, machine read only)
├── metadata.toml # Contains the edition, the MAJOR.MINOR.PATCH version number of the SpacetimeDB executable that created this directory. (Human readable, machine written only)
├── program-bytes # STANDALONE ONLY! Wasm modules aka `ProgramStorage` /var/lib/spacetime/data/standalone/2/program_bytes (NOTE: renamed from program_bytes)
│   └── d6
│       └── d9e66a8a285a416abd87e847c48b0990c6db6a5e0d5670c79a13f75dcabbe6
├── control-db # STANDALONE ONLY! Store information about the SpacetimeDB instances (NOTE: renamed from control_db)
│   ├── blobs/ # Blobs storage
│   ├── conf # Configuration for the Sled database
│   └── db # Sled database
├── cache
│   └── wasmtime
├── logs
│   └── spacetimedb-standalone.2024-07-08.log  # filename format: `spacetimedb-{edition}.YYYY-MM-DD.log`
└── replicas
    ├── 1 # Database `replica_id`, unique per cluster
    │   ├── clog # `CommitLog` files
    │   │   └── 00000000000000000000.stdb.log
    │   ├── module_logs # Module logs
    │   │   └── 2024-07-08.log # filename format: `YYYY-MM-DD.log`
    │   └── snapshots # Snapshots of the database
    │       └── 00000000000000000000.snapshot_dir # BSATN-encoded `Snapshot`
    │           ├── 00000000000000000000.snapshot_bsatn
    │           └── objects # Objects storage
    │               └── 01
    │                   └── 040a8585e6dc2c579c0c8f6017c7e6a0179a5d0410cd8db4b4affbd7d4d04f
    └── 34 # Database `replica_id`, unique per cluster
        ├── clog # `CommitLog` files
        │   └── 00000000000000000000.stdb.log
        ├── module_logs # Module logs
        │   └── 2024-07-08.log # filename format: `YYYY-MM-DD.log`
        └── snapshots # Snapshots of the database
            └── 00000000000000000000.snapshot_dir # BSATN-encoded `Snapshot`
                ├── 00000000000000000000.snapshot_bsatn
                └── objects # Objects storage directory trie
                    └── 01
                        └── 040a8585e6dc2c579c0c8f6017c7e6a0179a5d0410cd8db4b4affbd7d4d04f

Dependencies

~1–13MB
~88K SLoC