#table #iceberg #metadata #view #catalog #update #apache

iceberg-rust

Unofficial rust implementation of the Iceberg table format

20 releases (6 breaking)

new 0.7.0 Feb 19, 2025
0.6.1 Dec 10, 2024
0.5.8 Oct 18, 2024
0.5.1 Jul 12, 2024
0.1.0 Dec 20, 2022

#499 in Database interfaces


Used in 13 crates

Apache-2.0

615KB
12K SLoC

Iceberg-rust

Low level implementation of the Apache iceberg specification.

Featues:

  • catalog trait as an interface to the different iceberg catalogs
  • table structs to simplify the access to the iceberg table metadata
  • table transactions to guarantee atomic changes to the iceberg table metadata
  • view structs to define iceberg views

lib.rs:

Apache Iceberg implementation in Rust

This crate provides a native Rust implementation of Apache Iceberg, a table format for large analytic datasets. Iceberg manages large collections of files as tables, while providing atomic updates and concurrent writes.

Features

  • Table operations (create, read, update, delete)
  • Schema evolution
  • Hidden partitioning
  • Time travel and snapshot isolation
  • View and materialized view support
  • Multiple catalog implementations (REST, AWS Glue, File-based)

Components

The main components of this crate are:

  • table - Core table operations and management
  • catalog - Catalog implementations for metadata storage
  • arrow - Integration with Apache Arrow
  • view - View and materialized view support
  • error - Error types and handling

Example

use iceberg_rust::table::Table;
use iceberg_rust::catalog::Catalog;

// Create a new table
let mut table = Table::builder()
    .with_name("example_table")
    .with_schema(schema)
    .build()
    .await?;

// Start a transaction
table.new_transaction(None)
    .update_schema(new_schema)
    .commit()
    .await?;

Dependencies

~52MB
~1M SLoC