fimo (file-mongo)

Fimo is a fast and flexible CLI tool written in Rust that transforms CSV data into MongoDB documents using YAML-based field mappings and Jinja2-style templating. It's ideal for bulk inserts, updates, and upserts with full control over document structure.

🚀 Features

• ✅ RFC 4180-compliant CSV parsing (including headers, quoting, escaped quotes)
• 🛠️ Field mapping via YAML configuration
• 🧠 Custom transformation logic using MiniJinja
• 📦 MongoDB insert, update, and upsert support
• 🧪 Validate-only and dry-run modes
• 🔄 Batch processing support for large files
• 🔐 Supports Extended JSON and BSON types
• 🔣 Configurable CSV delimiter and quote characters
• 📊 Debug and verbose output for development and testing

📦 Installation

cargo install fimo

Or clone and build:

git clone https://github.com/fimo-org/fimo.git
cd fimo
cargo build --release

📝 Usage

fimo \
  --input tests/data/extended.csv \
  --mapping tests/mapping/extended.yaml \
  --template-dir tests/templates \
  --mongo-uri mongodb://localhost:27017 \
  --db testdb \
  --collection testcol \
  --operation upsert \
  --extended-json \
  --debug

🧪 Example: With Templates and Extended JSON

📁 data.csv

_id,price,created_at,name,active
507f1f77bcf86cd799439011,12.34,2024-01-01T10:00:00Z,Alice,yes

🧩 mapping.yaml

_id:
  type: objectId
price:
  type: decimal
created_at:
  type: date
name:
  type: string
active:
  type: bool
  truthy: ["yes", "true", "1", "Y"]
  falsy: ["no", "false", "0", "N"]

🧾 templates/upsert.j2

{
  "filter": { "_id": {{ row._id }} },
  "update": {
    "$set": {
      "price": {{ row.price }},
      "created_at": {{ row.created_at }},
      "name": "{{ row.name }}",
      "active": {{ row.active }}
    },
    "$setOnInsert": {
      "created_at": {{ row.created_at }}
    }
  }
}

▶️ Run the Import

fimo \
  --input data.csv \
  --mapping mapping.yaml \
  --template-dir templates \
  --mongo-uri mongodb://localhost:27017 \
  --db testdb \
  --collection customers \
  --operation upsert \
  --extended-json

🧪 Example: Raw Insert (No Templates)

📁 simple.csv

name,age,active
Bob,25,true

🧩 simple.yaml

name:
  type: string
age:
  type: int
active:
  type: bool

▶️ Raw Insert Command

fimo \
  --input simple.csv \
  --mapping simple.yaml \
  --mongo-uri mongodb://localhost:27017 \
  --db demo \
  --collection people \
  --operation insert \
  --raw-insert

🔧 CLI Options

🧠 Truthy/Falsy Mapping for Booleans

In mapping.yaml, you can define per-field truthy/falsy values:

active:
  type: bool
  truthy: ["yes", "1", "true"]
  falsy: ["no", "0", "false"]

This allows more natural mapping from "yes"/"no", "Y"/"N" strings into true/false.

📁 Project Structure

.
├── src/
│   ├── main.rs             # CLI entry point
│   ├── cli.rs              # Command-line argument parsing
│   ├── mongo.rs            # MongoDB connection
│   ├── transform.rs        # Mapping, templating, BSON conversion
│   ├── mapping.rs          # YAML field type parsing
│   └── template.rs         # Jinja environment loader
├── mappings/               # Sample mapping YAML files
├── templates/              # Sample Jinja templates
├── tests/                  # Sample CSV input for testing
└── Cargo.toml              # Project manifest

📚 RFC 4180 Compatibility

Fimo is fully compatible with RFC 4180:

• Comma-separated fields (configurable)
• Quoted fields with escape support
• Optional headers
• Uniform field count (recommended but not enforced)

📜 License

MIT ©