tail-s3

A command-line utility that monitors and streams content from an S3 object in real-time, similar to the Unix tail -f command but for S3 objects.

Features

• Stream content from S3 objects in real-time
• Support for AWS profiles and regions
• Automatic handling of range requests
• Real-time monitoring of appended content
• Error handling for invalid ranges

Installation

cargo install tail-s3

Usage

tail-s3 s3://my-bucket/my-key

Arguments

• s3_url: S3 URL in the format s3://bucket/key (required)
• --region, -r: AWS region (optional, uses default from AWS configuration)
• --profile, -p: AWS profile name (optional, defaults to "default")

Examples

# Monitor a log file in S3
tail-s3 s3://my-bucket/logs/application.log

# Use specific AWS profile and region
tail-s3 s3://my-bucket/logs/app.log --profile prod --region us-west-2

# Monitor training outputs
tail-s3 s3://ml-bucket/training/output.log --profile ml

AWS Configuration

The tool uses the AWS SDK for Rust and follows standard AWS configuration practices. Make sure you have:

1. AWS credentials configured (~/.aws/credentials)
2. AWS configuration set (~/.aws/config)

How it works

tail-s3 uses S3 range requests to efficiently fetch only new content that has been appended to the object. It polls the S3 object every second for new content and prints it to stdout, making it ideal for monitoring growing log files or other append-only files in S3.

License

MIT