8 releases

0.2.0 Oct 17, 2024
0.1.2 Mar 6, 2024
0.1.1 Feb 8, 2024
0.1.0-beta.2 Jan 29, 2024

#219 in HTTP client

Download history 71/week @ 2024-11-16 39/week @ 2024-11-23 87/week @ 2024-11-30 66/week @ 2024-12-07 18/week @ 2024-12-14 2/week @ 2024-12-21 28/week @ 2024-12-28 56/week @ 2025-01-04 128/week @ 2025-01-11 28/week @ 2025-01-18 42/week @ 2025-01-25 142/week @ 2025-02-01 155/week @ 2025-02-08 64/week @ 2025-02-15 53/week @ 2025-02-22 115/week @ 2025-03-01

404 downloads per month
Used in bed-reader

MIT/Apache

46KB
387 lines

cloud-file

github crates.io docs.rs build status

Simple reading of cloud files in Rust

Highlights

Install

cargo add cloud-file

Examples

Find the size of a cloud file.

use cloud_file::CloudFile;
# Runtime::new().unwrap().block_on(async {  // '#' needed for doctest

let url = "https://raw.githubusercontent.com/fastlmm/bed-sample-files/main/toydata.5chrom.fam";
let cloud_file = CloudFile::new(url)?;
let file_size = cloud_file.read_file_size().await?;
assert_eq!(file_size, 14_361);
# Ok::<(), Box<dyn std::error::Error>>(()) }).unwrap();
# use {cloud_file::CloudFileError, tokio::runtime::Runtime};

Find the number of lines in a cloud file.

use cloud_file::CloudFile;
use futures::StreamExt; // Enables `.next()` on streams.
# Runtime::new().unwrap().block_on(async { // '#' needed for doctest

let url = "https://raw.githubusercontent.com/fastlmm/bed-sample-files/main/toydata.5chrom.fam";
let cloud_file = CloudFile::new_with_options(url, [("timeout", "30s")])?;
let mut chunks = cloud_file.stream_chunks().await?;
let mut newline_count: usize = 0;
while let Some(chunk) = chunks.next().await {
    let chunk = chunk?;
    newline_count += bytecount::count(&chunk, b'\n');
}
assert_eq!(newline_count, 500);
# Ok::<(), Box<dyn std::error::Error>>(()) }).unwrap();
# use {cloud_file::CloudFileError, tokio::runtime::Runtime};   

More examples

Example Demonstrates
line_count Read a file as binary chunks.
nth_line Read a file as text lines.
bigram_counts Read random regions of a file, without regard to order.
aws_file_size Find the size of a file on AWS.

Dependencies

~9–18MB
~243K SLoC