#python #file #pyo3 #read-file #object #file-path #helper

pyo3-file

A small helper library for working with python file-like objects with rust

14 releases (8 breaking)

new 0.10.0 Nov 23, 2024
0.8.1 Jun 11, 2024
0.7.0 Jun 26, 2023
0.5.0 Apr 16, 2022
0.2.0 May 20, 2019

#71 in FFI

Download history 1463/week @ 2024-08-03 580/week @ 2024-08-10 734/week @ 2024-08-17 1137/week @ 2024-08-24 518/week @ 2024-08-31 1244/week @ 2024-09-07 836/week @ 2024-09-14 875/week @ 2024-09-21 1054/week @ 2024-09-28 808/week @ 2024-10-05 1586/week @ 2024-10-12 785/week @ 2024-10-19 752/week @ 2024-10-26 546/week @ 2024-11-02 562/week @ 2024-11-09 1122/week @ 2024-11-16

3,131 downloads per month

MIT/Apache

13KB
224 lines

crates.io

PyO3-File

This is a small utility library to facilitate working with python file-like objects with rust.

Example

An example use case for this is when a file is opened in python, and needs to be passed to a rust library.

We could support both by introspecting the PyObject, and pick the correct behavior.

We would like this to work:

from path_or_file_like import accepts_path_or_file_like

def main():
    # should open `some_file.txt`.
    accepts_path_or_file_like("./some_file.txt")

    # should read from the python handle.
    f = open('./some_file.txt')
    accepts_path_or_file_like(f)

We could use pyo3_file to extend an existing a pyo3 module.

use pyo3_file::PyFileLikeObject;
use pyo3::types::PyString;

use pyo3::prelude::*;
use pyo3::wrap_pyfunction;
use std::io::Read;
use std::fs::File;

/// Represents either a path `File` or a file-like object `FileLike`
#[derive(Debug)]
enum FileOrFileLike {
    File(String),
    FileLike(PyFileLikeObject),
}

impl FileOrFileLike {
    pub fn from_pyobject(path_or_file_like: PyObject) -> PyResult<FileOrFileLike> {
        Python::with_gil(|py| {
            // is a path
            if let Ok(string_ref) = path_or_file_like.downcast::<PyString>(py) {
                return Ok(FileOrFileLike::File(
                    string_ref.to_string_lossy().to_string(),
                ));
            }

            // is a file-like
            match PyFileLikeObject::with_requirements(path_or_file_like, true, false, true, false) {
                Ok(f) => Ok(FileOrFileLike::FileLike(f)),
                Err(e) => Err(e)
            }
        })
    }
}

#[pyfunction]
/// Opens a file or file-like, and reads it to string.
fn accepts_path_or_file_like(
    path_or_file_like: PyObject,
) -> PyResult<String> {
    Python::with_gil(|py| {
        match FileOrFileLike::from_pyobject(path_or_file_like) {
            Ok(f) => match f {
                FileOrFileLike::File(s) => {
                    println!("It's a file! - path {}", s);
                    let mut f = File::open(s)?;
                    let mut string = String::new();

                    let read = f.read_to_string(&mut string);
                    Ok(string)
                }
                FileOrFileLike::FileLike(mut f) => {
                    println!("Its a file-like object");
                    let mut string = String::new();

                    let read = f.read_to_string(&mut string);
                    Ok(string)
                }
            },
            Err(e) => Err(e),
        }
    })
}

#[pymodule]
fn path_or_file_like(_py: Python, m: &PyModule) -> PyResult<()> {
    m.add_wrapped(wrap_pyfunction!(accepts_path_or_file_like))?;

    Ok(())
}


# fn main() {}

Dependencies

~2–3.5MB
~65K SLoC