8 releases (4 stable)
1.0.3 | Apr 9, 2023 |
---|---|
1.0.1 | Feb 17, 2023 |
1.0.0 | Jan 6, 2023 |
0.1.3 | Dec 11, 2022 |
#219 in Network programming
10,192 downloads per month
Used in 2 crates
61KB
1K
SLoC
buf-list
A segmented list of bytes::Bytes
chunks.
Overview
This crate provides a BufList
type that is a list of Bytes
chunks. The
type implements bytes::Buf
, so it can be used in any APIs that use Buf
.
The main use case for BufList
is to buffer data received as a stream of chunks without
having to copy them into a single contiguous chunk of memory. The BufList
can then be passed
into any APIs that accept Buf
.
If you've ever wanted a Vec<Bytes>
or a VecDeque<Bytes>
, this type is for you.
Cursors
This crate also provides Cursor
, which is a cursor type around a BufList
. A Cursor
around a BufList
implements Seek
, Read
and
BufRead
, similar to std::io::Cursor
.
Examples
Gather chunks into a BufList
, then write them all out to standard error in one go:
use buf_list::BufList;
use tokio::io::AsyncWriteExt;
let mut buf_list = BufList::new();
buf_list.push_chunk(&b"hello "[..]);
buf_list.push_chunk(&b"world"[..]);
buf_list.push_chunk(&b"!"[..]);
let mut stderr = tokio::io::stderr();
stderr.write_all_buf(&mut buf_list).await?;
Collect a fallible stream of Bytes
into a BufList
:
use buf_list::BufList;
use bytes::Bytes;
use futures::TryStreamExt;
// A common example is a stream of bytes read over HTTP.
let stream = futures::stream::iter(
vec![
Ok(Bytes::from_static(&b"laputa, "[..])),
Ok(Bytes::from_static(&b"castle "[..])),
Ok(Bytes::from_static(&b"in the sky"[..]))
],
);
let buf_list = stream.try_collect::<BufList>().await?;
assert_eq!(buf_list.num_chunks(), 3);
Converting to Stream
s
A BufList
can be converted into a futures::Stream
, or a TryStream
, of Bytes
chunks. Use
this recipe to do so:
(This will be exposed as an API on BufList
once Stream
and/or TryStream
become part of
stable Rust.)
use buf_list::BufList;
use bytes::Bytes;
use futures::{Stream, TryStream};
fn into_stream(buf_list: BufList) -> impl Stream<Item = Bytes> {
futures::stream::iter(buf_list)
}
fn into_try_stream<E>(buf_list: BufList) -> impl TryStream<Ok = Bytes, Error = E> {
futures::stream::iter(buf_list.into_iter().map(Ok))
}
Optional features
-
tokio1
: With this feature enabled,Cursor
implements thetokio
crate'sAsyncSeek
,AsyncRead
andAsyncBufRead
. -
futures03
: With this feature enabled,Cursor
implements thefutures
crate'sAsyncSeek
,AsyncRead
andAsyncBufRead
.Note that supporting
futures03
means exporting 0.x types as a public interface. This violates the C-STABLE guideline. However, the maintainer ofbuf-list
considers that acceptable sincefutures03
is an optional feature and not critical tobuf-list
. As newer versions of thefutures
crate are released,buf-list
will support their versions of the async traits as well.
Minimum supported Rust version
The minimum supported Rust version (MSRV) is 1.39, same as the bytes
crate. Optional
features may cause a bump in the MSRV.
The MSRV is not expected to change in the future. If the MSRV changes, it will be accompanied by
a major version bump to buf-list
.
Contributing
Pull requests are welcome! Please follow the code of conduct.
License
buf-list is copyright 2022 The buf-list Contributors. All rights reserved.
Copied and adapted from linkerd2-proxy; original code written by Eliza Weisman. linkerd2-proxy is copyright 2018 the linkerd2-proxy authors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use these files except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Dependencies
~0.1–5.5MB
~22K SLoC