iroh-blobs implements
request/response and streaming transfers of arbitrary-sized byte blobs, using
BLAKE3-verified streams and content-addressed links.
Concepts
- Blob: an opaque sequence of bytes (no embedded metadata).
- Link: a 32-byte BLAKE3 hash that identifies a blob.
- HashSeq: a blob that contains a sequence of links (useful for chunking/trees).
- Provider / Requester: provider serves data; requester fetches it. An endpoint can be both.
Installation
Tutorial
Let’s dive into iroh by building a simple peer-to-peer file transfer tool in rust! At the end we should be able to transfer a file from one device by running this:Get set up
We’ll assume you’ve set up rust and cargo on your machine. Initialize a new project by running:src/main.rs file.
Create an iroh::Endpoint
To start interacting with other iroh endpoints, we need to build an iroh::Endpoint.
This is what manages the possibly changing network underneath, maintains a connection to the closest relay, and finds ways to address devices by EndpointId.
Using an existing protocol: iroh-blobs
Instead of writing our own protocol from scratch, let’s use iroh-blobs, which already does what we want: It loads files from your file system and provides a protocol for seekable, resumable downloads of these files.Learn more about what we mean by “protocol” on the protocol documentation page.
Endpoint.
At this point what we want to do depends on whether we want to accept incoming iroh connections from the network or create outbound iroh connections to other endpoints.
Which one we want to do depends on if the executable was called with send as an argument or receive, so let’s parse these two options out from the CLI arguments and match on them:
todo!()s!
Getting ready to send
If we want to make a file available over the network with iroh-blobs, we first need to hash this file.What does this step do?It hashes the file using BLAKE3 and remembers a so-called “outboard” for that file.
This outboard contains information about hashes of parts of this file.
All of this enables some extra features with iroh-blobs like automatically verifying the integrity of the file while it’s streaming, verified range downloads and download resumption.
For other use cases, there are other ways of importing blobs into iroh-blobs, you’re not restricted to pulling them from the file system!
You can see other options available, such as
add_slice.
Make sure to also check out the options you can pass and their documentation for some interesting tidbits on performance.tag contains the final piece of information such that another endpoint can fetch a blob from us.
We’ll use a BlobTicket to put the file’s BLAKE3 hash and our endpoint’s EndpointId into a single copy-able string:
Router.
Similar to routers in webserver libraries, it runs a loop accepting incoming connections and routes them to the specific handler.
However, instead of handlers being organized by HTTP paths, it routes based on “ALPNs”.
Read more about ALPNs and the router on the protocol page.
In our case, we only need a single protocol, but constructing a router also takes care of running the accept loop, so that makes our life easier:
Ctrl+C in the console and once they do so, we shut down the router gracefully.
Connecting to the other side to receive
On the connection side, we got theticket and the path from the CLI arguments and we can parse them into their struct versions.
With them parsed
- we first construct a
Downloader(that can help us coordinate multiple downloads from multiple peers if we’d want to) - and then call
.downloadwith the information contained in the ticket and wait for the download to finish:
Reusing the same downloader across multiple downloads can be more efficient, e.g. by reusing existing connections.
In this example we don’t see this, but it might come in handy for your use case.
The return value of
.download() is DownloadProgress.
You can either .await it to wait for the download to finish, or you can stream out progress events instead, e.g. if you wanted to use this for showing a nice progress bar!This first downloads the file completely into memory, then copies it from memory to file in a second step.There’s ways to make this work without having to store the whole file in memory!
This would involve setting up an
FsStore instead of a MemStore and using .export_with_opts with ExportMode::TryReference.
Something similar can be done on the sending side!
We’ll leave these changes as an exercise to the reader 😉That’s it!
You’ve now successfully built a small tool for peer-to-peer file transfers! 🎉 The full example with the very latest version of iroh and iroh-blobs can be viewed on github. If you’re hungry for more, check out- the iroh rust documentation,
- Video streaming with iroh-roq,
- Writing your own protocols,
- other examples, or