downlowd

Downloading a file is easy. Just make an HTTP request, and write the results to a file, right? That works, but it doesn't cover a lot of corner cases. downlowd supports:

• Streaming the file to disk, instead of downloading it to memory and then writing it to disk. This is both faster and far more memory efficient for large files.
• Progress callback for displaying progress bar.
• Resuming file downloads.
• Files are written to disk as "filename.part" and then renamed to "filename" on completion, to make it obvious the file isn't complete.
• Automatic retries for flakey network connections and servers, with exponential backoff.
• Uses content-disposition header to retrieve the name of the file.
• Support for bandwidth restrictions.
• Blocking client which is based on ureq so the blocking case doesn't depend on reqwest or tokio, making a smaller executable and faster build times.

Documentation

See the documentation at docs.rs.

Crate Features

• async - Enabled by default. Provides Client for downloading files. This uses reqwest and tokio under the hood.
• blocking - Provides the blocking::Client for downloading files, which is based on ureq. To use the blocking client, install will cargo add downlowd --no-default-features -F blocking.

Usage

This is the simplest example:

# tokio_test::block_on(async {
use downlowd::Client;
# use temp_dir::TempDir;
# let dir = TempDir::new()?;
# let dirname = dir.path();

let client = Client::new();
let result = client
    .get("http://localhost:8089/hello.txt")
    .destination(dirname)
    .send()
    .await?;

assert_eq!(&result.path, &dirname.join("hello.txt"));
let file_contents = tokio::fs::read_to_string(&result.path).await?;
assert_eq!(file_contents, "hello world");
# Ok::<(), Box<dyn std::error::Error>>(())
# }).unwrap()

This is a short example, but it has a lot packed into it. First, since we've passed in a directory as the destination, this will work out what filename the file should be saved as In this case, the filename is derived from the URL, but if the server responds to a HEAD request with a Content-Disposition header with a filename, we'll use that filename. Note here we could also specify a filename instead of a directory name, and then downlowd would write our file to the specified filename.

If the file already exists, and has the correct length, then downlowd will just report success right away! If not, then we'll start downloading the file into a file named hello.txt.part. The file will be renamed to hello.txt once the download is complete. While the download is in progress, a "sidecar" file named hello.txt.downloadinfo will be written alongside the file which will contain cache information about the file (the etag header, the last-modified header, etc...). The sidecar file is used to help determine whether or not a file has changed on the server if the download is interrupted and needs to be resumed.

If there's an error during the download, such as a network error, or the transfer is interrupted, or the server returns a 5xx error, then downlowd will automatically retry the file, with an exponential backoff between retries. By default, downlowd will retry forever. Calling max_retries() will set a maximum number of tries, but note that downlowd will reset the retry counter if any progress is made downloading the file.

Reporting Progress

There are a couple of ways you can hook into downlowd to report on progress. The on_progress handler is called once at the start of the download, and then whenever bytes are downloaded.

# tokio_test::block_on(async {
# use downlowd::Client;
# use temp_dir::TempDir;
# let dir = TempDir::new()?;
# let dirname = dir.path();

let client = Client::new();
let result = client
    .get("http://localhost:8089/hello.txt")
    .destination(dirname.join("file.txt"))
    .on_progress(|progress| {
        println!(
            "Downloaded {} of {} bytes",
            progress.bytes(),
            progress.remote_length().unwrap()
        );
    })
    .send()
    .await?;

assert_eq!(&result.path, &dirname.join("file.txt"));
# Ok::<(), Box<dyn std::error::Error>>(())
# }).unwrap();

The progress handle can also be used to cancel a download via progress.cancel(). There's quite a bit of data about the download that can be retrieved from the progress handle. You can see an example in the examples folder which uses indicatif to render a pretty download progress bar. If you've cloned the repo, you can run it with something like:

cargo run https://releases.ubuntu.com/24.04.3/ubuntu-24.04.3-desktop-amd64.iso .

Customizing Retries

You can also use the on_retry() method to register a handler that will be run immediately prior to a retry. This can be used to customize the backoff, or cancel the download:

# tokio_test::block_on(async {
# use std::time::Duration;
# use temp_dir::TempDir;
use downlowd::{Client, Error};
# let dir = TempDir::new()?;
# let dirname = dir.path();

let client = Client::new();
let result = client
    .get("http://localhost:8089/hello.txt")
    .destination(dirname)
    .on_retry(|r| {
        if matches!(r.error(), Error::FileChanged { .. }) {
            // No delay if the file changed.
            r.set_delay(Duration::ZERO);
        } else {
            r.set_delay(downlowd::exponential_backoff(
                Duration::from_secs(1),
                Duration::from_secs(30),
                r.retries(),
            ));
        }
    })
    .send()
    .await?;

# Ok::<(), Box<dyn std::error::Error>>(())
# }).unwrap();

Again, you can call r.cancel() here to not retry at all, and instead fail the entire download.

Client Options

You can create a custom client using the ClientBuilder:

# tokio_test::block_on(async {
use downlowd::{ClientBuilder, Client};
# use temp_dir::TempDir;
# let dir = TempDir::new()?;
# let dirname = dir.path();

let client = ClientBuilder::new()
    .user_agent("my-cool-app")
    .header("Authorization", "Bearer secret-token")
    .build()?;

let result = client
    .get("http://localhost:8089/hello.txt")
    // Can set headers at the request level, too.
    .header("x-my-custom-header", "canon")
    .destination(dirname.join("file.txt"))
    .send()
    .await?;
# Ok::<(), Box<dyn std::error::Error>>(())
# }).unwrap()