This is not a straight followup of my previous HTSP implementation post! It is more of a skew followup that can perfectly be read separately.

A lot of time has passed since Februari 2017, when I first started experimenting with Rust. I’ve been using Rust on an almost-daily basis since!

As with most hobby projects, my htsp-rs implementation became a side-side-side project, and it will probably stay that way. Another property of those hobby projects is that I now-and-then pick them up again, and htsp-rs has come to that point now. The reason for picking it up again is different from the reason of its existence though, and that’s a good thing: it means I have multiple reasons for the crate’s existence. Initially, I wanted it as a back-end for a livestreaming app on SailfishOS. Today, I want it as back-end for mpd’s TVHeadend support. If you don’t know what TVHeadend is, both that issue and my previous post on the topic have a quick introduction. Come back quickly though, because the code in there is quite old.

Some quick history

Between Februari 2017 and now, I’ve started yet another Rust project: compass. Compass is meant to put TVHeadend DVR recordings in their respective directories, and uses the HTTP API of TVHeadend (among some other APIs) for that. For me, it made sense to merge the hts-rs (HTTP) part of compass and the separate htsp-rs crate. How both parts should integrate, I don’t know yet.

But there’s a problem: there’s two years between htsp-rs and compass, and there’s three months (!) between compass and today. Both crates used nightly features from around their own time. For htsp-rs, that was just plain futures and tokio, which was quite revolutionary for Rust at the time. await! was still not very stable, I had a separate branch that tracked the then-famous futures-await crate on git, and that was about the last. It still used a lot of manual Future chaining, which made working asynchronously very tedious. Compass is a lot more recent, from the time that std::futures was just around the corner, and the await! macro was also a nightly feature. But even in these three months stuff has changed. await! got (notoriously) replaced by .await; it started a small civil war inside the Rust community!

Updating the ancient three-month ond project

I’ve just finished updating compass to the .await syntax, and made another awesome change with it. You see, I wanted to use the std::await! macro three months ago. It was fancy, new, and almost the standard. tokio had a compatibily layer for it, because it’s still running on futures 0.1 (the “new” futures are referred to as futures 0.3), but it meant that I had to constantly convert between 0.3 and 0.1. I made a convenience macro:

macro_rules! spawn_to_result {
    ($e:expr) => { {
        let mut rt = Runtime::new().unwrap();
        let f = Compat::new(Box::pin($e));
        rt.block_on(f)
    }};
}

I used that in test cases to “conveniently” test the HTTP API libraries. I couldn’t use the new-‘n’-fancy romio, because reqwest relies on the incompatible tokio.

But the async workgroup got some other experiment running: runtime. runtime acts as glue between any asynchronous runtime (e.g. tokio) and async syntax.

I can now write

#[runtime::test(runtime_tokio::Tokio)]
async fn unauthorized() -> Result<(), Error> {
    let server = dotenv::var("HTS_URL").unwrap();
    let username = "nonexistent-user";
    let password = "garbage password";

    let client = Client::new(server, (username, password));

    let recordings = client.finished_recordings().await;
    let error = recordings.expect_err("Unauthorized");
    let error: hts::HtsClientError = error.downcast().expect("HtsClientError");

    assert_eq!(error, hts::HtsClientError::Unauthorized);
    Ok(())
}

It allowed me to move to .async easily, write my tests as first-class citizens, get rid of my ugly conversion macro, and drop 50 lines of code all at once!

UPDATE: I’m still working a bit on translating the bits and pieces of htsp-rs to .await and the new tokio crates. One of the things I notice is that a lot more methods can be implemented with async fn vs returning a raw Box<Future>, which does seem to reduce the amount of code by a lot. The usual strategy of moving the code, and reading and fixing compiler errors works quite well. It also gives quite the opportunity to fix up old “rusty” (⸘) code, and refactor some things that desparately needed refactoring.

Some other thing I notice i that Tokio threw out quite some features. For example, the Service trait – which I used – has disappeared, but is quite easily replaced.

If you want to look at what I do, have a look at the htsp merge request on the repository, while I go along.

I will probably overthrow some design decisions of back then, partly because the futures design has changed, partly because .await support makes certain patterns easier. For example, I used a (type)state machine for encoding and enforcing the current state of the protocol at compile time:

pub struct ConnectionStateMachine<S> {
    state: S,
    service: HtspService,
}

The state was one of three structs:

pub struct Connected;
pub struct Negotiated {
    serverdata: ServerData,
}
pub struct Authenticated {
    serverdata: ServerData,
}

First, I’m moving the Authenticated state into the Negotiated state (as a sub-state, let’s say). Second, I also had an enum around the ConnectionStateMachine, nthat allowed me to handle the state at runtime. Since we have pretty first-class async support now, it’s not needed as much anymore. In fact, I’m not even sure anymore why I wanted the states evaluated at runtime. Anyhow, I managed to rewrite

let connector =
    htsp::Connection::connect(&addr, &handle.clone())
    .and_then(|connection| {
        println!("TCP Connected! Saying hello");
        connection.hello("Connect demo".into(), "0.1.0".into())
    })
    .and_then(|connection| {
        println!("Response from connection:");
        println!(" Negotiated version: {}", connection.version().unwrap());
        println!(" Server name: {} {}", connection.server_name().unwrap(),
                                        connection.server_version().unwrap());
        connection.authenticate(&username, &password)
    })
    .map_err(|err| {
        println!("Could not authenticate as {}", username);
        err
    });

as

let client = TcpStream::connect(&addr).await?;
info!("Tcp connected");

let client = hts::HtspClient::bind_client(client)
    .hello("hts/tests/htsp.rs", env!("CARGO_PKG_VERSION")).await?;
info!("Htsp connected");
info!(" Negotiated version: {}", client.version()?);
info!(" Server name: {} {}", client.server_name()?, connection.server_version()?);

let client = client.authenticate(username, password).await?;
info!("Htsp authenticated");

Disclaimer: the actual RPC has not been ported yet, but I’m quite confident that this will work.

Another redesign that should be possible, is to make RPC methods take &mut self in a clean way (without allocating extra channels), since futures::Sink now also implements its method using &mut self, instead of taking ownership and returning self.

The Rust async ecosystem is really gaining some beautiful code now, and is starting to feel quite productive. I really feel that the new futures API works quite well, especially now with runtime. I can now write both hts-rs and htsp-rs using the exact same tools, and that’s really awesome. I’m looking forward a lot to what Rusts asynchronous programming will bring.