It's called sans-io in Python land, which is where I heard it first.

https://sans-io.readthedocs.io/

I did it for one of my Rust projects back in 2018 https://github.com/Arnavion/k8s-openapi/commit/9a4fbb718b119... , and it's older than that in Python land.

The above sans-io page links to this PyCon 2016 talk:

Cory Benfield - Building Protocol Libraries The Right Way https://youtu.be/7cC3_jGwl_U

Seems like a rediscovery of "pure functions" from the FP world?

Well, only "pure" in the sense no IO effect happens. I doubt mentioned library neglects state or global variables

I don't mean to be snarky in any way. I think this is actually great development.

But isn't this just good old inversion of control, modularity with maybe some inspiration from Functional Programming. Or even more generally, good Software architecture and engineering?

Anyway, I'm very happy to see this, the more code is architected this way, the better for all our industry.

> One of the interesting patterns happening in Rust is io-less libraries.

Not to join in too much on the "but we already had this!" bandwagon ... but actually to join in the bandwagon - we had these sorts of patterns in C 20 years ago. I worked with a TLS implementation like this sometime around 2005.

Either you work with buffer-in/buffer-out types of scenario, or you register callbacks to do the real IO.

It's a great pattern and means that you can do stuff like change transport mechanism, or put in proxying, or whatever you want really, without having to change the library.

With embedded C stuff it was also relatively common to handle memory allocation this way - buy in a library that does whatever proprietrary thing you need from a vendor, then register your custom allocator with it so when it needed heap memory, you could provide that in line with whatever platform restrictions you had going.

How is this implemented in practice? Special care to keep io on the outermost layers? Never thought about software in this way. Seems really tough, but interesting

Wonder how well it scales to larger applications. Ie is there a codesize where io-less becomes too difficult? Perhaps performance concerns? Hmm

It's not really an "application" thing. It's meant to be a design for libraries that implement protocols of some sort. All the library API acts on byte buffers and leaves the network socket etc stuff to the library user. So when the library needs to write data to a socket, the API instead returns a byte buffer to the caller, and the caller writes it to the network socket. When the library needs to read data from a socket, it instead expects the caller to do that and then give the populated byte buffer to a library function to ingest it.

Also, quite the opposite, it's *easier* to design a library this way because it's strictly less code the library needs to contain. Specifically in Rust it also has the advantage that the library becomes agnostic to sync vs async I/O since that's handled by the library user. Correspondingly, it is slightly harder for the library user to use such a library, but it's usually just a matter of writing a tiny generic wrapper around the network socket type to connect it to the library functions.

Nice from a testing standpoint too, since you can trivially mock out the hardware.

That said, a lot of what’s actually hard about IO is the error/fault handling, imposing timeouts and backoffs and all that jazz. At a certain point I’d wonder if extracting this out to a separate interface might obscure the execution flow in some of these scenarios.

> That said, a lot of what’s actually hard about IO is the error/fault handling, imposing timeouts and backoffs and all that jazz.

Application-level timeout/backoff handling is always scary to me, because I don't know how to make robust tests for it. I wonder if you couldn't use the same I/O-less approach, and split the logic out into pure functions that take the time passed/error state/... as value arguments, instead of measuring the physical time using OS APIs. It's probably not something for reusable libraries, but it could still be a nice benefit to be able to unit test in detail.

Split the re-triable action into one function, make a wrapper function that re-tries if needed, and use a third function that makes the decision to re-try and how long to back off.

Then you can test the decision function trivially, the re-try function by mocking the action and decision, and the action function itself without back off interfering.

That’s what you suggested, just saying that I did that in a Python API client with the backoff library and the result is pretty neat.

I love the idea of it all being totally abstract but in my experience this stuff is usually tied in with application level behaviours too, so you could end up with a pretty messy API between the layers.

If the property of "io-lessness" becomes something statically verifiable as part of dependency handling, it also seems potentially beneficial as a guard against supply-chain attacks.

A compromised IO-less file system library can still synthetize malware files on a volume.

... but only on the volume it is explicitly given access to. So, if the library was IO-less (and didn't use unsafe code), you could embed it in some tool, e.g. for forensics, and not have to worry about it compromising the security of the "host" system.