Motherduck.com has a pop-up promotion to get the book for free.

Does anyone know how well does DuckDB's persistent mode compare to SQLite for example?

Does it write synchronously on COMMIT and never lose data? Is it reliable enough to use instead of SQLite?

DuckDB targets analytical workloads. Your question sounds like your workload is transactional to me, so I'd recommend sticking with SQLite.

Agree, in fact this wonderful book calls this out, stating:

  As DuckDB is an analytics database, it has only minimal support for transactions and parallel write access. You therefore couldn’t use it in applications and APIs that process and store input data arriving arbitrarily. Similarly when multiple concurrent processes read from a writeable database.

How do you decide when to reach for Polars vs DuckDB?

In Python, I think of them as, DuckDB is for getting the data you want, in the form you want it, from a file / remote place. Polars is for when you want to do something with that data, like visualize it.

`duckdb.sql("SELECT foo, bar * 2 as bar_times_2 FROM ...").pl()` (now in polars format) -> other stuff

In Rust, it's a bit fuzzier to me, as DuckDB is a pretty heavy dependency. I'm looking more and more fondly at DataFusion.

Do you mean Polars depends on/uses DuckDB pretty heavily in Rust? I'm only just now dabbling in Rust myself so I'm not familiar.

No not at all. Polars is not dependent on DuckDB.

DuckDB is a heavy dependency in terms of size. It's written in C++, so you can't work with it like a native rust dep.

Or you use Ibis and switch between the two at will!

DuckDB is nice but I’d like to see some alternatives to RAPIDS in the GPU space.