Hacker News new | past | comments | ask | show | jobs | submit login

Author here, happy to answer any questions! Here's a few links that may be useful:

Direct GitHub link: https://github.com/asg017/sqlite-loadable-rs

A soon-to-be-released sqlite-loadable extension that's the fasted CSV parser for SQLite, and rivals DuckDB's parser at non-analytical queries: https://github.com/asg017/sqlite-xsv

A regular expression extension in Rust, the fastest SQLite regex implementation: https://github.com/asg017/sqlite-regex

And in the near future, expect more extensions for postgres, parquet, XML, S3, image manipulation, and more!




The API for virtual tables is tricky, because you don't ever consume the API in Rust: SQLite does it. However, as it is presented the interface is unsafe, I think.

In the sqlite-xsv VTab::open impl for XsvTable [0], you pass a &'vtab XsvTable to XsvCursor. But the means that if open is called again, to create a second cursor, then a `&mut XsvTable` and a `&XsvTable` reference exist at the same time, which is unsafe. Note that sqlite-xsv doesn't actually keep the `&XsvTable`, so it's fine, but it serves as an example where a safety problem could surface.

The same problem also applies to VTabWriteable::update. The fix is to make both of these methods receive an immutable reference and force implementors to use interior mutability. Note this isn't hypothetical, it's actually unsafe, and would appear if you had a unit test that implemented a writable virtual table backed by a rust data structure, and attempted to iterate and update at the same time.

I have an as-yet-unpublished code base that experiments with this. It's not published yet because it doesn't cover everything I want to and I'm trying to avoid publishing something where the API might have to break. My goals are explicitly safety first, performance second. https://github.com/CGamesPlay/sqlite3_ext

[0] https://github.com/asg017/sqlite-xsv/blob/main/src/xsv.rs#L9...


I went ahead and published the crate so that it is browsable via docs.rs. Feel free to adopt anything from it; the code is public domain. https://docs.rs/sqlite3_ext/latest/sqlite3_ext/


I think "undefined behavior" is the standard term for the "is unsafe" situation that you mention.


> And in the near future, expect more extensions

Please consider GIS/spatial extension. SpatiaLite is pretty bad, for example the KNN feature is deprecated and the replacement has not yet made it to a release yet.

https://www.gaia-gis.it/fossil/libspatialite/wiki?name=KNN




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: