Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

If I'm not mistaken, Arrow is an in-memory format and not necessarily a storage format. Though it is common to use Arrow with the Parquet serialization format, and the benchmarks here show a much faster read performance than Parquet while achieving around the same file size. From my read, that seems to be the impetus for the project (with comparisons made on Linkedin by the author as well) and so in theory Arrow and this could be complementary like Arrow and Parquet if someone setup the links.


> the benchmarks here show a much faster read performance than Parquet

That felt bit surprising.. is parquet slow to read in general or is the julia implementation just slow?


Arrow.jl, when compression is off, is just MMAP. I don't know how it can be faster than that.


The benchmark comparison is vs Feather.jl.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: