Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Hi, how does this compare to Vaex? https://vaex.io/docs/tutorial.html

Vaex can join larger than memory datasets, but it doesn't support SQL per-se, it uses it's own lazy dataframe DSL.



I didn't dive into Vaex's implementation, but based on the example code, I would say they are similar in the sense that they all provide a Dataframe interface for end users to perform compute on relational data.

It looks like Vaex focuses more on end users like data scientists while Datafusion focuses more on being a composable embedded library for building analytical engines. For example, InfluxDB IOx, Ballista and ROAPI all uses Datafusion as the compute engine.

On top of that, Datafusion also comes with a builtin SQL planner so users can choose between Dataframe and SQL interfacts.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: