Hacker News new | past | comments | ask | show | jobs | submit login

Much of the value of Arrow is in the things that will get built after Arrow is widely supported by data warehouses. Much of the data ecosystem we have today was designed to avoid the cost of moving data between systems. The whole Hadoop ecosystem is written in Java and shoehorned into map-reduce for this reason.

Imagine if, for example, you could use Mathematica or R to analyze data in your Snowflake cluster, with no bottleneck reading data from the warehouse even for giant datasets. This is the future that’s going to be enabled by Arrow.




Is it not what Presto (now Trino) is solving as well (among other things) ? Even though it is focused only on analytics and not on ML use cases.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: