Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The sole fact that pandas, while trying to catch up to polars, now sort-of supports arrow as it's in-memory backend doesn't make it a "pandas dataframe binary format" in any possible way.


The whole point of Arrow is that it is language agnostic; so of course it is not a "pandas binary format". However, pandas didn't start supporting Arrow "to try to catch up to polars". The creator of pandas, Wes McKinney, came up with the Arrow idea because of the interoperability limitations he came across while working with pandas and the big data ecosystem. Supporting Arrow has been in the works for a long time: https://wesmckinney.com/blog/pandas-and-apache-arrow/ Pandas is an older and larger library with less maneuver capability and millions of users relying on things not being broken.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: