I love Elixir! One of the main reasons I haven’t been able to sell it as much was it’s lack of good numerical computational support, but with the release of nx now getting to a very mature point it’s really hard for me to not get excited about this ecosystem.
I also think that livebook has a couple of really killer features now that make anyone who spends a significant amount of their time in Jupyter notebooks to stop and take a second look (eg it’s so easy to share a livebook compared with sharing a jupyter notebook, or even easier to extract code from a livebook prototype to turn into a resilient production elixir module).
Yes there is still a ways to go to catch up to the maturity and breadth of the community that Python has in this domain, but that gap is closing way faster than I could have thought.
I think the biggest challenge for adoption is its functional nature. If you take the time to learn about elixir and the BEAM you are rewarded with a pretty complete runtime which ships with its own DB, in-memory Store and message queue (with a very easy path to adopting more battle tested versions of these when you outgrow what is included in OTP). This means spin up time prototyping new systems is spent less on vendor choices (which type of DB should I use) and more on solving your problem domain.
I wish I had know about all this stuff 5 years ago…
Regarding numeric stuff. I discussed this matter with some folks and it was quite interesting because just like Erlang/OTP was used at control plane for phone switches while the data plane was implemented in something else, one could theoretically leverage it to oversee ML processes (perhaps Python programs written with PyTorch or something else)
The topology is exactly the same
In fact, porting your vetted numeric code to Elixir is probably going to make the risk go sky high, but running the same code with supervision structure in Elixir would in fact reduce risk so is a no-brainer
I do exactly this. Elixir supervises Ports to python and C binaries and manages all the communication between processes including buffering and maintaining state when necessary. Using libraries like Horde even allows you to run these processes on remote nodes very easily. It has worked fantastically well.
Elixir and Explorer using the Rust implementation of Polars under the hood is practically the same as Python and the Polars Python bindings. The rest of the programming language makes a difference though. Python rejected a large portion of CS research and rejected to do a proper implementation for many things like pattern matching or lambda functions. I am pretty sure they had sound reasons to do this.
There may be such a source already, but it would help if the powerful BEAM features had a guide showing what good use cases were (for each of the features it provides). Bonus would be examples.
I also think that livebook has a couple of really killer features now that make anyone who spends a significant amount of their time in Jupyter notebooks to stop and take a second look (eg it’s so easy to share a livebook compared with sharing a jupyter notebook, or even easier to extract code from a livebook prototype to turn into a resilient production elixir module).
Yes there is still a ways to go to catch up to the maturity and breadth of the community that Python has in this domain, but that gap is closing way faster than I could have thought.
I think the biggest challenge for adoption is its functional nature. If you take the time to learn about elixir and the BEAM you are rewarded with a pretty complete runtime which ships with its own DB, in-memory Store and message queue (with a very easy path to adopting more battle tested versions of these when you outgrow what is included in OTP). This means spin up time prototyping new systems is spent less on vendor choices (which type of DB should I use) and more on solving your problem domain.
I wish I had know about all this stuff 5 years ago…