I'm personally afraid that the python community has bifurcated so far that a single 'correct' solution is doomed to fail. The needs of people writing and deploying HTTP/REST servers is just so different from the people writing PyTorch models and numerical simulations that no tool will ever satisfy the very different needs of both camps. The worst part is that many people developing these tools don't seem to realise this and blindly claim that their tool is The Tool! without having any deep insights into the needs of the other camps.
> The needs of people writing and deploying HTTP/REST servers is just so different from the people writing PyTorch models and numerical simulations that no tool will ever satisfy the very different needs of both camps.
Is this true though? At the end of it they both need python packages and some system dependencies installed (let's ignore models and data for now).
Why is there umpteen tools for doing this in python when there isn't in other languages? I have to deploy both web apps and ML models, the first thing I do is convert any project to use pyproject and poetry.
Whilst Poetry comes with it's own issues I've not had a project yet that this doesn't work for and wish the wider community would just settle on one method. Instead we have stuff coming in with various conda incantations, pip, pipx, poetry, setuptools, setup.py!
Deployment of ML models needs a decent solution too, half the ML code I get goes and fetches stuff from NLTK or Huggingface at runtime. Some of it (like the LLAMA models) needs various API keys set and EULAs agreed to before it'll run, then pings back to Huggingface each time you run it to check the EULA again! This makes life difficult when trying to deploy and adds this massive dependency on 3rd party services.
But how are you handling those system dependencies (is R and its various packages one?), and how do you know that the packages you install/manage via the python ecosystem work with them?
I think the reason Python has so many things is simply because most other languages throw their arms up and say "system stuff, not my problem" (rust is an excellent example of this, the build.rs is basically the same thing as a setup.py, except less standard, and currently lacks from what I've seen is any kind of systematic solution like cibuildwheel), whereas Python has always been trying to do something to address it.
