Databricks is janky, but so much better than the Azure services for data.

If you're buying from microsoft, it won't be cheap either way, might as well treat yourself a little bit.