Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

when you got out of Spark, what did you go to?


BigQuery ELT, the org I went to was rather immature in their data practice, and I sold them on getting some proper orchestration (Dataform, their preference over DBT, and Airflow), and keeping the architecture coherent.

I'd have rather stuck with Spark just because I prefer Scala or Python to SQL (and that comes with e.g. being far easier to unit test), but life happened and that ecosystem was getting disrupted anyway.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: