Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Cool... though that would make it difficult to get the hundred or so CSVs into a single table, since it isn't supported I guess stitching them before processing would be easy enough... also, no idea when that feature became available.


It’s never been a good idea to batch ingest a lot of little single files using any ETL process on AWS, whether it be DDB, Aurora MySQL/Postgres using “load data from S3…”, Redshift batch import from S3, or just using Athena (yeah I’ve done all of them).


These weren't "little" single files... just separated by tld iirc.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: