Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

There's newer json support in clickhouse where you can load arbitrary json objects into a JSON column type:

https://clickhouse.com/blog/getting-data-into-clickhouse-par...

Behind the scenes it flattens things into real columns with auto detected types.

I suppose if you planned on running many queries that would be faster, but I bet octosql could also convert it to something like parquet for similar gains.

There's also some tricks you can do with json filtering if strings are involved, if you have a query like "select path from http_logs where hostname = 'news.ycom...'" you can read a whole chunk of data, do a `memmem` for the string and if no matches, not even bother decoding ANY of the json.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: