I see roughly the type of issue you face. For the case where there's some good value/effort ratio to invent internal representations, Haskell's being "strong" on parsing tasks make it a good fit. When this ratio is less clear: I found lenses, albeit a bit foreign to typical Haskell code, are a really good addition (for me it's akin to a new language, like `jq` for arbitrary in-memory structures) for extract/read tasks.
Anecdotal evidence: I recently had to turbo-build a tool to generate statistics over five-digit numbers of very complex business objects (including dates, strings, IDs, boolean as whatever string, ill-named columns) scattered in a number of systems (some web apis plus some CSV extracts from you don't really-know where). Using 'raw structures' like Aeson's JSON AST with lenses was more than good enough. Lenses have typically solved the "dynamic / cumbersone shape". Then I had to create a CSV extratc with like 50 boolean fields, reaching to intermediate techniques like type-level symbols allowed to really cheaply ensure I was not mixing two lines when adding a new column. I could even reach to Haxl from Facebook to write a layer of "querying" over my many heterogeneous source to automatically make concurrent and cached fetches for it.
The main difficulty in this setup is to keep the RAM usage under control because of two things. On the one hand, AST representations are costly. On the other hand, concurrency and caching means more work done in the same memory space.
Overall, got the data on time at relatively low effort (really low compared to previous attempt - to a point that some people with mileage at my company thought it would be impossible to build timely). Pretty good experience, would recommend to a friend.
Anecdotal evidence: I recently had to turbo-build a tool to generate statistics over five-digit numbers of very complex business objects (including dates, strings, IDs, boolean as whatever string, ill-named columns) scattered in a number of systems (some web apis plus some CSV extracts from you don't really-know where). Using 'raw structures' like Aeson's JSON AST with lenses was more than good enough. Lenses have typically solved the "dynamic / cumbersone shape". Then I had to create a CSV extratc with like 50 boolean fields, reaching to intermediate techniques like type-level symbols allowed to really cheaply ensure I was not mixing two lines when adding a new column. I could even reach to Haxl from Facebook to write a layer of "querying" over my many heterogeneous source to automatically make concurrent and cached fetches for it.
The main difficulty in this setup is to keep the RAM usage under control because of two things. On the one hand, AST representations are costly. On the other hand, concurrency and caching means more work done in the same memory space.
Overall, got the data on time at relatively low effort (really low compared to previous attempt - to a point that some people with mileage at my company thought it would be impossible to build timely). Pretty good experience, would recommend to a friend.