I’ve been thinking along these lines and if it’ll become more common with smaller but more tailored language models? The advantage seems obvious in that they would have significantly lower resource requirements both as for prediction and fine tuning, which matters when companies dealing with sensitive data (or even just comply with strict EU laws) wants to run them locally.

It makes sense that this specialization is seen first with language coverage because only GPT-4 is often considered truly fit for professional multilingual use; an LLM that is expensive and in the cloud.