I think there will be a trend where model's size will shrink due to better optim...

		tiborsaas on June 23, 2022 \| parent \| context \| favorite \| on: YaLM-100B: Pretrained language model with 100B par... I think there will be a trend where model's size will shrink due to better optimization / compression while hardware specs keep increasing. You can already see this with Chinchilla: https://towardsdatascience.com/a-new-ai-trend-chinchilla-70b...