SICSA Seminar – From Large to Small Building Affordable Language Models with Limited Resources

This talk aims to question the limitations and harms of Large Language Models, followed by a review of Small Language Models, covering prominent examples, their key techniques, and their capabilities. It will also give an overview of even smaller ‘baby’ language models. Finally, the talk will conclude by presenting some recent studies in which we developed baby language models using very small amounts of data.