Densing law of LLMs

Nature Machine Intelligence – Xiao et al. introduce ‘capability density’, defined as capability per parameter, as a metric for evaluating large language models. They report an empirical…