Publications

Rigorous statistical evaluations of large language models (LLMs), including valid error bars and significance testing, are essential …

Calibration of language models is typically studied at the token level, with scalar temperature scaling serving as the primary approach …

Merging datasets across institutions is a lengthy and costly procedure, especially when it involves private information. Data hosts may …

Bayesian experimental design (BED) provides a powerful and general framework for optimizing the design of experiments. However, its …

We develop a semi-amortized, policy-based, approach to Bayesian experimental design (BED) called Step-wise Deep Adaptive Design …

We propose a method to improve the efficiency and accuracy of amortized Bayesian inference by leveraging universal symmetries in the …

We introduce a gradient-based approach for the problem of Bayesian optimal experimental design to learn causal models in a batch …

We formalize the problem of contextual optimization through the lens of Bayesian experimental design and propose CO-BED – a …

Our method is used for the data-efficient evaluation of the regret of past treatment assignments. Unlike approaches such as A/B testing, our method avoids assigning treatments that are known to be highly sub-optimal, whilst engaging in some exploration to gather pertinent information. We achieve this by introducing an information-based design objective, which we optimise end-to-end.

iDAD allows us to practically run Bayesian optimal experiments with implicit (likelihood-free) models in real-time. Previous methods either relied on an explicit likelihood model of the outcomes, or were too computationally costly to run in real-time.

DAD is the first policy-based approach to BOED that enables adaptive experiments to be performed in real-time.