Expert Data (Dis)agreement

Elicitation is the process of extracting knowledge about the parameters in the statistical model. This information can then be used to provide input for the prior distribution needed for Bayesian analysis. Several methods of prior elicitation are used in practice including the use of experts.

Most studies, however, rely on results of previous publications as input for the prior distributions. In the current project I propose to use experts’ knowledge to provide the necessary background knowledge. When expert knowledge and the estimates solely based on the data agree, the Bayesian results lead to similar estimates as compared to ML-estimation, but with an increase in statistical power and smaller confidence/credibility intervals. However, when expert knowledge and collected data disagree the posterior estimates are influenced by the expert knowledge. This is because the Bayesian results are a compromise between the two sources of information. Such a disagreement is what is called a prior-data conflict, but I propose to use the term expert-data conflict when it concerns experts opinions. Whereas other fields increasingly appreciate this expert-data conflict, in the social sciences use of expert knowledge in this way is hardly ever used. The main reason might be that the estimates based on the data are trusted more than the subjective opinions of experts. But these arguments only hold if the data is truly a random sample from the population, which is almost never the case with limited data. In these situations, the data may be quite unreliable and I argue balancing the two sub-optimal sources of information via Bayesian statistics leads to more reliable conclusions than relying on imperfect data alone. Also, by quantifying the expert-data conflict a new type of research question can be answered: which expert, or group of experts disagree most with the data? Answering such question opens a new area of research which can be especially useful to identify biased experts or for policy makers who can then focus their campaign on specific misconceptions between groups of experts.

Sub-Projects

Ongoing

VIDI

In my VIDI*-project we work on several projects related to small data. [more information will follow soon]

*The VIDI is part of the innovational research incentives scheme of The Netherlands Organization for Scientific Research. The VIDI is the second (out of three) grants to develop their own research group.

Team members

(Inter)national collaborators

Milica Miočević Assistant Professor

In her current projects, Milica is focusing on optimal methods for data synthesis from non-exchangeable studies, on the consequences of specifying inaccurate priors in mediation models, and on issues that arise in applications of Bayesian mediation analysis with informative prior distributions in small samples.

Read more
Duco Veen PhD Student

Ducos PhD project with Rens focusses on providing a solution to small data problems in latent growth curve models. With few data points, it can be impossible to estimate the model of interest. Yet asking a different (simplified) question the data can answer may not be desirable.

In her PhD project, Mariëlle focuses on including prior knowledge in statistical analyses (informative Bayesian research) and confronting prior knowledge with new data.

Sarah Depaoli Assistant Professor at the University of California, Merced Sarah’s research interests are largely focused on issues surrounding Bayesian estimation of latent variable models. She has a particular interest in estimation issues arising from nonlinear growth patterns over time. She is also interested in improving accuracy of uncovering unobserved (latent) groups of individuals. She is currently working with several students that are involved in research spanning a wide range of methodological topics . Jeremy OakleyMarit Sijbrandij Diederik StoelProfitwise Lars TummersUSBO – add description Sonja Winter Former team member