The trial-and-roulette method is a popular method to extract experts’ beliefs about a statistical parameter. However, most studies examining the validity of this method only use ‘perfect’ elicitation results. In practice, it is sometimes hard to obtain such neat elicitation results. In our project about predicting fraud and questionable research practices among PhD candidates, we ran into issues with imperfect elicitation results. The goal of the current chapter is to provide an overview of the solutions we used for dealing with these imperfect results, so that others can benefit from our experience. We present information about the nature of our project, the reasons for the imperfect results, and how we resolved these supported by annotated R-syntax.

van de Schoot, R., Griffioen, E., & Winter, S. D. (2019, September 20). Dealing with imperfect elicitation results. Retrieved from DOI: 10.31234/

We put all the relevant information needed to replicate our findings, on the Open Science Framework (OSF; see ).

Former team member

After working with Rens on various research projects related to Bayesian Estimation and latent growth modeling I developed an interest in researching both of these further.