Developed by Lion Behrens and Rens van de Schoot
This tutorial expects:
- Basic knowledge of hypothesis testing
- Basic knowledge of Bayesian inference
- An installed version of JASP release 0.8.2.0 or later on your electronic device
This tutorial provides the reader with a basic introduction to the software JASP (JASP Team 2017). The tutorial assumes that JASP is already installed on the used device and ready to use. The reader will be guided through the process of loading data into JASP and conducting a classical and Bayesian T-Test. The used dataset as well as an editable version of this exercise can be found under 'Assignment Files' above.
The data we will be using for this exercise is based on a study about predicting PhD-delays (Van de Schoot, Yerkes, Mouw and Sonneveld 2013). Among many other questions, the researchers asked the Ph.D. recipients how long it took them to finish their Ph.D. thesis (n=333). It appeared that Ph.D. recipients took an average of 59.8 months (five years and four months) to complete their Ph.D. trajectory. The variable B3_difference_extra measures the difference between planned and actual project time in months (mean=9.96, minimum=-31, maximum=91, sd=14.43). For the current exercise we will answer the question why some Ph.D. recipients took longer than others by investigating whether having had any children (up to the age 18) throughout the Ph.D. trajectory affects delays (0=No, 1=Yes). Out of the 333 respondents, 18% reported to have had at least one child.
Step 1. Start by downloading the dataset 'phd delays.csv' and loading this datafile into JASP. Do so by clicking on File -> Open. Have a look at the descriptive statistics. Has all data been loaded in correctly? The image below shows you how your output should look like. Click on the image if you would like to enlarge it.
Step 2. We will first do a frequentist T-test before we move to the Bayesian alternative including Bayes Factors. The image below shows you where to find the classical T-test.
Task: Perform an independent samples T-test and interpret the output. Using a significance criterion of 0.05, is there a significant difference between the two groups? Recently, a group of 72 notable statisticians proposed to shift the significance threshold to 0.005 (Benjamin et al. 2017). How does your conclusion change if you follow this advice?
Step 3. In the window that opens, you can easily request additional statistics and group-specific descriptives. Request discriptives, the mean difference between the two groups, a 95% confidence interval and the effect size (Cohen's d).
Task: Interpret your newly obtained output. Which group took longer to complete their Ph.D. trajectory? What about the size of the effect?
Step 5. Do you have any prior knowledge on how parenthood affects the achievement of individual mid-term life goals (like a Ph.D.)?
Task: Think about a reasonable prior distribution. How would your prior distribution for the effect of parenthood look like? Give reasons for your choice.
Step 6. Now, perform your Bayesian T-test in JASP. Long time, a crucial feature of the JASP software is that users weren’t able to specify individual prior distributions. Instead, JASP exclusively used default priors, which peak at 0 (no effect). Since release 0.8.2.0, users have the choice between default and informed prior distributions, and can choose between Cauchy, normal and t-distributions. In this tutorial, we will first start by using the default prior of JASP. To do so, set your dependent and independent variable in the respective boxes and choose a hypothesis to test (right below).
Task: Interpret your output. What does the Bayes Factor mean? How does its interpretation differ from the classical p-value above? What are potential pitfalls to the interpretation of a Bayes Factor? After having collected your own ideas, have a look at Konijn et al. (2015) for further reasoning.
Step 7. As a last step, get a better understanding of the evidence and its robustness by ticking 'Descriptives' under 'Additional Statistics' and all available boxes under 'Plots'. By doing this, a large amount of output is created.
Task: Work through the entire output and try to understand every single diagram and number. If you have difficulties with some interpretation, consult the solutions file under 'Answers and Explanation' on the top of this exercise. How would you evaluate the effect of parenthood based on your new insights? How robust is this effect?
Step 8. Repeat steps 6 and 7 but now using your individual informative prior distribution. How can you best translate your prior distribution that you designed in Step 5 into the features provided by JASP? How do your results change under the usage of an informative prior? Why did they change? Try to note all changes compared to your results of Steps 6 and 7 and to understand why the happen.
Benjamin, D. J., Berger, J., Johannesson, M., Nosek, B. A., Wagenmakers, E.-J., Berk, R., … Johnson, V. (2017, July 22). Redefine statistical significance. Retrieved from psyarxiv.com/mky9j
JASP Team (2017). JASP (Version 0.8.2)[Computer software].
Konijn, Elly A., van de Schoot, Rens, Winter, Sonja D., Ferguson, Christopher J. (2015). Possible Solution to Publication Bias Through Bayesian Statistics, Including Proper Null Hypothesis Testing. Communication Methods and Measures, 9(4): 280 - 302.
van de Schoot, R., Kaplan, D., Denissen, J., Asendorpf, J. B., Neyer, F. J. and van Aken, M. A.G. (2014). A Gentle Introduction to Bayesian Analysis: Applications to Developmental Research. Child Dev, 85: 842–860. doi:10.1111/cdev.12169
van de Schoot R, Yerkes MA, Mouw JM, Sonneveld H (2013). What Took Them So Long? Explaining PhD Delays among Doctoral Candidates. PLoS ONE 8(7): e68839. https://doi.org/10.1371/journal.pone.0068839
Other software exercises you might be interested in