Recently I participated in a study for which many scientists were asked to analyse the same dataset to investigate whether the association between religion and wellbeing depends on cultural norms. The paper reporting those results is now published (The MARP team, 2022), together with several commentary articles. In my commentary I showed why researchers should not ignore skewness and measurement error when running statistical analyses based on construct measured with questionnaire item scores. Researchers commonly study interaction effects between constructs measured with items on an ordinal measurement level and skewed score distributions. Here, I show that ignoring skewness in item scores often produces biased regression estimates, especially for interaction effects. I simulated 800 datasets with scores on two independent (X and Z) and one dependent (Y) latent variables. In the structural model I simulated a main effect (X) and an interaction effect (X*Z) on Y. When generating item scores from a factor model, I varied item skewness by using symmetrical or asymmetrical threshold parameters. Simulation parameters were based on empirical estimates (The MARP team). I analysed each simulated dataset using three methods: (1) a linear regression on sum scores, (2) a structural equation model (SEM) and (3) a SEM for ordered categorical items (CATSEM). Sum score regression produced underestimated effects, especially interactions. SEM was primarily biased when item scores were skewed. CATSEM produced the least biased estimates. These findings highlight the importance of inspecting item score distributions and questionnaire reliability before choosing a statistical model. Researchers are recommended to use CATSEM when testing associations between latent variables that are measured with skewed ordinal item scores. Not doing so risks underestimated associations, especially when estimating interaction effects
Together with colleagues we investigated the psychological predictors of the willingness to get a COVID-19 vaccination. Our study was recently published open access in the Journal of Anxiety Disorders. We specifically examined whether fear of COVID-19 predicts vaccination willingness. We followed 938 participants for 14 months and measured their fear for COVID-19 in April 2020 and vaccination willingness was measured in June 2021. Approximately 11% of the participants indicated that they were not willing to get vaccinated. Results of a logistic regression showed that increased fear of COVID-19 predicts vaccination willingness 14 months later, even when controlling for several anxious personality traits, infection control perceptions, risks for loved ones, self-rated health, previous infection, media use, and demographic variables. These results show that fear of COVID-19 is a relevant construct to consider for predicting and possibly influencing vaccination willingness. Nonetheless, sensitivity and specificity of fear of COVID-19 to predict vaccination willingness were quite low and only became slightly better when fear of COVID-19 was measured concurrently. This indicates that other potential factors, such as perceived risks of the vaccines, probably also play a role in explaining vaccination willingness.
My new track Callisto is included on a beautiful ambient compilation released on Nuno dos Santos’ label Something Happening Somewhere. The release is available digitally or on cassette tape: https://sohaso.bandcamp.com/album/somewhere-v
In this newly published study, me and my collaborators studied the differences between two methods that are commonly used in the literature to estimate a Type D personality effect. Type D personality, operationalised as high scores on negative affectivity (NA) and social inhibition (SI), has been associated with various medical and psychosocial outcomes. The recent failure to replicate several earlier findings could result from the various methods used to assess the Type D effect. Despite recommendations to analyse the continuous NA and SI scores, a popular approach groups people as having Type D personality or not. This method does not adequately detect a Type D effect as it is also sensitive to main effects of NA or SI only, suggesting the literature contains false positive Type D effects. Here, we systematically assess the extent of this problem. We conducted a systematic review including 44 published studies assessing a Type D effect with both a continuous and dichotomous operationalisation. The dichotomous method showed poor agreement with the continuous Type D effect. Of the 89 significant dichotomous method effects, 37 (41.6%) were Type D effects according to the continuous method. The remaining 52 (58.4%) are therefore likely not Type D effects based on the continuous method, as 42 (47.2%) were main effects of NA or SI only. Half of the published Type D effect according to the dichotomous method may be false positives, with only NA or SI driving the outcome. Our findings suggest that a large part of the Type D literature should be reanalysed to find out whether the results remain unchanged when using the continuous method.
In this newly published study we focused on three popular methods to model interactions between two constructs containing measurement error in predicting an observed binary outcome: logistic regression using (1) observed scores, (2) factor scores, and (3) Structural Equation Modeling (SEM). It is still unclear how they compare with respect to bias and precision in the estimated interaction when item scores underlying the interaction constructs are skewed and ordinal. In this article, we investigated this issue using both a Monte Carlo simulation and an empirical illustration of the effect of Type D personality on cardiac events. Our results indicated that the logistic regression using SEM performed best in terms of bias and confidence interval coverage, especially at sample sizes of 500 or larger. Although for most methods bias increased when item scores were skewed and ordinal, SEM produced relatively unbiased interaction effect estimates when items were modeled as ordered categorical.
You can download the article here.
A re-evaluation of the Type D personality effect
Below is a short summary of a new study I published in the Personality and Individual Differences. This study is a sequel to my study published in the Journal of Psychosomatic Research, earlier this year. Click here to download the published article
Type D personality has been associated with various medical and psychosocial outcomes. Type D’s underlying personality traits negative affectivity (NA) and social inhibition (SI) are hypothesized to either additively (NA + SI) or synergistically (NA ∗ SI) affect an outcome. As some of the methods used to assess a Type D effect have been criticized in the past, this study aimed to investigate for all commonly used methods their tendency of producing false positive Type D effect. 324,000 datasets were generated using a Monte Carlo Simulation. Each dataset was analyzed using various methods to assess a Type D effect. Each method’s performance was assessed in terms of absolute bias and the percentage of false-positive findings. An online application was developed where readers can easily experiment with this simulation. Our simulation showed that all commonly used methods risk producing false-positive Type D effects. The only method with adequate false-positive rates included the continuous NA and SI main effects, as well as their quadratic effects and their interaction. All commonly used methods to assess a Type D personality effect showed inflated false-positive rates in realistic simulation scenarios. All earlier research based only on these methods should be reconsidered.
I am truly honored to have been awarded the Elsevier & EAPM Young Investigator Award for my recently published article in the Journal of Psychosomatic Research. I would like to thank EAPM and Elsevier for making this award possible. Also many thanks to my supervisors Prof. Dr. Jelte Wicherts and Dr. Nina Kupper for their support and inspiration. Lastly, special thanks to the late Prof. Dr. Johan Denollet, without whom this study would never have taken place. I dedicate this award to him.
Below is a short summary of a new study I published in the Journal of Psychosomatic Research. In this study, I investigated several methods used to assess a Type D personality effect and showed that the commonly used methods based on personality subgroups may result in false positive Type D effects.
In research on Type D personality, its subcomponents negative affectivity (NA) and social inhibition (SI) are hypothesised to have a synergistic effect on various medical and psychosocial outcomes. As some methods to analyse Type D personality have been criticised, this study investigated whether these methods adequately detect a Type D effect. We used a simulation and two empirical illustrations to investigate each method’s performance (bias, power and false positives) in detecting the Type D effect. Our simulation showed that the two most commonly used methods to assess the Type D effect (subgroup methods) were primarily picking up the presence of NA or SI main effects, indicating that these methods might falsely suggest synergistic Type D effects. Moreover, these methods failed to detect the combined presence of the NA and SI main effects, resulting in significant Type D effects when only one of the NA/SI main effects was present. The method that best detected Type D effects modeled the continuous NA/SI main effects and their statistical interaction in a regression analysis. Reanalysis of two empirical Type D personality datasets confirmed the patterns found in our simulation. This study showed that Type D effects should be modeled with a continuous interaction approach. Other approaches showed either more bias, more false positive findings or lower power. We recommend against using subgroup approaches to operationalise Type D personality, as these methods are biased, regardless of whether the Type D effect is synergistic or additive in nature.
Below is a short summary of a new study I published with my collaborators in the Journal Multivariate Behavioral Research. The study reports a simulation study investigating the performance of various approaches to model interaction effects in structural equation models. It also includes an empirical investigation of how the two personality traits negative affectivity and social inhibition interact in influencing depressive and anxiety symptoms.
Modeling Interactions Between Latent Variables in Research on Type D Personality: A Monte Carlo Simulation and Clinical Study of Depression and Anxiety
Several approaches exist to model interactions between latent variables. However, it is unclear how these perform when item scores are skewed and ordinal. Research on Type D personality serves as a good case study for that matter. In Study 1, we fitted a multivariate interaction model to predict depression and anxiety with Type D personality, operationalized as an interaction between its two subcomponents negative affectivity (NA) and social inhibition (SI). We constructed this interaction according to four approaches: (1) sum score product; (2) single product indicator; (3) matched product indicators; and (4) latent moderated structural equations (LMS). In Study 2, we compared these interaction models in a simulation study by assessing for each method the bias and precision of the estimated interaction effect under varying conditions. In Study 1, all methods showed a significant Type D effect on both depression and anxiety, although this effect diminished after including the NA and SI quadratic effects. Study 2 showed that the LMS approach performed best with respect to minimizing bias and maximizing power, even when item scores were ordinal and skewed. However, when latent traits were skewed LMS resulted in more false-positive conclusions, while the Matched PI approach adequately controlled the false-positive rate.
Lodder, P., Denollet, J., Emons, W. H., Nefs, G., Pouwer, F., Speight, J., & Wicherts, J. M. (2019). Modeling Interactions Between Latent Variables in Research on Type D Personality: A Monte Carlo Simulation and Clinical Study of Depression and Anxiety. Multivariate behavioral research, 1-29.
Below is a short summary of a new study I published with my collaborators in the Journal of Experimental Psychology: General. In this study we combined the results of dozens of studies investigating how money may affect our thoughts, emotions and behaviors, including the results of 100 unpublished experiments. Overall, this influence of money appeared to be important. However, these results should be interpreted carefully because published studies showed much larger effects than unpublished studies, a phenomenon called publication bias.
A Comprehensive Meta-analysis of Money Priming
Research on money priming typically investigates whether exposure to money-related stimuli
can affect people’s thoughts, feelings, motivations and behaviors. Our study answers the call for a comprehensive meta-analysis examining the available evidence on money priming. By conducting a systematic search of published and unpublished literature on money priming, we sought to achieve three key goals. First, we aimed to assess the presence of biases in the available published literature (e.g., publication bias). Second, in the case of such biases, we sought to derive a more accurate estimate of the effect size after correcting for these biases. Third, we aimed to investigate whether design factors such as prime type and study setting moderated the money priming effects. Our overall meta-analysis included 246 suitable experiments and showed a significant overall effect size estimate (Hedges’ g = .31, 95%CI = [0.26, 0.36]). However, publication bias and related biases are likely given the asymmetric funnel plots, Egger’s test and two other tests for publication bias. Moderator analyses offered insight into the variation of the money priming effect, suggesting for various types of study designs whether the effect was present, absent, or biased. We found the largest money priming effect in lab studies investigating a behavioral dependent measure using a priming technique in which participants actively handled money. Future research should use sufficiently powerful pre-registered studies to replicate these findings.
Lodder, P., Ong, H. H., Grasman, R. P., & Wicherts, J. (2019). A comprehensive meta-analysis of money priming. Journal of Experimental Psychology: General.