How to assess the temporal stability of psychological constructs

markus-spiske-IiEFmIXZWSw-unsplash

 

In this new publication in the Journal of Research in Personality, we illustrate how to assess temporal stability of psychological constructs. We discuss common methods based on a review of the personality traits negative affectivity and social inhibition. Most methods ignore the non-normal distributions and measurement error in the questionnaire item scores. We illustrate how to handle these issues using three longitudinal latent variable models. We further highlight the importance of testing the often overlooked assumption of longitudinal measurement invariance. Lastly, we apply several longitudinal measurement invariance models, univariate and multivariate latent growth curves models, and latent trait-state occasions models to data from 2625 cancer survivors, to assess the temporal stability of negative affectivity, social inhibition, depression, anxiety, across a period of four years.

 

Why researchers should not ignore skewness and measurement error in questionnaire scores

skew

Recently I participated in a study for which many scientists were asked to analyse the same dataset to investigate whether the association between religion and wellbeing depends on cultural norms. The paper reporting those results is now published (The MARP team, 2022), together with several commentary articles. In my commentary I showed why researchers should not ignore skewness and measurement error when running statistical analyses based on constructs measured with questionnaire item scores. Researchers commonly study interaction effects between constructs measured with items on an ordinal measurement level and skewed score distributions. Here, I show that ignoring skewness in item scores often produces biased regression estimates, especially for interaction effects. I simulated 800 datasets with scores on two independent (X and Z) and one dependent (Y) latent variables. In the structural model I simulated a main effect (X) and an interaction effect (X*Z) on Y. When generating item scores from a factor model, I varied item skewness by using symmetrical or asymmetrical threshold parameters. Simulation parameters were based on empirical estimates (The MARP team). I analysed each simulated dataset using three methods: (1) a linear regression on sum scores, (2) a structural equation model (SEM) and (3) a SEM for ordered categorical items (CATSEM). Sum score regression produced underestimated effects, especially interactions. SEM was primarily biased when item scores were skewed. CATSEM produced the least biased estimates. These findings highlight the importance of inspecting item score distributions and questionnaire reliability before choosing a statistical model. Researchers are recommended to use CATSEM when testing associations between latent variables that are measured with skewed ordinal item scores. Not doing so risks underestimated associations, especially when estimating interaction effects

Fear of COVID-19 predicts vaccination willingness

Screenshot 2022-05-13 at 11.34.53
Together with colleagues I investigated the psychological predictors of the willingness to get a COVID-19 vaccination.  Our study was recently published open access in the Journal of Anxiety Disorders. We specifically examined whether fear of COVID-19 predicts vaccination willingness. We followed 938 participants for 14 months and measured their fear for COVID-19 in April 2020 and vaccination willingness was measured in June 2021. Approximately 11% of the participants indicated that they were not willing to get vaccinated. Results of a logistic regression showed that increased fear of COVID-19 predicts vaccination willingness 14 months later, even when controlling for several anxious personality traits, infection control perceptions, risks for loved ones, self-rated health, previous infection, media use, and demographic variables. These results show that fear of COVID-19 is a relevant construct to consider for predicting and possibly influencing vaccination willingness. Nonetheless, sensitivity and specificity of fear of COVID-19 to predict vaccination willingness were quite low and only became slightly better when fear of COVID-19 was measured concurrently. This indicates that other potential factors, such as perceived risks of the vaccines, probably also play a role in explaining vaccination willingness.

Comparing the findings of two popular methods used to estimate a Type D personality effect

tingey-injury-law-firm-yCdPU73kGSc-unsplash

In this new study published in General Hospital Psychiatry, we examined the differences between two methods that are commonly used in the literature to estimate a Type D personality effect. Type D personality, operationalised as high scores on negative affectivity (NA) and social inhibition (SI), has been associated with various medical and psychosocial outcomes. The recent failure to replicate several earlier findings could result from the various methods used to assess the Type D effect. Despite recommendations to analyse the continuous NA and SI scores, a popular approach groups people as having Type D personality or not. This method does not adequately detect a Type D effect as it is also sensitive to main effects of NA or SI only, suggesting the literature contains false positive Type D effects. Here, we systematically assess the extent of this problem. We conducted a systematic review including 44 published studies assessing a Type D effect with both a continuous and dichotomous operationalisation. The dichotomous method showed poor agreement with the continuous Type D effect. Of the 89 significant dichotomous method effects, 37 (41.6%) were Type D effects according to the continuous method. The remaining 52 (58.4%) are therefore likely not Type D effects based on the continuous method, as 42 (47.2%) were main effects of NA or SI only. Half of the published Type D effect according to the dichotomous method may be false positives, with only NA or SI driving the outcome. Our findings suggest that a large part of the Type D literature should be reanalysed to find out whether the results remain unchanged when using the continuous method.

Latent logistic interaction modeling of skewed ordinal item scores

skeword

In this new study published in the journal Structural Equation Modeling we focused on three popular methods to model interactions between two constructs containing measurement error in predicting an observed binary outcome: logistic regression using (1) observed scores, (2) factor scores, and (3) Structural Equation Modeling (SEM). It is still unclear how they compare with respect to bias and precision in the estimated interaction when item scores underlying the interaction constructs are skewed and ordinal. In this article, we investigated this issue using both a Monte Carlo simulation and an empirical illustration of the effect of Type D personality on cardiac events. Our results indicated that the logistic regression using SEM performed best in terms of bias and confidence interval coverage, especially at sample sizes of 500 or larger. Although for most methods bias increased when item scores were skewed and ordinal, SEM produced relatively unbiased interaction effect estimates when items were modeled as ordered categorical.

You can download the article here.

How to assess a Type D personality effect (part 2)

 

effect

A new study I published in the journal Personality and Individual Differences is a sequel to my study published in the Journal of Psychosomatic Research, earlier this year. Type D personality has been associated with various medical and psychosocial outcomes. Type D’s underlying personality traits negative affectivity (NA) and social inhibition (SI) are hypothesized to either additively (NA + SI) or synergistically (NA ∗ SI) affect an outcome. As some of the methods used to assess a Type D effect have been criticized in the past, this study aimed to investigate for all commonly used methods their tendency of producing false positive Type D effect. 324,000 datasets were generated using a Monte Carlo Simulation. Each dataset was analyzed using various methods to assess a Type D effect. Each method’s performance was assessed in terms of absolute bias and the percentage of false-positive findings. An online application was developed where readers can easily experiment with this simulation. Our simulation showed that all commonly used methods risk producing false-positive Type D effects. The only method with adequate false-positive rates included the continuous NA and SI main effects, as well as their quadratic effects and their interaction. All commonly used methods to assess a Type D personality effect showed inflated false-positive rates in realistic simulation scenarios. All earlier research based only on these methods should be reconsidered.

 

Young Investigator Award

I am truly honored to have been awarded the Elsevier & EAPM Young Investigator Award for my recently published article in the Journal of Psychosomatic Research. I would like to thank EAPM and Elsevier for making this award possible. Also many thanks to my supervisors Prof. Dr. Jelte Wicherts and Dr. Nina Kupper for their support and inspiration. Lastly, special thanks to the late Prof. Dr. Johan Denollet, without whom this study would never have taken place. I dedicate this award to him.

Screen Shot 2020-06-25 at 17.01.35

How to assess a Type D personality effect

Below is a short summary of a new study I published in the Journal of Psychosomatic Research. In this study, I investigated several methods used to assess a Type D personality effect and showed that the commonly used methods based on personality subgroups may result in false positive Type D effects.

Screenshot 2022-05-13 at 11.43.20

In research on Type D personality, its subcomponents negative affectivity (NA) and social inhibition (SI) are hypothesised to have a synergistic effect on various medical and psychosocial outcomes. As some methods to analyse Type D personality have been criticised, this study investigated whether these methods adequately detect a Type D effect. We used a simulation and two empirical illustrations to investigate each method’s performance (bias, power and false positives) in detecting the Type D effect. Our simulation showed that the two most commonly used methods to assess the Type D effect (subgroup methods) were primarily picking up the presence of NA or SI main effects, indicating that these methods might falsely suggest synergistic Type D effects. Moreover, these methods failed to detect the combined presence of the NA and SI main effects, resulting in significant Type D effects when only one of the NA/SI main effects was present. The method that best detected Type D effects modeled the continuous NA/SI main effects and their statistical interaction in a regression analysis. Reanalysis of two empirical Type D personality datasets confirmed the patterns found in our simulation. This study showed that Type D effects should be modeled with a continuous interaction approach. Other approaches showed either more bias, more false positive findings or lower power. We recommend against using subgroup approaches to operationalise Type D personality, as these methods are biased, regardless of whether the Type D effect is synergistic or additive in nature.

Click here to download the published article

 

How to model interactions between latent variables?

Below is a short summary of a new study I published with my collaborators in the Journal Multivariate Behavioral Research. The study reports a simulation study investigating the performance of various approaches to model interaction effects in structural equation models. It also includes an empirical investigation of how the two personality traits negative affectivity and social inhibition interact in influencing depressive and anxiety symptoms.

Modeling Interactions Between Latent Variables in Research on Type D Personality: A Monte Carlo Simulation and Clinical Study of Depression and Anxiety

Screen Shot 2020-03-14 at 10.44.32

Several approaches exist to model interactions between latent variables. However, it is unclear how these perform when item scores are skewed and ordinal. Research on Type D personality serves as a good case study for that matter. In Study 1, we fitted a multivariate interaction model to predict depression and anxiety with Type D personality, operationalized as an interaction between its two subcomponents negative affectivity (NA) and social inhibition (SI). We constructed this interaction according to four approaches: (1) sum score product; (2) single product indicator; (3) matched product indicators; and (4) latent moderated structural equations (LMS). In Study 2, we compared these interaction models in a simulation study by assessing for each method the bias and precision of the estimated interaction effect under varying conditions. In Study 1, all methods showed a significant Type D effect on both depression and anxiety, although this effect diminished after including the NA and SI quadratic effects. Study 2 showed that the LMS approach performed best with respect to minimizing bias and maximizing power, even when item scores were ordinal and skewed. However, when latent traits were skewed LMS resulted in more false-positive conclusions, while the Matched PI approach adequately controlled the false-positive rate.

Lodder, P., Denollet, J., Emons, W. H., Nefs, G., Pouwer, F., Speight, J., & Wicherts, J. M. (2019). Modeling Interactions Between Latent Variables in Research on Type D Personality: A Monte Carlo Simulation and Clinical Study of Depression and Anxiety. Multivariate behavioral research, 1-29.

Click here to download the published article