Knowing What We're Talking About
Facilitating Decentralized, Unequivocal Publication of and Reference to Psychological Construct Definitions and Instructions
DOI:
https://doi.org/10.15626/MP.2022.3638Keywords:
Constructs, Theory Development, Jingle-Jangle Fallacy, Measurement, Validity, Epistemic IterationAbstract
A theory crisis and measurement crisis have been argued to be root causes of psychology's replication crisis. In both, the lack of conceptual clarification and the jingle-jangle jungle at the construct definition level as well the measurement level play a central role. We introduce a conceptual tool that can address these issues: Decentralized Construct Taxonomy specifications (DCTs). These consist of comprehensive specifications of construct definitions, corresponding instructions for quantitative and qualitative research, and unique identifiers. We discuss how researchers can develop DCT specifications as well as how DCT specifications can be used in research, practice, and theory development. Finally, we discuss the implications and potential for future developments to answer the call for conceptual clarification and epistemic iteration. This contributes to the move towards a psychological science that progresses in a cumulative fashion through discussion and comparison.
References
Alexandrova, A., & Haybron, D. (2016). Is construct validation valid? Philosophy of Science, 83(5), 1098–1109. https://doi.org/10/gg33wb
Altgassen, E., Geiger, M., & Wilhelm, O. (2023). Do you mind a closer look? A jingle-jangle fallacy perspective on mindfulness. European Journal of Personality, 089020702311745. https://doi.org/10/kcrc
Borgstede, M., & Eggert, F. (2022). Squaring the circle: From latent variables to theory-based measurement. Theory & Psychology. https://doi.org/10/grhv9p
Borsboom, D., Cramer, A., Kievit, R., Scholten, A., & Frani ́c, S. (2009). The end of construct validity. In R. Lissitz (Ed.), The concept of validity: Revisions, new directions, and applications (pp. 135–170). IAP Information Age Publishing.
Borsboom, D., Van der Maas, H., Dalege, J., Kievit, R., & Haig, B. (2021). Theory construction methodology: A practical framework for building theories in psychology. Perspectives on Psychological Science, 16(4), 756–766. https://doi.org/10/gh65nr
Box, G. E. P. (1979). Robustness in the strategy of scientific model building. In R. Launer & G. Wilkinson (Eds.), Robustness in Statistics (pp. 201–236). Academic Press, Inc. https://doi.org/10/gd32bv
Brembs, B., Lenardic, A., Murray-Rust, P., Chan, L., & Irawan, D. E. (2023). Mastodon over Mammon: Towards publicly owned scholarly knowledge. Royal Society Open Science, 10(7), 230207. https://doi.org/10/gstkx9
Brick, C., Hood, B., Ekroll, V., & De-Wit, L. (2022). Illusory essences: A bias holding back theorizing in psychological science. Perspectives on Psychological Science, 17(2), 491–506. https://doi.org/10/gmdj5f
Bringmann, L., Elmer, T., & Eronen, M. (2022). Back to basics: The importance of conceptual clarification in psychological science. Current Directions in Psychological Science, 31(4), 340–346. https://doi.org/10/gqmqg5
Bschir, K., & Lohse, S. (2022). Pandemics, policy, and pluralism: A Feyerabend-inspired perspective on COVID-19. Synthese, 200(6), 441. https://doi.org/10/gq5cz9
Cartwright, N. (2021). Rigour versus the need for evidential diversity. Synthese, 199, 13095–13119. https://doi.org/10/gstkzb
Castanho Silva, B., Jungkunz, S., Helbling, M., & Littvay, L. (2020). An empirical comparison of seven populist attitudes scales. Political Research Quarterly, 73(2), 409–424. https://doi.org/10/ghx8nm
Chang, H. (2007). Inventing temperature: Measurement and scientific progress. Oxford University Press.
Crutzen, R., & Peters, G.-J. Y. (2023a). The regression trap: Why regression analyses are not suitable for selecting determinants to target in behavior change interventions. Health Psychology and Behavioral Medicine, 11(1). https://doi.org/10/k2x9
Crutzen, R., & Peters, G.-J. Y. (2023b). A lean method for selecting determinants when developing behavior change interventions. Health Psychology and Behavioral Medicine, 11(1), 2167719. https://doi.org/10/js9b
De Graaf, A., Van den Putte, B., Nguyen, M.-J., Zebregs, S., Lammers, J., & Neijens, P. (2017). The effectiveness of narrative versus informational smoking education on smoking beliefs, attitudes and intentions of low-educated adolescents. Psychology & Health, 32(7), 810–825. https://doi.org/10/gstkzd
Devezer, B., Nardin, L., Baumgaertner, B., & Buzbas, E. (2019). Scientific discovery in a model-centric framework: Reproducibility, innovation, and epistemic diversity. PLOS ONE, 14(5), e0216125. https://doi.org/10/gf86cs
Diener, E., Emmons, R. A., Larsen, R. J., & Griffin, S. (1985). The Satisfaction With Life Scale. Journal of Personality Assessment, 49(1), 71–75. https://doi.org/10/fqqbmr
Earp, B., & Trafimow, D. (2015). Replication, falsification, and the crisis of confidence in social psychology. Frontiers in Psychology, 6, 621. https://doi.org/10/gfpn82
Eronen, M., & Bringmann, L. (2021). The theory crisis in psychology: How to move forward. Perspectives on Psychological Science, 16(4), 779–788. https://doi.org/10/ghw2x3
Eronen, M., & Romeijn, J.-W. (2020). Philosophy of science and the formalization of psychological theory. Theory & Psychology, 30(6), 786–799. https://doi.org/10/ghqcr8
Feyerabend, P. (1965). Problems of empiricism. In R. Colodny (Ed.), Beyond the edge of certainty. Essays in contemporary science and philosophy. (pp. 145–260). Prentice-Hall.
Fishbein, M., & Ajzen, I. (2010). Predicting and Changing Behavior: The Reasoned Action Approach. Taylor & Francis Group.
Flake, J. K., & Fried, E. I. (2020). Measurement Schmeasurement: Questionable Measurement Practices and How to Avoid Them. Advances in Methods and Practices in Psychological Science, 3(4), 456–465. https://doi.org/10/ghnbdg
Flake, J. K., Pek, J., & Hehman, E. (2017). Construct Validation in Social and Personality Research: Current Practice and Recommendations. Social Psychological and Personality Science, 8(4), 370–378. https://doi.org/10/gbf8nx
Fried, E. (2017a). The 52 symptoms of major depression: Lack of content overlap among seven common depression scales. Journal of Affective Disorders, 208, 191–197. https://doi.org/10/gcsk3p
Fried, E. (2017b). What are psychological constructs? On the nature and statistical modeling of emotions, intelligence, personality traits and mental disorders. Health Psychology Review, 11(2), 130–134. https://doi.org/10/gfj8tz
Fried, E. (2020). Lack of theory building and testing impedes progress in the factor and network literature. Psychological Inquiry, 31(4), 271–288. https://doi.org/10/ghsbbk
Fried, E., & Nesse, R. (2015). Depression sum-scores don’t add up: Why analyzing specific depression symptoms is essential. BMC Medicine, 13(1), 72. https://doi.org/10/f67sz8
Gibbs, G. (2007). Analyzing qualitative data. Sage Publications Ltd.
Glaw, X., Inder, K., Kable, A., & Hazelton, M. (2017). Visual Methodologies in Qualitative Research: Autophotography and Photo Elicitation Applied to Mental Health Research. International Journal of Qualitative Methods, 16(1). https://doi.org/10/gfvmq8
Gruijters, S., Fleuren, B., & Peters, G.-J. (2021). Crossing the seven Cs of internal consistency: Assessing the reliability of formative instruments. PsyArXiv. https://doi.org/10/g5sh
Guest, O., & Martin, A. (2021). How computational modeling can force theory building in psychological science. Perspectives on Psychological Science, 16(4), 789–802. https://doi.org/10/ghvcwr
Hagger, M. (2014). Avoiding the “déjà-variable” phenomenon: Social psychology needs more guides to constructs. Frontiers in Psychology, 5, 52. https://doi.org/10/gd7fkx
Hale, J., Hastings, J., West, R., Lefevre, C., Direito, A., Bohlen, L., Godinho, C., Anderson, N., Zink, S., Groarke, H., & Michie, S. (2020). An ontology-based modelling system (OBMS) for representing behaviour change theories applied to 76 theories. Wellcome Open Research, 5, 177. https://doi.org/10/ghgnb5
Hawkins-Elder, H., & Ward, T. (2020). Theory construction in the psychopathology domain: A multiphase approach. Theory & Psychology, 30(1), 77–98. https://doi.org/10/ghvmvg
Hodson, G. (2021). Construct jangle or construct mangle? Thinking straight about (nonredundant) psychological constructs. Journal of Theoretical Social Psychology, 5(4), 576–590. https://doi.org/10/gng6pw
Holcombe, A. O., Kovacs, M., Aust, F., & Aczel, B. (2020). Documenting contributions to scholarly articles using CRediT and tenzing (C. R. Sugimoto, Ed.). PLOS ONE, 15(12), e0244611. https://doi.org/10/gk347c
Huls, S., Van Osch, S., Brouwer, W., Van Exel, J., & Stiggelbout, A. (2022). Psychometric evaluation of the Health-Risk Attitude Scale (HRAS-2013): Assessing the reliability, dimensionality and validity in the general population and a patient population. Psychology & Health, 37(1), 34–50. https://doi.org/10/gstkzf
Kok, G., & Ruiter, R. (2014). Who has the authority to change a theory? Everyone! A commentary on Head and Noar. Health Psychology Review, 8(1), 61–64. https://doi.org/10/gstkzg
Kruisbrink, M. (2022). Towards enhanced management of fear of falling in older people: Unravelling interventions and measuring related avoidance of activity [Doctoral dissertation, Maastricht University] [ISBN: 9789464217124]. https://doi.org/10/kxkf
Lakens, D. (2019). The value of preregistration for psychological science: A conceptual analysis. Japanese Psychological Review, 62(3), 221–230. https://doi.org/10/gk9jpb
Lakens, D., & DeBruine, L. (2021). Improving transparency, falsifiability, and rigor by making hypothesis tests machine-readable. Advances in Methods and Practices in Psychological Science, 4(2), 251524592097094. https://doi.org/10/gmbq55
Landy, J., Jia, M., Ding, I., Viganola, D., Tierney, W., Dreber, A., Johannesson, M., Pfeiffer, T., Ebersole, C., Gronau, Q., Ly, A., Van den Bergh, D., Marsman, M., Derks, K., Wagenmakers, E.-J., Proctor, A., Bartels, D., Bauman, C., Brady, W., ... Uhlmann, E. (2020). Crowdsourcing hypothesis tests: Making transparent how design choices shape research results. Psychological Bulletin, 146(5), 451–479. https://doi.org/10/ggj87k
Lawson, K., & Robins, R. W. (2021). Sibling constructs: What are they, why do they matter, and how should you handle them? Personality and Social Psychology Review, 25(4), 344–366. https://doi.org/10/gm62mr
Leising, D., & Borgstede, M. (2019). Hypothetical constructs. In V. Zeigler-Hill & T. Shackelford (Eds.), Encyclopedia of Personality and Individual Differences (pp. 1–6). Springer. https://doi.org/10/kxkb
Lonsdorf, T., Merz, C., & Fullana, M. (2019). Fear extinction retention: Is it what we think it is? Biological Psychiatry, 85(12), 1074–1082. https://doi.org/10/ggw7x6
Marques, M. M., Wright, A. J., Corker, E., Johnston, M., West, R., Hastings, J., Zhang, L., & Michie, S. (2023). The Behaviour Change Technique Ontology: Transforming the Behaviour Change Technique Taxonomy v1. Wellcome Open Research, 8, 308. https://doi.org/10/kxj9
Maul, A. (2017). Rethinking traditional methods of survey validation. Measurement: Interdisciplinary Research and Perspectives, 15(2), 51–69. https://doi.org/10/gf5stn
Metz, G., Peters, G.-J., & Crutzen, R. (2022). Acyclic Behavior Change Diagrams: A tool to report and analyze interventions. Health Psychology and Behavioral Medicine. https://doi.org/10/hrz4
Miyakawa, T. (2020). No raw data, no science: Another possible source of the reproducibility crisis. Molecular Brain, 13, 24. https://doi.org/10/ggmgw6
Moreau, D., & Wiebels, K. (2022). Psychological constructs as local optima. Nature Reviews Psychology, 1(4), 188–189. https://doi.org/10/grhkxq
Murray-Rust, P. (2018). Open data in science. Nature Precedings. https://doi.org/10/gnkn6x
Nasa, P., Jain, R., & Juneja, D. (2021). Delphi methodology in healthcare research: How to decide its appropriateness. World Journal of Methodology, 11(4), 116–129. https://doi.org/10/gmpjvz
Nosek, B., & Bar-Anan, Y. (2012). Scientific Utopia: I. Opening scientific communication [arXiv: 1205.1055]. Psychological Inquiry, 23(3), 217–243. https://doi.org/10/gcsk27
Nosek, B., Ebersole, C., DeHaven, A., & Mellor, D. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600–2606. https://doi.org/10/gc6xk8
Nosek, B., Spies, J., & Motyl, M. (2012). Scientific Utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science, 7(6), 615–631. https://doi.org/10/f4fc2k
Oberauer, K., & Lewandowsky, S. (2019). Addressing the theory crisis in psychology. Psychonomic Bulletin & Review, 26(5), 1596–1618. https://doi.org/10/gf8dr6
Onwuegbuzie, A., & Leech, N. (2005). On becoming a pragmatic researcher: The importance of combining quantitative and qualitative research methodologies. International Journal of Social Research Methodology, 8(5), 375–387. https://doi.org/10/bwmj6n
Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. https://doi.org/10/68c
Oude Maatman, F. (2021). Psychology’s theory crisis, and why formal modelling cannot solve it. PsyArXiv. https://doi.org/10/jspz
Panasiuk, S. (2023). Life Satisfaction: DCT specification [Publisher: Zenodo]. https://doi.org/10/kqtb
Panc, T., Mihalcea, A., & Panc, I. (2012). Self-efficacy survey: A new assessment tool. Procedia - Social and Behavioral Sciences, 33, 880–884. https://doi.org/10/gk6fwr
Pashler, H., & Wagenmakers, E.-J. (2012). Editors’ introduction to the special section on replicability in psychological science: A crisis of confidence? Perspectives on Psychological Science, 7(6), 528–530. https://doi.org/10/gckf56
Peters, G.-J. Y., Crutzen, R., Roozen, S., & Kok, G. (2020). The Reasoned Action Approach represented as a Decentralized Construct Taxonomy (DCT). https://a-bc.gitlab.io/dct-raa/
Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2016). Recommendations for Creating Better Concept Definitions in the Organizational, Behavioral, and Social Sciences. Organizational Research Methods, 19(2), 159–203. https://doi.org/10/f8fqdb
Ponnock, A., Muenks, K., Morell, M., Seung Yang, J., Gladstone, J., & Wigfield, A. (2020). Grit and conscientiousness: Another jangle fallacy. Journal of Research in Personality, 89, 104021. https://doi.org/10/ghcrsz
Reschly, A., & Christenson, S. (2012). Jingle, jangle, and conceptual haziness: Evolution and future directions of the engagement construct. In S. Christenson, A. Reschly, & C. Wylie (Eds.), Handbook of Research on Student Engagement (pp. 3–19). Springer. https://doi.org/10/gmxbks
Robinaugh, D., Haslbeck, J., Ryan, O., Fried, E., & Waldorp, L. (2021). Invisible hands and fine calipers: A call to use formal theory as a toolkit for theory construction. Perspectives on Psychological Science, 16(4), 725–743. https://doi.org/10/gh65nn
Rohrer, J., Tierney, W., Uhlmann, E., DeBruine, L., Heyman, T., Jones, B., Schmukle, S., Silberzahn, R., Willén, R., Carlsson, R., et al. (2021). Putting the self in self-correction: Findings from the loss-of-confidence project. Perspectives on Psychological Science, 16(6), 1255–1269. https://doi.org/10/gh6f6r
Romppel, M., Herrmann-Lingen, C., Wachter, R., Edelmann, F., Düngen, H.-D., Pieske, B., & Grande, G. (2013). A short form of the General Self-Efficacy Scale (GSE-6): Development, psychometric properties and validity in an intercultural non-clinical sample and a sample of patients at risk for heart failure. GMS Psycho-Social-Medicine, 10. https://doi.org/10/ggbbw8
Santor, D., Gregus, M., & Welch, A. (2006). Eight decades of measurement in depression. Measurement: Interdisciplinary Research and Perspectives, 4, 135–155. https://doi.org/10/fcmm9d
Scheel, A. M. (2022). Why most psychological research findings are not even wrong. Infant and Child Development, 31(1). https://doi.org/10/gn8sks
Schnell, T., Spitzenstätter, D., & Krampe, H. (2022). Compliance with COVID-19 public health guidelines: An attitude-behaviour gap bridged by personal concern and distance to conspiracy ideation. Psychology & Health, 37(12), 1680–1701. https://doi.org/10/gm35dt
Siegling, A., & Petrides, K. (2016). Zeroing in on mindfulness facets: Similarities, validity, and dimensionality across three independent measures. PLOS ONE, 11(4), e0153073. https://doi.org/10/f8w3hz
Skinner, E. (1996). A guide to constructs of control. Journal of Personality and Social Psychology, 71(3), 549–570. https://doi.org/10/d9q7cv
Smaldino, P. (2008). Models are stupid, and we need more of them. In Computational social psychology (pp. 311–331). Routledge.
Smaldino, P. (2016). Not even wrong: Imprecision perpetuates the illusion of understanding at the cost of actual understanding. Behavioral and Brain Sciences, 39, e163. https://doi.org/10/gjfx4j
Snippe, M. H. M., Peters, G.-J. Y., & Kok, G. (2021). The operationalization of self-identity in reasoned action models: A systematic review of self-identity operationalizations in three decades of research. Health Psychology and Behavioral Medicine, 9(1), 48–69. https://doi.org/10/gmzj98
Snippe, M. H. M., Peters, G.-J. Y., & Kok, G. (2023). Determining the Conceptual Independence of Self-Identity in Reasoned Action Models. https://doi.org/10/kdwx
Spadaro, G., Tiddi, I., Columbus, S., Jin, S., Ten Teije, A., CoDa Team, & Balliet, D. (2022). The Cooperation Databank: Machine-Readable Science Accelerates Research Synthesis. Perspectives on Psychological Science, 17(5), 1472–1489. https://doi.org/10/gstkzh
Szollosi, A., Kellen, D., Navarro, D., Shiffrin, R., Van Rooij, I., Van Zandt, T., & Donkin, C. (2020). Is preregistration worthwhile? Trends in Cognitive Sciences, 24(2), 94–95. https://doi.org/10/ggf3pd
Taylor, J., Nailer, E., Cohen, C., Redman, C., & Sherman, S. (2022). HPV vaccination and cervical screening: The knowledge and attitudes of mothers of adolescent girls. Psychology & Health. https://doi.org/10/gstkzj
Uhlmann, E. L., Ebersole, C., Chartier, C., Errington, T., Kidwell, M., Lai, C., McCarthy, R., Riegelman, A., Silberzahn, R., & Nosek, B. (2019). Scientific Utopia III: Crowdsourcing science. Perspectives on Psychological Science, 14(5), 711–733. https://doi.org/10/gf4jhq
UNESCO. (2021). UNESCO Recommendation on Open Science (tech. rep.). UNESCO. https://unesdoc.unesco.org/ark:/48223/pf0000379949
Van Der Maas, H. L. J., Molenaar, D., Maris, G., Kievit, R. A., & Borsboom, D. (2011). Cognitive psychology meets psychometric theory: On the relation between process models for decision making and latent variable models for individual differences. Psychological Review, 118(2), 339–356. https://doi.org/10/drf5nw
Van Rooij, I., & Baggio, G. (2020). Theory development requires an epistemological sea change. Psychological Inquiry, 31(4), 321–325. https://doi.org/10/gjzbzq
Van Tuijl, P., Verboon, P., & Van Lankveld, J. (2023). Initial development and validation of item banks to measure problematic hypersexuality. Open Research Europe, 3, 129. https://doi.org/10/gstkzk
Van Tuijl, P., Verboon, P., & Van Lankveld, J. J. D. M. (2023). Three Quarks for Hypersexuality Research. Sexes, 4(1), 118–132. https://doi.org/10/kqjb
van Es, K. (2023). Unpacking tool criticism as practice, in practice. Digital Humanities Quarterly, 017(2). https://digitalhumanities.org/dhq/vol/17/2/000692/000692.html
Vazire, S. (2018). Implications of the credibility revolution for productivity, creativity, and progress. Perspectives on Psychological Science, 13(4), 411–417. https://doi.org/10/gd59dt
Vazire, S., Schiavone, S., & Bottesini, J. (2022). Credibility beyond replicability: Improving the four validities in psychological science. Current Directions in Psychological Science, 31(2), 162–168. https://doi.org/10/grht5z
Warnell, K., & Redcay, E. (2019). Minimal coherence among varied theory of mind measures in childhood and adulthood. Cognition, 191, 103997. https://doi.org/10/gf3787
Weidman, A., Steckler, C., & Tracy, J. (2017). The jingle and jangle of emotion assessment: Imprecise measurement, casual scale usage, and conceptual fuzziness in emotion research. Emotion, 17(2), 267–295. https://doi.org/10/f9w6ff
West, R. (2021). PAT: An on-line paper authoring tool for writing up randomized controlled trials. Addiction, 116(8), 1938–1940. https://doi.org/10/gstkzm
West, R., Godinho, C., Bohlen, L., Carey, R., Hastings, J., Lefevre, C., & Michie, S. (2019). Development of a formal system for representing behaviour-change theories. Nature Human Behaviour, 3(5), 526. https://doi.org/10/gf5fmd
Williams, D., & Rhodes, R. (2016). The confounded self-efficacy construct: Review, conceptual analysis, and recommendations for future research. Health Psychology Review, 10(2), 113–128. https://doi.org/10/ggzktz
Wolf, M. G., Ihm, E. D., Maul, A., & Taves, A. (2023). The Response Process Evaluation Method (preprint). PsyArXiv. https://doi.org/10/rbd2x
Wood, C., Conner, M., Sandberg, T., Godin, G., & Sheeran, P. (2014). Why does asking questions change health behaviours? The mediating role of attitude accessibility. Psychology & Health, 29(4), 390–404. https://doi.org/10/gf83n2
Yarkoni, T. (2020). Implicit realism impedes progress in psychology: Comment on Fried (2020). Psychological Inquiry, 31(4), 326–333. https://doi.org/10/ghzsjk
Zollman, K. (2010). The epistemic benefit of transient diversity. Erkenntnis, 72(1), 17–35. https://doi.org/10/dnjk7f
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Gjalt-Jorn Peters, Rik Crutzen
This work is licensed under a Creative Commons Attribution 4.0 International License.
