Zwischen Definition und Empirie
Theoretische Probleme der Fragebogenkonstruktion
In: Vierteljahrsschrift für wissenschaftliche PädagogikSearch for other papers by Matthias Borgstede in
Current site
Google Scholar
Abstract
Between Definition and Empirical Research. Theoretical Problems of Questionnaire Design
Within the field of quantitative educational science, theoretical terms are often defined operationally by scales or questionnaires, which have been constructed using the formal background of Classical Test Theory. This article takes a critical perspective on the current modus operandi of scale construction.
It is argued that classical quality criteria (reliability, validity, and objectivity) are usually represented by empirical correlations between scales or items of a scale. Apart from some theoretical inconsistencies, using these criteria (e.g. Chronbach’s Alpha) to optimize a scale results in a set of items with high intercorrelations, regardless of their content or meaning.
Because high inter-item correlations are neither necessary nor sufficient for an adequate operational definition, the utility of the classical approach to scale construction is questioned on a fundamental level. It is further argued that operational definitions do not pose a statistical problem in the first place, but a theoretical problem. Hence, we do not need a method to optimize scales for high inter-item-correlations, but rather a way to translate theoretical terms (which may be inherently vague) to empirically observable phenomena. This is accomplished by the semantic approach to psychological tests by Buntins, Buntins and Eggert (2016), which provides a formal alternative to classical quality criteria.
Purchase
Buy instant access (PDF download and unlimited online access):
Institutional Login
Log in with Open Athens, Shibboleth, or your institutional credentials
Personal login
Log in with your brill.com account
-
Borsboom, Denny (2005): Measuring the Mind. Conceptual Issues in Contemporary Psychometrics. Cambridge: Cambridge University Press.
-
Borsboom, Denny (2017): A Network Theory of Mental Disorders. In: World Psychiatry. Official Journal of the World Psychiatric Association (WPA) 1/2017, pp. 5–13.
-
Borsboom, Denny/Mellenbergh, Gideon J./van Heerden, Jaap (2003): The Theoretical Status of Latent Variables. In: Psychological Review 2/2003, pp. 203–219.
-
Bourdieu, Pierre (2012): Ökonomisches Kapital, kulturelles Kapital, soziales Kapital. In: Ullrich Bauer/Uwe H. Bittlingmayer/Albert Scherr (Hrsg.): Handbuch Bildungs- und Erziehungssoziologie. Wiesbaden: Springer VS, S. 229–242.
-
Bridgman, Percy W. (1927): The Logic of Modern Physics. New York: Macmillan.
-
Bühner, Markus (2011): Einführung in die Test- und Fragebogenkonstruktion (3., aktualisierte und erweiterte Aufl.). München: Pearson Studium.
-
Buntins, Matthias (2014): Psychologische Tests und mehrwertige Logik. Ein alternativer Ansatz zur Quantifizierung psychologischer Konstrukte. Wiesbaden: Springer VS.
-
Buntins, Matthias/Buntins, Katja/Eggert, Frank (2016): Psychological Tests from a (Fuzzy-) Logical Point of View. In: Quality & Quantity 6/2016, pp. 2395–2416.
-
Buntins, Matthias/Buntins, Katja/Eggert, Frank (2017): Clarifying the Concept of Validity. From Measurement to Everyday Language. In: Theory & Psychology 6/2017, pp. 703–710.
-
Carnap, Rudolph (1928): The Logical Structure of the World. Berkeley: University of California Press.
-
Cramer, Angélique O.J./Kendler, Kenneth S./Borsboom, Denny (2012): A Constructionist Account of Emotional Disorders. In: The Behavioral and Brain Sciences 3/2012, pp. 146–147.
-
Cronbach, Lee J./Mehl, Paul E. (1955): Construct Validity in Psychological Tests. In: Psychological Bulletin 4/1955, pp. 281–302.
-
Weltgesundheitsorganisation (2011): Internationale Klassifikation psychischer Störungen. ICD-10 Kapitel V (F). Diagnostische Kriterien für Forschung und Praxis (5., überarbeitete Aufl.). Bern: Huber.
-
Feynman, Richard P. (1974): Cargo Cult Science. In: Engineering and Science 7/1974, pp. 10–13.
-
Gulliksen, Harold (1950): Theory of Mental Tests. Hoboken: John Wiley & Sons Inc.
-
Klein, Richard A./Ratliff, Kate A./Vianello, Michelangelo/Adams, Reginald B./Bahník, Štěpán/Bernstein, Michael J. et al. (2014): Investigating Variation in Replicability. In: Social Psychology 3/2014, pp. 142–152.
-
Koller, Ingrid/Haberkorn, Kerstin/Rohm, Theresa (2014): NEPS Technical Report for Reading: Scaling results of Starting Cohort 6 for adults in main study 2012 (NEPS Working Paper No. 48). Bamberg: Leibniz Institute for Educational Trajectories.
-
Lord, F.M./Novick, M.R. (1968): Statistical Theories of Mental Test Scores. Oxford: Addison-Wesley.
-
Michell, Joel (1999): Measurement in Psychology. Cambridge: Cambridge University Press.
-
Moosbrugger, Helfried/Kelava, Augustin (2012): Testtheorie und Fragebogenkonstruktion (2., aktualisierte und überarbeitete Aufl.). Berlin: Springer.
-
Open Science Collaboration (2015): Estimating the Reproducibility of Psychological Science. In: Science 349/6251, pp. aac4716-1–aac4716-8.
-
Organisation for Economic Co-operation and Development (2012): PISA 2009 Technical Report. Paris: OECD Publishing.
-
Rost, Jürgen (2004): Lehrbuch Testtheorie – Testkonstruktion (2., überarbeitete und erweiterte Aufl.). Bern: Huber.
-
Schmittmann, Verena D./Cramer, Angélique O.J./Waldorp, Lourens J./Epskamp, Sacha/Kievit, Rogier A./Borsboom, Denny (2013): Deconstructing the Construct. A Network Perspective on Psychological Phenomena. In: New Ideas in Psychology 1/2013, pp. 43–53.
-
Slaney, Kathleen L./Racine, Timothy P. (2013): What’s in a Name? Psychology’s Ever Evasive Construct. In: New Ideas in Psychology 1/2013, pp. 4–12.
-
van Fraassen, Bas C. (1980): The Scientific Image. Oxford: Clarendon Press.
-
Martin, Michael O./Mullis, Ina V.S./Hooper, Martin (Eds.) (2016): Methods and Procedures in TIMSS 2015. Boston College. URL: https://timss.bc.edu/publications/timss/2015-methods.html (Datum des letzten Abrufs: 27. September 2018).
| All Time | Past 365 days | Past 30 Days | |
|---|---|---|---|
| Abstract Views | 1085 | 179 | 25 |
| Full Text Views | 287 | 51 | 4 |
| PDF Views & Downloads | 287 | 29 | 0 |
Zwischen Definition und Empirie
Theoretische Probleme der Fragebogenkonstruktion
In: Vierteljahrsschrift für wissenschaftliche PädagogikSearch for other papers by Matthias Borgstede in
Current site
Google Scholar
PubMed
Abstract
Between Definition and Empirical Research. Theoretical Problems of Questionnaire Design
Within the field of quantitative educational science, theoretical terms are often defined operationally by scales or questionnaires, which have been constructed using the formal background of Classical Test Theory. This article takes a critical perspective on the current modus operandi of scale construction.
It is argued that classical quality criteria (reliability, validity, and objectivity) are usually represented by empirical correlations between scales or items of a scale. Apart from some theoretical inconsistencies, using these criteria (e.g. Chronbach’s Alpha) to optimize a scale results in a set of items with high intercorrelations, regardless of their content or meaning.
Because high inter-item correlations are neither necessary nor sufficient for an adequate operational definition, the utility of the classical approach to scale construction is questioned on a fundamental level. It is further argued that operational definitions do not pose a statistical problem in the first place, but a theoretical problem. Hence, we do not need a method to optimize scales for high inter-item-correlations, but rather a way to translate theoretical terms (which may be inherently vague) to empirically observable phenomena. This is accomplished by the semantic approach to psychological tests by Buntins, Buntins and Eggert (2016), which provides a formal alternative to classical quality criteria.
| All Time | Past 365 days | Past 30 Days | |
|---|---|---|---|
| Abstract Views | 1085 | 179 | 25 |
| Full Text Views | 287 | 51 | 4 |
| PDF Views & Downloads | 287 | 29 | 0 |
