The Data Generation Mechanism: Relationship Between Constructs and Their Indicators

O Mecanismo de Geração de Dados: Relação entre Construtos e seus Indicadores

Abstract

The choice of statistical data analysis should be guided by a critical analysis that supports the theoretical relationship between the construct and its indicators. This theoretical article reviews the three main existing psychometric paradigms and their proposals for explaining the relationship between indicators and their constructs. The discussion begins with the standard paradigm that guides the construction and analysis of data in psychology, reflective model. Then, a description of the formative models is performed and finally the Network Analysis as an alternative. The definitions, consequences, and limitations of the use of each measurement model are presented such as a reflection on making decisions about which data generation mechanisms are more appropriate.

Keywords:
psychometrics; data analysis; data generation mechanism; reflective models; formative models; network analysis

Resumo

A escolha da análise estatística de dados deveria ser guiada por uma análise crítica que fundamenta a relação teórica entre construto e seus indicadores. Este teórico artigo faz uma revisão dos três principais paradigmas psicométricos e suas propostas de explicação da relação entre os indicadores e seus construtos. A discussão é iniciada com o paradigma padrão que guia a construção e análise de dados na psicologia, os modelos reflexivos. Em seguida, é realizada uma descrição dos modelos formativos e, por fim, a proposta da Análise de Redes como alternativa. São apresentadas as definições, consequências e limitações do uso de cada modelo de medida, bem como uma reflexão na tomada de decisão sobre quais mecanismos de geração de dados são mais apropriados.

Palavras-chave:
psicometria; análise de dados; mecanismo de geração de dados; modelos reflexivos; modelos formativos; análise de redes

One of the focuses of psychological science is the comprehension of unobservable constructs, known as latent traits. To study them, one of the methods commonly employed is the use of psychological instruments (Primi, 2003). It is understood that items of different natures (verbal, non-verbal, and iconographic, among others) may represent these factors, depending on a consistent theoretical construction. Two psychometric models have been commonly cited in the literature (Rhemtulla et al., 2020) to understand the relationship between the latent trait and the items or indicators that compose it: the reflective model and the formative model.

In the reflective model, it is understood that indicators or items are caused by the latent variable. In the formative model, however, it is assumed that the indicators cause the latent variable (Rhemtulla et al., 2020). Comprehending this difference in the relationship between the construct and the measurement variation, as well as the meaning of the instrument scores (Borsboom, 2005; Borsboom et al., 2009; Christensen et al., 2020) is a central issue for the appropriate choices of the analysis techniques employed and for a better interpretation of the score of the instruments used, or even to comprehend the view of human beings related to latent traits.

The use of certain statistical data analysis applied to psychological instruments presupposes a theory about the data generation mechanism. For example, when analyzing the data using Confirmatory Factor Analysis (CFA) it is assumed that there is a latent trait that explains the covariance between indicators of the same factor. This analytical choice is preponderant in psychology studies that include the reflective model as a data generating mechanism (Bollen, 2002). However, the reflective model is not the only one available and some studies have demonstrated the effects of its indiscriminate use (Rhemtulla et al., 2020).

The use of a reflective measurement model, when the true data generating mechanism is potentially another (formative model or in networks), results in bias in the research. The results of a study using Monte Carlo simulation indicated that incorrect specification of the measurement model can inflate estimates of non-standard structural parameters by up to 400% or reduce them by up to 80% and, consequently, lead to both type I and type II inference errors (MacKenzie et al., 2005). Rhemtulla et al. (2020) found that when a common factor model is fit to a set of items that represent the components or causes of the construct, the common factor model overestimates the structural correlations. The authors also found that the greater the unique variance, the greater the possibility of bias. That is, the incorrect specification makes the inference about the data invalid.

If the modeling of the construct is carried out assuming a formative model, when in fact it should be done assuming a reflective model, there is an erroneous increase in the explained variance. This leads to a reduction in bivariate correlations with other variables (Cole & Preacher, 2014). This has numerous consequences for the modeling of the constructs among themselves, which may lead to imprecise conclusions about their structural relationships (Law & Wong, 1999). Poor specification is not detected by most fit indices (MacKenzie et al., 2005) and can even occur when they are perfect.

As the model fit is not a reliable indicator of the degree of bias in the estimated parameters, there is a need for the data generation mechanism to be justified with theoretical bases in addition to the statistics used (Rhemtulla et al., 2020). Accordingly, this study aims to discuss the three main existing psychometric paradigms and the interpretations and analysis of data that can be carried out from each one in order to enable a critical analysis of what is being produced.

The Reflective Models

In psychology, for the most part, it is assumed that variations in the construct lead to variations in indicators. For example, variations in the level of commitment of a person lead to variations in the responses to the items of the commitment questionnaire. Reflective models are commonly presented as measurement models in modern and classical test theories (Item Response Theory and Classical Test Theory, Mellenbergh, 1994). Consequently, guides for the construction of scales are based on reflective indicators (Spector, 1992). Therefore, the so-called Reflective Models are those in which the indicators are the effect. That is, the scores produced by a questionnaire are effects caused by the common factor, in this case, the construct (Bollen & Lennox, 1991). Reflective models are also called common factor models by some authors (Rhemtulla et al., 2020). In reflective models, the observed indicators are modeled as a function of an unobserved common latent variable and the item has a specific error variance.

This reflective measurement model is also currently understood as the model underlying the Classical Test Theory (CTT; Lord et al., 1968; Novick, 1966), which formalizes the so-called observed score. That is, the subject’s effective response in the measurement instrument, obtained by summing the true score and the error. A researcher who understands the score of a job satisfaction instrument, for example, as the true score plus the error, assumes that the subject’s level of satisfaction on the test is its expected value (Borsboom, 2005). Strictly following the CTT, the reliability will always be unique in the population tested (Thompson, 2003), with no possibility of generalizations of the specific scores of each test. It would also be impossible to calibrate different items to construct a scale that accesses the same construct. Furthermore, any error associated with the measurement, by definition, is considered random (Borsboom, 2008). With advances in the understanding of the CTT, the generalizability theory started to allow the researcher to separate the various sources of error, something that in the initial proposition was random (Brennan, 1992). Generalizability theory is part of the scope of the Modern Theory of psychometrics (IRT), as opposed to the CTT (Spearman, 1904).

The reflective model is also the model underlying the latent variable model (Spearman, 1904), with this being the data generating mechanism that explains the use of analyses such as Confirmatory Factor Analysis (Jöreskog, 1971), Item Response Theory (p . e.g.; Rasch, 1960) and Generalized Linear Item Response Theory (Mellenbergh, 1994), among others. In the latent variable model, there is a theory regarding the data generation mechanism with the formalization of the relationship between the latent variable and the observed scores (Borsboom, 2005).

Among the latent trait definitions that offer a mathematical formalization, according to Bollen (2002), local independence is one of the most common ways to define the latent variable. The idea is that there is one or more variables that create the association between the observed indicators (the responses to the questionnaire items) and, when these variables are kept constant, the observed indicators are independent. In other words, the indicators are only related to the existence of the latent variable. By controlling the effect of the unobserved variable, the observed variables will be independent, therefore this variable will be understood as latent. Latent variables are locally independent due to their ability to explain the association between the observed variables (McDonald, 1996). That said, the need to assume local independence is evident when carrying out analyses that assume the common cause model as a paradigm. If the latent variable is the real cause of the association between variables, it is necessary for it to explain the entire association between the indicators, working analogously to an unobserved common cause (Pearl, 2000). However, this comprehension of the latent variable as the cause is not consensual.

Authors who take a more descriptive approach to the phenomenon claim that one common factor should not be interpreted as a real-world entity, but rather as a mere shared variance, a parsimonious summary of the data (Jonas & Markon, 2016). The main argument is that if the latent variable is a part of the variance shared among the indicators or is its cause, both result in the same statistical model. However, this argument proved to be flawed in simulations, implying a greater error associated with the incorrect specification of models (Rhemtulla et al., 2020).

In a practical way, a researcher who comprehends job satisfaction as a latent variable understands that the common cause of the responses to the different questionnaires is due to the individual’s underlying satisfaction. From this perspective, measurement error is associated with the indicators and, in this case, with the scores. When treating satisfaction as a latent variable, it is assumed that a person’s location in this variable cannot be inferred with certainty from the data (Borsboom, 2008), as there is always an error associated with the scores.

Because all indicators reflect the subject’s satisfaction, in the modeling process, reflective indicators must have the same antecedents and consequences in relation to other phenomena and constructs (MacKenzie et al., 2005). Even though they present a varied factor loading, the relationship established with the latent construct remains and the set of items of the same dimension are representative of this latent trait (Bollen, 1989), as they all reflect the same underlying construct. The concept of the psychological construct as a common cause of observed behaviors works well as a proxy to explain associations between certain behaviors, however, although the understanding made in the reflective model is the standard in psychology, some authors have indicated Formative Modeling as an alternative in which the causal relationship between the indicator and the latent variable is inverted (Bollen & Diamantopoulos, 2017; Bollen & Lennox, 1991).

The Formative Models

Both CTT and more modern theories are widespread, so it seems natural to simply use them. However, a reflection on the descriptors, items and the construct that one intends to access leads to the conclusion that the relationship between the measured attribute and the observed data is not always evident. For example, is the pattern of responses to the items in a questionnaire that addresses a job characteristic that generates satisfaction the result of a variation in the satisfaction latent trait or is it the characteristic of the job itself that causes the positive assessment?

Formative models differ from reflective models in several respects. Essentially, while the indicator is caused by the latent variable in the application of reflective models, in formative models the indicator causes the latent variable (Bollen & Bauldry, 2011). In other words, in reflective modeling the indicator is an effect and in formative modeling it is a cause. The discussion of what formative models are seems to be far from exhausted, and the authors themselves differ on the mathematical formalization of formative models (see Bollen & Diamantopoulos, 2017; Hanafiah, 2020).

There is a central distinction between two types of formative indicators that have guided the literature on the subject: the concept of models with Composite Formative indicators and models with Causal Formative indicators (Bollen & Bauldry, 2011; Bollen & Lennox, 1991). The variation in the nomenclature is due to the inclusion of the residuals in the model, as, in the Composite Formative models there are composite indicators, when the proxy/factor/index has no residual variation and in the Causal Formative models there are Causal indicators, when they have residual variation (Rhemtulla et al., 2020). Composite Formative Models refer to indicators in which the weighted sum equals the construct itself. The construct created is nothing more than the result of this sum of indicators, for example, the sum of education, income and occupation indicators as a representative of socioeconomic status. The composite forms a concept and disregards measurement errors, it is just a convenient way of presenting results (Bollen & Bauldry, 2011). The final score is entirely determined by the observed indicators (H. van der Maas et al., 2014).

Conceptualizing measurement as the process by which a concept/construct is linked to one or more latent variables and also to observed variables (Bollen, 1989), composite formative indicators are not psychological measures (Bollen & Bauldry, 2011), therefore they can be neither causes nor effects. There is no error and, unless the items are considered objective indicators, there is no way to suppose that the response to each item in the questionnaire does not have an associated error, even if the source of the error is unknown. Conversely, in Causal Formative Models, also known as causal indicators, the model is not totally determined by the set of observed variables and can be formalized and operationalized as latent (Bollen & Bauldry, 2011; Bollen & Lennox, 1991). The indicators have a certain conceptual unit and cause the latent construct. A widely used example in the literature is stress indicators (for example, changing jobs and divorce) that would be better understood as causes of stress and not caused by it.

Although these relationships are more complex than the examples presented and few variables are purely reflective or formative, network modeling emerges as an alternative. The formative model proposal is clearly the opposite of the reflective model, as it does not assume that all measures are caused by a single underlying construct. It is assumed that all measures have an impact (or cause) on the construct. That is, the direction of causality flows from the indicators to the latent construct, and the indicators, as a group, jointly determine the conceptual and empirical meaning of the construct, while the error is associated with the latent variable and not with the indicator (Bollen & Bauldry, 2011).

There are several contexts in which the application of formative modeling is adequate, such as in marketing research (Jarvis et al., 2003) and studies involving functional cognitive execution (Ikanga et al., 2017; Willoughby & Blair, 2016). For some authors, context variables such as performance and role conflict measures, among others, should also be understood as formative (MacKenzie et al., 2005). The usefulness of this type of model has been demonstrated, despite the various questions regarding its use (Edwards, 2011). For guides on how to use formative models, see Diamantopoulos and Temme (2013) and Peterson et al. (2017).

From a practical point of view, a researcher, when conducting a job satisfaction study following a Causal Formative Model, understands that each item captures a unique aspect of the construct. From this perspective, the items about satisfaction with the division of tasks are not interdependent with the items on satisfaction with task autonomy and both cause the latent trait satisfaction.

While in Reflective Models items must be interdependent and share something in common so that each captures the essence of the construct repeatedly, in Formative Models the indicators may not share a common theme and each may capture a unique aspect of the conceptual domain. Consequently, the factor may or may not contain items with a high correlation (Bollen & Lennox, 1991). As the construct is composed of all indicators, the removal of an item in the formative indicator model can omit a specific part of the conceptual domain and change the meaning of the variable. Unlike the practice carried out in reflective models, in formative models it is not possible to determine the relative importance of each item to construct a factor and, consequently, reduce the scale. Very high correlations between formative indicators can make it difficult to separate the distinct impact of the individual indicators on the construct (Bollen & Lennox, 1991). Consequently, causal indicators should not be evaluated for their internal consistency-commonly accessed using Cronbach’s alpha. Examples of data analysis that can be performed by understanding the formative model as a generating mechanism are Principal Component Analysis, Multiple Indicators Multiple Causes Model (MIMIC; Jöreskog & Goldberger, 1975) and “cluster” techniques that group indicators (Schmittmann et al., 2013; Benassi et al., 2020).

Although formative models are an alternative, some authors do not recommend them, claiming that they are fallacious models (Edwards, 2011) and problematic in several aspects. According to Howell et al. (2007), measures of formative constructs do not need to have the same nomological network. Forcing them to form a single compound may be inadvisable. Chang et al. (2016), using data simulations, showed that employing reflective models for patterns that could be considered formative would not be as harmful as the opposite.

According to Rhemtulla et al. (2020), a measurement model that uses causal indicators cannot be estimated alone as the proxy cannot be uniquely identified by a set of causes. Unknown parameters of the model include the covariances among the indicators, the weights of the causal indicators and the residual variation of the latent variable. There is sufficient information in the set of measured variables to estimate the covariances between the indicators, however, the weights and residual variation are either underdetermined or unidentified. In practice, researchers who choose to use a formative model look for two or more reflective indicators of a construct so that the model can be identified. Therefore, in recent years, a more prominent model has emerged from the two previously mentioned and more traditional models in the social sciences, this being, network analysis, assuming a complex interconnection between variables that can interact with each other (De Beurs et al., 2019).

Psychometric Network Analysis

Reflective and formative models, whether causal or composite, do not exhaust the available possibilities that explain the relationship between psychological attributes and observable variables (Schmittmann et al., 2013). In the previous two decades, studies using network analysis have gained ground among researchers of psychological variables (Borsboom et al., 2021). Network Psychometrics, or Network Analysis, emerged as a new way of analyzing data and, mainly, as a new perspective on understanding the nature of the relationship between the attribute and its indicator. Network analysis provides an innovative opportunity, as it conceives psychological constructs not as effects or causes of a latent entity, but as a mutual interaction of its attributes and/or indicators (Borsboom & Cramer, 2013).

An important distinction is needed between network analysis and other analyses that have similar nomenclature. As explained by Epskamp et al., (2018b), psychometric network analysis is markedly different from network structures commonly used in graph theory - electrical networks (Watts & Strogatz, 1998), social networks (Wasserman & Faust, 1994), Social Network Analysis (SNA; Zagenczyk et al, 2010) or ecological networks (Barzel & Biham, 2009) - in which the nodes (of the network) represent entities and the connections (vertices) are observed and known. The big difference is that in psychological networks the strength of the connection between two nodes is a parameter to be estimated from the data.

From the perspective of networks, psychological attributes are conceptualized as directly related observable networks (Schmittmann et al., 2013). The theoretical framework underlying the use of network analysis is also a psychometric theory. This is because it is not just an analysis, but a theory about the data generating mechanism. However, psychological network analyses can also be used to explore multicollinearity and predictive mediation and to highlight the presence of latent variables (Epskamp & Fried, 2018).

The first studies that started using network analysis questioned the paradigm of symptoms being caused by disorders. In network approaches to psychopathology, for example, disorders result from the causal interaction between symptoms involving feedback loops (Borsboom & Cramer, 2013). Changes are understood as causally connected symptom systems rather than as effects of a latent disorder. Consequently, in network theory, the idea that symptoms share a single causal background is deconstructed (Borsboom, 2017).

The use of the network perspective in the analysis of psychopathology data resulted in an alternative way of conceptualizing mental disorders, to the point where a theory of the network of mental disorders was proposed (the network theory of psychiatric disorders; Borsboom, 2017). In it, the concepts and comprehension of the diagnosis and treatment were redefined. The foundation of the network modeling of mental disorders is that psychiatric symptoms interact in such a way that the activation of one symptom promotes the activation of another symptom or symptoms (Kalisch et al., 2019). These interactions can occur through biological, psychological and social mechanisms (Borsboom, 2017; Fried & Cramer, 2017) and give new understanding to the psychological and psychiatric clinical practice.

The application of the network perspective enables new comprehensions of different theories. In social psychology, the causal attitude network (CAN) has been proposed as a comprehensive model for measuring attitudes, conceptualizing this construct as networks of causally connected evaluative reactions (Dalege et al., 2019). In the CAN model proposal, the strength of an attitude is the central aspect that sets it apart - while some are durable and have an impact, others are irrelevant and easily changeable. According to the model, the connectivity hypothesis applies to them: networks with highly connected attitudes correspond to stronger attitudes.

The network model also potentially presents a unifying theory of the emotions. After a review, Lange et al. (2020) suggested that only a psychometric network model, in which emotions are conceptualized as systems of emotional components interacting causally, can integrate the theories regarding emotions. Each area of psychology proposes a theory on how to comprehend its variables and in this sense, the use of network analysis constitutes an advance. Returning to the examples, in psychopathology the network is formed by the symptom-symptom interaction; in social psychology attitudes are causally connected reactions and emotions are systems of components interacting causally.

In practical terms, networks are structures made up of nodes connected in vertices. In the analysis, nodes represent variables relevant to a given phenomenon - for example, resilience (Kalisch et al., 2019), attitude (Dalege et al., 2019), symptom, psychopathology (McNally, 2016), personality (Christensen et al., 2020), longitudinal data (Epskamp, 2020) - or another indicator. Therefore, they represent psychological variables and the links represent unknown statistical relationships that can be estimated from the data (Epskamp et al., 2018a).

The indicator represented by a node can be the only item of a scale, subscale or composite scale. The scale can be dichotomous, ordinal or scalar. The decision about its representation is theoretical, however, it also obeys some assumptions of network analysis. If two or more variables are strongly correlated, measuring the same construct, they must be represented by only one node in the network (Fried & Cramer, 2017). If items that measure the same latent trait are represented by several nodes, there may be distortions in the centrality estimates. The estimation in network analysis is commonly performed with partial correlations (Epskamp et al., 2018), while inferences are based on centrality indices, the latter being statistical parameters that highlight which nodes are more influential (Robinaugh et al., 2016).

Final Considerations

Constructs defined with well-specified models enable conclusions consistent with reality and can explain them. The difference between formative and reflective is a central issue that must be addressed before the models are empirically tested (Edwards & Bagozzi, 2000). Consequently, a construct can have its meaning changed according to erroneous specifications of the measurement models adopted (Petter et al., 2012). The prior distinction between data generation mechanisms is important, as many of the scale development procedures recommended in the literature only apply to constructs with reflective measures (MacKenzie et al., 2005). As recommended by Mackenzie et al. (2005), the first issue to be addressed is whether the indicators are defining characteristics of the construct or manifestations of it. Considering the possibility of the network analysis paradigm, it would also be necessary to investigate whether there is a need to highlight a relationship with an abstract (latent) cause or whether the interaction between indicators is the actual source of information.

To analyze each of the models, Table 1 presents a summary of their definitions, what is measured, the characteristics of the indicators, the relationship between them, the relationships with antecedents and consequences, and the most prominent criticisms.

Table 1
Comparison of the main characteristics of the Reflective Model, Formative Model and Network Analysis

Formative Models	Reflective Models	Network Analysis
What is the measure?
The measures represent defining characteristics that collectively explain the meaning of the construct.	The measures are manifestations of the latent construct, in that they are determined by it.	The measures are functions of each other. The psychological constructs refer to groups of behaviors that directly influence each other.
Characteristic of the indicators
They do not necessarily share a common theme, each indicator can capture a unique aspect of the conceptual domain	They must be caused by a construct in common and each indicator must capture the essence of the domain of the construction. The indicators are samples of the same conceptual domain.	There is a need for investigation into the nature of individual indicators, as well as their causal dynamics.
Relationship between indicators
There are no predictions about the relationships between the indicators, but they should not show a high correlation.	The theory explicitly states that the indicators must be correlated.	The question that must be asked is whether two indicators in a network differentiate themselves, if so, they must be aggregated, otherwise they measure two different constructs and must be modeled and understood as distinct nodes within the network.
Antecedents and consequences
They do not necessarily have the same antecedents and consequences.	All indicators must have the same antecedents and consequences.	The relationship can be established between and among indicators.
Most prominent criticisms
Formative measurement models are not identifiable, regardless of the number of measurements used. To achieve identification, the model must be complemented by at least two reflective measures that are caused directly or indirectly by the latent variable (Bollen & Davis, 2009; MacCallum & Browne, 1993).	A reflective measurement model is identified as long as it has at least three measures, the indicators they are independent and a scale is defined for the latent variable (Bollen, 1989). The most prominent criticism concerns the need for the underlying cause to fully explain the covariation between the indicators (local independence), something considered implausible.	A pure form of the network model postulates that the co-occurrence between symptoms is due solely to the causal interactions between the symptoms. Taking into account the various factors that can trigger multiple symptoms at the same time, this is considered unlikely.

Despite the comprehension of the relationship between the latent trait and its indicators, until then widely used in reflective and formative psychometric models, network analysis has proposed an advance in the understanding of psychological phenomena, providing additional information of interest to researchers in the area and for professional interventions.

Although the choice for a given paradigm is restricted to a theoretical assumption, which cannot be put to the test with current resources, there is a recommendation that a critical analysis of the relationship between the construct and its indicator is made. Prior to using the instrument, it is important to reflect on whether interventions in the construct should change the indicator values, resulting in a reflective model, or the opposite, resulting in a formative model (see also Bollen, 1989; Edwards & Bagozzi, 2000). It is also important to identify whether there are mutual cause relationships between indicators following a network model (Borsboom, 2008; Borsboom et al., 2021). In terms of the analysis procedure, the first thing that should be done is to examine the patterns of association between the indicators (Bollen & Lennox, 1991; Diamantopoulos & Winklhofer, 2001). This procedure, which signals an important criticism or the need for further reflection on the models and analyses resulting from this choice, has been widely used in psychology research.

Regarding network analysis, despite being innovative, it does not solve some central issues of psychological measurement, and this is a subject still under debate. Considering its relationship with the latent trait theory, some enthusiasts say that there is no proxy variable that represents the construct (Borsboom & Cramer, 2013; Cramer & Borsboom, 2015) and that comprehending the attributes, through mutual interaction and indicators, presents markedly different theoretical implications (Kruis & Maris, 2016). However, some authors have demonstrated the mathematical equivalence between models (Golino & Epskamp, 2017) and claim that both reflective and network models generate the same results (Molenaar, 2010; van der Maas et al., 2006). Regardless of the equivalence between some special cases of these models, it is necessary that the choice of data analysis is based not only on its mathematical and statistical properties, but mainly on the theoretical impact generated.

Not everything can be fully comprehended in a network. More recent proposals have highlighted the need to understand some constructs in a hybrid way. For example, a disorder is a common cause of a range of symptoms, however, its maintenance is fueled by direct interactions that produce vicious circles (Fried & Cramer, 2017). The idea of a hybrid model also comprehends that the constitution of the set of interactions of, for example, a disorder, would be the cause of a latent variable, which, in this case, would be a formative latent variable (van Rooij et al., 2017). Despite this contribution, we also have some critiques about this systematically analysis of variables such as symptoms, traits and beliefs (see Neal et al., Preprint). Therefore, the combined use of the three types of models could be the best alternative for some psychology research scenarios and certainly more investments on researches to understand the limitations and necessary advances of psychological measurements.

References

• Barzel, B., & Biham, O. (2009). Quantifying the connectivity of a network: The network correlation function method. Physical Review E, 80(4), 046104-046104. https://doi.org/10.1103/PhysRevE.80.046104
  » https://doi.org/10.1103/PhysRevE.80.046104
• Benassi, M., Garofalo, S, Ambrosini, F., Sant'Angelo, R.P., Raggini, R., De Paoli, G., Ravani, C., Giovagnoli, S., Orsoni, M., Piraccini, G. (2020). Using Two-Step Cluster Analysis and Latent Class Cluster Analysis to Classify the Cognitive Heterogeneity of Cross-Diagnostic Psychiatric Inpatients. Frontiers in Psychology, 10(11), 1085. https://doi.org/10.3389/fpsyg.2020.01085
  » https://doi.org/10.3389/fpsyg.2020.01085
• Bollen, K. A. (1989). Structural equations with latent variables (Vol. 210). John Wiley & Sons. https://doi.org/10.1002/9781118619179
  » https://doi.org/10.1002/9781118619179
• Bollen, K. A. (2002). Latent variables in psychology and the social sciences. Annual Review of Psychology, 53(1), 605-634. https://doi.org/10.1146/annurev.psych.53.100901.135239
  » https://doi.org/10.1146/annurev.psych.53.100901.135239
• Bollen, K. A., & Bauldry, S. (2011). Three Cs in measurement models: causal indicators, composite indicators, and covariates. Psychological Methods, 16(3), 265-265. https://doi.org/10.1037/a0024448
  » https://doi.org/10.1037/a0024448
• Bollen, K. A., & Davis, W. R. (2009). Causal indicator models: Identification, estimation, and testing. Structural Equation Modeling: A Multidisciplinary Journal, 16(3), 498- 522. https://doi.org/10.1080/10705510903008253
  » https://doi.org/10.1080/10705510903008253
• Bollen, K. A., & Diamantopoulos, A. (2017). In defense of causal-formative indicators: A minority report. Psychological Methods, 22(3), 581-581. https://doi.org/10.1037/met0000056
  » https://doi.org/10.1037/met0000056
• Bollen, K. A., & Lennox, R. (1991). Conventional wisdom on measurement: A structural equation perspective. Psychological Bulletin, 110(2), 305-305. https://doi.org/10.1037/0033-2909.110.2.305
  » https://doi.org/10.1037/0033-2909.110.2.305
• Borsboom, D. (2005). Measuring the Mind: Conceptual Issues in Contemporary Psychometrics Cambridge University Press. https://doi.org/10.1017/CBO9780511490026
  » https://doi.org/10.1017/CBO9780511490026
• Borsboom, D. (2008). Latent Variable Theory. Measurement: Interdisciplinary Research and Perspectives, 6, 25-35. https://doi.org/10.1080/15366360802035497
  » https://doi.org/10.1080/15366360802035497
• Borsboom, D. (2017). A network theory of mental disorders. World Psychiatry, 16(1), 5-13. https://doi.org/10.1002/wps.20375
  » https://doi.org/10.1002/wps.20375
• Borsboom, D., & Cramer, A. O. (2013). Network analysis: an integrative approach to the structure of psychopathology. Annual Review of Clinical Psychology, 9, 91-121. https://doi.org/10.1146/annurev-clinpsy-050212-185608
  » https://doi.org/10.1146/annurev-clinpsy-050212-185608
• Borsboom, D., Deserno, M. K., Rhemtulla, M., Epskamp, S., Fried, E. I., McNally, R. J., Robinaugh, D. J., Perugini, M., Dalege, J., Costantini, G. et al. (2021). Network analysis of multivariate data in psychological science. Nature Reviews Methods Primers, 1(58). https://doi.org/10.1038/s43586-021-00055-w
  » https://doi.org/10.1038/s43586-021-00055-w
• Brennan, R. L. (1992). Generalizability theory. Educational Measurement: Issues and Practice, 11(4), 27-34. https://doi.org/10.1111/j.1745-3992.1992.tb00260.x
  » https://doi.org/10.1111/j.1745-3992.1992.tb00260.x
• Chang, W., Franke, G. R., & Lee, N. (2016). Comparing reflective and formative measures: New insights from relevant simulations. Journal of Business Research, 69(8), 3177-3185. https://doi.org/10.1016/j.jbusres.2015.12.006
  » https://doi.org/10.1016/j.jbusres.2015.12.006
• Christensen, A. P., Golino, H., & Silvia, P. J. (2020). A psychometric network perspective on the validity and validation of personality trait questionnaires. European Journal of Personality, 34(6), 1095-1108. https://doi.org/10.1002/per.2265
  » https://doi.org/10.1002/per.2265
• Cole, D. A., & Preacher, K. J. (2014). Manifest variable path analysis: Potentially serious and misleading consequences due to uncorrected measurement error. Psychological Methods, 19(2), 300-315. https://doi.org/10.1037/a0033805
  » https://doi.org/10.1037/a0033805
• Cramer, A. O., & Borsboom, D. (2015). Problems attract problems: A network perspective on mental disorders. Emerging Trends in the Social and Behavioral Sciences: An Interdisciplinary, Searchable, and Linkable Resource, 1-15. https://doi.org/10.1002/9781118900772.etrds0264
  » https://doi.org/10.1002/9781118900772.etrds0264
• Dalege, J., Borsboom, D., van Harreveld, F., & van der Maas, H. L. (2019). A network perspective on attitude strength: Testing the connectivity hypothesis. Social Psychological and Personality Science, 10(6), 746-756. https://doi.org/10.1177/1948550618781062
  » https://doi.org/10.1177/1948550618781062
• Darrow, J. B., & Behrend, T. S. (2017). Person-environment fit is a formative construct. Journal of Vocational Behavior, 103, 117-131. https://doi.org/10.1016/j.jvb.2017.08.007
  » https://doi.org/10.1016/j.jvb.2017.08.007
• De Beurs, D., Fried, E. I., Wetherall, K., Cleare, S., O' Connor, D. B., Ferguson, E., … O' Connor, R. C. (2019). Exploring the psychology of suicidal ideation: A theory driven network analysis. Behaviour Research and Therapy, 120, 103419. https://doi.org/10.1016/j.brat.2019.103419
  » https://doi.org/10.1016/j.brat.2019.103419
• Diamantopoulos, A., & Temme, D. (2013). MIMIC models, formative indicators and the joys of research. AMS Review, 3(3), 160-170. https://doi.org/10.1007/s13162-013-0050-0
  » https://doi.org/10.1007/s13162-013-0050-0
• Diamantopoulos, A., & Winklhofer, H. M. (2001). Index construction with formative indicators: An alternative to scale development. Journal of Marketing Research, 38(2), 269-277. https://doi.org/10.1509/jmkr.38.2.269.18845
  » https://doi.org/10.1509/jmkr.38.2.269.18845
• Edwards, J. R. (2011). The fallacy of formative measurement. Organizational Research Methods, 14(2), 370-388. https://doi.org/10.1177/1094428110378369
  » https://doi.org/10.1177/1094428110378369
• Edwards, J. R., & Bagozzi, R. P. (2000). On the nature and direction of relationships between constructs and measures. Psychological Methods, 5(2), 155-155. https://doi.org/10.1037/1082-989X.5.2.155
  » https://doi.org/10.1037/1082-989X.5.2.155
• Epskamp, S., Borsboom, D., & Fried, E. I. (2018). Estimating psychological networks and their accuracy: A tutorial paper. Behavior Research Methods, 50(1), 195-212. https://doi.org/10.3758/s13428-017-0862-1
  » https://doi.org/10.3758/s13428-017-0862-1
• Epskamp, S., & Fried, E. I. (2018). A tutorial on regularized partial correlation networks. Psychological Methods, 23(4), 617-617. https://doi.org/10.1037/met0000167
  » https://doi.org/10.1037/met0000167
• Epskamp, S. (2020). Psychometric network models from time-series and panel data. Psychometrika, 85(1), 1-34. https://doi.org/10.31234/osf.io/8ha93
  » https://doi.org/10.31234/osf.io/8ha93
• Fried, E. I., & Cramer, A. O. J. (2017). Moving forward: Challenges and directions for psychopathological network theory and methodology. Perspectives on Psychological Science, 12, 999-1020. https://doi.org/10.1177/1745691617705892
  » https://doi.org/10.1177/1745691617705892
• Golino, H. F., & Epskamp, S. (2017). Exploratory graph analysis: A new approach for estimating the number of dimensions in psychological research. PloS One, 12(6), e0174035-e0174035. https://doi.org/10.1371/journal.pone.0174035
  » https://doi.org/10.1371/journal.pone.0174035
• Hanafiah, M.H. (2020). Formative Vs. Reflective Measurement Model: Guidelines for Structural Equation Modeling Research. International Journal of Analysis and Applications, 18, 876-889. https://doi.org/10.28924/2291-8639-18-2020-876
  » https://doi.org/10.28924/2291-8639-18-2020-876
• Howell, R. D., Breivik, E., & Wilcox, J. B. (2007). Reconsidering formative measurement. Psychological Methods, 12(2), 205-218. https://doi.org/10.1037/1082-989X.12.2.205
  » https://doi.org/10.1037/1082-989X.12.2.205
• Ikanga, J., Hill, E. M., & MacDonald, D. A. (2017). The conceptualization and measurement of cognitive reserve using common proxy indicators: Testing some tenable reflective and formative models. Journal of Clinical and Experimental Neuropsychology, 39(1), 72-83. https://doi.org/10.1080/13803395.2016.1201462
  » https://doi.org/10.1080/13803395.2016.1201462
• Jarvis, C. B., MacKenzie, S. B., & Podsakoff, P. M. (2003). A critical review of construct indicators and measurement model misspecification in marketing and consumer research. Journal of Consumer Research, 30(2), 199-218. https://doi.org/10.1086/376806
  » https://doi.org/10.1086/376806
• Jonas, K. G., & Markon, K. E. (2016). A descriptivist approach to trait conceptualization and inference. Psychol Rev ., 123(1), 90-96. https://doi.org/10.1037/a0039542
  » https://doi.org/10.1037/a0039542
• Jöreskog, K. G. (1971). Simultaneous factor analysis in several populations. Psychometrika, 36(4), 409-426. https://doi.org/10.1007/BF02291366
  » https://doi.org/10.1007/BF02291366
• Jöreskog, K. G., & Goldberger, A. S. (1975). Estimation of a model with multiple indicators and multiple causes of a single latent variable. Journal of the American Statistical Association, 70(351a), 631-639. https://doi.org/10.2307/2285946
  » https://doi.org/10.2307/2285946
• Kalisch, R., Cramer, A. O., Binder, H., Fritz, J., Leertouwer, I., Lunansky, G., & van Harmelen, A. L. (2019). Deconstructing and reconstructing resilience: a dynamic network approach. Perspectives on Psychological Science, 14(5), 765-777. https://doi.org/10.1177/1745691619855637
  » https://doi.org/10.1177/1745691619855637
• Kruis, J., & Maris, G. (2016). Three representations of the Ising model. Scientific Reports, 6(1), 1-11. https://doi.org/10.1038/srep34175
  » https://doi.org/10.1038/srep34175
• Lange, J., Dalege, J., Borsboom, D., van Kleef, G. A., & Fischer, A. H. (2020). Toward an Integrative Psychometric Model of Emotions. Perspectives on Psychological Science, 15(2), 444-468. https://doi.org/10.1177/1745691619895057
  » https://doi.org/10.1177/1745691619895057
• Law, K. S., & Wong, C.-S. (1999). Multidimensional Constructs M Structural Equation Analysis: An Illustration Using the Job Perception and Job Satisfaction Constructs. Journal of Management, 25(2), 143-160. https://doi.org/10.1177/014920639902500202
  » https://doi.org/10.1177/014920639902500202
• Lord, F. M., Novick, M. R., & Birnbaum, A. (1968). The axioms and principal results of classical test theory Addison-Wesley. https://doi.org/10.1002/j.2333-8504.1965.tb00132.x
  » https://doi.org/10.1002/j.2333-8504.1965.tb00132.x
• MacCallum, R. C., & Browne, M. W. (1993). The use of causal indicators in covariance structure models: some practical issues. Psychological Bulletin, 12, 413-429. https://doi.org/10.1037/0033-2909.114.3.533
  » https://doi.org/10.1037/0033-2909.114.3.533
• MacKenzie, S. B., Podsakoff, P. M., & Jarvis, C. B. (2005). The problem of measurement model misspecification in behavioral and organizational research and some recommended solutions. Journal of Applied Psychology, 90(4), 710-710. https://doi.org/10.1037/0021-9010.90.4.710
  » https://doi.org/10.1037/0021-9010.90.4.710
• McDonald, R. P. (1996). Latent traits and the possibility of motion. Multivariate Behavioral Research, 31(4), 593-601. https://doi.org/10.1207/s15327906mbr3104_12
  » https://doi.org/10.1207/s15327906mbr3104_12
• McNally, r. j. (2016). Can network analysis transform psychopathology?, Behaviour Research and Therapy, 86, 95-104. https://doi.org/10.1016/j.brat.2016.06.006
  » https://doi.org/10.1016/j.brat.2016.06.006
• Mellenbergh, G. J. (1994). Generalized linear item response theory. Psychological Bulletin, 115(2), 300-307. https://doi.org/10.1037/0033-2909.115.2.300
  » https://doi.org/10.1037/0033-2909.115.2.300
• Molenaar, P. C. M. (2010). Latent variable models are net- work models. Behavioral and Brain Sciences, 33(2-3), 166-166. doi:10.1017/S0140525X10000798
  » https://doi.org/10.1017/S0140525X10000798
• Neal, Z. P., Forbes, M. K., Neal, J. W., Brusco, M., Krueger, R., Markon, K., Steinley, D., Wasserman, S., & Wright, A. G. C. Critiques of network analysis of multivariate data in psychological science. (2022). Nat Rev Methods Primers, 2, 90. https://doi.org/10.1038/s43586-022-00177-9
  » https://doi.org/10.1038/s43586-022-00177-9
• Novick, M. R. (1966). The axioms and principal results of classical test theory. Journal of Mathematical Psychology, 3(1), 1-18. https://doi.org/10.1016/0022-2496(66)90002-2
  » https://doi.org/10.1016/0022-2496(66)90002-2
• Pearl, J. (2000). Causal inference without counterfactuals: Comment. Journal of the American Statistical Association, 95(450), 428-431. https://doi.org/10.2307/2669380
  » https://doi.org/10.2307/2669380
• Peterson, C. H., Gischlar, K. L., & Peterson, N. A. (2017). Item construction using reflective, formative, or rasch measurement models: Implications for group work. Journal for Specialists in Group Work, 42(1), 17-32. https://doi.org/10.1080/01933922.2016.1264523
  » https://doi.org/10.1080/01933922.2016.1264523
• Petter, S., Rai, A., & Straub, D. (2012). The Critical Importance of Construct Measurement Specification: A Response to Aguirre-urreta and Marakas. MIS Quarterly, 36(1), 147-156. https://doi.org/10.2307/41410411
  » https://doi.org/10.2307/41410411
• Primi, R. (2003). Inteligência: avanços nos modelos teóricos e nos instrumentos de medida. Avaliação Psicológica, 2(1), 67-77.
• Rasch, G. (1960). Studies in mathematical psychology: I. Probabilistic models for some intelligence and attainment tests Nielsen & Lydiche. Available in: https://psycnet.apa.org/record/1962-07791-000
  » https://psycnet.apa.org/record/1962-07791-000
• Rhemtulla, M., van Bork, R., & Borsboom, D. (2020). Worse than measurement error: Consequences of inappropriate latent variable measurement models. Psychological Methods, 25(1), 30-30. https://doi.org/10.1037/met0000220
  » https://doi.org/10.1037/met0000220
• Robinaugh, D. J., Millner, A. J., & McNally, R. J. (2016). Identifying highly influential nodes in the complicated grief network. Journal of Abnormal Psychology, 125(6), 747-747. https://doi.org/10.1037/abn0000181
  » https://doi.org/10.1037/abn0000181
• Schmittmann, V. D., Cramer, A. O., Waldorp, L. J., Epskamp, S., Kievit, R. A., & Borsboom, D. (2013). Deconstructing the construct: a network perspective on psychological phenomena. New Ideas in Psychology, 31(1), 43-53. https://doi.org/10.1016/j.newideapsych.2011.02.007
  » https://doi.org/10.1016/j.newideapsych.2011.02.007
• Spearman, C. (1904). "General Intelligence", objectively determined and measured. American Journal of Psychology, 15, 201-293. https://doi.org/10.2307/1412107
  » https://doi.org/10.2307/1412107
• Spector, P. E. (1992). Summated Rating Scale Construction: An Introduction Sage CA. https://doi.org/10.4135/9781412986038
  » https://doi.org/10.4135/9781412986038
• Thompson, B. (2003). Guidelines for authors reporting score reliability estimates. In B. Thompson (Ed.), Reliability: Contemporary thinking on reliability issues(91-102). Thousand Oaks, CA: Sage. https://dx.doi.org/10.4135/9781412985789
  » https://dx.doi.org/10.4135/9781412985789
• van der Maas, H. L., Kan, K.-J., & Borsboom, D. (2014). Intelligence Is What the Intelligence Test Measures. Seriously. Journal of Intelligence, 2(1), 12-15. https://doi.org/10.3390/jintelligence2010012
  » https://doi.org/10.3390/jintelligence2010012
• van der Maas, H. L., Dolan, C. v., Grasman, R. P., Wicherts, J. M., Huizenga, H. M., & Raijmakers, M. E. (2006). A dynamical model of general intelligence: the positive manifold of intelligence by mutualism. Psychological Review, 113(4), 842-842. https://doi.org/10.1037/0033-295X.113.4.842
  » https://doi.org/10.1037/0033-295X.113.4.842
• van Rooij, A. J., van Looy, J., & Billieux, J. (2017). Internet Gaming Disorder as a formative construct: Implications for conceptualization and measurement. Psychiatry and Clinical Neurosciences, 71(7), 445-458. https://doi.org/10.1111/pcn.12404
  » https://doi.org/10.1111/pcn.12404
• Wasserman, S., & Faust, K. (1994). Social Network Analysis: Methods and Applications Cambridge University Press. https://doi.org/10.1017/CBO9780511815478
  » https://doi.org/10.1017/CBO9780511815478
• Watts, D. J., & Strogatz, S. H. (1998). Collective dynamics of 'small-world'networks. Nature, 393(6684), 440-442. https://doi.org/10.1038/30918
  » https://doi.org/10.1038/30918
• Willoughby, M. T., & Blair, C. B. (2016). Measuring executive function in early childhood: A case for formative measurement. Psychological Assessment, 28(3), 319-330. https://doi.org/10.1037/pas0000152
  » https://doi.org/10.1037/pas0000152
• Zagenczyk, T. J., Scott, K. D., Gibney, R., Murrell, A. J., & Thatcher, J. B. (2010). Social influence and perceived organizational support: A social networks analysis.Organizational Behavior and Human Decision Processes,111(2), 127-138. https://doi.org/10.1016/j.obhdp.2009.11.004
  » https://doi.org/10.1016/j.obhdp.2009.11.004

Publication Dates

• Publication in this collection
  23 June 2023
• Date of issue
  2023

History

• Received
  13 Dec 2021
• Accepted
  06 Sept 2022