thomas richardson

Challenges posed byStructural Equation Models : 

Challenges posed by Structural Equation Models Thomas Richardson Department of Statistics University of Washington Joint work with Mathias Drton, UC Berkeley Peter Spirtes, CMU

Overview: 

Overview Challenges for Likelihood Inference Problems in Model Selection and Interpretation Partial Solution sub-class of path diagrams: ancestral graphs

Problems for Likelihood Inference: 

Problems for Likelihood Inference Likelihood may be multimodal e.g. the bi-variate Gaussian Seemingly Unrelated Regression (SUR) model: may have up to 3 local maxima. Consistent starting value does not guarantee iterative procedures will find the MLE.

Problems for Likelihood Inference: 

Problems for Likelihood Inference Discrete latent variable models are not curved exponential families C X1 X2 X3 X4 binary observed variables ternary latent class variable 15 parameters in saturated model 14 model parameters BUT model has 2d.f. (Goodman) Usual asymptotics may not apply

Problems for Likelihood Inference: 

Problems for Likelihood Inference Likelihood may be highly multimodal in the asymptotic limit After accounting for label switching/aliasing C X1 X2 X3 X4 Why report one mode ? d.f. may vary as a function of model parameters

Problems for Model Selection: 

Problems for Model Selection SEM models with latent variables are not curved exponential families Standard c2 asymptotics do not necessarily apply e.g. for LRTs Model selection criteria such as BIC are not asymptotically consistent The effective degrees of freedom may vary depending on the values of the model parameters

Problems for Model Selection: 

Problems for Model Selection Many models may be equivalent: X1 X2 Y1 Y2 X1 X2 Y1 Y2 X1 X2 Y1 Y2

Problems for Model Selection: 

Problems for Model Selection Models with different numbers of latents may be equivalent: e.g. unrestricted error covariance within blocks

Problems for Model Selection: 

Problems for Model Selection Models with different numbers of latents may be equivalent: e.g. unrestricted error covariance within blocks X1 Xp Y1 Yq x w X1 Xp Y1 Yq y Wegelin & Richardson (2001)

Two scenarios: 

Two scenarios A single SEM model is proposed and fitted. The results are reported.

Two scenarios: 

Two scenarios A single SEM model is proposed and fitted. The results are reported. The researcher fits a sequence of models, making modifications to an original specification. Model equivalence implies: Final model depends on initial model chosen Sequence of changes is often ad hoc Equivalent models may lead to very different substantive conclusions Often many equivalence classes of models give reasonable fit. Why report just one?

Partial Solution: 

Partial Solution Embed each latent variable model in a ‘larger’ model without latent variables characterized by conditional independence restrictions. We ignore non-independence constraints and inequality constraints. Latent variable model Model imposing only independence constraints on observed variables Sets of distributions

The Generating graph: 

a b t c d Toy Example: G +others The Generating graph Begin with a graph, and associated set of independences

Marginalization: 

a b t c d G a t d t b c t +others hidden: ‘Unobserved’ independencies in red Marginalization Suppose now that some variables are unobserved Find the independence relations involving only the observed variables Toy Example:

Marginalization: 

a b t c d G a t d t b c t +others hidden: ‘Unobserved’ independencies in red Marginalization Suppose now that some variables are unobserved Find the independence relations involving only the observed variables Toy Example:

‘Graphical Marginalization’: 

a b t c d a b c d G G* ‘Graphical Marginalization’ Now construct a graph that represents the conditional independence relations among the observed variables. Bi-directed edges are required. represents Toy Example: all and only the distributions in which these independencies hold

Equivalence re-visited: 

Equivalence re-visited Restrict model class to path diagrams including only observed variables characterized by conditional independence Ancestral Graph Markov models For such models we can: Determine the entire class of equivalent models Identify which features they have in common Models are curved exponential: usual asymptotics do apply

Slide18: 

A T A B C D A B C D Ancestral Graph

Slide19: 

A V A B C D T A B C D U A B C D A B C D Equivalent ancestral graphs Þ

Slide20: 

A V A B C D T A B C D U Q A B C D P R A B C D Markov Equiv. Class of Graphs with Latent Variables Þ Equivalent ancestral graphs

Slide21: 

A V A B C D T A B C D U + infinitely many others Q A B C D P R A B C D N A B C D M R L Markov Equiv. Class of Graphs with Latent Variables Þ Equivalence Classes Equivalent ancestral graphs

Slide22: 

A B C D A V A B C D T A B C D U + infinitely many others Q A B C D P R A B C D N A B C D M R L Markov Equiv. Class of Graphs with Latent Variables ß Equivalence class of Ancestral Graphs Partial Ancestral Graph

Slide23: 

A B C D Partial Ancestral Graph A V A B C D T A B C D U + infinitely many others Q A B C D P R A B C D Equivalence class of Ancestral Graphs N A B C D M R L Markov Equiv. Class of Graphs with Latent Variables ß

Measurement models: 

Measurement models If we have pure measurement models with several indicators per latent: May apply similar search methods among the latent variables (Spirtes et al. 2001; Silva et al.2003)

Other Related Work: 

Other Related Work Iterative ML estimation methods exist Guaranteed convergence Multimodality is still possible Implemented in R package ggm (Drton & Marchetti, 2003) Current work: Extension to discrete data Parameterization and ML fitting for binary bi-directed graphs already exist Implementing search procedures in R

References: 

References Richardson, T., Spirtes, P. (2002) Ancestral graph Markov models, Ann. Stat., 30: 962-1030 Richardson, T. (2003) Markov properties for acyclic directed mixed graphs. Scand. J. Statist. 30(1), pp. 145-157 Drton, M., Richardson T. (2003) A new algorithm for maximum likelihood estimation in Gaussian graphical models for marginal independence. UAI 03, 184-191 Drton, M., Richardson T. (2003) Iterative conditional fitting in Gaussian ancestral graph models. UAI 04 130-137. Drton, M., Richardson T. (2004) Multimodality of the likelihood in the bivariate seemingly unrelated regressions model. Biometrika, 91(2), 383-92. Marchetti, G., Drton, M. (2003) ggm package. Available from http://cran.r-project.org