×

Finite mixtures in confirmatory factor-analysis models. (English) Zbl 0890.62047

Summary: Various types of finite mixtures of confirmatory factor-analysis models are proposed for handling data heterogeneity. Under the proposed mixture approach, observations are assumed to be drawn from mixtures of distinct confirmatory factor-analysis models. But each observation does not need to be identified to a particular model prior to model fitting. Several classes of mixture models are proposed. These models differ by their unique representations of data heterogeneity.
Three different sampling schemes for these mixture models are distinguished. A mixed type of these three sampling schemes is considered throughout this article. The proposed mixture approach reduces to regular multiple-group confirmatory factor-analysis under a restrictive sampling scheme, in which the structural equation model for each observation is assumed to be known. By assuming a mixture of multivariate normals for the data, maximum likelihood estimation using the EM (Expectation-Maximization) algorithm and the AS (Approximate-Scoring) method are developed, respectively. Some mixture models were fitted to a real data set for illustrating the application of the theory. Although the EM algorithm and the AS method gave similar sets of parameter estimates, the AS method was found computationally more efficient than the EM algorithm. Some comments on applying the mixture approach to structural equation modeling are made.

MSC:

62H25 Factor analysis and principal components; correspondence analysis
62P15 Applications of statistics to psychology
Full Text: DOI

References:

[1] Aitkin, M., & Wilson, G. T. (1980). Mixture models, outliers, & the EM algorithm.Technometrics, 22, 325–331. · Zbl 0466.62034 · doi:10.2307/1268316
[2] Behboodian, J. (1970). On a mixture of normal distributions.Biometrika, 57, 215–217. · Zbl 0193.18104 · doi:10.1093/biomet/57.1.215
[3] Bentler, P. M., Lee, S.-Y., & Weng, L.-J. (1987). Multiple population covariance structure analysis under arbitrary distribution theory.Communications in Statistics-Theory and Methods, 16, 1951–1964. · Zbl 0625.62032 · doi:10.1080/03610928708829482
[4] Bhattacharya, C. G. (1967). A simple method of resolution of a distribution into Gaussian components.Biometrics, 23, 115–135. · doi:10.2307/2528285
[5] Blåfield, E. (1980). Clustering of observations from finite mixtures with structural information.Jyvaskyla Studies in Computer Science, Economics & Statistics 2. Jyvaskyla University, Finland.
[6] Bollen, K. A. (1989).Structural Equations with Latent Variables. New York: Wiley. · Zbl 0731.62159
[7] Browne, M. W. (1982). Covariance structures. In D. M. Hawkins (Ed.),Topics in applied multivariate analysis (pp. 72–141). London: Cambridge University Press.
[8] Browne, M. W. (1984). Asymptotically distribution-free methods for the analysis of covariance structures.British Journal of Mathematical and Statistical Psychology, 37, 62–83. · Zbl 0561.62054
[9] Choi, K. (1969). Estimators for the parameters of a finite mixture of distributions.The Annals of Institute of Statistical Mathematics, 21, 107–116. · Zbl 0183.48301 · doi:10.1007/BF02532235
[10] Choi, K., & Bulgren, W. B. (1968). An estimation procedure for mixtures of distributions.Journal of the Royal Statistical Society, Series B, 30, 444–460. · Zbl 0187.15804
[11] Crawford, S. L., DeGroot, M. H., Kadane, J. B., & Small, M. J. (1992). Modeling lake-chemistry distribution: Approximate Bayesian methods for estimating a finite mixture model.Technometrics, 34, 441–455. · doi:10.2307/1268943
[12] Day, N. E. (1969). Estimating the components of a mixture of normal distributions.Biometrika, 56, 463–474. · Zbl 0183.48106 · doi:10.1093/biomet/56.3.463
[13] Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood estimation from incomplete data via the EM algorithm.Journal of the Royal Statistical Society, Series B, 39, 1–38. · Zbl 0364.62022
[14] Do, K., & McLachlan, G. J. (1984). Estimation of mixing proportions: A case study.Applied Statistics, 33, 134–140. · doi:10.2307/2347437
[15] Everitt, B. S., & Hand, D. J. (1981).Finite mixture distributions. London: Chapman and Hall. · Zbl 0466.62018
[16] Fryer, J. G., & Robertson, C. A. (1972). A comparison of some methods for estimating mixed normal distributions.Biometrika, 59, 639–648. · Zbl 0255.62035 · doi:10.1093/biomet/59.3.639
[17] Furman, W. D., & Lindsay, B. G. (1994b). Measuring the relative effectiveness of moment estimators as starting values in maximizing mixture likelihoods.Computational Statistics and Data Analysis, 17, 473–492. · Zbl 0937.62557 · doi:10.1016/0167-9473(94)90144-9
[18] Ganesalingam, S., & McLachlan, G. J. (1981). Some efficiency results for the estimation of the mixing proportion in a mixture of two normal distributions.Biometrics, 37, 23–33. · Zbl 0498.62049 · doi:10.2307/2530519
[19] Goldfeld, S. M., & Quandt, R. E. (1976).Studies in Nonlinear Estimation. Cambridge, MA: Ballinger. · Zbl 0418.62090
[20] Hartigan, J. A. (1977). Distribution problems in clustering. In J. van Ryzin (Ed.),Classification and clustering (pp. 54–71). New York: Academic Press.
[21] Hartley, M. J. (1978). Comments on a paper by Quandt and Ramsey.Journal of the American Statistical Association, 73, 738–741. · doi:10.2307/2286267
[22] Hasselblad, V. (1966). Estimation of parameters for a mixture of normal distributions.Technometrics, 8, 431–444. · doi:10.2307/1266689
[23] Hasselblad, V. (1967).Finite mixtures of distributions from the exponential family. Unpublished doctoral dissertation, UCLA, California.
[24] Hathaway, R. J. (1985). A constrained formulation of maximum-likelihood estimation for normal mixture distributions.The Annals of Statistics, 13, 795–800. · Zbl 0576.62039 · doi:10.1214/aos/1176349557
[25] Hathaway, R. J. (1986). A constrained EM algorithm for univariate normal mixtures.Journal of Statistical Computation and Simulation, 23, 211–230. · doi:10.1080/00949658608810872
[26] Holzinger, K. J., & Swineford, F. (1939). A study in factor analysis: The stability of a bi-factor solution.Supplementary Educational Monographs, 48.
[27] Hosmer, D. W. (1974). Maximum likelihood estimates of the parameters of a mixture of two regression lines.Communication in Statistics-Theory and Methods, 3, 995–1006. · Zbl 0294.62085 · doi:10.1080/03610927408827201
[28] Hosmer, D. W., & Dick, N. P. (1977). Information and mixtures of two normal distributions.Journal of Statistical Computation and Simulation, 6, 137–148. · Zbl 0366.62038 · doi:10.1080/00949657708810178
[29] John, S. (1970). On identifying the population of origin of each observation in a mixture of observations from two normal populations.Technometrics, 12, 553–563. · doi:10.2307/1267202
[30] Johnson, R. A., & Wichern, D. W. (1988).Applied Multivariate Statistical Analysis (2nd ed.). New Jersey: Prentice Hall. · Zbl 0663.62061
[31] Jöreskog, K. G. (1971). Simultaneous factor analysis in several populations.Psychometrika, 57, 409–426. · Zbl 0227.62061 · doi:10.1007/BF02291366
[32] Kano, Y., Berkane, M., & Bentler, P. M. (1990). Covariance structure analysis with heterogeneous kurtosis parameters.Biometrika, 77, 575–585. · Zbl 0724.62057 · doi:10.1093/biomet/77.3.575
[33] Kiefer, N. M. (1978). Discrete parameter variation: Efficient estimation of a switching regression model.Econometrica, 46, 427–434. · Zbl 0408.62058 · doi:10.2307/1913910
[34] Lee, S.-Y., & Tsui, K. L. (1982). Covariance structure analysis in several populations.Psychometrika, 47, 297–308. · Zbl 0498.62050 · doi:10.1007/BF02294161
[35] Lehmann, E. L. (1980). Efficient Likelihood Estimators.The American Statistician, 34, 233–235. · doi:10.2307/2684068
[36] Lindsay, B. G. (1989). Moment matrices: Applications in mixtures.The Annals of Statistics, 13, 435–475. · Zbl 0672.62063
[37] Lindsay, B. G., & Basak, P. (1993). Multivariate normal mixtures: A fast consistent method of moments.Journal of the American Statistical Association, 88, 468–476. · Zbl 0773.62037 · doi:10.2307/2290326
[38] Magnus, J. R., & Neudecker, H. (1988).Matrix differential calculus with applications in statistics and econometrics. Chichester: Wiley. · Zbl 0651.15001
[39] McLachlan, G. J. (1982). The classification and mixture mixture likelihood approaches to cluster analysis. In P. R. Krishnaiah & L. N. Kanal (Eds.):Handbook of statistics, Vol.2 (pp. 199–208). · Zbl 0513.62064 · doi:10.1016/S0169-7161(82)02012-4
[40] McLachlan, G. J., & Basford, K. E. (1988).Mixture models: Inference and applications to clustering. New York: Marcel Dekker. · Zbl 0697.62050
[41] Muthén, B. O. (1989). Latent variable modeling in heterogeneous populations.Psychometrika, 54, 557–585. · doi:10.1007/BF02296397
[42] Odell, P. L., & Basu, J. P. (1976). Concerning several methods for estimating crop acreages using remote sensing data.Communications in Statistics-Theory and Methods, 5, 1091–1114. · Zbl 0364.62066 · doi:10.1080/03610927608827427
[43] Pearson, K. (1894). Contribution to the mathematical theory of evolution.Philosophical Transactions of the Royal Society, Series A, 185, 71–110. · JFM 25.0347.02 · doi:10.1098/rsta.1894.0003
[44] Please, N. W. (1973). Comparison of factor loadings in different populations.British Journal of Mathematical and Statistical Psychology, 26, 61–89. · Zbl 0261.92008
[45] Quandt, R. E. (1972). A new approach to estimating switching regressions.Journal of the American Statistical Association, 67, 306–310. · Zbl 0237.62047 · doi:10.2307/2284373
[46] Quandt, R. E., & Ramsey, J. B. (1978). Estimating mixtures of normal distributions and switching regressions.Journal of the American Statistical Association, 73, 730–738. · Zbl 0401.62024 · doi:10.2307/2286266
[47] Rajagopalan, M., & Loganathan, A. (1991). Bayes estimates of mixing proportions in finite mixture distributions.Communications in Statistics-Theory and Methods, 20, 2337–2349. · doi:10.1080/03610929108830636
[48] Rao, C. R. (1952).Advanced statistical methods in biometric research. New York: Wiley. · Zbl 0047.38601
[49] Redner, R. A., & Walker, H. F. (1984). Mixture densities, maximum likelihood and the EM algorithm.SIAM Review, 26, 195–239. · Zbl 0536.62021 · doi:10.1137/1026034
[50] Rubin, D. B., & Thayer, D. T. (1982). EM algorithms for ML factor analysis.Psychometrika, 47, 69–76. · Zbl 0483.62046 · doi:10.1007/BF02293851
[51] SAS Institute (1990).SAS/IML Software: Usage and Reference (version 6). Cary, NC: Author.
[52] Schoenberg, R., & Richtand, C. (1984). Application of the EM method.Sociological Methods and Research, 13, 127–150. · doi:10.1177/0049124184013001006
[53] Schork, N. (1992). Bootstrapping likelihood ratios in quantitative genetics. In R. LePage & L. Billard (Eds.),Exploring the limits of bootstrap (pp. 389–396). New York: Wiley. · Zbl 0960.62558
[54] Sclove, S. C. (1977). Population mixture models and clustering algorithms.Communications in Statistics-Theory and Methods, Series A, 6, 417–434. · Zbl 0368.62036 · doi:10.1080/03610927708827502
[55] Scott, A. J., & Symons, M. J. (1971). Clustering methods based on likelihood ratio criteria.Biometrics, 27, 238–397.
[56] Smith, A. F. M., & Makov, U. E. (1978). A quasi-Bayes sequential procedures for mixtures.Journal of the Royal Statistical Society, Series B, 40, 106–111. · Zbl 0377.62045
[57] Sörbom, D. (1974). A general method for studying differences in factor means and factor structures between groups.British Journal of Mathematical and Statistical Psychology, 27, 229–239. · Zbl 0285.62031
[58] Sundberg, R. (1976). An iterative method for solution of the likelihood equations for incomplete data from exponential families.Communications in Statistics-Simulation and Computation, 5, 55–64. · Zbl 0352.62014 · doi:10.1080/03610917608812007
[59] Symons, M. J. (1981). Clustering criteria and multivariate normal mixtures.Biometrics, 37, 35–43. · Zbl 0473.62048 · doi:10.2307/2530520
[60] Tan, W. Y., & Chang, W. C. (1972). Some comparisons of the method of moments and the method of maximum likelihood in estimating parameters of a mixture of two normal densities.Journal of the American Statistical Association, 67, 702–708. · Zbl 0245.62039 · doi:10.2307/2284472
[61] Teicher, H. (1960). On the mixture of distributions.The Annals of the Mathematical Statistics, 31, 55–73. · Zbl 0107.13501 · doi:10.1214/aoms/1177705987
[62] Teicher, H. (1961). Identifiability of mixtures.The Annals of the Mathematical Statistics, 32, 244–248. · Zbl 0146.39302 · doi:10.1214/aoms/1177705155
[63] Teicher, H. (1963). Identifiability of finite mixtures.The Annals of the Mathematical Statistics, 34, 1265–1269. · Zbl 0137.12704 · doi:10.1214/aoms/1177703862
[64] Titerington, D. M. (1990). Some recent research in the analysis of mixture distributions.Statistics, 21, 619–641. · Zbl 0714.62023 · doi:10.1080/02331889008802274
[65] Titterington, D. M., Smith, A. F. M., & Makov, U. E. (1985).Statistical analysis of finite mixture distributions. Chichester: Wiley. · Zbl 0646.62013
[66] Wolfe, J. H. (1970). Pattern clustering by multivariate mixture analysis.Multivariate Behavioral Research, 5, 329–350. · doi:10.1207/s15327906mbr0503_6
[67] Yakowitz, S. J. (1969). A consistent estimators for the identification of finite mixtures.The Annals of Mathematical Statistics, 39, 209–214. · Zbl 0155.25703 · doi:10.1214/aoms/1177698520
[68] Yakowitz, S. J., & Spragins, J. D. (1968). On the identifiability of finite mixtures.The Annals of Mathematical Statistics, 39, 1728–1735. · Zbl 0155.25703 · doi:10.1214/aoms/1177698520
[69] Yung, Y. F. (1995).Finite mixtures in confirmatory factor-analytic models (microfilm). Ann Arbor, MI: Univesity Microfilms.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.