This project is supported by a grant awarded to Zhiyong Zhang and Ke-Hai Yuan through the grant program on Statistical and Research Methodology in Education from the Institute of Education Sciences of U.S. Department of Education .

The title of the project is A General Framework for Statistical Power Analysis with Non-normal and Missing Data through Monte Carlo Simulation.

The importance of conducting a statistical power analysis at the beginning of a study is universally accepted (e.g., Cohen, 1988; Hedges & Rhoads, 2009). Without careful planning, a study can easily fail to detect an existing effect by chance. The increasing complexity of education research driven by education practice poses great challenges on existing methods of statistical power analysis. For example, education research often involves longitudinal and multilevel designs as well as advanced techniques such as structural equation and multilevel models. Furthermore, practical data in education are often not normally distributed and are incomplete. Regarding the nature of real data, Micceri (1989) reported that among 440 large-sample achievement and psychometric measures taken from journal articles, research projects, and tests, all were significantly non-normally distributed. In impact evaluations funded by the National Center for Educational Evaluation and Regional Assistance (NCEE), student achievement outcomes are often missing for 10-20 percent (e.g., Puma, 2009). Without careful consideration of the complexity of study designs and the impact of non-normal data and missing data, the validity of education research can be harmed.

This project develops a general framework for statistical power analysis with non-normal and missing data through Monte Carlo simulation. It enables data generation from a given target population as well as model fitting using cutting-edge methodology. The simulation is carried out for a sufficient larger number of times and the proportion of times that a null hypothesis is rejected is used as an estimate of statistical power. In this project, we address three critical components of such a method. (1) We develop a general method to enable the specification of models including structural equation models and multilevel models as well missing data mechanisms and non-normality of data through drawing path diagrams. (2) We develop methods for simulating data, including non-normal data and missing data, from a supplied model. (3) We develop methods for statistical power analysis that are robust to non-normal data and missing data. Compared to the commonly used methods in the literature, our method will have better controlled type I error rates and greater statistical power.

This project also develops software MCpower (now known as WebPower) to conduct power analysis within the proposed framework. MCpower runs as a web application and, therefore, can be used locally on a personal computer or remotely on a Web server within a web browser. (1) MCpower includes a graphical user interface that allows the specification of structural equation models and multilevel models with non-normal and missing data through drawing path diagrams. (2) MCpower implements different algorithms for data simulation from an arbitrary model. (3) MCpower conducts automatic Monte Carlo simulation to estimate statistical power.

The project is expected to offer the community of education researchers an easy-to-use and general-purpose tool to conduct sophisticated statistical power analysis for structural equation and multilevel models with non-normal and missing data.

1. Zhang, Z., & Mai, Y. (2018). WebPower: Basic and Advanced Statistical Power Analysis. Retrieved from https://CRAN.R-project.org/package=WebPower.
2. Zhang, Z., & Keenan, A. (2018). WebPower: An Android App for Power Analysis. Available on play store: https://play.google.com/store/apps/details?id=org.psychstat.webpower&hl=en
3. Zhang, Z., Yuan, K.-H., & Mai, Y. (2014-2017). WebPower: Statistical power analysis online. Retrieved from http://webpower.psychstat.org.
4. Zhang, Z., Yuan, K.-H., & Cain, M. (2016). Software for estimating univariate and multivariate skewness and kurtosis. Retrieved from http://psychstat.org/nonnormal
5. Mai, Y., Zhang, Z., & Yuan, K.-H. (2015) An Online Interface for Drawing Path Diagrams for Structural Equation Modeling. Retrieved from http://semdiag.psychstat.org
6. Zhang, Z., & Yuan, K.-H. (2015). coefficientalpha: Robust Cronbach's alpha and McDonald's omega for non-normal and missing data. http://CRAN.R-project.org/package=coefficientalpha

1. Qu, W., Liu, H., & Zhang, Z. (accepted). A Method of Generating Multivariate Non-normal Random Numbers with Desired Multivariate Skewness and Kurtosis. Behavior Research Methods.
2. Tong, X., & Zhang, Z. (accepted). Robust Bayesian approaches in growth curve modeling: Using Student's t distributions versus semiparametric methods. Structural Equation Modeling.
3. Liu, H., Jin, I. K., & Zhang, Z.(2018). Structural Equation Modeling of Social Networks: Specification, Estimation, and Application. Multivariate Behavioral Research, 53(5), 714–730.
4. Mai, Y., & Zhang, Z. (2018). Review of Software Packages for Bayesian Multilevel Modeling. Structural Equation Modeling, 25(4), 650–658. http://www.tandfonline.com/eprint/6u84fbxfzJPCGa6eUUgS/full
5. Cain, M., Zhang, Z., & Bergeman, C. S. (2018). Time and Other Considerations in Mediation Design. Educational and Psychological Measurement, 78(6), 952-972
6. Ke, Z., & Zhang, Z. (2018). Testing Autocorrelation and Partial Autocorrelation: Asymptotic Methods versus Resampling Techniques. British Journal of Mathematical and Statistical Psychology, 71(1), 96–116.
7. Mai, Y., Zhang, Z., & Wen, Z. (2018). Comparing Exploratory Structural Equation Modeling and Existing Approaches for Multiple Regression with Latent Variables. Structural Equation Modeling, 25(5), 737-749. https://www.tandfonline.com/eprint/6u84fbxfzJPCGa6eUUgS/full
8. Tong, X., & Zhang, Z. (2017). Outlying Observation Diagnostics in Growth Curve Modeling. Multivariate Behavioral Research, 52(6), 768–788. http://www.tandfonline.com/eprint/43NdXgKr7Pywnv8SKYie/full
9. Cain, M., Zhang, Z., & Yuan, K. (2017). Univariate and Multivariate Skewness and Kurtosis for Measuring Nonnormality: Prevalence, Influence and Estimation. Behavior Research Methods, 49(5), 1716–1735.
10. Liu, H., & Zhang, Z. (2017). Logistic Regression with Misclassification in Binary Outcome Variables: A Method and Software. Behaviormetrika, 44(2), 447–476.
11. Zhang, Z., Jiang, K., Liu, H., & Oh, I.-S. (2017). Bayesian meta-analysis of correlation coefficients through power prior. Communications in Statistics – Theory and Methods, 46(24), 11988-12007.
12. Yuan, K.-H., Zhang, Z., & Zhao, Y. (2017). Reliable and More Powerful Methods for Power Analysis in Structural Equation Modeling. Structural Equation Modeling, 24(3), 315-330.
13. Zhang, Z. & Yuan, K.-H. (2016). Robust Coefficients Alpha and Omega and Confidence Intervals with Outlying Observations and Missing Data: Methods and Software. Educational and Psychological Measurement, 76(3), 387-411.
14. Zhang, Z. (2016). Modeling Error Distributions of Growth Curve Models through Bayesian Methods. Behavior Research Methods, 48(2), 427-444.
15. Mai, Y., Zhang, Z., & Yuan, K. (Under revision). An Online Interface for Drawing Path Diagrams for Structural Equation Modeling

1. Zhang, Z. (2018). Moments of a Distribution. The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation. (p.1084-1085)
2. Zhang, Z., & Liu, H. (invited chapter). Sample Size and Measurement Occasion Planning for Latent Change Score Models through Monte Carlo Simulation. In E. Ferrer, S. M. Boker, and K. J. Grimm (Eds.) Advances in Longitudinal Models for Multivariate Psychology: A Festschrift for Jack McArdle.
3. Mai, Y., & Zhang, Z. (2017). Statistical Power Analysis for Comparing Means with Binary or Count Data Based on Analogous ANOVA. In L. A. van der Ark, M. Wiberg, S. A. Culpepper, J. A. Douglas, and W.-C. Wang (Eds.) Quantitative Psychology - The 81st Annual Meeting of the Psychometric Society, Asheville, North Carolina, 2016. Springer Proceedings in Mathematics & Statistics. (pp. 381-393).
4. Du, H., Zhang, Z., & Yuan, K.-H. (2017). Power analysis for t-test with non-normal data and unequal variances. In L. A. van der Ark, M. Wiberg, S. A. Culpepper, J. A. Douglas, and W.-C. Wang (Eds.) Quantitative Psychology - The 81st Annual Meeting of the Psychometric Society, Asheville, North Carolina, 2016. Springer Proceedings in Mathematics & Statistics. (pp. 373-380).
5. Zhang, Z., & Yuan, K.-H. (2017) Practical Statistical Power Analysis using R and WebPower. ISDSA Press.

1. Zhang, Z. (2016, August). Practical Statistical Power Analysis for Simple and Complex Models. Workshop conducted for the American Psychological Association Convention in Denver, Colorado, August 4-7, 2016.

1. Zhang, Z. (2018, July). A Blessing or a Curse? An Overview of Non-normal Data and Missing Data Invited talk at 2018 International Conference on Management and Operations Research. Beijing, China.
2. Zhang, Z. (2018, May). A New Software Program for Practical Statistical Power Analysis. Poster presented at the 30th Annual Convention of the American Psychological Society, San Francisco, CA.
3. Tzakis, T., Liu, H., & Zhang, Z. (2018, May). A Review of Social Network Analysis in Psychological Research. Poster presented at the 30th Annual Convention of the American Psychological Society, San Francisco, CA.
4. Tzakis, T., & Zhang, Z. (2018, March). A Review of Social Network Analysis in Psychological Research. Poster presented at the Michigan Academy 2018 Conference, Alma, MI.
5. Zhang, Z. (2017, Oct). Two-Stage Bayesian Estimation in Structural Equation Modeling. Paper presented at 2017 SMEP meeting, Minneapolis, MN.
6. Zhang, Z. (2017, August). Practical Statistical Power Analysis for Multilevel Modeling: Methods and Software. Poster presented at the 125th Annual Convention of the American Psychological Association, Washington DC.
7. Zhang, Z. (2017, June). Modeling Non-normal Distributions in Mixed-effects and Multilevel Models. Paper presented at the 2017 ICSA Applied Statistics Symposium, Chicago, IL.
8. Zhang, Z. (2017, May). Statistical Methods and Software for Handling Non-normal in Social, Behavioral and Economic Sciences. Invited talk at Henan University, Kaifeng, China.
9. Yuan, K.-H., & Zhang, Z. (May, 2017). Statistics in Social Sciences: Present and Future. Beijing, China.
10. Zhang, Z., & Liu, H. (2017, May). Sample Size Planning for Latent Change Score Models through Monte Carlo Simulation. Poster presented at the 29th Annual Convention of the American Psychological Society, Boston, MA.
11. Cain, M. K., & Zhang, Z. (2017, May). Fit for a Bayesian: An Evaluation of PPP and DIC. Poster presented at the 2017 Modern Modeling Methods Conference, in Storrs, CT.
12. Zhang, Z. (2017, Mar). Two-Stage Bayesian Estimation in Structural Equation Modeling. Invited talk at ACMS Statistics Seminar, Department of ACMS, University of Notre Dame, Notre Dame, IN.
13. Zhang, Z. (2016, October). Sample size planning for latent change score models through Monte Carlo simulation. Invited talk at the Conference on Advances in Longitudinal Models for Multivariate Psychology: A Festschrift for Jack McArdle, October 18, 2016, Richmond, VA.
14. Zhang, Z. (2016, October). Practical Statistical Power Analysis for Structural Equation Modeling: Methods and Software. Poster presented at the 87th Annual Meeting of the Indiana Academy of the Social Sciences, October 7, 2015, Westville, IN.
15. Liu, H., & Zhang, Z. (2016, May). Logistic Regression with Misclassification in Binary Outcome Variables: Method and Software. Paper presented at the Annual Meeting of the Psychometric Society, July 12-15, Asheville, NC.
16. Mai, Y. (2016, July). Statistical Power for ANOVA with Binary and Count Data. Paper presented at the Annual Meeting of the Psychometric Society, July 12-15, Asheville, NC.
17. Du, H. (2016, July). Power analysis for t-test with non-normal data and unequal variances. Paper presented at the Annual Meeting of the Psychometric Society, July 12-15, Asheville, NC.
18. Zhang, Z. (2016, July). Statistical Power Analysis for Mediation with Non-normal and Missing Data. Paper presented at the Annual Meeting of the Psychometric Society, July 12-15, Asheville, NC.
19. Yuan, K.-H. (2016, July). Power Analysis for Structural Equation Modeling. Paper presented at the Annual Meeting of the Psychometric Society, July 12-15, Asheville, NC.
20. Yang, M. (2016, July). Statistical power analysis for multilevel modeling. Paper presented at the Annual Meeting of the Psychometric Society, July 12-15, Asheville, NC.
21. Cain, M. K., & Zhang, Z. (2016, May). Time and Other Considerations in Mediation Design. Poster presented at the 2017 Modern Modeling Methods Conference, May 23-26, 2016, in Storrs, CT.
22. Liu, H., & Zhang, Z. (2016, May). Power to detect misclassification rates in logistic regression modeling. Poster presented at the 28th APS Annual Convention, May 26-29, 2016, in Chicago, IL.
23. Zhang, Z. (2016, May) Statistical Power Analysis for Complex Models. Poster presented at the 28th APS Annual Convention, May 26-29, 2016, in Chicago, IL.
24. Mai, Y., & Zhang, Z. (2016, May). Multilevel Modeling Through Path Diagramming: An Online Graphical Interface. Poster presented at the 28th APS Annual Convention, May 26-29, 2016, in Chicago, IL.
25. Mai, Y. (2016, April). A Unified R package for Fitting Multilevel Models with Different Data and Optimization Methods. Paper presented to the Department of Psychology, University of Notre Dame.
26. Zhang, Z. & Yuan, K.-H. (2015, December). WebPower: Online Statistical Power Analysis for Simple and Complex Models. Software demonstration at the IES PI meeting in Washington, DC.
27. Zhang, Z. (2015, June). Statistical Power Analysis for Mediation Effects through WebPower. Invited talk at the Renmin University of China. Beijing, China.
28. Cain, M. (2015, May). Unicorns and Normal Distributions in Psychology and Education: A Meta-Analysis, Paper presented at the 27th Annual Convention of the American Psychological Society, New York, NY.
29. Mai, Y. (2015, May). A General Framework for Graphical Representation of a Statistical Model with Non-normal Data, Paper presented at the 27th Annual Convention of the American Psychological Society, New York, NY.
30. Yuan, K.-H. (2015, May). Power Analysis for Structural Equation Modeling with Non-normal Data, Paper presented at the 27th Annual Convention of the American Psychological Society, New York, NY.
31. Zhang, Z. (2015, May). Monte Carlo Based Statistical Power Analysis for Mediation Analysis with Non-normal Data: Methods and Software, Paper presented at the 27th Annual Convention of the American Psychological Society, New York, NY.
32. Wang, X. (2015, April). Development of online software for statistical power analysis. Undergraduate Scholars Conference at the University of Notre Dame.
33. Yang, M. (2015, March). Statistical power analysis for multilevel models. Paper presented to the Department of Psychology, University of Notre Dame.
34. Liu, H. (2015, February). Statistical power analysis for logistic regression. Paper presented to the Department of Psychology, University of Notre Dame.