Machine learning (ML) methods have shown substantial promise in advancing psychological research by capturing complex, nonlinear relationships that traditional regression models often overlook. However, the comparative advantages of ML models in theory-building contexts remain underexplored, particularly in comprehensive evaluations across multiple methods and relationship patterns. This study addresses this gap by utilizing rigorous Monte Carlo simulations to evaluate the performance of Random Forests, Boosting, Regularization, Neural Networks, and regression methods. The computational model simulated predictor-criterion relationships across six distinct patterns: no effect, linear, exponential, logarithmic, quadratic, and cubic.
By embedding true underlying latent relationships within the simulations, this study uniquely enables direct comparisons of model outputs against known patterns. Results revealed that ML methods consistently outperformed regression in capturing nonlinear dynamics, particularly in exponential, quadratic, and cubic conditions. Traditional regression models failed to accurately characterize key nonlinear relationships and often produced simplified or inaccurate representations of variable patterns. These findings highlight ML's superior capacity to uncover nuanced behavioral relationships, offering significant implications for advancing psychological theories by identifying mechanisms and configurations that remain obscured under linear constraints.
This presentation will delve into the computational modeling process, present comparative analyses of model performances, and discuss the interpretability of ML insights, leveraging model-agnostic interpretability methods. This research underscores ML's transformative potential in refining traditional methodological frameworks, expanding theory-building capabilities, and fostering hypothesis generation beyond the regression paradigm. These insights establish a foundation for applying ML in both theoretical advancements and hypothesis testing within psychological science.

Authors:
Jordan Epistola, University of Maryland College Park, United States
Paul Hanges, University of Maryland College Park, United States

About the Presenter(s)
Dr. Jordan Epistola is a postdoctoral fellow with the US Army and University of Maryland, and a former Sr. Analyst at Walmart HQ. He specializes in AI/ML implementation in IO Psychology, focusing on measurement validity/fairness for theory-building.

Connect on Linkedin
https://www.linkedin.com/in/jordanepistola/

See this presentation on the full schedule – Wednesday Schedule

Bookmark