Surrogate-Assisted Global Optimization for Expensive Engineering Systems: From Trust Regions to Bayesian Optimization
Keywords:
Surrogate modeling, Bayesian optimization, trust-region methods, multi-fidelity modeling, uncertainty quantification, physics-informed neural networks, engineering design optimizationAbstract
This review aims to synthesize methodological and conceptual advances in surrogate-assisted global optimization (SAGO) for computationally expensive engineering systems, highlighting the evolution from deterministic trust-region frameworks to probabilistic Bayesian optimization approaches. A qualitative systematic review design was employed using content analysis of peer-reviewed literature. Twenty articles published between 2010 and 2025 were selected through purposive sampling after comprehensive database searches in Scopus, Web of Science, IEEE Xplore, and ScienceDirect. Only studies addressing surrogate-assisted strategies for expensive or multi-fidelity optimization were included. Data collection relied exclusively on literature review, and theoretical saturation was achieved after analyzing 20 studies. The qualitative coding and thematic synthesis were conducted using Nvivo 14 software, following open, axial, and selective coding procedures to extract major conceptual themes related to surrogate frameworks, optimization strategies, and robustness mechanisms. Three overarching themes emerged: (1) Evolution of Surrogate Modeling Frameworks—the transition from polynomial and RBF surrogates to probabilistic Kriging, multi-fidelity, and deep learning-based surrogates such as physics-informed neural networks; (2) Global Optimization Strategies and Trust-Region Adaptation—the convergence of deterministic trust-region methods with Bayesian acquisition-based algorithms that integrate uncertainty-aware exploration and exploitation; and (3) Robustness, Generalization, and Application Integration—the expansion of surrogate-assisted methods into real-world workflows emphasizing uncertainty quantification, transfer learning, and digital twin integration. Together, these themes reveal a paradigm shift toward scalable, adaptive, and hybrid optimization systems that unify physics-based modeling with data-driven intelligence. Surrogate-assisted optimization has evolved from local curve-fitting into a data-efficient, uncertainty-aware framework fundamental to modern engineering design. The field now converges toward hybrid, physics-informed, and AI-integrated paradigms that enable robust, automated decision-making in computationally intensive environments.
Downloads
References
Botorch Developers. (2020). BoTorch: A framework for efficient Monte Carlo Bayesian optimization. arXiv preprint arXiv:2002.08418.
Conn, A. R., Gould, N. I. M., & Toint, P. L. (2000). Trust-region methods. SIAM.
Emmerich, M., & Deutz, A. (2018). A tutorial on multiobjective optimization: Fundamentals and evolutionary methods. Natural Computing, 17(3), 585–609.
Eriksson, D., Pearce, M., Gardner, J., Turner, R., & Poloczek, M. (2019). Scalable global optimization via local Bayesian optimization. Advances in Neural Information Processing Systems, 32, 5496–5507.
Feurer, M., & Hutter, F. (2019). Hyperparameter optimization. In Automated Machine Learning (pp. 3–33). Springer.
Forrester, A. I. J., & Keane, A. J. (2009). Recent advances in surrogate-based optimization. Progress in Aerospace Sciences, 45(1–3), 50–79.
Forrester, A., Sobester, A., & Keane, A. (2008). Engineering design via surrogate modelling: A practical guide. John Wiley & Sons.
Frazier, P. I. (2018). A tutorial on Bayesian optimization. arXiv preprint arXiv:1807.02811.
Ginsbourger, D., Le Riche, R., & Carraro, L. (2010). Kriging is well-suited to parallelize optimization. In Computational Intelligence in Expensive Optimization Problems (pp. 131–162). Springer.
Goh, C. J., Teo, K. L., & Rehbock, V. (2013). Surrogate-assisted evolutionary algorithms for engineering design optimization. Applied Soft Computing, 13(4), 1819–1830.
Hernández-Lobato, J. M., Hoffman, M. W., & Ghahramani, Z. (2017). Predictive entropy search for efficient global optimization of black-box functions. Advances in Neural Information Processing Systems, 27, 918–926.
Jin, Y. (2011). Surrogate-assisted evolutionary computation: Recent advances and future challenges. Swarm and Evolutionary Computation, 1(2), 61–70.
Jones, D. R., Schonlau, M., & Welch, W. J. (1998). Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13(4), 455–492.
Kandasamy, K., Dasarathy, G., Oliva, J. B., Schneider, J., & Póczos, B. (2018). Multi-fidelity Bayesian optimization with continuous approximations. International Conference on Machine Learning, 2545–2554.
Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S., & Yang, L. (2021). Physics-informed machine learning. Nature Reviews Physics, 3(6), 422–440.
Kennedy, M. C., & O’Hagan, A. (2000). Predicting the output from a complex computer code when fast approximations are available. Biometrika, 87(1), 1–13.
Knowles, J. (2006). ParEGO: A hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE Transactions on Evolutionary Computation, 10(1), 50–66.
Lam, R., Allaire, D., & Willcox, K. (2015). Multifidelity optimization using statistical surrogate modeling for nonhierarchical information sources. SIAM Journal on Scientific Computing, 37(1), A212–A239.
Mockus, J. (2012). Bayesian approach to global optimization: Theory and applications. Springer.
Perdikaris, P., & Karniadakis, G. E. (2016). Model inversion via multi-fidelity Bayesian optimization: A new paradigm for parameter estimation in physics-based models. SIAM Journal on Scientific Computing, 38(5), A3163–A3194.
Perdikaris, P., Raissi, M., Damianou, A., Lawrence, N. D., & Karniadakis, G. E. (2017). Nonlinear information fusion algorithms for data-efficient multi-fidelity modeling. Proceedings of the Royal Society A, 473(2198), 20160751.
Queipo, N. V., Haftka, R. T., Shyy, W., Goel, T., Vaidyanathan, R., & Tucker, P. K. (2005). Surrogate-based analysis and optimization. Progress in Aerospace Sciences, 41(1), 1–28.
Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378, 686–707.
Regis, R. G., & Shoemaker, C. A. (2007). A stochastic radial basis function method for the global optimization of expensive functions. INFORMS Journal on Computing, 19(4), 497–509.
Sacks, J., Welch, W. J., Mitchell, T. J., & Wynn, H. P. (1989). Design and analysis of computer experiments. Statistical Science, 4(4), 409–423.
Santner, T. J., Williams, B. J., & Notz, W. I. (2003). The design and analysis of computer experiments. Springer.
Sankararaman, S., & Mahadevan, S. (2013). Integration of model uncertainty and parameter uncertainty for reliability assessment and design. Reliability Engineering & System Safety, 112, 187–199.
Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., & De Freitas, N. (2016). Taking the human out of the loop: A review of Bayesian optimization. Proceedings of the IEEE, 104(1), 148–175.
Simpson, T. W., Peplinski, J. D., Koch, P. N., & Allen, J. K. (2001). Metamodels for computer-based engineering design: Survey and recommendations. Engineering with Computers, 17(2), 129–150.
Surjanovic, S., & Bingham, D. (2013). Virtual library of simulation experiments: Test functions and datasets. Retrieved from https://www.sfu.ca/~ssurjano
Wang, G. G., & Shan, S. (2007). Review of metamodeling techniques in support of engineering design optimization. Journal of Mechanical Design, 129(4), 370–380.
Wang, Y., Niu, S., & Zhang, Y. (2022). Quantum-inspired surrogate modeling for engineering optimization. Applied Soft Computing, 116, 108307.
Willcox, K., & Megretski, A. (2019). Predictive digital twins: Data-driven modeling for complex systems. AIAA Journal, 57(11), 4732–4751.
Zhao, H., & Xue, D. (2019). A multi-surrogate approach for global optimization of expensive engineering design problems. Structural and Multidisciplinary Optimization, 60(5), 1825–1843.
