Thesis

Developing a novel hyperparameter optimisation method using learning curve prediction to enhance decision making

Downloadable Content

Download PDF
Creator
Rights statement
Awarding institution
  • University of Strathclyde
Date of award
  • 2024
Thesis identifier
  • T17174
Person Identifier (Local)
  • 202090569
Qualification Level
Qualification Name
Department, School or Faculty
Abstract
  • In response to the slow adoption of artificial intelligence (AI) trends in the construction sector, this study is one of the pioneers to tackle the challenges of subjective decision making practice in construction project management using machine learning (ML) techniques. A neural network-based Decision Support System (DSS) is developed to model significant correlations among various decision factors as well as identifying critical success factors (CSFs) such that decision outcomes can be enhanced with greater accuracy and reduced subjectivity. Existing methods for HPO using learning curve prediction are limited in their ability to predict unseen learning curves on the same dataset. The current gap in predicting full learning curves without running all configurations limits the efficiency of these approaches and constrains their application. A key contribution of this study is the development of a novel hyperparameter optimisation (HPO) algorithm, namely SEquential Learning Curve Training (SELECT), grounded in learning curve prediction which can help to improve both modelling efficiency and effectiveness. Leveraging a Convolutional Gated Recurrent Neural Network (CGRNN), the SELECT method predicts learning curves for unseen hyperparameter configurations without the need to train them. Comparative validation of SELECT against existing HPO methods such as Tree Parzen’s Estimator, Bayesian Optimisation with Gaussian Process, Hyperband and Random Search were conducted, with prediction accuracies ranging between 7%-68% better than the benchmarks in the experiments. Further to this, the computational expense for the SELECT method is less than that of the benchmarks, with the closest benchmark requiring 25% more time to find optimum hyperparameters, averaging over all datasets. The consistency of allocated computational resources is also another benefit with the standard deviation between experiments being 81s for the SELECT method, while the closest benchmark had a standard deviation of 427s averaged over 5 datasets and 5-fold splits of each. This underscores its superiority in prediction accuracy and computational efficiency. The SELECT algorithm exhibits the capability to find high performing hyperparameter configurations across different well-known datasets, including synthetic and real-world scenarios, and demonstrates a high capability for identifying CSFs through feature importance analysis. The validation of the DSS, involving feedback from senior industry experts, reflects positive performance evaluations, with an average score of 3.93 out of 5 on a Likert scale over all questions with a standard deviation of 0.84. These experts, intrigued by the system's potential, express strong interest in collaborative efforts for future development. This research, adeptly navigating industry challenges, provides not only objective decision support in construction project management but also introduces a novel HPO approach that transcends the confines of the construction sector, with applicability in the greater field of AI.
Advisor / supervisor
  • Wong, Andy
Resource Type
Note
  • Previously held under moratorium from 17 December 2024 until 17 December 2025.
DOI

Relations

Items

Thumbnail Title Date Uploaded Visibility Actions
file details PDF of thesis T17174 2025-01-20 Public Download