Thesis
Deep learning with human oversight for time-series data
- Creator
- Rights statement
- Awarding institution
- University of Strathclyde
- Date of award
- 2025
- Thesis identifier
- T17566
- Person Identifier (Local)
- 202170766
- Qualification Level
- Qualification Name
- Department, School or Faculty
- Abstract
- Lately we are witnessing a rapid improvement in the performance of artificial intelligence (AI) for tasks that are time consuming or challenging for humans. However, AI typically relies on large datasets and high-quality labels. Data collection itself is often relatively straightforward, while labelling poses a challenge. Additionally, once deployed, algorithm performance deteriorates if the underlying conditions (i.e. data statistics) change. Moreover, there is a strong initiative for lawful, ethical and robust AI algorithms instead ofblack-box systems. To address these challenges, human-in-the-loop approaches such as active learning, interactive learning, and machine teaching have been proposed to optimise handling of training data, i.e., minimise the amount of data that needs labelling without compromising the performance, while including human in the design cycle of algorithms. In this thesis we first design and test oracle-based active learning frameworks, including various ways of selecting data samples for algorithm training, transferability of algorithms to new environments, and using simplified labels that apply to larger parts of signals instead of per-sample labels. Optimal trade-off between algorithm performance and labelling effort is achieved with the amount of labelled data reduced by 85-95% for the non-intrusive load monitoring problem with regular, fine grained labels; by 82.6-98.5% with simplified labels covering larger signal parts, and 83% for the micro-seismic event detection problem. Next, we move towards human-in-the-loop active learning approaches, including domain experts in the labelling task during active learning. We address practical considerations of active learning, i.e., existence of an oracle providing absolutely correct labels, variable difficulty of labelling available data samples, and errors introduced during labelling if there is no oracle. We design a stopping mechanism for the active learning process, to avoid unnecessary labelling. We propose several ways to mitigate introduced errors - using expert’s confidence to suppress the effects of labels which are difficult to assign, and using a mechanism to detect potentially wrong labels and send them for re-labelling. We validate the proposed solutions for the non-intrusive load monitoring problem in experiments with three domain experts. The results show that the proposed methodology significantly improves model transferability with labelling effort reduced by 61-93%. Lastly, we design a machine teaching framework with a hybrid humanmachine teacher. The domain expert (human teacher) makes a selection of just several representative data samples to lead the algorithm training process, based on which the machine teacher creates labels and curates the training dataset for learning in stages, resembling real-world teaching. Applied for the problem of micro-seismic event classification, we demonstrate the efficiency of the approach, outperforming the random teacher (F1 score of 0.64) and active learning (F1 score of 0.71) approaches with the same labelling effort, achieving F1 score of 0.78. The work presented in this thesis aligns with several of the United Nations Sustainable Development Goals (SDGs), promoting peace and prosperity for people and the planet. Research applied to the non-intrusive load monitoring problem aligns with goals 7 - “Ensure access to affordable, reliable, sustainable and modern energy for all” and 12 - “Ensure sustainable consumption and production patterns” by providing users with a clear and easy-to-understand summary of their energy expenses. This will help them see when and how much energy is consumed as well as how its carbon footprint. With this information, users can change their habits and adopt more sustainable practices, ultimately reducing CO2 emissions from their homes. Research on micro-seismic event classification with human oversight will strengthen resilience and adaptive capacity to climate-related hazards and natural disasters such as landslides, supporting SDG 13 - “Take urgent action to combat climate change and its impacts”.
- Advisor / supervisor
- Stankovic, Lina
- Stanković, Vladimir
- Resource Type
- DOI
Relations
Items
| Thumbnail | Title | Date Uploaded | Visibility | Actions |
|---|---|---|---|---|
|
|
PDF of Thesis T17566. | 2026-01-15 | Public | Download |