PrimaVera Demonstrators
- Primavera Product Catalogue – Primavera has developed a product catalogue. The PrimaVera Product Catalogue synthetises and displays the outputs of the PrimaVera project. It allows both researchers and industry partners to learn what was done in PrimaVera, while enabling interested parties to contact researchers about specific products.
- Infrastructure Demonstrator – PrimaVera is developing a web app that showcases research applications in diagnostics, prognostics, and maintenance policy optimization and logistics. The work-in-progress web application can be visited via the link above.
List of scientific publications
2026 |
|
| 78. | Giacomo Barbieri; Alexandra Mulder; Panashe Mangezi: From Sinterklaas to Smart Maintenance Organizational Enablers. In: Work in progress, 2026. (Type: Journal Article | BibTeX | Tags: )@article{nokey, |
| 77. | Bas Van Oudenhoven; David Sanchez-Londono; Giacomo Barbieri; Jan Braaksma: The Knowledge Product Catalogue as a Tool to Support Dissemination in Inter-Organizational Research Projects. In: Work in progress, 2026. (Type: Journal Article | BibTeX | Tags: )@article{nokey, |
2025 |
|
| 76. | Nubia Nale Alves da Silveira; Annemieke Angelique Meghoe; Tiedo Tinga: Quantifying the suitability and feasibility of predictive maintenance approaches. In: Computers & industrial engineering, vol. 194, iss. 110342, 2025. (Type: Journal Article | Abstract | Links | BibTeX | Tags: Maintenance optimization)@article{daSilveira2025,Predictive maintenance is a promising concept for maintenance optimization which requires reliable and accurate predictions of a system's lifetime. Despite the many studies on this topic, selecting the best approach is still a topic of debate. In general, predictive maintenance approaches can be roughly classified as knowledge-based, data analytics and physics-based models. However, this classification does not provide maintainers clear guidance on how to select the most suitable approach for a specific case. This work, therefore, presents a selection method for this process. For that purpose, a list of selection criteria was established, and six predictive maintenance approaches were analysed. The proposed selection process is based on two main groups of criteria: the suitability criteria check the match with the desired ambition level of predictive maintenance, while the feasibility criteria identify whether this can be realized, given the labour, models and data available. Finally, three case studies are presented, demonstrating that the tool effectively guides to an optimal approach. |
| 75. | Bram Ton; Rick Akster: Large Scale Asset Detection Within Railway Scene Point Cloud Data From Mobile Laser Scanning. In: IEEE Access, 2025, ISSN: 2169-3536. (Type: Journal Article | Links | BibTeX | Tags: )@article{Ton2025b, |
| 74. | Bas van Oudenhoven; Evangelia Demerouti; Rob Basten; Philippe Van de Calseyde: Preparing for predictive maintenance: Employee perspectives on job demands and resources before and after its implementation. In: Applied Ergonomics, vol. 129, pp. 104561, 2025. (Type: Journal Article | Abstract | Links | BibTeX | Tags: )@article{nokey,Predictive maintenance (PdM) technologies promise to be valuable resources for maintenance employees. However, employees often view these technologies with skepticism. We conducted 35 interviews with maintenance employees at three different organizations to gauge their perspectives on PdM implementation. Using Job Demands-Resources theory and Carayon's Work System model as frameworks, we explored per work domain which demands (hindering work characteristics) and resources (facilitating work characteristics) employees experience under their current, non-PdM working procedures and what they expect to experience under PdM. The results show that employees currently view their decision-support system and organization as demands; however, they expect both to become resources under PdM. Nevertheless, employees express concerns that PdM may undermine their job autonomy and expertise, and they warn of the potential negative effects of false alarms generated by PdM technologies. Organizations can use these insights to (re)design PdM systems, thereby improving the acceptance and effectiveness of PdM technologies. |
| 73. | Zhao Kang: Robust Spare Parts Inventory Management. 2025, ISBN: 978-94-6510-653-3. (Type: PhD Thesis | Links | BibTeX | Tags: Spare part management)@phdthesis{Kang2025, |
| 72. | Luc Stefan Keizers: Hybrid Prognostics For Predictive Maintenance. Combining Physics-Based And Data-Driven Methods To Overcome Prognostic Challenges. 2025, ISBN: 978-90-365-6574-5. (Type: PhD Thesis | Abstract | Links | BibTeX | Tags: Maintenance optimization, Prognostics)@phdthesis{Keizers2025,Predictive maintenance is a growing research field, aiming to perform maintenance only when it is required. Prognostic algorithms are essential to achieve this, as predictions of upcoming failures help to increase availability of systems, utilize the full life time of equipment and improve maintenance logistics. Since sensors are getting cheaper and data storage and processing have become cheaper and more efficient during the fourth industrial revolution, there is a lot of interest in data-driven prognostic algorithms. However, high data requirements limit applicability is many practical applications. Physics-based prognostic models yield quantitative relations between system usage and degradation independent from historical data, but the development of such physics-of-failure models is complex and expensive. Because both data-driven and physics-based models have their advantages and limitations, combinations of both types of methods have the potential to get rid of the limitations and profit from the benefits. When both loads and the condition of a component can be monitored, physics-of-failure models can be updated in real-time using Bayesian filtering algorithms. This results in updated quantitative relations between loads and degradation, calibrated for a specific component. The first part of this thesis describes how such methods can be applied for components in variable operating conditions and implements it in a generic prognostic framework. These Bayesian filters preferably receive a direct measure of degradation, such as crack length or the amount of removed material. In many practical applications it is only possible to measure indirect consequences of degradation, such as increased vibration levels, elevated temperatures or acoustic emissions. Therefore, the second part of this thesis focuses on improving quantitative diagnostics to act as input for prognostic algorithms. Because of their modularity and applicability in multiple physical domains, bond graphs are proposed to simulate faults to enhance quantitative diagnostics. A combined diagnostic and prognostics framework is developed which is suitable for prognostics under varying operating conditions, when only limited historical run-to-failure data are available. One of the biggest challenges remains validation of the methods on a real-world case study, as the lack of real-world data is one of the biggest challenges from which this research partly originated. |
| 71. | Roel Bouman: Rethinking Anomaly Detection: From Theory to Practice. 2025, ISBN: 9789465150765. (Type: PhD Thesis | Abstract | Links | BibTeX | Tags: Anomaly Detection)@phdthesis{Bouman2025,The field of anomaly detection is rapidly evolving with numerous algorithms and applications. However, the fundamental principles behind anomaly detection are less frequently studied. This thesis aims to evaluate algorithm performance, assess the reliability of autoencoders, and introduce a novel application in power grid load estimation. A comprehensive comparative study on real-world data identifies that only a few algorithms are necessary for effective anomaly detection. Specifically, k-nearest neighbors and extended isolation forest perform well in detecting common anomalies. Autoencoders are increasingly used for anomaly detection, especially in computer vision due to their feature extraction capabilities. However, we analyze the underlying assumptions of autoencoder-based anomaly detection and conclude that these assumptions do not hold in practice, making autoencoders unreliable for this purpose. Applying anomaly detection to the energy sector, we utilize statistical process control and binary segmentation to identify measurement errors and power rerouting in load estimation. By filtering these anomalies, we improve the reliability of power grid load estimates in the Netherlands. Enhancing load estimation through interpretable methods helps reduce unused grid capacity, providing necessary flexibility for the ongoing energy transition. |
| 70. | Bas van Oudenhoven: Human-Centric Predictive Maintenance. 2025, ISBN: 978-90-386-6315-9. (Type: PhD Thesis | Links | BibTeX | Tags: Maintenance optimization, Predictive Maintenance)@phdthesis{vanOudenhoven2025, |
| 69. | Thom Badings: Robust Verification of Stochastic Systems. 2025, ISSN: 2950-2772. (Type: PhD Thesis | Abstract | Links | BibTeX | Tags: Markov Decision Process, Stochastic Systems, Verification)@phdthesis{Badings2025,Verifying that systems are safe and reliable is crucial in today’s world. For example, we want to prove that an autonomous drone will safely reach its target, or that a manufacturing system will not break down. Classical algorithms for verifying such properties often rely on a precise mathematical model of the system, for example, in the form of a Markov chain or a Markov decision process (MDP). Such Markov models are probabilistic transition systems and are ubiquitous in many areas, including control theory, artificial intelligence (AI), formal methods, and operations research. However, as systems become increasingly complex with more cyber-physical and AI components, uncertainty about the system’s behavior is inevitable. As a result, transition probabilities in Markov models are subject to uncertainty, rendering many existing analysis algorithms inapplicable. In this thesis, we fill this gap by developing novel verification methods for Markov models in the presence of uncertainty. To capture this uncertainty, we use parametric Markov models, where probabilities are described as functions over parameters. We study two perspectives on rendering verification methods robust against uncertainty: (1) set-bounded uncertainty, where only the set of possible parameter values is known and one can be robust in a worst-case sense, and (2) stochastic uncertainty, where the parameter values are described by a probability distribution and one can be robust in a probabilistic sense. By combining techniques from formal methods, AI, and control theory, our contributions span the following general problem settings: 1. We develop robust abstraction techniques for solving control problems for Markov models with continuous state and action spaces, and with set-bounded uncertain parameters. Based on the notion of probabilistic simulation relations, we show that such continuous control problems can be solved by only analyzing a finite-state abstraction, formalized as an MDP with sets of transition probabilities. 2. We present novel and scalable verification techniques for parametric Markov models with a prior distribution over the parameters. Our approaches are sampling-based, do not require any assumptions on the parameter distribution, and provide probably approximately correct (PAC) guarantees on the verification results. 3. We show that parametric models can be used to improve the sample efficiency of data-driven learning methods. We leverage tools from convex optimization to perform a sensitivity analysis, where we measure sensitivity in terms of partial derivatives of the polynomial function that describes a measure of interest. 4. We study continuous-time Markov chains where the initial state must be inferred from a set of (possibly uncertain) state observations. This setting is particularly relevant in runtime monitoring. We compute upper and lower bounds on reachability probabilities, conditioned on these observations. Our approach is based on a robust abstraction into an MDP with intervals of transition probabilities. In conclusion, we develop verification methods that reason over uncertainty without sacrificing guarantees. While dealing with uncertainty can be computationally expensive, providing such guarantees is crucial for designing safe and reliable systems with cyber-physical and AI components. The aim of this thesis is to contribute to a better understanding of dealing with uncertainty in stochastic verification problems. |
| 68. | Bram Ton; Niek Tempert; Danny Plass: Immersive visualisation of point cloud data of railway environments. In: 2024 10th International Conference on Virtual Reality (ICVR), IEEE, 2025, ISBN: 979-8-3503-6423-1. (Type: Proceedings Article | Links | BibTeX | Tags: )@inproceedings{Ton2025, |
| 67. | Lisandro Arturo Jimenez-Roa: Reliability and Maintenance for Engineering Systems: Fault Trees, Degradation Modelling and Maintenance Optimisation. 2025, ISBN: 978-90-365-6407-6. (Type: PhD Thesis | Abstract | Links | BibTeX | Tags: Deep Reinforcement Learning, Degradation Modelling, Fault tree analysis, Maintenance optimization, Prognostics and Health Management)@phdthesis{Jimenez-Roa2025,Modern infrastructures, machines, and manufacturing processes require effective management through sustainable policies under constrained resources, where deter- mining when and how to intervene becomes crucial. The Prognostics and Health Management (PHM) paradigm provides a systematic framework for leveraging data collection and computational models, supporting the management of virtually any engineering component or system. This dissertation delves into three key aspects of PHM: Reliability Modelling, Markov Process-based Prognostics, and Maintenance Optimisation. Data-driven techniques are crucial in these areas, enhancing the automation of model development and deployment. Part I centres on Reliability Modelling, specifically the automatic inference of Fault Tree (FT) models. Traditionally, graph-based models like FTs are manually constructed through iterative collaboration between system experts and FT modellers. However, this manual approach is prone to human error and may result in incomplete models. With the increasing availability of data, methodologies that attempt to automate this process, discover patterns and reduce dependency on manual intervention have gained significant relevance. Thus, in Part I of this dissertation, we focus on how to obtain efficient and compact Fault Tree models from failure datasets in a robust and scalable manner. For this matter, we propose, for the first time, using Multi-Objective Evolutionary Algorithms (MOEAs) to automatically infer FT models and cast the optimisation as a multi-objective task. This resulted in the FT-MOEA algorithm (Chapter 2), focusing on three optimisation metrics, including FT size and accuracy-related metrics. FT-MOEA consistently produced compact FT structures, but faced scalability issues. To address this, we developed the SymLearn toolchain (Chapter 3), which uses a ‘divide-and-conquer’ approach by identifying modules and symmetries in the failure dataset, breaking the inference problem into smaller tasks. Additionally, to improve robustness and scalability, the FT-MOEA-CM extension (Chapter 4) includes additional metrics from the confusion matrix. Our approaches in Part I of this dissertation contribute to automating FT model construction, revealing compact structures. These consistent structures can help uncover relationships between basic and intermediate events, providing valuable insights for asset managers to improve reliability modelling. Part II focuses on Markov Process-based Prognostics, specifically the stochastic deterioration modelling of sewer mains. Sewer systems are critical to social welfare but pose significant challenges due to their extensive scale, slow degradation, and limited capacity to monitor the entire network. Accurate modelling of the deterioration profile is crucial for optimising inspections and maintenance, thereby enhancing the reliability and availability of the networks. Various deterioration models are discussed in the literature, ranging from physics-based to data-driven approaches, each with distinct advantages and limitations. In Part II of this dissertation, we address how and to what extent it is possible to accurately model Multi-State Deterioration with applications in sewer mains. For this, we focus on Markov chains, widely used to model stochastic sequences through states and transitions. Since the 1990s, they have been applied to represent damage severity levels in sewer mains using inspection data from Closed Circuit Television cameras. Nonetheless, further evaluation of their assumptions and properties is required. We present a case study of a Dutch sewer network (Chapter 5), starting with Discrete-Time Markov Chains for deterioration modelling and examining two Markov chain structures. Given challenges such as interval-censored data, advanced analysis was necessary. In Chapter 6, we use the Turnbull estimator for non-parametric analysis to establish a ground truth. Although both homogeneous and inhomogeneous-time Markov chains are employed for sewer mains deterioration, no prior studies have compared their performance on the same dataset. Chapter 6 addresses this by demonstrating that inhomogeneous-time Markov chains are more versatile at capturing non-linear stochastic behaviour, while also highlighting issues like overfitting that reduce predictive accuracy. Part II provides a real-world case study, emphasising the need to critically evaluate modelling assumptions to enhance deterioration modelling of sewer mains using Markov chains. Finally, Part III focuses on Maintenance Optimisation of sewer mains, where obtaining optimal maintenance policies for such large-scale systems is a complex task. This complexity arises, among others, from the system’s scale and simplifications in the deterioration model. Among the various techniques available, Reinforcement Learning (RL) approaches remain largely unexplored for devising strategic main- tenance actions in sewer mains. Thus, in Part III of this dissertation, we focus on how to devise optimal maintenance strategies for components with Multi-State Deterioration such as sewer mains using Deep Reinforcement Learning.In Chapter 7, we frame the sequential decision-making problem using Deep Reinforcement Learning (DRL) for component-level maintenance of sewer mains. This framework considers damage severity levels, testing different deterioration model assumptions and evaluating their impact on maintenance policy. Our results show that agent-based policies outperformed heuristics by learning optimal sequences of maintenance actions. Part III provides evidence that DRL-based techniques o!er a flexible framework with the potential to improve heuristics and support maintenance decision-making for sewer mains. However, training these models to achieve the desired behaviour remains a challenging task. |
| 66. | Natália Ribeiro Marinho; Richard Loendersloot; Jan Willem Wiegman; Frank Grooteman; Tiedo Tinga: Evaluating sensor performance for impact identification in composites: a comprehensive comparison of FBGs with PZTs. In: Structural health monitoring, pp. 22, 2025. (Type: Journal Article | Abstract | Links | BibTeX | Tags: Prognostics and Health Management)@article{Marinho2025,Aerospace composite components require effective monitoring techniques to detect possible internal damage from impact events. To ensure reliable impact identification, sensor measurements can provide valuable information about impact energy and identify potential issues that may require further investigation. However, selecting the most appropriate sensor technology to measure impact force and energy is a challenge. In this article, a systematic and structured approach is presented to compare the expected performance of sensors and their metrological parameters in terms of their ability for impact identification in aerospace composites. The proposed methodology is demonstrated using an application example where fibre Bragg grating (FBG) are compared with piezoelectric (PZT) sensors through comprehensive tests. These tests include the correlation test, the sensitivity test, and the factor test. The correlation test showed a high agreement between FBG and PZT sensors in the time and frequency domain. The sensitivity test indicated a significant correlation between the signal features and the impact energy levels in the energy profiling diagrams, revealing nonlinearities and energy losses indicative of damage. Furthermore, these results emphasise the superior resolution of the FBG sensors and the comparable repeatability of the two sensor types. Finally, the factor test showed that FBG sensors are sensitive to different angles of incidence, while PZT sensors have a more stable directivity. Further analysis also showed that the signal strength of both sensor types decreases with increasing distance from the impact source. Overall, the proposed approach enables a thorough evaluation of the capabilities and limitations of both sensor types. Consequently, it provides information to make an informed decision on the most suitable sensor for impact monitoring systems. |
| 65. | Luc Stefan Keizers; Richard Loendersloot; Tiedo Tinga: Bayesian filtering based prognostic framework incorporating varying loads. In: Mechanical systems and signal processing Volume, vol. 224, no. 111992, pp. 25, 2025. (Type: Journal Article | Abstract | Links | BibTeX | Tags: Bayesian filters, Prognostics)@article{Keizers2025b,Unexpected system failures are costly and preventing them is crucial to guarantee availability and reliability of complex assets. Prognostics help to increase the availability and reliability. However, existing methods have their limitations: physics-based methods have limited adaptivity to specific applications, while data-driven methods heavily rely on (scarcely available) historical data, which reduces their prognostic performance. Especially when operational conditions change over time, existing methods do not always perform well. As a solution, this paper proposes a new framework in which loads are explicitly incorporated in a prognostic method based on Bayesian filtering. This is accomplished by zooming in on the failure mechanism on the material level, thus establishing a quantitative relation between usage and degradation rates. This relation is updated using a Bayesian filter and measured loads, but also allows accurate degradation predictions by considering future (changing) loads. This enables decision support on either operational use or maintenance activities. The performance of the proposed load-controlled prognostic method is demonstrated in an atmospheric corrosion use case, based on a public real data set constructed from annual corrosion measurements on carbon steel specimens. The developed load-controlled particle filter (LCPF) is demonstrated to outperform a method based on a regular particle filter, a regression model and an ARIMA model for this specific scenario with changing operating conditions. The generalization of the framework is demonstrated by two additional conceptual case studies on crack propagation and seal wear. |
2024 |
|
| 64. | Sabari Nathan Anbalagan: SPECTRUM: Towards Self-aware Industrial IoT Systems. 2024, ISBN: 978-90-365-6335-2. (Type: PhD Thesis | Abstract | Links | BibTeX | Tags: Artificial Intelligence, Fourth Industrial Revolution, Internet of Things, Wireless Communication)@phdthesis{Anbalagan2024,The Fourth Industrial Revolution (4IR) has sparked remarkable technological advancements, particularly in integrating the Internet of Things (IoT) into manufacturing industries, known as the Industrial Internet of Things (IIoT). This integration holds potential for revolutionizing industrial processes and boosting human progress through advanced applications within factory settings. However, the diverse Quality of Service (QoS) requirements of 4IR applications, including data rate, latency, reliability, power efficiency, scalability, and mobility, present significant challenges. The IIoT network must support these diverse requirements while operating in harsh and dynamic industrial indoor environments characterized by metallic infrastructure, thick concrete pillars, and complex pipe networks. Additionally, environmental fluctuations like temperature and humidity, process dynamics such as spinning and smelting, and the mobility of operators and indoor vehicles contribute to the dynamic nature of these environments. Achieving the diverse QoS requirements in such challenging conditions necessitates intelligent and versatile solutions. The latest telecommunications standard, 5G, holds promise as a solution to these challenges. This thesis addresses these challenges by focusing on the optimization of industrial use cases for 5G-based IIoT systems. It begins with an exploration of the significance and applications of 4IR in manufacturing industries, identifying the potential of 5G technology augmented by varying levels of Artificial Intelligence (AI) to meet the diverse demands of IIoT networks. The problem statement emphasizes the necessity for intelligent methodologies that adapt to the dynamic nature of industrial environments while supporting a wide range of industrial applications. The thesis proposes a multifaceted approach to tackle these challenges, incorporating varying levels of context awareness within IIoT systems. Chapters delve into the evolution of IIoT systems, presenting a framework for assessing system maturity and providing practical guidance for implementation. The development of heuristics-based, coarse-grained, and fine-grained Channel Quality Prediction (CQP) techniques is explored, offering insights into optimizing wireless communication parameters for diverse 5G use cases. Simulations and analyses demonstrate the efficacy of these methodologies in enhancing throughput, resource utilization, and power efficiency across various industrial scenarios. In conclusion, this thesis contributes to the advancement of wireless communication optimization in industrial settings, offering practical solutions and insights for realizing the potential of 4IR initiatives. By addressing a spectrum of challenges, requirements and use case scenarios, the research offers insights for future 5G and 6G releases. |
| 63. | Ragnar Hans Eggertsson: Advances in Asset Management: Maintenance Optimization under Incomplete Information and Sustainable Technology Selection. 2024, ISBN: 978-90-386-6159-9. (Type: PhD Thesis | Links | BibTeX | Tags: Inspection optimization, Maintenance optimization, Technology Selection)@phdthesis{nokey, |
| 62. | Bram Ton: Point taken: Translating the physical rail domain to cyber space using point clouds from mobile laser scanning. 2024, ISBN: 978-90-365-6199-0. (Type: PhD Thesis | Abstract | Links | BibTeX | Tags: LiDAR, Point Cloud, Railway)@phdthesis{nokey,There is an active interest to digitise the rail infrastructure, because it is expected that the coming years rail transport will be an important key factor to combat climate change. Compared to other means of transport, such as air or road transportation, rail transport is viewed as being more environment-friendly. With the envisioned increase of passenger and freight rail traffic in the near future, there will be an increased burden on the railway infrastructure. To ensure continued reliability, availability, maintainability and safety of the railway network in an efficient way, it is vital to have an accurate and up-to-date digital representation of the railway environment. Often such a digital representation is also referred to as an `as-is' model or representation because it reflects the current state. These as-is representations can be used for various tasks, such as planning work, monitoring the infrastructural health, automated inventory assessment, robotised construction, and eventually predictive maintenance. Automated creation of these as-is models is challenging because railway environments pose a complex scene structure and span several thousands of kilometres. Scenes contain discrete objects, such as signals, catenary arches, and relay cabinets on the one hand. On the other hand there are continuous objects, such as overhead catenary wires and rail tracks. Adding to the complexity, is the fact that the number of distinct objects around the track is large. Especially for countries with a dense rail network and a long history of rail transportation such as the Netherlands. These factors, combined with the large intra-class variance, make the digitalisation of the rail environment a challenging task. To accomplish the task of digitising the rail environment, point cloud data is considered a viable data format. Point clouds are captured using Light Detection and Ranging (LiDAR) technology. This is done through laser scanning, which operates by emitting a usually non-visible laser light pulse and measuring its time-of-flight. This time is directly proportional to the distance of the object. This distance, combined with the known angles of direction of the laser are used to calculate the 3D position of the reflection occurrence relative to the sensor. Point clouds are able to capture the geometric surface structure of a scene with high accuracy, under varying illumination conditions, and produces an immediate representation. This thesis focuses on digitising the rail environment using point cloud data. It takes a top-down approach, first objects are detected within the point cloud data with a low level of detail. Thereafter this thesis focuses on dissecting these large objects into smaller semantic meaningful pieces, providing a higher level of detail. These two steps conclude the translation from the physical world to cyber space. To complete the cycle, going from cyber space to the physical world again, this thesis also focuses on how information from cyber space can be visualised in an intuitive, immersive way using a head mounted display. The thesis starts by looking at how large objects can be detected at large scale from point cloud data. To do so, existing models which have been successful in the domain of autonomous driving vehicles are evaluated for this task. A key insight from this evaluation is that the absolute positional accuracy required for `as-is' models cannot be obtained from these existing models. Another insight is that models do not generalise well to new scenarios, and that to combat this issue, transfer learning is very suitable to train new models using fewer labels. To further scrutinise these large objects and thereby increase the level of detail of the digitisation process, semantic segmentation can be used. Semantic segmentation assigns a class to each point within the point cloud. This decomposition can then for instance be used to retrieve CAD models from a CAD library to create an accurate digital representation. This thesis looks at the semantic segmentation of catenary arches which are captured at high resolution. Catenary arches are decomposed into fourteen distinct classes and labelled accordingly. An overall mIoU of 71% was accomplished for this task. Object detection and semantic segmentation conclude the process of translating the physical world to cyberspace. Once a representation is available in cyberspace, a wide range of applications become available. For instance, planning maintenance, robotised construction, predictive maintenance, and computerised clearance inspections. To complete the cycle, and go from cyberspace to the physical world again, this thesis explores immersive visualisation of point cloud data. A proof-of-concept for this immersive visualisation is presented in this thesis. It is based on a head-mounted display and enables the user to interact with the point cloud using hand gestures and voice commands. To conclude, this thesis has explored the digitisation of the rail environment using point cloud data. It provides valuable insights and pointers to move this relatively new research area to the next level. One promising research direction to highlight is the use of self-supervised learning (SSL). By using SSL, models can learn some `common sense' about the data without external labels. This alleviates the expensive, and tedious task of labelling large amounts of data. |
| 61. | Marek Skarupski; Quinhao Wu: Confidence bounds for compound Poisson process. Stat Papers, 2024. (Type: Proceedings | Links | BibTeX | Tags: Reliability engineering)@proceedings{Skarupski2024, |
| 60. | Luc Stefan Keizers; Richard Loendersloot; Tiedo Tinga: Towards a Hybrid Framework For Prognostics With Limited Run-to-Failure Data. Proceedings of the 8th European Conference of the Prognostics and Health Management Society 2024, vol. 8, PHM Society, 2024, ISBN: 78-1-936263-40-0. (Type: Conference | Abstract | Links | BibTeX | Tags: Prognostics, Prognostics and Health Management)@conference{Keizers2024,The introduction of cyber-physical systems with increased availability of sensor data creates a lot of research interest in prognostic algorithms for predictive maintenance. Although a lot of algorithms are successfully applied to benchmark case studies based on simulated data and experimental set-ups, deployment in industry lags behind. From a comparison between three benchmark case studies with two real-world case studies based on prognostic metrics (monotonicity, prognosability and trendability), two main issues are observed: 1) the lack of run-to-failures and 2) low prognostic metrics due to a low signal-to-noise ratio of degradation trends, as a result of unexplained physical phenomena. To make prognostics feasible, a hybrid framework is proposed that focuses on improving system knowledge. The framework consists of a quantitative diagnostic assessments, guided by (modular) system models in which damage is induced. This quantitative damage assessment provides input for prognostics based on Bayesian filtering, enabling prognostics for assets in varying operational conditions. Implementation and validation of the framework requires investments, but modularity within the framework can accelerate development for new systems. |
| 59. | Henk Akkermans; Rob Basten; Quan Zhu; Luk Van Wassenhove: Transition paths for condition-based maintenance-driven smart services. In: Journal of Operations Management, vol. n/a, no. n/a, 2024. (Type: Journal Article | Abstract | Links | BibTeX | Tags: Maintenance optimization, smart services)@article{https://doi.org/10.1002/joom.1295,Abstract This research investigates growth inhibitors for smart services driven by condition-based maintenance (CBM). Despite the fast rise of Industry 4.0 technologies, such as smart sensoring, internet of things, and machine learning (ML), smart services have failed to keep pace. Combined, these technologies enable CBM to achieve the lean goal of high reliability and low waste for industrial equipment. Equipment located at customers throughout the world can be monitored and maintained by manufacturers and service providers, but so far industry uptake has been slow. The contributions of this study are twofold. First, it uncovers industry settings that impede the use of equipment failure data needed to train ML algorithms to predict failures and use these predictions to trigger maintenance. These empirical settings, drawn from four global machine equipment manufacturers, include either under- or over-maintenance (i.e., either too much or too little periodic maintenance). Second, formal analysis of a system dynamics model based on these empirical settings reveals a sweet spot of industry settings in which such inhibitors are absent. Companies that fall outside this sweet spot need to follow specific transition paths to reach it. This research discusses these paths, from both a research and practice perspective. |
| 58. | R. Stribos; R. Bouman; L. A. Jimenez-Roa; M. Slot; Mariëlle Stoelinga: A Comparison of Anomaly Detection Algorithms with Applications on Recoater Streaking in an Additive Manufacturing Process. In: Rapid Prototyping Journal, 2024, (Submitted). (Type: Journal Article | Abstract | Links | BibTeX | Tags: )@article{stribo2024comparison,Purpose Powder bed additive manufacturing has recently seen substantial growth, yet consistently producing high-quality parts remains challenging. Recoating streaking is a common anomaly that impairs print quality. Several data-driven models for automatically detecting this anomaly have been proposed, each with varying effectiveness. However, comprehensive comparisons among them are lacking. Additionally, these models are often tailored to specific data sets. This research addresses this gap by implementing and comparing these anomaly detection models for recoating streaking in a reproducible way. This study aims to offer a clearer, more objective evaluation of their performance, strengths and weaknesses. Furthermore, this study proposes an improvement to the Line Profiles detection model to broaden its applicability, and a novel preprocessing step was introduced to enhance the models’ performances. Design/methodology/approach All found anomaly detection models have been implemented along with several preprocessing steps. Additionally, a new universal benchmarking data set has been constructed. Finally, all implemented models have been evaluated on this benchmarking data set and the effect of the different preprocessing steps was studied. Findings This comparison shows that the improved Line Profiles model established it as the most efficient detection approach in this study’s benchmark data set. Furthermore, while most state-of-the-art neural networks perform very well off the shelf, this comparison shows that specialised detection models outperform all others with the correct preprocessing. Originality/value This comparison gives new insights into different recoater streaking (RCS) detection models, showcasing each one with its strengths and weaknesses. Furthermore, the improved Line Profiles model delivers compelling performance in detecting RCS. |
| 57. | Thom S. Badings; Matthias Volk; Sebastian Junges; Mariëlle Stoelinga; Nils Jansen: CTMCs with Imprecisely Timed Observations. In: TACAS 2024, 2024. (Type: Proceedings Article | Links | BibTeX | Tags: Model checking, Robustness)@inproceedings{Badings2024TACAS, |
| 56. | Sabari Nathan Anbalagan; Melissa Schwarz; Rob Bemthuis; Paul Havinga: Assessing Factory's Industry 4.0 Readiness: A Practical Method for IIoT Sensor and Network Analysis. In: Procedia Computer Science, vol. 232, pp. 2730–2739, 2024. (Type: Journal Article | Abstract | Links | BibTeX | Tags: Internet of Things, Sensors)@article{ANBALAGAN20242730,Manufacturing industries are aware of the benefits of Industry 4.0 (I4.0) and the pivotal role of the Industrial Internet of Things (IIoT) to facilitate the transition. However, they often underestimate the intricate connection between the implementation of IIoT and their existing sensing and communication systems. Hindering their progress towards I4.0 implementation, many factories still rely on rigid sensing systems, characterized by limited adaptability and inflexibility. This raises questions regarding the industrial readiness for I4.0 implementation and its ability to meet the sensing and connectivity standards of IIoT. Specifically, it is unclear whether the current brownfield installations, consisting of sensors and networking systems, adequately support the demanding I4.0 applications that necessitate a multi-functional and collaborative IIoT system. Our objective is to assess a factory's progress towards I4.0 by examining the readiness of its sensing system and the communication capabilities of its network. We propose a two-stage analysis of factory readiness, including a functional segment-based sensor analysis and an application class-based network analysis. We present a case study conducted at an iron-making plant in The Netherlands to illustrate our method. Key findings of the case study include: (1) a lack of multi-functionality across segments for the majority of sensors (90%), (2) a considerable portion of network traffic (73%) requires high reliability, and (3) only 3% of the current network traffic necessitate ultra-reliable, low latency communication. Furthermore, we discuss how our method provides decision-makers with valuable guidance for the digital transformation of established and newly built manufacturing industries. |
| 55. | Eyuel Debebe Ayele; Stylianos Gavriel; Javier Ferreira Gonzalez; Wouter B. Teeuw; Panayiotis Philimis; Ghayoor Gillani: Emerging Industrial Internet of Things Open-Source Platforms and Applications in Diverse Sectors. In: Telecom, vol. 5, no. 2, pp. 369–399, 2024, ISSN: 2673-4001. (Type: Journal Article | Links | BibTeX | Tags: Internet of Things)@article{telecom5020019, |
| 54. | Thom S. Badings; Licio Romao; Alessandro Abate; Nils Jansen: A Stability-Based Abstraction Framework for Reach-Avoid Control of Stochastic Dynamical Systems with Unknown Noise Distributions. In: CoRR, vol. abs/2404.01726, 2024. (Type: Journal Article | Abstract | Links | BibTeX | Tags: Decision-making under uncertainty, Model checking)@article{DBLP:journals/corr/abs-2404-01726,Finite-state abstractions are widely studied for the automated synthesis of correct-by-construction controllers for stochastic dynamical systems. However, existing abstraction methods often lead to prohibitively large finite-state models. To address this issue, we propose a novel abstraction scheme for stochastic linear systems that exploits the system's stability to obtain significantly smaller abstract models. As a unique feature, we first stabilize the open-loop dynamics using a linear feedback gain. We then use a model-based approach to abstract a known part of the stabilized dynamics while using a data-driven method to account for the stochastic uncertainty. We formalize abstractions as Markov decision processes (MDPs) with intervals of transition probabilities. By stabilizing the dynamics, we can further constrain the control input modeled in the abstraction, which leads to smaller abstract models while retaining the correctness of controllers. Moreover, when the stabilizing feedback controller is aligned with the property of interest, then a good trade-off is achieved between the reduction in the abstraction size and the performance loss. The experiments show that our approach can reduce the size of the graph of abstractions by up to 90% with negligible performance loss. |
| 53. | L. A. Jimenez-Roa; T. D. Simão; Z. Bukhsh; T. Tinga; H. Molegraaf; N. Jansen; M. Stoelinga: Maintenance Strategies for Sewer Pipes with Multi-State Deterioration and Deep Reinforcement Learning. In: 8th European Conference of the Prognostics and Health Management Society 2024, PHME24, 2024. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: )@inproceedings{jimenez2024maintenance,Large-scale infrastructure systems are crucial for societal welfare, and their effective management requires strategic forecasting and intervention methods that account for various complexities. Our study addresses two challenges within the Prognostics and Health Management (PHM) framework applied to sewer assets: modeling pipe degradation across severity levels and developing effective maintenance policies. We employ Multi-State Degradation Models (MSDM) to represent the stochastic degradation process in sewer pipes and use Deep Reinforcement Learning (DRL) to devise maintenance strategies. A case study of a Dutch sewer network exemplifies our methodology. Our findings demonstrate the model's effectiveness in generating intelligent, cost-saving maintenance strategies that surpass heuristics. It adapts its management strategy based on the pipe's age, opting for a passive approach for newer pipes and transitioning to active strategies for older ones to prevent failures and reduce costs. This research highlights DRL's potential in optimizing maintenance policies. Future research will aim improve the model by incorporating partial observability, exploring various reinforcement learning algorithms, and extending this methodology to comprehensive infrastructure management. |
| 52. | L. A. Jimenez-Roa; N. Rusnac; M. Volk; M. Stoelinga: Fault Tree inference using Multi-Objective Evolutionary Algorithms and Confusion Matrix-based metrics. In: Formal Methods for Industrial Critical Systems (FMICS), Springer, 2024. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: )@inproceedings{jimenez2024fault,In the domain of reliability engineering and risk assessment, the development of fault tree (FT) models is pivotal for decision-making in complex systems. Traditional FT model development, relying on manual efforts and expert collaboration, is both time-consuming and error-prone. The era of Industry 4.0 introduces capabilities for automatically deriving FTs from inspection and monitoring data. This paper presents FT-MOEA-CM, an extension of the FT-MOEA algorithm for inferring FT models from failure data using multi-objective optimization. FT-MOEA-CM enhances its predecessor by integrating confusion matrix-derived metrics and incorporating parallelization and caching mechanisms. Our evaluation on six FTs from diverse application areas showcases that FT-MOEA-CM exhibits (1) enhanced robustness, (2) faster convergence and (3) better scalability than FT-MOEA, suggesting its potential in efficiently inferring larger FT models. |
| 51. | L. A. Jimenez-Roa; T. Heskes; T. Tinga; M. Stoelinga: Comparing Homogeneous and Inhomogeneous Time Markov Chains for Modeling Deterioration in Sewer Pipe Networks. In: 34th European Safety and Reliability Conference, ESREL 2024: Advances in Reliability, Safety and Security, 2024. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: )@inproceedings{jimenez2024comparing,Sewer pipe systems are essential for social and economic welfare. Managing these systems requires robust predictive models for degradation behaviour. This study focuses on probability-based approaches, particularly Markov chains, for their ability to associate random variables with degradation. Literature predominantly uses homogeneous and inhomogeneous Markov chains for this purpose. However, their effectiveness in sewer pipe degradation modelling is still debatable. Some studies support homogeneous Markov chains, while others challenge their utility. We examine this issue using a large-scale sewer network in the Netherlands, incorporating historical inspection data. We model degradation with homogeneous discrete and continuous time Markov chains, and inhomogeneous-time Markov chains using Gompertz, Weibull, Log-Logistic and Log-Normal density functions. Our analysis suggests that, despite their higher computational requirements, inhomogeneous-time Markov chains are more appropriate for modelling the nonlinear stochastic characteristics related to sewer pipe degradation, particularly the Gompertz distribution. However, they pose a risk of over-fitting, necessitating significant improvements in parameter inference processes to effectively address this issue. |
| 50. | Sabari Nathan Anbalagan; Alessandro Chiumento; Paul Havinga: Fine Grained vs Coarse Grained Channel Quality Prediction: A 5G-RedCap Perspective for Industrial IoT Networks. In: 20th IEEE International Conference on Factory Communication Systems (WFCS 2024), 2024. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: Internet of Things, Sensors)@inproceedings{nokey,The article evaluates the effectiveness of coarsegrained channel quality prediction (CQP) for 5G-RedCap/5G NR-Light devices within industrial IoT (IIoT) networks. Finegrained predictions refine real-time communication, enhancing throughput and reducing resource utilization (RU), albeit with increased computation complexity. In contrast, coarse-grained CQP offers low computational overhead while optimizing longterm network characteristics, such as redundancy planning. Our study investigates the potential applications of coarsegrained CQP in real-time communication within the IIoT context, aiming to enhance the efficiency of simple devices (5G-RedCap) without adding the computational overhead. The varying traffic profiles and quality of service levels across diverse 5G use cases, including massive Machine Type Communication (mMTC), enhanced Mobile Broadband (eMBB), and Ultra-Reliable and Low Latency Communication (URLLC), present different challenges. For mMTC devices, coarse-grained CQP demonstrates comparable RU gains to fine-grained CQP (with up to a 50% reduction in RU), showcasing its effectiveness without added complexity. However, in the eMBB scenario, where throughput is paramount, it yields only marginal improvements. Similarly, RU gains for URLLC devices are negligible due to their stricter QoS requirements. The effectiveness of coarse-grained CQP is intricately linked to the variability in experienced channel quality across different scenarios within an indoor IIoT network. This research underscores the potential of AI applications for enhancing the performance of simple 5G-RedCap/5G NR-Light devices without compromising device complexity. |
2023 |
|
| 49. | Zhao Kang; Ahmadreza Marandi; Rob J. I. Basten; Ton De Kok: Robust Spare Parts Inventory Management. In: Management Science (submitted), vol. 15, 2023. (Type: Journal Article | Links | BibTeX | Tags: Maintenance optimization, Robustness)@article{Kang2023RobustManagement, |
| 48. | Nubia Nale Alves da Silveira; Annemieke A. Meghoe; Tiedo Tinga: Integration of multiple failure mechanisms in a life assessment method for centrifugal pump impellers. In: Advances in mechanical engineering, vol. 15, no. 6, 2023, ISSN: 1687-8132. (Type: Journal Article | Links | BibTeX | Tags: Failure mechanisms, Prognostics)@article{Silveria2023AME, |
| 47. | Natália Ribeiro Marinho; Dario Di Maio; Richard Loendersloot; Tiedo Tinga: Exploiting high frequency waves for SHM using 3D SLDV system: what can we learn more?. Measuring by Light 2023: International meeting on optical measurement techniques and industrial applications, 2023. (Type: Conference | Abstract | Links | BibTeX | Tags: Structural health monitoring)@conference{Marinho2023,This research work will present a study on the use of a 3D Scanning Laser Doppler Vibrometer (3D-SLDV) to visualise the wave field in threedimensional composite structures. Knowledge of the wave field in structures, induced either actively by for example piezo-electric transducers or by an impact, supports the design of integrated Structural Health Monitoring (SHM) sensor systems on aerospace components to detect damage precursors. Reconstruction of an impact location and the possibly associated damage, may be relatively straightforward in simple geometries of homogeneous materials, whereby sensor and signal analysis are not biased by any geometrical feature other than the location of the impact and damage in the structure. The more complex the geometry, the more complex the wave field. Any additional feature to a simple geometry can bias the measurement interpretation. Understanding of the wave field is often supported by numerical models, which are also limited by these geometrical complexities. Experimental validation of the calculated wave fields is therefore of great importance. Oneof the goals of this research is to exploit the 3D-SLDV to overcome the identified challenges. It is believed that better measurement strategies for SHM can be achieved by investigating how vibrations and acoustic waves generate and manifest in complex geometrical structures. SLDV based experimental validation is an important tool assisting the development of SHM systems in aerospace applications |
| 46. | Bas van Oudenhoven; Philippe Van de Calseyde; Rob Basten; Evangelia Demerouti: Predictive maintenance for industry 5.0: Behavioural inquiries from a work system perspective. In: International Journal of Production Research, vol. 61, iss. 22, pp. 7846-7865, 2023. (Type: Journal Article | Abstract | Links | BibTeX | Tags: )@article{nokey,Predictive Maintenance (PdM) solutions assist decision-makers by predicting equipment health and scheduling maintenance actions, but their implementation in industry remains problematic. Specifically, prior research repeatedly indicates that decision-makers often refuse to adopt the data-driven, system-generated advice in their working procedures. In this paper, we address these acceptance issues by studying how PdM implementation changes the nature of decision-makers’ work and how these changes affect their acceptance of PdM systems. We build on the human-centric Smith-Carayon Work System model to synthesise literature from research areas where system acceptance has been explored in more detail. Consequently, we expand the maintenance literature by investigating the human-, task-, and organisational characteristics of PdM implementation. Following the literature review, we distil ten propositions regarding decision-making behaviour in PdM settings. Next, we verify each proposition’s relevance through in-depth interviews with experts from both academia and industry. Based on the propositions and interviews, we identify four factors that facilitate PdM adoption: trust between decision-maker and model (maker), control in the decision-making process, availability of sufficient cognitive resources, and proper organisational allocation of decision-making. Our results contribute to a fundamental understanding of acceptance behaviour in a PdM context and provide recommendations to increase the effectiveness of PdM implementations. |
| 45. | Reza Soltani; Matthias Volk; Leonardo Diamonte; Milan Lopuhaä-Zwakenberg; Mariëlle Stoelinga: Optimal Spare Management via Statistical Model Checking: A Case Study in Research Reactors. In: FMICS, pp. 205–223, Springer, 2023. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: Maintenance optimization, Spare part management)@inproceedings{DBLP:conf/fmics/SoltaniVDLS23,Systematic spare management is important to optimize the twin goals of high reliability and low costs. However, existing approaches to spare management do not incorporate a detailed analysis of the effect on the absence of spares on the system’s reliability. In this work, we combine fault tree analysis with statistical model checking to model spare part management as a stochastic priced timed game automaton (SPTGA). We use Uppaal Stratego to find the number of spares that minimizes the total costs due to downtime and spare purchasing; the resulting SPTGA model can then additionally be analyzed according to other metrics like expected availability. We apply these techniques to the emergency shutdown system of a research nuclear reactor. Our methods find the optimal spare management for a subsystem in a matter of minutes, minimizing cost while ensuring an expected availability of 99.96%. |
| 44. | Sabari Nathan Anbalagan; Paul JM Havinga; Alessandro Chiumento: VarOLLA: Maximizing Throughput for 5G-RedCap Devices in IIoT Networks. In: IEEE 9th World Forum on Internet of Things, IoT 2023, 2023. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: Internet of Things)@inproceedings{anbalagan2023varolla,In this paper, we propose VarOLLA, a novel approach aimed at maximizing the throughput of 5G-RedCap devices in Industrial Internet of Things (IIoT) environments. VarOLLA addresses the sub-optimal spectral efficiency of Outer Loop Link Adaptation (OLLA) by introducing a ‘Throughput Factor’ that incentivizes adaptive decision-making based on throughput considerations. Through extensive simulations, we demonstrate significant improvements in throughput, with gains of up to 35% compared to traditional OLLA techniques, particularly in scenarios with high channel outdatedness. Moreover, VarOLLA effectively reduces consecutive transmission failures (rBLER) and achieves substantial reductions in control message overhead, up to 87.5%. Our findings highlight the strong potential of VarOLLA in IIoT networks and its significant contribution to the realization of high-performance applications during the Fourth Industrial Revolution (4IR) in the manufacturing sector. |
| 43. | Bram Dekker; Bram Ton; Joanneke Meijer; Nacir Bouali; Jeroen Linssen; Faizan Ahmed: Point Cloud Analysis of Railway Infrastructure: A Systematic Literature Review. In: IEEE Access, vol. 11, pp. 134355-134373, 2023. (Type: Journal Article | Links | BibTeX | Tags: Sensors)@article{10328827, |
| 42. | Rob Basten Ragnar Eggertsson; Geert-Jan Houtum: Maintenance optimization for capital goods when information is incomplete and environment-dependent. In: IISE Transactions, vol. 0, no. 0, pp. 1-16, 2023. (Type: Journal Article | Links | BibTeX | Tags: Maintenance optimization)@article{doi:10.1080/24725854.2023.2257245, |
| 41. | Yannick Hogewind; Thiago D. Simão; Tal Kachman; Nils Jansen: Safe Reinforcement Learning From Pixels Using a Stochastic Latent Representation. In: ICLR, 2023. (Type: Proceedings Article | Links | BibTeX | Tags: Decision-making under uncertainty, Machine learning)@inproceedings{Hogewind2023safe, |
| 40. | Thom Badings; Licio Romao; Alessandro Abate; David Parker; Hasan A. Poonawala; Marielle Stoelinga; Nils Jansen: Robust Control for Dynamical Systems with Non-Gaussian Noise via Formal Abstractions. In: Journal of Artificial Intelligence Research (to appear), pp. 1–29, 2023. (Type: Journal Article | Abstract | Links | BibTeX | Tags: Decision-making under uncertainty, Machine learning, Robustness)@article{Badings2023JAIR,Controllers for dynamical systems that operate in safety-critical settings must account for stochastic disturbances. Such disturbances are often modeled as process noise in a dynamical system, and common assumptions are that the underlying distributions are known and/or Gaussian. In practice, however, these assumptions may be unrealistic and can lead to poor approximations of the true noise distribution. We present a novel controller synthesis method that does not rely on any explicit representation of the noise distributions. In particular, we address the problem of computing a controller that provides probabilistic guarantees on safely reaching a target, while also avoiding unsafe regions of the state space. First, we abstract the continuous control system into a finite-state model that captures noise by probabilistic transitions between discrete states. As a key contribution, we adapt tools from the scenario approach to compute probably approximately correct (PAC) bounds on these transition probabilities, based on a finite number of samples of the noise. We capture these bounds in the transition probability intervals of a so-called interval Markov decision process (iMDP). This iMDP is, with a user-specified confidence probability, robust against uncertainty in the transition probabilities, and the tightness of the probability intervals can be controlled through the number of samples. We use state-of-the-art verification techniques to provide guarantees on the iMDP and compute a controller for which these guarantees carry over to the original control system. In addition, we develop a tailored computational scheme that reduces the complexity of the synthesis of these guarantees on the iMDP. Benchmarks on realistic control systems show the practical applicability of our method, even when the iMDP has hundreds of millions of transitions. |
| 39. | Thiago D. Simão; Marnix Suilen; Nils Jansen: Safe Policy Improvement for POMDPs via Finite-State Controllers. In: AAAI, 2023. (Type: Proceedings Article | Links | BibTeX | Tags: Machine learning)@inproceedings{Simao2023safe, |
| 38. | Steven Carr; Nils Jansen; Sebastian Junges; Ufuk Topcu: Safe Reinforcement Learning via Shielding for POMDPs. In: To be presented at AAAI 2023, 2023. (Type: Proceedings Article | Links | BibTeX | Tags: Decision-making under uncertainty, Machine learning)@inproceedings{DBLP:journals/corr/abs-2204-00755, |
| 37. | Thom S. Badings; Licio Romao; Alessandro Abate; Nils Jansen: Probabilities Are Not Enough: Formal Controller Synthesis for Stochastic Dynamical Models with Epistemic Uncertainty. In: AAAI, 2023. (Type: Proceedings Article | Links | BibTeX | Tags: Decision-making under uncertainty, Robustness)@inproceedings{DBLP:journals/corr/abs-2210-05989, |
| 36. | Thom Badings; Thiago D. Simão; Marnix Suilen; Nils Jansen: Decision-making under uncertainty: beyond probabilities. Challenges and Perspectives. In: STTT, 2023. (Type: Journal Article | Abstract | Links | BibTeX | Tags: Decision-making under uncertainty, Machine learning, Robustness)@article{Badings2023Decision,This position paper reflects on the state-of-the-art in decision-making under uncertainty. A classical assumption is that probabilities can sufficiently capture all uncertainty in a system. In this paper, the focus is on the uncertainty that goes beyond this classical interpretation, particularly by employing a clear distinction between aleatoric and epistemic uncertainty. The paper features an overview of Markov decision processes (MDPs) and extensions to account for partial observability and adversarial behavior. These models sufficiently capture aleatoric uncertainty, but fail to account for epistemic uncertainty robustly. Consequently, we present a thorough overview of so-called uncertainty models that exhibit uncertainty in a more robust interpretation. We show several solution techniques for both discrete and continuous models, ranging from formal verification, over control-based abstractions, to reinforcement learning. As an integral part of this paper, we list and discuss several key challenges that arise when dealing with rich types of uncertainty in a model-based fashion. |
| 35. | Merlijn Krale; Thiago D. Simão; Nils Jansen: Act-Then-Measure: Reinforcement Learning for Partially Observable Environments with Active Measuring. In: ICAPS, pp. 212-220, 2023. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: Decision-making under uncertainty, Machine learning)@inproceedings{Krale2023act,We study Markov decision processes (MDPs), where agents control when and how they gather information, as formalized by action-contingent noiselessly observable MDPs (ACNO-MPDs). In these models, actions have two components: a control action that influences how the environment changes and a measurement action that affects the agent's observation. To solve ACNO-MDPs, we introduce the act-then-measure (ATM) heuristic, which assumes that we can ignore future state uncertainty when choosing control actions. To decide whether or not to measure, we introduce the concept of measuring value. We show how following this heuristic may lead to shorter policy computation times and prove a bound on the performance loss it incurs. We develop a reinforcement learning algorithm based on the ATM heuristic, using a Dyna-Q variant adapted for partially observable domains, and showcase its superior performance compared to prior methods on a number of partially-observable environments. |
| 34. | Cevahir Koprulu; Thiago D. Simão; Nils Jansen; Ufuk Topcu: Risk-aware Curriculum Generation for Heavy-tailed Task Distributions. In: UAI, pp. 1132–1142, 2023. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: Machine learning)@inproceedings{Koprulu2023,Automated curriculum generation for reinforcement learning (RL) aims to speed up learning by designing a sequence of tasks of increasing difficulty. Such tasks are usually drawn from probability distributions with exponentially bounded tails, such as uniform or Gaussian distributions. However, existing approaches overlook heavy-tailed distributions. Under such distributions, current methods may fail to learn optimal policies in rare and risky tasks, which fall under the tails and yield the lowest returns, respectively. We address this challenge by proposing a risk-aware curriculum generation algorithm that simultaneously creates two curricula: 1) a primary curriculum that aims to maximize the expected discounted return with respect to a distribution over target tasks, and 2) an auxiliary curriculum that identifies and over-samples rare and risky tasks observed in the primary curriculum. Our empirical results evidence that the proposed algorithm achieves significantly higher returns in frequent as well as rare tasks compared to the state-of-the-art methods. |
| 33. | Zaharah Allah Bukhsh; Hajo Molegraaf; Nils Jansen: A Maintenance Planning Framework using Online and Offline Deep Reinforcement Learning. In: Neural Computing and Applications, 2023. (Type: Journal Article | Abstract | Links | BibTeX | Tags: Machine learning)@article{Bukshh2023maintenance,Cost-effective asset management is an area of interest across several industries. Specifically, this paper develops a deep reinforcement learning (DRL) solution to automatically determine an optimal rehabilitation policy for continuously deteriorating water pipes. We approach the problem of rehabilitation planning in an online and offline DRL setting. In online DRL, the agent interacts with a simulated environment of multiple pipes with distinct lengths, materials, and failure rate characteristics. We train the agent using deep Q-learning (DQN) to learn an optimal policy with minimal average costs and reduced failure probability. In offline learning, the agent uses static data, e.g., DQN replay data, to learn an optimal policy via a conservative Q-learning algorithm without further interactions with the environment. We demonstrate that DRL-based policies improve over standard preventive, corrective, and greedy planning alternatives. Additionally, learning from the fixed DQN replay dataset in an offline setting further improves the performance. The results warrant that the existing deterioration profiles of water pipes consisting of large and diverse states and action trajectories provide a valuable avenue to learn rehabilitation policies in the offline setting, which can be further fine-tuned using the simulator. |
| 32. | Luke Rickard; Thom S. Badings; Licio Romao; Nils Jansen; Alessandro Abate: Formal Controller Synthesis for Markov Jump Linear Systems with Uncertain Dynamics. In: QEST, 2023. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: Decision-making under uncertainty, Robustness)@inproceedings{Rickard2023QEST,Automated synthesis of provably correct controllers for cyber-physical systems is crucial for deployment in safety-critical scenarios. However, hybrid features and stochastic or unknown behaviours make this problem challenging. We propose a method for synthesising controllers for Markov jump linear systems (MJLSs), a class of discrete-time models for cyber-physical systems, so that they certifiably satisfy probabilistic computation tree logic (PCTL) formulae. An MJLS consists of a finite set of stochastic linear dynamics and discrete jumps between these dynamics that are governed by a Markov decision process (MDP). We consider the cases where the transition probabilities of this MDP are either known up to an interval or completely unknown. Our approach is based on a finite-state abstraction that captures both the discrete (mode-jumping) and continuous (stochastic linear) behaviour of the MJLS. We formalise this abstraction as an interval MDP (iMDP) for which we compute intervals of transition probabilities using sampling techniques from the so-called ‘scenario approach’, resulting in a probabilistically sound approximation. We apply our method to multiple realistic benchmark problems, in particular, a temperature control and an aerial vehicle delivery problem. |
| 31. | Bram Ton; Faizan Ahmed; Jeroen Linssen: Semantic Segmentation of Terrestrial Laser Scans of Railway Catenary Arches: A Use Case Perspective. In: Sensors, vol. 23, no. 1, 2023, ISSN: 1424-8220. (Type: Journal Article | Abstract | Links | BibTeX | Tags: Sensors)@article{Ton2023Sensors,Having access to accurate and recent digital twins of infrastructure assets benefits the renovation, maintenance, condition monitoring, and construction planning of infrastructural projects. There are many cases where such a digital twin does not yet exist, such as for legacy structures. In order to create such a digital twin, a mobile laser scanner can be used to capture the geometric representation of the structure. With the aid of semantic segmentation, the scene can be decomposed into different object classes. This decomposition can then be used to retrieve cad models from a cad library to create an accurate digital twin. This study explores three deep-learning-based models for semantic segmentation of point clouds in a practical real-world setting: PointNet++, SuperPoint Graph, and Point Transformer. This study focuses on the use case of catenary arches of the Dutch railway system in collaboration with Strukton Rail, a major contractor for rail projects. A challenging, varied, high-resolution, and annotated dataset for evaluating point cloud segmentation models in railway settings is presented. The dataset contains 14 individually labelled classes and is the first of its kind to be made publicly available. A modified PointNet++ model achieved the best mean class Intersection over Union (IoU) of 71% for the semantic segmentation task on this new, diverse, and challenging dataset. |
| 30. | Qisong Yang; Thiago D. Simão; Nils Jansen; Simon H. Tindemans; Matthijs T. J. Spaan: Reinforcement Learning by Guided Safe Exploration. In: ECAI, 2023. (Type: Proceedings Article | Links | BibTeX | Tags: Decision-making under uncertainty, Machine learning)@inproceedings{Yang2023reinforcement, |
| 29. | Patrick Wienhöft; Marnix Suilen; Thiago D. Simão; Clemens Dubslaff; Christel Baier; Nils Jansen: More for Less: Safe Policy Improvement with Stronger Performance Guarantees. In: IJCAI, pp. 4406–4415, 2023. (Type: Proceedings Article | Links | BibTeX | Tags: Decision-making under uncertainty, Machine learning)@inproceedings{Wienhoft2023more, |