PrimaVera Demonstrators
- Infrastructure Demonstrator – PrimaVera is developing a web app that showcases research applications in diagnostics, prognostics, and maintenance policy optimization and logistics. The work-in-progress web application can be visited via the link above.
2024 |
|
57. | Henk Akkermans; Rob Basten; Quan Zhu; Luk Van Wassenhove: Transition paths for condition-based maintenance-driven smart services. In: Journal of Operations Management, vol. n/a, no. n/a, 2024. (Type: Journal Article | Abstract | Links | BibTeX | Tags: Maintenance optimization, smart services)@article{https://doi.org/10.1002/joom.1295, Abstract This research investigates growth inhibitors for smart services driven by condition-based maintenance (CBM). Despite the fast rise of Industry 4.0 technologies, such as smart sensoring, internet of things, and machine learning (ML), smart services have failed to keep pace. Combined, these technologies enable CBM to achieve the lean goal of high reliability and low waste for industrial equipment. Equipment located at customers throughout the world can be monitored and maintained by manufacturers and service providers, but so far industry uptake has been slow. The contributions of this study are twofold. First, it uncovers industry settings that impede the use of equipment failure data needed to train ML algorithms to predict failures and use these predictions to trigger maintenance. These empirical settings, drawn from four global machine equipment manufacturers, include either under- or over-maintenance (i.e., either too much or too little periodic maintenance). Second, formal analysis of a system dynamics model based on these empirical settings reveals a sweet spot of industry settings in which such inhibitors are absent. Companies that fall outside this sweet spot need to follow specific transition paths to reach it. This research discusses these paths, from both a research and practice perspective. |
56. | Thom S. Badings; Matthias Volk; Sebastian Junges; Mariëlle Stoelinga; Nils Jansen: CTMCs with Imprecisely Timed Observations. In: TACAS 2024, 2024. (Type: Proceedings Article | Links | BibTeX | Tags: Model checking, Robustness)@inproceedings{Badings2024TACAS, |
55. | Sabari Nathan Anbalagan; Melissa Schwarz; Rob Bemthuis; Paul Havinga: Assessing Factory's Industry 4.0 Readiness: A Practical Method for IIoT Sensor and Network Analysis. In: Procedia Computer Science, vol. 232, pp. 2730–2739, 2024. (Type: Journal Article | Abstract | Links | BibTeX | Tags: Internet of Things, Sensors)@article{ANBALAGAN20242730, Manufacturing industries are aware of the benefits of Industry 4.0 (I4.0) and the pivotal role of the Industrial Internet of Things (IIoT) to facilitate the transition. However, they often underestimate the intricate connection between the implementation of IIoT and their existing sensing and communication systems. Hindering their progress towards I4.0 implementation, many factories still rely on rigid sensing systems, characterized by limited adaptability and inflexibility. This raises questions regarding the industrial readiness for I4.0 implementation and its ability to meet the sensing and connectivity standards of IIoT. Specifically, it is unclear whether the current brownfield installations, consisting of sensors and networking systems, adequately support the demanding I4.0 applications that necessitate a multi-functional and collaborative IIoT system. Our objective is to assess a factory's progress towards I4.0 by examining the readiness of its sensing system and the communication capabilities of its network. We propose a two-stage analysis of factory readiness, including a functional segment-based sensor analysis and an application class-based network analysis. We present a case study conducted at an iron-making plant in The Netherlands to illustrate our method. Key findings of the case study include: (1) a lack of multi-functionality across segments for the majority of sensors (90%), (2) a considerable portion of network traffic (73%) requires high reliability, and (3) only 3% of the current network traffic necessitate ultra-reliable, low latency communication. Furthermore, we discuss how our method provides decision-makers with valuable guidance for the digital transformation of established and newly built manufacturing industries. |
54. | Eyuel Debebe Ayele; Stylianos Gavriel; Javier Ferreira Gonzalez; Wouter B. Teeuw; Panayiotis Philimis; Ghayoor Gillani: Emerging Industrial Internet of Things Open-Source Platforms and Applications in Diverse Sectors. In: Telecom, vol. 5, no. 2, pp. 369–399, 2024, ISSN: 2673-4001. (Type: Journal Article | Links | BibTeX | Tags: Internet of Things)@article{telecom5020019, |
53. | Thom S. Badings; Licio Romao; Alessandro Abate; Nils Jansen: A Stability-Based Abstraction Framework for Reach-Avoid Control of Stochastic Dynamical Systems with Unknown Noise Distributions. In: CoRR, vol. abs/2404.01726, 2024. (Type: Journal Article | Abstract | Links | BibTeX | Tags: Decision-making under uncertainty, Model checking)@article{DBLP:journals/corr/abs-2404-01726, Finite-state abstractions are widely studied for the automated synthesis of correct-by-construction controllers for stochastic dynamical systems. However, existing abstraction methods often lead to prohibitively large finite-state models. To address this issue, we propose a novel abstraction scheme for stochastic linear systems that exploits the system's stability to obtain significantly smaller abstract models. As a unique feature, we first stabilize the open-loop dynamics using a linear feedback gain. We then use a model-based approach to abstract a known part of the stabilized dynamics while using a data-driven method to account for the stochastic uncertainty. We formalize abstractions as Markov decision processes (MDPs) with intervals of transition probabilities. By stabilizing the dynamics, we can further constrain the control input modeled in the abstraction, which leads to smaller abstract models while retaining the correctness of controllers. Moreover, when the stabilizing feedback controller is aligned with the property of interest, then a good trade-off is achieved between the reduction in the abstraction size and the performance loss. The experiments show that our approach can reduce the size of the graph of abstractions by up to 90% with negligible performance loss. |
52. | L. A. Jimenez-Roa; T. D. Simão; Z. Bukhsh; T. Tinga; H. Molegraaf; N. Jansen; M. Stoelinga: Maintenance Strategies for Sewer Pipes with Multi-State Deterioration and Deep Reinforcement Learning. In: 8th European Conference of the Prognostics and Health Management Society 2024, PHME24, 2024. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: )@inproceedings{jimenez2024maintenance, Large-scale infrastructure systems are crucial for societal welfare, and their effective management requires strategic forecasting and intervention methods that account for various complexities. Our study addresses two challenges within the Prognostics and Health Management (PHM) framework applied to sewer assets: modeling pipe degradation across severity levels and developing effective maintenance policies. We employ Multi-State Degradation Models (MSDM) to represent the stochastic degradation process in sewer pipes and use Deep Reinforcement Learning (DRL) to devise maintenance strategies. A case study of a Dutch sewer network exemplifies our methodology. Our findings demonstrate the model's effectiveness in generating intelligent, cost-saving maintenance strategies that surpass heuristics. It adapts its management strategy based on the pipe's age, opting for a passive approach for newer pipes and transitioning to active strategies for older ones to prevent failures and reduce costs. This research highlights DRL's potential in optimizing maintenance policies. Future research will aim improve the model by incorporating partial observability, exploring various reinforcement learning algorithms, and extending this methodology to comprehensive infrastructure management. |
51. | L. A. Jimenez-Roa; N. Rusnac; M. Volk; M. Stoelinga: Fault Tree inference using Multi-Objective Evolutionary Algorithms and Confusion Matrix-based metrics. In: Formal Methods for Industrial Critical Systems (FMICS), Springer, 2024. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: )@inproceedings{jimenez2024fault, In the domain of reliability engineering and risk assessment, the development of fault tree (FT) models is pivotal for decision-making in complex systems. Traditional FT model development, relying on manual efforts and expert collaboration, is both time-consuming and error-prone. The era of Industry 4.0 introduces capabilities for automatically deriving FTs from inspection and monitoring data. This paper presents FT-MOEA-CM, an extension of the FT-MOEA algorithm for inferring FT models from failure data using multi-objective optimization. FT-MOEA-CM enhances its predecessor by integrating confusion matrix-derived metrics and incorporating parallelization and caching mechanisms. Our evaluation on six FTs from diverse application areas showcases that FT-MOEA-CM exhibits (1) enhanced robustness, (2) faster convergence and (3) better scalability than FT-MOEA, suggesting its potential in efficiently inferring larger FT models. |
50. | L. A. Jimenez-Roa; T. Heskes; T. Tinga; M. Stoelinga: Comparing Homogeneous and Inhomogeneous Time Markov Chains for Modeling Deterioration in Sewer Pipe Networks. In: 34th European Safety and Reliability Conference, ESREL 2024: Advances in Reliability, Safety and Security, 2024. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: )@inproceedings{jimenez2024comparing, Sewer pipe systems are essential for social and economic welfare. Managing these systems requires robust predictive models for degradation behaviour. This study focuses on probability-based approaches, particularly Markov chains, for their ability to associate random variables with degradation. Literature predominantly uses homogeneous and inhomogeneous Markov chains for this purpose. However, their effectiveness in sewer pipe degradation modelling is still debatable. Some studies support homogeneous Markov chains, while others challenge their utility. We examine this issue using a large-scale sewer network in the Netherlands, incorporating historical inspection data. We model degradation with homogeneous discrete and continuous time Markov chains, and inhomogeneous-time Markov chains using Gompertz, Weibull, Log-Logistic and Log-Normal density functions. Our analysis suggests that, despite their higher computational requirements, inhomogeneous-time Markov chains are more appropriate for modelling the nonlinear stochastic characteristics related to sewer pipe degradation, particularly the Gompertz distribution. However, they pose a risk of over-fitting, necessitating significant improvements in parameter inference processes to effectively address this issue. |
49. | R. Stribos; R. Bouman; L. A. Jimenez-Roa; M. Slot; Mariëlle Stoelinga: A Comparison of Anomaly Detection Algorithms with Applications on Recoater Streaking in an Additive Manufacturing Process. In: Rapid Prototyping Journal, 2024, (Submitted). (Type: Journal Article | Abstract | Links | BibTeX | Tags: )@article{stribo2024comparison, Purpose Powder bed additive manufacturing has recently seen substantial growth, yet consistently producing high-quality parts remains challenging. Recoating streaking is a common anomaly that impairs print quality. Several data-driven models for automatically detecting this anomaly have been proposed, each with varying effectiveness. However, comprehensive comparisons among them are lacking. Additionally, these models are often tailored to specific data sets. This research addresses this gap by implementing and comparing these anomaly detection models for recoating streaking in a reproducible way. This study aims to offer a clearer, more objective evaluation of their performance, strengths and weaknesses. Furthermore, this study proposes an improvement to the Line Profiles detection model to broaden its applicability, and a novel preprocessing step was introduced to enhance the models’ performances. Design/methodology/approach All found anomaly detection models have been implemented along with several preprocessing steps. Additionally, a new universal benchmarking data set has been constructed. Finally, all implemented models have been evaluated on this benchmarking data set and the effect of the different preprocessing steps was studied. Findings This comparison shows that the improved Line Profiles model established it as the most efficient detection approach in this study’s benchmark data set. Furthermore, while most state-of-the-art neural networks perform very well off the shelf, this comparison shows that specialised detection models outperform all others with the correct preprocessing. Originality/value This comparison gives new insights into different recoater streaking (RCS) detection models, showcasing each one with its strengths and weaknesses. Furthermore, the improved Line Profiles model delivers compelling performance in detecting RCS. |
48. | Sabari Nathan Anbalagan; Alessandro Chiumento; Paul Havinga: Fine Grained vs Coarse Grained Channel Quality Prediction: A 5G-RedCap Perspective for Industrial IoT Networks. In: 20th IEEE International Conference on Factory Communication Systems (WFCS 2024), 2024. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: Internet of Things, Sensors)@inproceedings{nokey, The article evaluates the effectiveness of coarsegrained channel quality prediction (CQP) for 5G-RedCap/5G NR-Light devices within industrial IoT (IIoT) networks. Finegrained predictions refine real-time communication, enhancing throughput and reducing resource utilization (RU), albeit with increased computation complexity. In contrast, coarse-grained CQP offers low computational overhead while optimizing longterm network characteristics, such as redundancy planning. Our study investigates the potential applications of coarsegrained CQP in real-time communication within the IIoT context, aiming to enhance the efficiency of simple devices (5G-RedCap) without adding the computational overhead. The varying traffic profiles and quality of service levels across diverse 5G use cases, including massive Machine Type Communication (mMTC), enhanced Mobile Broadband (eMBB), and Ultra-Reliable and Low Latency Communication (URLLC), present different challenges. For mMTC devices, coarse-grained CQP demonstrates comparable RU gains to fine-grained CQP (with up to a 50% reduction in RU), showcasing its effectiveness without added complexity. However, in the eMBB scenario, where throughput is paramount, it yields only marginal improvements. Similarly, RU gains for URLLC devices are negligible due to their stricter QoS requirements. The effectiveness of coarse-grained CQP is intricately linked to the variability in experienced channel quality across different scenarios within an indoor IIoT network. This research underscores the potential of AI applications for enhancing the performance of simple 5G-RedCap/5G NR-Light devices without compromising device complexity. |
2023 |
|
47. | Zhao Kang; Ahmadreza Marandi; Rob J. I. Basten; Ton De Kok: Robust Spare Parts Inventory Management. In: Management Science (submitted), vol. 15, 2023. (Type: Journal Article | Links | BibTeX | Tags: Maintenance optimization, Robustness)@article{Kang2023RobustManagement, |
46. | Nubia Nale Alves da Silveira; Annemieke A. Meghoe; Tiedo Tinga: Integration of multiple failure mechanisms in a life assessment method for centrifugal pump impellers. In: Advances in mechanical engineering, vol. 15, no. 6, 2023, ISSN: 1687-8132. (Type: Journal Article | Links | BibTeX | Tags: Failure mechanisms, Prognostics)@article{Silveria2023AME, |
45. | Rob Basten Ragnar Eggertsson; Geert-Jan Houtum: Maintenance optimization for capital goods when information is incomplete and environment-dependent. In: IISE Transactions, vol. 0, no. 0, pp. 1-16, 2023. (Type: Journal Article | Links | BibTeX | Tags: Maintenance optimization)@article{doi:10.1080/24725854.2023.2257245, |
44. | Reza Soltani; Matthias Volk; Leonardo Diamonte; Milan Lopuhaä-Zwakenberg; Mariëlle Stoelinga: Optimal Spare Management via Statistical Model Checking: A Case Study in Research Reactors. In: FMICS, pp. 205–223, Springer, 2023. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: Maintenance optimization, Spare part management)@inproceedings{DBLP:conf/fmics/SoltaniVDLS23, Systematic spare management is important to optimize the twin goals of high reliability and low costs. However, existing approaches to spare management do not incorporate a detailed analysis of the effect on the absence of spares on the system’s reliability. In this work, we combine fault tree analysis with statistical model checking to model spare part management as a stochastic priced timed game automaton (SPTGA). We use Uppaal Stratego to find the number of spares that minimizes the total costs due to downtime and spare purchasing; the resulting SPTGA model can then additionally be analyzed according to other metrics like expected availability. We apply these techniques to the emergency shutdown system of a research nuclear reactor. Our methods find the optimal spare management for a subsystem in a matter of minutes, minimizing cost while ensuring an expected availability of 99.96%. |
43. | Natália Ribeiro Marinho; Richard Loendersloot; Tiedo Tinga; Frank Grooteman; Jan Willem Wiegman: A Comparison of Optical Sensing Systems with Piezo-Electric Sensors for Impact Identification of Composite Plates. In: 14th International Workshop on Structural Health Monitoring, IWSHM 2023, pp. 1127–1133, DEStech Publications, Inc 2023. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: Sensors)@inproceedings{marinho2023comparison, This paper proposes using Fibre Bragg Grating (FBG) and piezoelectric (PZT) sensors for impact identification. The main objective is to evaluate the ability of the state-ofthe- art sensors to estimate the impact energy, using the results of the PZTs as a reference. This approach allows a fair comparison and overcomes the inherent variability of different test runs of the same measurement. The comparison of sensor technologies consists of evaluating sensitivity to features for impact energy estimation, signal strength, repeatability, directivity, and signal correlation. Small-mass impacts were applied to a square composite plate at different locations and energies. The energy was kept low enough to avoid damaging the panel. The PZT and FGB sensors were placed at the same locations but on either side of the panel to compare signals evenly. The results showed that the energy from the measured response reflects the impact energy level. Moreover, FBGs and PZTs had comparable responses and an apparent similarity in time response, besides consistency in the frequency domain. The higher sampling rate for the PZTs allows for the analysis of higher frequency bands, compared to FBGs, showing relevant amplitudes above 10kHz. Future work will focus on developing and validating a force reconstruction algorithm and defining the optimal sensor configuration. |
42. | Matthias Volk; Muzammil Ibne Irshad; Joost-Pieter Katoen; Falak Sher; Mariëlle Stoelinga; Ahmad Zafar: SAFEST: the static and dynamic fault tree analysis tool. In: ESREL, pp. 193–200, Research Publishing, 2023. (Type: Proceedings Article | Links | BibTeX | Tags: Fault tree analysis)@inproceedings{ESREL_SAFEST, |
41. | Wouter Bos; Matthias Volk; Mariëlle Stoelinga; Marc Bouissou; Pavel Krčál: From Fault Trees to Piping and Instrumentation Diagrams. In: ESREL, pp. 1234–1235, Research Publishing, 2023. (Type: Proceedings Article | Links | BibTeX | Tags: Fault tree analysis)@inproceedings{ESREL_PID, |
40. | Thom S. Badings; Sebastian Junges; Ahmadreza Marandi; Ufuk Topcu; Nils Jansen: Efficient Sensitivity Analysis for Parametric Robust Markov Chains. In: CAV (3), pp. 62–85, Springer, 2023. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: Decision-making under uncertainty, Model learning)@inproceedings{DBLP:conf/cav/BadingsJMTJ23, We provide a novel method for sensitivity analysis of parametric robust Markov chains. These models incorporate parameters and sets of probability distributions to alleviate the often unrealistic assumption that precise probabilities are available. We measure sensitivity in terms of partial derivatives with respect to the uncertain transition probabilities regarding measures such as the expected reward. As our main contribution, we present an efficient method to compute these partial derivatives. To scale our approach to models with thousands of parameters, we present an extension of this method that selects the subset of k parameters with the highest partial derivative. Our methods are based on linear programming and differentiating these programs around a given value for the parameters. The experiments show the applicability of our approach on models with over a million states and thousands of parameters. Moreover, we embed the results within an iterative learning scheme that profits from having access to a dedicated sensitivity analysis. |
39. | Alberto Castellini; Federico Bianchi; Edoardo Zorzi; Thiago D. Simão; Alessandro Farinelli; Matthijs T. J. Spaan: Scalable Safe Policy Improvement via Monte Carlo Tree Search. In: ICML, pp. 3732–3756, 2023. (Type: Proceedings Article | Links | BibTeX | Tags: Machine learning)@inproceedings{Castellini2023, |
38. | Dennis Gross; Thiago D. Simão; Nils Jansen; Guillermo A. Pérez: Targeted Adversarial Attacks on Deep Reinforcement Learning Policies via Model Checking. In: ICAART, pp. 501–508, 2023. (Type: Proceedings Article | Links | BibTeX | Tags: Machine learning, Model checking)@inproceedings{Gross2023targeted, |
37. | Wietze Koops; Nils Jansen; Sebastian Junges; Thiago D. Simão: Recursive Small-Step Multi-Agent A* for Dec-POMDPs. In: IJCAI, pp. 5402–5410, 2023. (Type: Proceedings Article | Links | BibTeX | Tags: Decision-making under uncertainty)@inproceedings{Koops2023recursive, |
36. | Patrick Wienhöft; Marnix Suilen; Thiago D. Simão; Clemens Dubslaff; Christel Baier; Nils Jansen: More for Less: Safe Policy Improvement with Stronger Performance Guarantees. In: IJCAI, pp. 4406–4415, 2023. (Type: Proceedings Article | Links | BibTeX | Tags: Decision-making under uncertainty, Machine learning)@inproceedings{Wienhoft2023more, |
35. | Qisong Yang; Thiago D. Simão; Nils Jansen; Simon H. Tindemans; Matthijs T. J. Spaan: Reinforcement Learning by Guided Safe Exploration. In: ECAI, 2023. (Type: Proceedings Article | Links | BibTeX | Tags: Decision-making under uncertainty, Machine learning)@inproceedings{Yang2023reinforcement, |
34. | Bram Ton; Faizan Ahmed; Jeroen Linssen: Semantic Segmentation of Terrestrial Laser Scans of Railway Catenary Arches: A Use Case Perspective. In: Sensors, vol. 23, no. 1, 2023, ISSN: 1424-8220. (Type: Journal Article | Abstract | Links | BibTeX | Tags: Sensors)@article{Ton2023Sensors, Having access to accurate and recent digital twins of infrastructure assets benefits the renovation, maintenance, condition monitoring, and construction planning of infrastructural projects. There are many cases where such a digital twin does not yet exist, such as for legacy structures. In order to create such a digital twin, a mobile laser scanner can be used to capture the geometric representation of the structure. With the aid of semantic segmentation, the scene can be decomposed into different object classes. This decomposition can then be used to retrieve cad models from a cad library to create an accurate digital twin. This study explores three deep-learning-based models for semantic segmentation of point clouds in a practical real-world setting: PointNet++, SuperPoint Graph, and Point Transformer. This study focuses on the use case of catenary arches of the Dutch railway system in collaboration with Strukton Rail, a major contractor for rail projects. A challenging, varied, high-resolution, and annotated dataset for evaluating point cloud segmentation models in railway settings is presented. The dataset contains 14 individually labelled classes and is the first of its kind to be made publicly available. A modified PointNet++ model achieved the best mean class Intersection over Union (IoU) of 71% for the semantic segmentation task on this new, diverse, and challenging dataset. |
33. | Luke Rickard; Thom S. Badings; Licio Romao; Nils Jansen; Alessandro Abate: Formal Controller Synthesis for Markov Jump Linear Systems with Uncertain Dynamics. In: QEST, 2023. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: Decision-making under uncertainty, Robustness)@inproceedings{Rickard2023QEST, Automated synthesis of provably correct controllers for cyber-physical systems is crucial for deployment in safety-critical scenarios. However, hybrid features and stochastic or unknown behaviours make this problem challenging. We propose a method for synthesising controllers for Markov jump linear systems (MJLSs), a class of discrete-time models for cyber-physical systems, so that they certifiably satisfy probabilistic computation tree logic (PCTL) formulae. An MJLS consists of a finite set of stochastic linear dynamics and discrete jumps between these dynamics that are governed by a Markov decision process (MDP). We consider the cases where the transition probabilities of this MDP are either known up to an interval or completely unknown. Our approach is based on a finite-state abstraction that captures both the discrete (mode-jumping) and continuous (stochastic linear) behaviour of the MJLS. We formalise this abstraction as an interval MDP (iMDP) for which we compute intervals of transition probabilities using sampling techniques from the so-called ‘scenario approach’, resulting in a probabilistically sound approximation. We apply our method to multiple realistic benchmark problems, in particular, a temperature control and an aerial vehicle delivery problem. |
32. | Zaharah Allah Bukhsh; Hajo Molegraaf; Nils Jansen: A Maintenance Planning Framework using Online and Offline Deep Reinforcement Learning. In: Neural Computing and Applications, 2023. (Type: Journal Article | Abstract | Links | BibTeX | Tags: Machine learning)@article{Bukshh2023maintenance, Cost-effective asset management is an area of interest across several industries. Specifically, this paper develops a deep reinforcement learning (DRL) solution to automatically determine an optimal rehabilitation policy for continuously deteriorating water pipes. We approach the problem of rehabilitation planning in an online and offline DRL setting. In online DRL, the agent interacts with a simulated environment of multiple pipes with distinct lengths, materials, and failure rate characteristics. We train the agent using deep Q-learning (DQN) to learn an optimal policy with minimal average costs and reduced failure probability. In offline learning, the agent uses static data, e.g., DQN replay data, to learn an optimal policy via a conservative Q-learning algorithm without further interactions with the environment. We demonstrate that DRL-based policies improve over standard preventive, corrective, and greedy planning alternatives. Additionally, learning from the fixed DQN replay dataset in an offline setting further improves the performance. The results warrant that the existing deterioration profiles of water pipes consisting of large and diverse states and action trajectories provide a valuable avenue to learn rehabilitation policies in the offline setting, which can be further fine-tuned using the simulator. |
31. | Cevahir Koprulu; Thiago D. Simão; Nils Jansen; Ufuk Topcu: Risk-aware Curriculum Generation for Heavy-tailed Task Distributions. In: UAI, pp. 1132–1142, 2023. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: Machine learning)@inproceedings{Koprulu2023, Automated curriculum generation for reinforcement learning (RL) aims to speed up learning by designing a sequence of tasks of increasing difficulty. Such tasks are usually drawn from probability distributions with exponentially bounded tails, such as uniform or Gaussian distributions. However, existing approaches overlook heavy-tailed distributions. Under such distributions, current methods may fail to learn optimal policies in rare and risky tasks, which fall under the tails and yield the lowest returns, respectively. We address this challenge by proposing a risk-aware curriculum generation algorithm that simultaneously creates two curricula: 1) a primary curriculum that aims to maximize the expected discounted return with respect to a distribution over target tasks, and 2) an auxiliary curriculum that identifies and over-samples rare and risky tasks observed in the primary curriculum. Our empirical results evidence that the proposed algorithm achieves significantly higher returns in frequent as well as rare tasks compared to the state-of-the-art methods. |
30. | Merlijn Krale; Thiago D. Simão; Nils Jansen: Act-Then-Measure: Reinforcement Learning for Partially Observable Environments with Active Measuring. In: ICAPS, pp. 212-220, 2023. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: Decision-making under uncertainty, Machine learning)@inproceedings{Krale2023act, We study Markov decision processes (MDPs), where agents control when and how they gather information, as formalized by action-contingent noiselessly observable MDPs (ACNO-MPDs). In these models, actions have two components: a control action that influences how the environment changes and a measurement action that affects the agent's observation. To solve ACNO-MDPs, we introduce the act-then-measure (ATM) heuristic, which assumes that we can ignore future state uncertainty when choosing control actions. To decide whether or not to measure, we introduce the concept of measuring value. We show how following this heuristic may lead to shorter policy computation times and prove a bound on the performance loss it incurs. We develop a reinforcement learning algorithm based on the ATM heuristic, using a Dyna-Q variant adapted for partially observable domains, and showcase its superior performance compared to prior methods on a number of partially-observable environments. |
29. | Thom Badings; Thiago D. Simão; Marnix Suilen; Nils Jansen: Decision-making under uncertainty: beyond probabilities. Challenges and Perspectives. In: STTT, 2023. (Type: Journal Article | Abstract | Links | BibTeX | Tags: Decision-making under uncertainty, Machine learning, Robustness)@article{Badings2023Decision, This position paper reflects on the state-of-the-art in decision-making under uncertainty. A classical assumption is that probabilities can sufficiently capture all uncertainty in a system. In this paper, the focus is on the uncertainty that goes beyond this classical interpretation, particularly by employing a clear distinction between aleatoric and epistemic uncertainty. The paper features an overview of Markov decision processes (MDPs) and extensions to account for partial observability and adversarial behavior. These models sufficiently capture aleatoric uncertainty, but fail to account for epistemic uncertainty robustly. Consequently, we present a thorough overview of so-called uncertainty models that exhibit uncertainty in a more robust interpretation. We show several solution techniques for both discrete and continuous models, ranging from formal verification, over control-based abstractions, to reinforcement learning. As an integral part of this paper, we list and discuss several key challenges that arise when dealing with rich types of uncertainty in a model-based fashion. |
28. | Yannick Hogewind; Thiago D. Simão; Tal Kachman; Nils Jansen: Safe Reinforcement Learning From Pixels Using a Stochastic Latent Representation. In: ICLR, 2023. (Type: Proceedings Article | Links | BibTeX | Tags: Decision-making under uncertainty, Machine learning)@inproceedings{Hogewind2023safe, |
27. | Thom Badings; Licio Romao; Alessandro Abate; David Parker; Hasan A. Poonawala; Marielle Stoelinga; Nils Jansen: Robust Control for Dynamical Systems with Non-Gaussian Noise via Formal Abstractions. In: Journal of Artificial Intelligence Research (to appear), pp. 1–29, 2023. (Type: Journal Article | Abstract | Links | BibTeX | Tags: Decision-making under uncertainty, Machine learning, Robustness)@article{Badings2023JAIR, Controllers for dynamical systems that operate in safety-critical settings must account for stochastic disturbances. Such disturbances are often modeled as process noise in a dynamical system, and common assumptions are that the underlying distributions are known and/or Gaussian. In practice, however, these assumptions may be unrealistic and can lead to poor approximations of the true noise distribution. We present a novel controller synthesis method that does not rely on any explicit representation of the noise distributions. In particular, we address the problem of computing a controller that provides probabilistic guarantees on safely reaching a target, while also avoiding unsafe regions of the state space. First, we abstract the continuous control system into a finite-state model that captures noise by probabilistic transitions between discrete states. As a key contribution, we adapt tools from the scenario approach to compute probably approximately correct (PAC) bounds on these transition probabilities, based on a finite number of samples of the noise. We capture these bounds in the transition probability intervals of a so-called interval Markov decision process (iMDP). This iMDP is, with a user-specified confidence probability, robust against uncertainty in the transition probabilities, and the tightness of the probability intervals can be controlled through the number of samples. We use state-of-the-art verification techniques to provide guarantees on the iMDP and compute a controller for which these guarantees carry over to the original control system. In addition, we develop a tailored computational scheme that reduces the complexity of the synthesis of these guarantees on the iMDP. Benchmarks on realistic control systems show the practical applicability of our method, even when the iMDP has hundreds of millions of transitions. |
26. | Thiago D. Simão; Marnix Suilen; Nils Jansen: Safe Policy Improvement for POMDPs via Finite-State Controllers. In: AAAI, 2023. (Type: Proceedings Article | Links | BibTeX | Tags: Machine learning)@inproceedings{Simao2023safe, |
25. | Steven Carr; Nils Jansen; Sebastian Junges; Ufuk Topcu: Safe Reinforcement Learning via Shielding for POMDPs. In: To be presented at AAAI 2023, 2023. (Type: Proceedings Article | Links | BibTeX | Tags: Decision-making under uncertainty, Machine learning)@inproceedings{DBLP:journals/corr/abs-2204-00755, |
24. | Thom S. Badings; Licio Romao; Alessandro Abate; Nils Jansen: Probabilities Are Not Enough: Formal Controller Synthesis for Stochastic Dynamical Models with Epistemic Uncertainty. In: AAAI, 2023. (Type: Proceedings Article | Links | BibTeX | Tags: Decision-making under uncertainty, Robustness)@inproceedings{DBLP:journals/corr/abs-2210-05989, |
23. | Bram Dekker; Bram Ton; Joanneke Meijer; Nacir Bouali; Jeroen Linssen; Faizan Ahmed: Point Cloud Analysis of Railway Infrastructure: A Systematic Literature Review. In: IEEE Access, vol. 11, pp. 134355-134373, 2023. (Type: Journal Article | Links | BibTeX | Tags: Sensors)@article{10328827, |
22. | Sabari Nathan Anbalagan; Paul JM Havinga; Alessandro Chiumento: VarOLLA: Maximizing Throughput for 5G-RedCap Devices in IIoT Networks. In: IEEE 9th World Forum on Internet of Things, IoT 2023, 2023. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: Internet of Things)@inproceedings{anbalagan2023varolla, In this paper, we propose VarOLLA, a novel approach aimed at maximizing the throughput of 5G-RedCap devices in Industrial Internet of Things (IIoT) environments. VarOLLA addresses the sub-optimal spectral efficiency of Outer Loop Link Adaptation (OLLA) by introducing a ‘Throughput Factor’ that incentivizes adaptive decision-making based on throughput considerations. Through extensive simulations, we demonstrate significant improvements in throughput, with gains of up to 35% compared to traditional OLLA techniques, particularly in scenarios with high channel outdatedness. Moreover, VarOLLA effectively reduces consecutive transmission failures (rBLER) and achieves substantial reductions in control message overhead, up to 87.5%. Our findings highlight the strong potential of VarOLLA in IIoT networks and its significant contribution to the realization of high-performance applications during the Fourth Industrial Revolution (4IR) in the manufacturing sector. |
2022 |
|
21. | Luc Stefan Keizers; Richard Loendersloot; Tiedo Tinga: Atmospheric Corrosion Prognostics Using a Particle Filter. In: Proceedings of the 32nd European Safety and Reliability Conference (ESREL 2022), pp. 1259–1266, 2022. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: Bayesian filters, Prognostics)@inproceedings{Keizers2022ESREL, Unexpected system failures are costly and preventing them is crucial to guarantee availability and reliability of complex assets. Prognostics help to increase the availability and reliability. However, existing methods have their limitations: physics-based methods have limited adaptivity to specific applications, while data-driven methods heavily rely on (scarcely available) historical data, which reduces their prognostic performance. Especially when operational conditions change over time, existing methods do not always perform well. As a solution, this paper proposes a new framework in which loads are explicitly incorporated in a prognostic method based on Bayesian filtering. This is accomplished by zooming in on the failure mechanism on the material level, thus establishing a quantitative relation between usage and degradation rates. This relation is updated using a Bayesian filter and measured loads, but also allows accurate degradation predictions by considering future (changing) loads. This enables decision support on either operational use or maintenance activities. The performance of the proposed load-controlled prognostic method is demonstrated in an atmospheric corrosion use case, based on a public real data set constructed from annual corrosion measurements on carbon steel specimens. The developed load-controlled particle filter (LCPF) is demonstrated to outperform a method based on a regular particle filter, a regression model and an ARIMA model for this specific scenario with changing operating conditions. The generalization of the framework is demonstrated by two additional conceptual case studies on crack propagation and seal wear. |
20. | Norman Weik; Matthias Volk; Joost-Pieter Katoen; Nils Nießen: DFT modeling approach for operational risk assessment of railway infrastructure. In: Int. J. Softw. Tools Technol. Transf., vol. 24, no. 3, pp. 331–350, 2022. (Type: Journal Article | Links | BibTeX | Tags: Fault tree analysis)@article{DBLP:journals/sttt/WeikVKN22, |
19. | Daniel Basgöze; Matthias Volk; Joost-Pieter Katoen; Shahid Khan; Mariëlle Stoelinga: BDDs Strike Back – Efficient Analysis of Static and Dynamic Fault Trees. In: NFM, pp. 713–732, Springer, 2022. (Type: Proceedings Article | Links | BibTeX | Tags: Decision tree, Fault tree analysis)@inproceedings{DBLP:conf/nfm/BasgozeVKKS22, |
18. | Bas Oudenhoven; Philippe Van Calseyde; Rob Basten; Evangelia Demerouti: Predictive maintenance for industry 5.0: behavioural inquiries from a work system perspective. In: International Journal of Production Research, vol. 0, no. 0, pp. 1-20, 2022. (Type: Journal Article | Links | BibTeX | Tags: Human decision-making, Organizational behavior)@article{Oudenhoven2022, |
17. | Qisong Yang; Thiago D. Simão; Simon H. Tindemans; Matthijs T. J. Spaan: Safety-constrained reinforcement learning with a distributional safety critic. In: Machine Learning, pp. 1–29, 2022. (Type: Journal Article | Links | BibTeX | Tags: Machine learning)@article{Yang2022safety-constrained, |
16. | Marnix Suilen; Thiago D. Simão; David Parker; Nils Jansen: Robust Anytime Learning of Markov Decision Processes. In: NeurIPS, 2022. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: Decision-making under uncertainty, Machine learning)@inproceedings{DBLP:journals/corr/abs-2205-15827, Markov decision processes (MDPs) are formal models commonly used in sequential decision-making. MDPs capture the stochasticity that may arise, for instance, from imprecise actuators via probabilities in the transition function. However, in data-driven applications, deriving precise probabilities from (limited) data introduces statistical errors that may lead to unexpected or undesirable outcomes. Uncertain MDPs (uMDPs) do not require precise probabilities but instead use so-called uncertainty sets in the transitions, accounting for such limited data. Tools from the formal verification community efficiently compute robust policies that provably adhere to formal specifications, like safety constraints, under the worst-case instance in the uncertainty set. We continuously learn the transition probabilities of an MDP in a robust anytime-learning approach that combines a dedicated Bayesian inference scheme with the computation of robust policies. In particular, our method (1) approximates probabilities as intervals, (2) adapts to new data that may be inconsistent with an intermediate model, and (3) may be stopped at any time to compute a robust policy on the uMDP that faithfully captures the data so far. Furthermore, our method is capable of adapting to changes in the environment. We show the effectiveness of our approach and compare it to robust policies computed on uMDPs learned by the UCRL2 reinforcement learning algorithm in an experimental evaluation on several benchmarks. |
15. | Lisandro A. Jimenez-Roa; Tom Heskes; Tiedo Tinga; Mariëlle Stoelinga: Automatic inference of fault tree models via multi-objective evolutionary algorithms. In: IEEE Transactions on Dependable and Secure Computing, pp. 1-12, 2022. (Type: Journal Article | Links | BibTeX | Tags: Fault tree analysis, Machine learning)@article{9875105, |
14. | Lisandro Arturo Jimenez-Roa; Matthias Volk; Mariëlle Stoelinga: Data-Driven Inference of Fault Tree Models Exploiting Symmetry and Modularization. In: International Conference on Computer Safety, Reliability, and Security, pp. 46–61, Springer 2022. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: Fault tree analysis, Machine learning)@inproceedings{jimenez2022data, We present SymLearn, a method to automatically infer fault tree (FT) models from data. SymLearn takes as input failure data of the system components and exploits evolutionary algorithms to learn a compact FT matching the input data. SymLearn achieves scalability by leveraging two common phenomena in FTs: (i) We automatically identify symmetries in the failure data set, learning symmetric FT parts only once. (ii) We partition the input data into independent modules, subdividing the inference problem into smaller parts. We validate our approach via case studies, including several truss systems, which are symmetric structures commonly found in infrastructures, such as bridges. Our experiments show that, in most cases, the exploitation of modules and symmetries accelerates the FT inference from hours to under three minutes. |
13. | Lisandro A Jimenez-Roa; Tom Heskes; Tiedo Tinga; Hajo JA Molegraaf; Mariëlle Stoelinga: Deterioration modeling of sewer pipes via discrete-time Markov chains: A large-scale case study in the Netherlands. In: 32nd European Safety and Reliability Conference, ESREL 2022: Understanding and Managing Risk and Reliability for a Sustainable Future, pp. 1299–1306, 2022. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: Model learning, Prognostics)@inproceedings{jimenez2022deterioration, Sewer pipe network systems are an important part of civil infrastructure, and in order to find a good trade-off between maintenance costs and system performance, reliable sewer pipe degradation models are essential. In this paper, we present a large-scale case study in the city of Breda in the Netherlands. Our dataset has information on sewer pipes built since the 1920s and contains information on different covariates. We also have several types of damage, but we focus our attention on infiltrations, surface damage, and cracks. Each damage has an associated severity index ranging from 1 to 5. To account for the characteristics of sewer pipes, we defined 6 cohorts of interest. Two types of discrete-time Markov chains (DTMC), which we called Chain `Multi' and `Single' (where Chain `Multi' contains additional transitions compared to Chain `Single'), are commonly used to model sewer pipe degradation at the pipeline level, and we want to evaluate which suits better our case study. To calibrate the DTMCs, we define an optimization process using Sequential Least-Squares Programming to find the DTMC parameter that best minimizes the root mean weighted square error. Our results show that for our case study, there is no substantial difference between Chain `Multi' and `Single', but the latter has fewer parameters and can be easily trained. Our DTMCs are useful to compare the cohorts via the expected values, e.g., concrete pipes carrying mixed and waste content reach severe levels of surface damage more quickly compared to concrete pipes carrying rainwater, which is a phenomenon typically identified in practice. |
12. | Thom S. Badings; Murat Cubuktepe; Nils Jansen; Sebastian Junges; Joost-Pieter Katoen; Ufuk Topcu: Scenario-based verification of uncertain parametric MDPs. In: Int. J. Softw. Tools Technol. Transf., vol. 24, no. 5, pp. 803–819, 2022. (Type: Journal Article | Links | BibTeX | Tags: Model checking, Scenario optimization)@article{DBLP:journals/sttt/BadingsCJJKT22, |
11. | Thom S. Badings; Nils Jansen; Sebastian Junges; Mariëlle Stoelinga; Matthias Volk: Sampling-Based Verification of CTMCs with Uncertain Rates. In: CAV (2), pp. 26–47, Springer, 2022. (Type: Proceedings Article | Links | BibTeX | Tags: Decision-making under uncertainty, Model checking, Scenario optimization)@inproceedings{DBLP:conf/cav/BadingsJJSV22, |
10. | David Kerkkamp; Zaharah A. Bukhsh; Yingqian Zhang; Nils Jansen: Grouping of Maintenance Actions with Deep Reinforcement Learning and Graph Convolutional Networks. In: International Conference on Agents and Artificial Intelligence (ICAART), 2022. (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: Machine learning, Maintenance optimization)@inproceedings{Kerkkamp2021, Reinforcement learning (RL) has shown promising performance in several applications such as robotics and games. However, the use of RL in emerging real-world domains such as smart industry and asset management remains scarce. This paper addresses the problem of optimal maintenance planning using historical data. We propose a novel Deep RL (DRL) framework based on Graph Convolutional Networks (GCN) to leverage the inherent graph structure of typical assets. As demonstrator, we employ an underground sewer pipe network. In particular, instead of dispersed maintenance actions of individual pipes across the network, the GCN ensures the grouping of maintenance actions of geographically close pipes. We perform experiments using the distinct physical characteristics, deterioration profiles, and historical data of sewer inspections within an urban environment. The results show that combining Deep Q-Networks (DQN) with GCN leads to structurally more reliable networks and a higher degree of maintenance grouping, compared to DQN with fully-connected layers and standard preventive and corrective maintenance strategy that are often adopted in practice. Our approach shows potential for developing efficient and practical maintenance plans in terms of cost and reliability. |
9. | Thom S. Badings; Alessandro Abate; Nils Jansen; David Parker; Hasan A. Poonawala; Mariëlle Stoelinga: Sampling-Based Robust Control of Autonomous Systems with Non-Gaussian Noise. In: AAAI, pp. 9669–9678, AAAI Press, 2022. (Type: Proceedings Article | Links | BibTeX | Tags: Decision-making under uncertainty, Model checking, Robustness)@inproceedings{DBLP:conf/aaai/BadingsA00PS22, |
2021 |
|
8. | Lisandro A. Jimenez-Roa; Tom Heskes; Mariëlle Stoelinga: Fault Trees, Decision Trees, and Binary Decision Diagrams: A systematic comparison. In: Castanier, Bruno; Cepin, Marko; Bigaud, David; Bérenguer, Christophe (Ed.): Proceedings of the 31st European Safety and Reliability Conference (ESREL 2021), pp. 673–680, Research Publishing, 2021, (European Safety and Reliability Conference 2021, ESREL 2021 ; Conference date: 19-09-2021 Through 23-09-2021). (Type: Proceedings Article | Abstract | Links | BibTeX | Tags: Decision trees, Fault tree analysis, Reliability engineering)@inproceedings{JimenezRoa2021_ESREL, In reliability engineering, we need to understand system dependencies, cause-effect relations, identify critical components, and analyze how they trigger failures. Three prominent graph models commonly used for these purposes are fault trees (FTs), decision trees (DTs), and binary decision diagrams (BDDs). These models are popular because they are easy to interpret, serve as a communication tool between stakeholders of various backgrounds, and support decision-making processes. Moreover, these models help to understand real-world problems by computing reliability metrics, minimum cut sets, logic rules, and displaying dependencies. Nevertheless, it is unclear how these graph models compare. Thus, the goal of this paper is to understand the similarities and differences through a systematic comparison based on their (i) purpose and application, (ii) structural representation, (iii) analysis methods, (iv) construction, and (v) benefits & limitations. Furthermore, we use a running example based on a Container Seal Design to showcase the models in practice. Our results show that, given that FTs, DTs, and BDDs have different purposes and application domains, they adopt different structural representations and analysis methodologies that entail a variety of benefits and limitations, the latter can be addressed via conversion methods or extensions. Specific remarks are that BDDs can be considered as a compact representation of binary DTs, since the former allows sub-node sharing, which makes BDDs more efficient at representing logical rules than binary DTs. It is possible to obtain cut sets from BDDs and DTs and construct a FT using the (con/dis)junctive normal form, although this may result in a sub-optimal FT structure. |