Suraj Pawar
Title: HAM
Highlight: Outstanding graduate student award in Mechanical and Aerospace Engineering at Oklahoma State University. Numerous journal articles.
Eivind Meyer
Title: On Course Towards Model-Free Guidance: A Self-Learning Approach To Dynamic Collision Avoidance for Autonomous Surface Vehicles
Highlights: Best thesis award from the OpwnAI Lab, 2 Journal articles
Abstract: In this project, we show that Deep Reinforcement Learning (DRL) is applicable to the problem of training a reactive, autonomous vessel to navigate unknown waters, which en- tails not only the challenge of avoiding running ashore while efficiently making progress along the desired path, but also dynamic obstacle avoidance, i.e. control that mitigates col- lision risk upon ship encounters. A rangefinder sensor suite attached to the vessel, whose output is fed to the agent’s control policy network, is designed, implemented in software and efficiently pre-processed to reduce the dimensionality of the perception vector while maintaining sensing integrity. The contribution of this work is two-fold: First, we outline the design, implementation and training of the perception-based guidance agent, with the goal of making it capable of following priori known trajectories while avoiding collisions with other vessels. The reinforcement learning agent is trained to control the vessel’s actuators, which include both thrusters as well as rudder control surfaces. A carefully constructed reward func- tion, which balances the prioritization of path adherence versus that of collision avoidance (which can be considered competing objectives), is used to guide the agent’s learning pro- cess. Then, the state-of-the-art Proximal Policy Optimization (PPO) DRL algorithm is utilized for training the agent’s policy such that it, in the end, yields optimal actions with regards to maximizing the reward that the agent receives by the environment over time. Finally, we evaluate the trained agent’s performance in challenging, dynamic test scenar- ios, including ones that are reconstructed from real-world terrain and maritime traffic data from the Trondheim Fjord, an inlet of the Norwegian sea. Furthermore, The Python simulation framework gym-auv, which was developed to facil- itate this research, has a vast potential to enable further research in the field, and is thus covered extensively in this thesis. It provides not only a software foundation that can be easily expanded by new environments, reward function designs and vessel models, but also access to high-quality plotting and reporting functionality as well as access to real- time (and recorded) video rendering in both 2D and 3D.
Harsha Vaddireddy
Highlight: 1 Journal article appeared as Editor’s pick in Physics of Fluids
Simen Havenstrøm
Title: From Beginner to Expert: Deep Reinforcement Learning Controller for 3D Path-Following and Collision Avoidance by Autonomous Underwater Vehicles
Highlights: 1 Journal article published and 1 under consideration.
Duy Tan Tran
Title: Convolutional Neural Network and Generative Adversarial Networks Enabled Resolution Enhancement of Numerical Simulations
Highlights: Best poster award in the Deep Wind Conference 2020, 1 Journal article and ! confernce proceeding
Abstract: Fluid flows, like atmospheric flows around terrains in wind farms, are governed by a broad variety of spatio-temporal turbulent scales, thus making their real-time numerical model- ing computationally unmanageable owing to higher resolution required to capture all the scales. Hence, in this work, we demonstrate a novel approach to address this issue through a combination of fast coarse scale physics-based simulator and a family of advanced ma- chine learning algorithms like convolutional neural networks (CNNs) and generative ad- versarial networks (GANs). The physics-based simulator generates a coarse wind field in a real wind farm located on a complex terrain and then machine learning models enhance these results to a much finer resolution. The results from machine learning methods are compared with each other and against state-of-the-art interpolation methods with respect to ground truth, which shows the superiority of the approach. We also investigate interme- diate results within both deep learning models, gaining insight into how it reconstructs the fully-resolved 3D velocity fields from coarser scale while respecting the local terrain.
Eirik Ekjord Vesterkjær
Title: Combining grid-based uncertainty propagation and neural networks with uncertainty estimation
Abstract: We can apply grid-based methods for deterministic uncertainty propagation in combination with uncertainty estimation algorithms for neural networks. This can be used to model dynamical processes and their uncertainties using neural networks. Further, such methods can yield calibrated uncertainty estimates when forecasting with dissipative first order systems.
This master’s thesis in engineering cybernetics develops and applies an ap- proach to employing existing grid-based uncertainty propagation methods (e.g. the unscented transform) on machine learning algorithms which have a variable (heteroscedastic) output uncertainty. Further, the thesis explores a method of jointly learning the process dynamics and process uncertainty in a dynamical system. This method applies recent advances in predictive uncertainty estimation for neural networks in combination with grid-based uncertainty propagation.
The methods are applied on a real world dataset, to model the temperature evolution in the main bearing of wind turbines at a Norwegian wind farm. It’s found that the temperature confidence intervals predicted by a neural network- based method when simulating generalizes to other wind turbines. The results in- dicate that it is feasible to jointly model first order process dynamics and process uncertainty with neural networks. Though a trade-off between predictive accur- acy and calibration was observed, this can likely be mitigated by fine-tuning the uncertainty estimation methods and optimization procedures.
The main contributions from this thesis are: i) Derivation of equations for applying grid-based uncertainty propagation methods on systems with variable output uncertainty, ii) Demonstration of how this can be used to model process dynamics and process noise using neural networks, and iii) Application of the methods to model the temperature of a wind turbine main bearing with uncer- tainty estimates.
Peter Herman Stavelin
Title: Object Detection Applied to Marine Data for Utilizing Optic-Acoustic Optimization
Highlight: 1 Journal article
Abstract: With the increased focus on man made changes to our planet and wildlife, more and more emphasis is put on sustainable and responsible gathering of resources. In an effort to preserve marine wildlife, the Norwegian government has proclaimed a necessity for creating ecological maps, detailing the presence and amount of wildlife species in Norwegian fjords and oceans.
To this end, a submerged sonar system has been deployed in the Oslo Fjord, gathering vast amounts of marine data. Procuring labeled acoustic data is time consuming and expensive, and analysis is predominantly based on ad hoc mathematical methods that are difficult to verify. It is of interest to determine if a more cost effective labeling procedure can be devised, and if the recent breakthroughs within \gls{ml} enables improvements within classification, compared to classical mathematical methods.
In this thesis the author demonstrates techniques for acquiring and analysing marine data. A procedure for interweaving optic and acoustic data is developed and its validity demonstrated empirically. It is shown that the two data sources can be sufficiently related, spatially and temporally, yielding a rich dataset capable of harnessing the individual strengths of each data source. Deep learning techniques are employed and a \gls{nn} is developed and trained on opti-acoustic data. The results show that supervised classification of unlabeled acoustic data can be performed, utilizing cross-referencing with labeled optic data. The methods were able to correctly classify the presence of fish with an accuracy of 64.8 %, demonstrating a proof of concept.
Amalie Heiberg
Title: COLREG-Compliance for Autonomous Surface Vehicles using Deep Reinforcement Learning
Highlight: 1 Journal article, Another in the preparation
Abstract: Autonomous systems are becoming ubiquitous, and are now also gaining momen- tum within the marine sector. Since the electrification of transport is happening at the same time, the envisioned autonomous vessels promise reduced environ- mental impact, lower costs, and higher efficiency. Although close monitoring is still required to ensure safety, the ultimate goal would be total autonomy. One of the major hurdles is the development of a control system versatile enough to handle all possible weather and encounter situations, that is also robust and reli- able. Additionally, the International Regulations for Preventing Collisions at Sea (COLREGs) must be followed for successful interaction with human sailors. Since the COLREGs were written for the human mind to interpret, they are written in ambiguous prose and therefore not machine readable or verifiable.
Due to these challenges and the wide variety of situations to be tackled, classi- cal model-based approaches prove complicated to implement and computationally heavy. Within the field of artificial intelligence, deep reinforcement learning (DRL) has shown great potential for a wide range of applications. Its model-free and self- learning nature makes it a promising candidate for autonomous vessels. In this thesis, two ways of incorporating the COLREGs into a DRL-based path following and obstacle avoidance system are explored. First, the direct usage of sensor data combined with intuition is looked into. Then, a system based on readily available theory of collision risk is developed.
Both of the approaches provide good results in testing scenarios, adhering to the COLREG rules relevant to a single-agent environment – rules 14-16. This means that in addition to achieving excellent path following and collision avoidance per- formance in the face of static obstacles, the DRL agent adhered the implemented COLREGs in situations where the desired behaviour was clearly defined. In both cases, it was shown that a modular approach to reward function design works well in DRL applications with multiple objectives.
The successful inclusion of key COLREG rules into a well-functioning path fol- lowing and collision avoidance system is testament to the potential of DRL in autonomous vessels.
Thomas Nakken
Title: On the applicability of a perceptually driven generative-adversarial framework for super-resolution of wind fields in complex terrain
Abstract: Great strides have been made in recent years in single-image super-resolution (SISR) tasks by utilizing high-dimensional feature activations from pre-trained fea- ture extractors to introduce a perceptual loss in generative-adversarial networks (GANs). A perceptually driven GAN model was recently proposed for super- resolving wind fields in complex terrain. While the generative model was shown to produce plausible wind fields, no statistical analysis was shown, nor was the perceptual aspect of the model justified for application on wind data.
This thesis investigates the applicability of such a perceptually driven model to super-resolve low-resolution wind fields. An initial stability analysis found that the perceptual loss component for the generative model consistently failed to con- verge. Consequently, an in-depth analysis was performed on the wind data from the perspective of the convolutional feature extractor used to construct this percep- tual loss. Considering that the feature extractor was explicitly trained to classify RGB images, wind datasets were converted into an RGB representation to build an intuition for their equivalent visual complexity. It was hypothesized that the gen- erative model was unable to fully learn the visual characteristics of the presented wind data due to the significant difference between the dataset used to train the feature extractor and the wind data used to train the generative model. Thus, the variations in feature activations are thought to act as a source of noise for the generative model rather than helping it improve the accuracy in its super-resolved wind fields. By setting the model to super-resolve wind data from the top of the domain, it was able to converge as expected. Therefore, it was shown that the perceptual feedback from the feature extractor inhibits the model.
By enforcing an agreement evaluation between the Peak Signal-to-Noise Ratio (PSNR) and the Learned Perceptual Image Patch Similarity (LPIPS) metrics, it was shown that minimizing the perceptual loss is not synonymous with learning the governing equations of airflow in the relevant domain. In conclusion, the pro- posed model was deemed insufficient for the task of super-resolving wind fields in complex terrain.
Tiril Sundby
Title: Geometric Change Detection in Digital Twins
HIghlight: 1 Journal article
Abstract: Digital twins are meant to bridge the gap between real-world physical systems and virtual representations. Both stand-alone and descriptive digital twins incorporate 3D geometric models, which are the physical representations of objects in the digital replica. Digital twin applications are required to rapidly update internal parameters with the evolution of their physical counterpart. Due to an essential need for having high-quality geometric models for accurate physical representations, the storage and bandwidth requirements for storing 3D model information can quickly exceed the available storage and bandwidth capacity. In this work, we demonstrate a novel approach to geometric change detection in a digital twin context. We address the issue through a combined solution of dynamic mode decomposition (DMD) for motion detection, YOLOv5 for object detection, and 3D machine learning for pose estimation. DMD is applied for background subtraction, enabling detection of moving foreground objects in real-time. The video frames containing detected motion are extracted and used as input to the change detection network. The object detection algorithm YOLOv5 is applied to extract the bounding boxes of detected objects in the video frames. Furthermore, we estimate the rotational pose of each object in a 3D pose estimation network. A series of convolutional neural networks (CNNs) conducts feature extraction from images and 3D model shapes. Then, the network outputs the camera orientation’s estimated Euler angles concerning the object in the input image. By only storing data associated with a detected change in pose, we minimize necessary storage and bandwidth requirements while still recreating the 3D scene on demand. Our assessment of the new geometric detection framework shows that the proposed methodology could represent a viable tool in emerging digital twin applications.