Machine learning, and artificial neural networks in particular, have been used for astronomy research for two decades. More recently, deep neural networks (aka “deep learning”) that contain orders of 100 million parameters have been shown remarkable results in various artificial intelligent tasks such as image/speech recognition. At ICRAR we have developed deep learning algorithms and models (e.g. Region-based Convolutional Neural Networks) to detect complex radio sources for the citizen science project Radio Galaxy Zoo (RGZ).
Our current model is trained on a single Pawsey GPU node, requiring on average 18 ~ 36 hours of training time. Shortening this time will enable radio astronomers to explore alternative approaches (different network topology, pretrained models, etc), selectively ingesting various prior assumptions, and test many similar models for optimal performance. That’s why it’s crucial to enable our deep learning algorithms to either intelligently compress input data without compromising detection rate and/or utilise multiple GPUs across multiple machines for faster training, thus rapidly reducing the turnaround time for much needed productivity.
However, naively parallellising training workload through simple data parallelism will compromise training performance due to lack of inter-worker communication for global model synchronisation. On the other hand, a full-scale model synchronisation at each iteration will subject the entire training progress to a single “straggler”, defeating the whole purpose of parallelism for decent speedup.
This project aims to bring together techniques from machine learning, computer vision, distributed graph execution and high performance computing in order to solve a practical radio astronomy problem. The solution will be immediately and immensely useful for continuously training machine learning-based detectors, ultimately achieving or even surpassing human-level recognition capabilities.
This project presents a great opportunity for students to get into the “inner workings” of deep learning through an exciting citizen science project using “beautiful” interferometry data collected from an array of radio dishes quietly sitting in the desert.
Dr O. Ivy Wong
Adjunct Senior Research FellowRead More