ICT Program Charter

The ICRAR Information and Communication Technology Program is one of the four programs forming the ICRAR plan. The program includes three main projects, the Data Intensive Research Pathfinder, the High Performance Computing for Radio Astronomy and the Conceptual Design of the SKA Data System. The ICT program is working as part of a scientific research institute and is primarily supporting the ICRAR lead science programs for ASKAP and the archiving activity for the MWA. The experience gathered during the design and implementation of these activities will be used to design the SKA data system. In addition ICRAR has established a direct collaboration with the LSST project in the US to explore advanced and novel database technologies. More specifically the ICT team is working in specific areas in Data Treatment, Data Storage, Data Presentation, Data Mining and Persistent Archiving as outlined below.

Data treatment is a very wide field, but we are concentrating on implementation of new and optimisation and comparison of existing algorithms for radio data cube stacking, source finding and fast transient detection. In addition we are implementing data simulators in order to be able to test those algorithms.

In order to be able to efficiently store the vast amounts of data coming from ASKAP, the MWA and later also the SKA we are researching compression algorithms, multi-resolution techniques and various data formats. This research is carried out having in mind the features and limitations of the storage hardware and media used in HPC environments.

In this research area we are working on visualisation and dissemination techniques for very big data sets. In particular there is a PhD student working on ultra-scale visualisation technologies for out-of-core visualisation of multi-terabyte data cubes. We are also involved in the International Virtual Observatory Alliance (IVOA) and are committed to provide VO compatible data services for the data sets produced by ICRAR scientists.

The size of the data cubes produced by the SKA pre-cursors and the SKA requires better or new algorithms in order be able to find and quantify features of scientific interest. This might include known objects, but more excitingly completely new discoveries.  Apart from images and image cubes ASKAP, the MWA, LSST and SKA will also produce catalogue data of astronomical sources. In the case of LSST and SKA the number of sources will potentially reach 100 billion or more. In order to perform scientific research on such enormous databases we need to develop novel data mining techniques.

Historically astronomical data has been kept in libraries and archives for centuries and even now astronomers are going back to this historical data to compare the historical data with modern measurements. Many things in the universe are changing quite slowly compared to human life time and even centuries are fairly short, but some effects are starting to be visible .

A view inside the 6500 slot robotic tape library operated operated by iVEC. The StorageTek SL850 library is currently being updated with higher capacity tape drives which will allow for storage of up to 32.5 PB of uncompressed data.

Credit: Prof. Andreas Wicenec, ICRAR