Big Data in Astronomy

Linghe Kong is currently a Research Professor in Department of Computer Science and Engineering at Shanghai Jiao Tong University and an engineer in the scientific data processing group in SKA China. Before that, he was a postdoctoral researcher at Columbia University and McGill University. He received his Ph.D. degree from Shanghai Jiao Tong University, China, his Masters degree from TELECOM SudParis, France, and his B. E. degree from Xidian University, China. His research interests include big data, Internet of things, and mobile computing systems. He has published more than 60 papers in refereed journals and conferences, such as ACM MobiCom, IEEE INFOCOM, IEEE RTSS, IEEE ICDCS, IEEE TMC, and IEEE TPDS. He serves on the editorial boards of several journals including Springer Telecommunication Systems and KSII Transactions on Internet and Information Systems. He organized several special issues such as in IEEE Communications Magazine and in the Computer Journal. He is a senior member of IEEE. Tian Huang is Research Associate of the Astrophysics Group, Cavendish Lab, University of Cambridge. He takes part in multiple radio telescope array projects and mainly focuses on data preprocessing and quality metrics. In March 2016, he graduated from the School of Microelectronics at Shanghai Jiao Tong University, where he completed his PhD thesis. His main research interest is Data Mining for time series, including time series big data indexing, anomaly detecting, and computer architecture for time series data mining and statistical models for time series data. He has published 9 SCI journal and 18 EI conference papers. He has rich experience on software and hardware co-designing. Yongxin Zhu is a full Professor at Shanghai Advanced Research Institute, Chinese Academy of Sciences (CAS). He is also an Adjunct Professor with the School of Microelectronics at the Shanghai Jiao Tong University (SJTU). He is currently the technical leader of Chinese Consortium of Science Data Processor (SDP) for Square Kilometre Array Telescope. He has published over 130 English journal and conference papers, 40 Chinese journal papers and 20 China patent approvals in the areas of computer architecture, embedded systems, and big data processing. With around 1,000 citations of these works in recent years, he has received recognition in China and Asia with IEEE best paper award, Shanghai innovation award, SJTU Annual Outstanding Teacher Award and Bilingual Teaching Award. To date, he has received around 20 million RMB in grants from various funding agencies and industrial partners in China. Prior to his tenure with CAS and SJTU, he worked as a research fellow with the National University of Singapore in 2002-2005, a senior consultant with S1 Incorporation (inventor of the 1st Internet banking in the world) in 1999-2002 and a teaching assistant with the Department of Computer Science and Engineering, SJTU in 1994-1995. He is a guest editor of Journal of Systems Architecture, senior member of IEEE and China Computer Federation (CCF). He has been also a Visiting Professor with National University of Singapore since 2013. Shenghua Yu is an Associate Professor at National Astronomical Observatories, Chinese Academy of Sciences (CAS). He received his PhD degree in astrophysics from Queen’s University of Belfast in 2012, and worked as a post-doc fellow at University of Western Australia in 2014-2015. His main research interests include gravitational wave astrophysics and astronomy, double compact objects, radio emission from ultracool dwarfs and radiation mechanisms. He has published ~13 SCI journal papers, 3 Chinese journal and conference papers, 2 China patent approvals, and 3 China computer software copyrights in the research areas.

Part A: Fundamentals

Chapter 1: Introduction of Radio Astronomy

Chapter 2: Fundamentals of Big Data in Radio Astronomy

Part B: Big Data Processing

Chapter 3: Pre-processing Pipeline on FPGA

Chapter 4: Real-time stream processing in radio astronomy

Chapter 5: Digitization, Channelization and Packeting

Chapter 6: Processing Data of Correlation on GPU

Chapter 7: Data Calibration for single dish radio telescope

Chapter 8: Imaging Algorithm Optimization for Scale-out Processing

Part C: Computing Technologies

Chapter 9: Execution Framework Technology

Chapter 10: Application Design For Execution Framework

Chapter 11: Heterogeneous Computing Platform for Backend Computing Tasks

Chapter 12: High Performance Computing for Astronomical Big Data

Chapter 13: Spark and Dask Performance Analysis Based on ARL Image Library

Chapter 14: Applications of Artificial Intelligence in Astrnomical Big Data

Part D: Future Developments

Chapter 15: Mapping the Universe with 21cm Observations