The radio environment of the 21 Centimeter Array:RFI detection and mitigation

Detection and mitigation of radio frequency interference(RFI) is the first and also the key step for data processing in radio observations, especially for ongoing low frequency radio experiments towards the detection of the cosmic dawn and epoch of reionization(Eo R). In this paper we demonstrate the technique and efficiency of RFI identification and mitigation for the 21 Centimeter Array(21CMA), a radio interferometer dedicated to the statistical measurement of Eo R. For terrestrial, man-made RFI, we concentrate mainly on a statistical approach by identifying and then excising non-Gaussian signatures, in the sense that the extremely weak cosmic signal is actually buried under thermal and therefore Gaussian noise. We also introduce the so-called visibility correlation coefficient instead of conventional visibility, which allows a further suppression of rapidly time-varying RFI. Finally, we briefly discuss removals of the sky RFI, the leakage of sidelobes from off-field strong radio sources with time-invariant power and a featureless spectrum. It turns out that state of the art technique should allow us to detect and mitigate RFI to a satisfactory level in present low frequency interferometer observations such as those acquired with the 21 CMA, and the accuracy and efficiency can be greatly improved with the employment of low-cost, high-speed computing facilities for data acquisition and processing.     

基于小波变换的射频干扰消除方法的研究

在射电天文观测中,射频干扰(Radio Frequency Interference, RFI)会以多种形式混入望远镜接收系统,给观测带来误判或者降低观测信噪比.近年来国内国际射电天文快速发展,国内国际大型射电望远镜和阵列先后建设,观测灵敏度大为提高,射频干扰的影响尤为突出.随着科技发展和人类活动的加剧,射频干扰日益严重且不可逆转.提出利用2维离散小波变换的方法分析射电天文观测的数据,对望远镜系统输出的时间频率序列进行小波变换,根据小波系数分离出原始信号中各分量,每个分量统计得到相应的阈值,将各分量与阈值相比较识别干扰成分并标记去除.利用该方法对实际观测数据进行了处理,结果表明该方法能够很好地标记并消减干扰信号,且提高了观测的信噪比.     

Detection and mitigation of RFI in SBRS observation data

In view of the inconsistency of channel gains and a large amount of interference noise in Solar Broadband Radio Spectrometer(SBRS) observation data, they will seriously affect the analysis of SBRS data. In this paper, a method of Radio Frequency Interference(RFI) detection and mitigation for SBRS observation data is reported. Firstly, the SBRS observation data are preprocessed, a part of the observation data was selected to calculate the mean and variance to achieve the normalization of the entire observation data, which can avoid the influence of strong noise on the normalization result. Furthermore, we proposed an adaptive threshold RFI detection method based on fusion wavelet transform reconstruction and an RFI elimination method based on neighborhood weighted filling. It is worth mentioning that to detect RFI interference signals of different magnitudes, we adopted an iterative approach to the RFI detection and mitigation process. Through qualitative analysis of real observation data and quantitative analysis of simulated data, it is shown that the method proposed in this paper can effectively eliminate RFI in SBRS observation data, and improve the quality of observation data for further scientific analysis.     

快速射电暴观测数据干扰缓解方法研究

从海量的天文观测数据中快速搜寻罕见的快速射电暴(Fast Radio Burst, FRB)事件, 干扰缓解是其中一项关键而具有挑战的工作. 射频干扰(Radio Frequency Interference, RFI)会淹没真实的天文事件, 还会导致搜寻管线输出大量的假阳性候选体. 由于干扰来源及其种类的复杂性, 目前并没有一种通用的方法可以解决这个问题. 为了降低干扰对FRB观测搜寻的影响, 分析和研究了南山26m射电望远镜L波段观测数据中的干扰情况, 针对主要的窄带干扰和宽带干扰建立了3层次的干扰缓解处理流程, 从而有效缓解了观测数据的干扰污染情况. 将该流程嵌入到FRB色散动态谱搜寻(Dispersed Dynamic Spectra Search, DDSS)管线中, 实验结果表明, 搜寻管线的检测率和检测精度得到了进一步的提高. 该方法为FRB观测数据干扰缓解处理提供了有价值的参考.     

Study of Radio Frequency Interference Mitigation Method Based on Wavelet Transform

In radio astronomy observations, radio frequency interference (RFI) is mixed into the telescope receiving system in various forms. The existence of RFI brings misjudgment to the observation or reduces the observational signal-to-noise ratio. In recent years, the domestic and overseas radio astronomy has developed rapidly. Large-scale radio telescopes and telescope arrays at home and abroad have been constructed successively. Observation sensitivity is greatly improved, and the influence of RFI is particularly prominent. With the development of science and technology and the intensification of human activities, the RFI has become increasingly serious and irreversible. We propose to use the two-dimensional discrete wavelet transform method to analyze the data of radio astronomy observations, and to perform wavelet transform on the time-frequency sequence output by the telescope system. According to the wavelet coefficients, the every component in the original signal is separated, and the corresponding threshold value is obtained by the statistics on each component. Each component is compared with the threshold to identify the interference component, and to mark it for removal. This method is used to process the actual observation data. The results show that this method can well mark and eliminate the interference signal, and improve the observational signal-to-noise ratio.     

Radio frequency self-interference from a data processing centre at a radio telescope site

Apart from externally generated Radio Frequency Interference (RFI), the occurrence of self-interference is a major concern at any modern radio telescope site. Antenna servo motor controllers, data acquisition processors, and fast computing capabilities operate very close to extremely sensitive and wideband radio astronomical receivers. In this paper, we present a set of measurements of the RFI level generated by a cluster of computers that will be installed at the site of the Sardinia Radio Telescope (SRT). The measured levels are compared to Recommendation ITU-R RA.769-2, which gives the threshold levels for interference detrimental to radio astronomy observations. Our analysis shows that, with proper shielding of the noisiest devices, it will be possible to preserve the present excellent RFI conditions of the SRT site.     

The SumThreshold Method for Radio Frequency Interference Detection

Radio frequency interference (RFI) is one of the main challenges in the search of radio targets and their accurate analysis. The efficient RFI detection and mitigation techniques are required in the radio data processing. Existing RFI mitigation algorithms typically fall into three categories: component decomposition methods, threshold-based methods, and machine learning methods. The threshold-based algorithms are widely used in real applications because of its clear principle, simple structure, and easily implementation. Especially, the SumThreshold method is becoming more concerned for its good performance in RFI detection. Therefore, this work investigates the principles and algorithm of SumThreshold, and discusses its characteristics and applicability.     

射频干扰检测的SumThreshold算法

射频干扰(Radio Frequency Interference, RFI)是射电目标搜寻和精确分析研究的关键影响因素,因此RFI检测是相关数据处理中的一个重要环节.已有RFI检测方法可分为成分分解法、阈值分析法、机器学习类方法.从应用广泛性和可解释性方面考虑,阈值分析法最具代表性,特别是SumThreshold是近年备受关注的一个RFI检测方法.从SumThreshold方法的原理、算法设计、优化要点、适用性等方面进行介绍和探讨,以供同行参考.     

Selection of radio astronomical observation sites and its dependence on human generated RFI

We investigate the influence of population density on radio-frequency interference(RFI) affecting radio astronomy.We use a new method to quantify the threshold of population density in order to determine the most suitable lower limit for site selection of a radio quiet zone(RQZ).We found that there is a certain trend in the population density-RFI graph that increases rapidly at lower values and slows down to almost flat at higher values.We use this trend to identify the thresholds for population density that produce RFI.Using this method we found that,for frequencies up to 2.8 GHz,low,medium and high population densities affecting radio astronomy are below 150 ppl km-2,between 150 ppl km-2and 5125 ppl km-2,and above5125 ppl km-2respectively.We also investigate the effect of population density on the environment of RFI in three astronomical windows,namely the deuterium,hydrogen and hydroxyl lines.We find that a polynomial fitting to the population density produces a similar trend,giving similar thresholds for the effect of population density.We then compare our interference values to the standard threshold levels used by the International Telecommunication Union within these astronomical windows.     

RFI detection by automated feature extraction and statistical analysis

In this paper we present an interference detection toolbox consisting of a high dynamic range Digital Fast‐Fourier‐Transform spectrometer (DFFT, based on FPGA‐technology) and data analysis software for automated radio frequency interference (RFI) detection. The DFFT spectrometer allows high speed data storage of spectra on time scales of less than a second. The high dynamic range of the device assures constant calibration even during extremely powerful RFI events. The software uses an algorithm which performs a two‐dimensional baseline fit in the time‐frequency domain, searching automatically for RFI signals superposed on the spectral data. We demonstrate, that the software operates successfully on computer‐generated RFI data as well as on real DFFT data recorded at the Effelsberg 100‐m telescope. At 21‐cm wavelength RFI signals can be identified down to the 4σ _rms level. A statistical analysis of all RFI events detected in our observational data revealed that: (1) mean signal strength is comparable to the astronomical line emission of the Milky Way, (2) interferences are polarised, (3) electronic devices in the neighbourhood of the telescope contribute significantly to the RFI radiation. We also show that the radiometer equation is no longer fulfilled in presence of RFI signals. (© 2007 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

Scalable framework of intelligent RFI flagging for large-scale HI survey data from FAST

Radio frequency interference (RFI) identification is a key step in radio data processing. In order to efficiently process huge volumes of data produced by modern large radio telescopes, such as the Five-hundred-meter Aperture Spherical radio Telescope (FAST), exceptional balance between accuracy and performance (throughput) is required for RFI flagging algorithms. RFI-Net is a single-process RFI identification package based on deep learning technique, and has achieved a higher flagging accuracy than the classical SumThreshold method. In this paper, we present a scalable RFI flagging toolkit, which can drive parallel workflows on multi-CPU and multi-GPU clusters, with RFI-Net as its core detector. It can automatically schedule the workload and aggregate itself after errors according to the running environment. Moreover, its main components are all pluggable, and can be easily customized according to requirements. The experiments with real data of FAST showed that using eight parallel workflows, the toolkit can process sky survey data at a speed of 66.79 GB/h, which means quasi-real-time RFI flagging can be achieved considering the data rate of FAST extragalactic spectral line observations.     

Radio frequency interference mitigation using pseudoinverse learning autoencoders

Radio frequency interference(RFI) is an important challenge in radio astronomy. RFI comes from various sources and increasingly impacts astronomical observation as telescopes become more sensitive. In this study, we propose a fast and effective method for removing RFI in pulsar data. We use pseudo-inverse learning to train a single hidden layer auto-encoder(AE). We demonstrate that the AE can quickly learn the RFI signatures and then remove them from fast-sampled spectra, leaving real pulsar signals. This method has the advantage over traditional threshold-based filter method in that it does not completely remove contaminated channels, which could also contain useful astronomical information.     

大口径射电望远镜电磁兼容评估技术研究

电磁兼容性是设备或系统的重要性能指标, 也是保障系统的工作效能和提高系统可靠性的重要因素. 大口径射电望远镜运行阶段, 台址周围无线电业务及内部潜在的电磁干扰会降低观测系统灵敏度、影响天文观测的质量. 本论文针对拟建的新疆110 m全向可动射电望远镜(Qi Tai raido Telescope, QTT)开展了系统电磁兼容评估技术及控制方法研究, 具有重要的工程应用价值. 首先, 依据现有电波环境测量方法的不足, 深入分析了仪器设备的关键参数配置方法及测量时间计算方法, 采用Y因子法校准测量数据, 提出一种准实时电波环境测量方法. 面向高重复性宽带频谱, 分析了宽带频谱信号和噪声特征, 结合标准差理论, 提出一种基于邻值比较的信噪分离方法, 并采用邻值统计方法优化关键参数, 提高信噪分离精度. 针对QTT台址, 开发了自动化电波环境监测系统, 该系统6 GHz以下频段系统增益大于40 dB, 系统噪声系数小于2 dB, 测量不确定度小于1.49 dB, 具有极高的系统灵敏度和测量精度; 分析了频谱监测数据流, 设计了基于HDF5 (Hierarchical Data Format version 5)的数据存储格式, 开发了自动化电波环境测量和监控软件及数据处理软件. 依据QTT台址长期监测数据, 评估分析了台址电磁环境、主要干扰源特征及其影响. 其次, 提出大口径射电望远镜馈源口面干扰电平限值量化方法, 建立了基于台址地形的电波传播模型, 分析了现有电波传播模型的优缺点及适应性, 结合QTT台址实际地形及地质特征, 采用Longley-Rice和Two-Ray电波传播模型, 预测分析了QTT台址潜在干扰区域电磁干扰达到射电望远镜的电波路径衰减, 结合大口径射电望远镜天线增益量化方法, 提出设备所在位置干扰电平限值量化方法, 运用该方法对QTT台址潜在干扰区域的干扰电平限值进行量化. 依据设备所在位置干扰电平限值, 调研分析了国内外军用、民用电磁兼容测量标准, 结合电磁干扰对射电天文观测的影响, 提出一种大口径射电望远镜电磁兼容控制方法, 解决了现有电波暗室测量系统无法直接测量评估电子设备电磁兼容的问题, 该电磁兼容控制方法计划应用于QTT建设及运行阶段, 确保系统拥有良好的电磁兼容性. 最后, 依据QTT台址潜在干扰区域干扰电平限值, 结合典型电子设备电磁辐射频谱, 分析了QTT电磁兼容设计需求, 提出电磁兼容设计初步方案. 另外, 针对台址建筑设施内的中低电磁辐射干扰源, 提出一种低成本建筑屏蔽方法, 应用于QTT台址现有建筑.     

Radio frequency interference measurements in Indonesia

We report the first measurements of radio frequency spectrum occupancy performed at sites aimed to host the future radio astronomy observatory in Indonesia. The survey is intended to obtain the radio frequency interference (RFI) environment in a spectral range from low frequency 10 MHz up to 8 GHz. The measurements permit the identification of the spectral occupancy over those selected sites in reference to the allocated radio spectrum in Indonesia. The sites are in close proximity to Australia, the future host of Square Kilometre Array (SKA) at low frequency. Therefore, the survey was deliberately made to approximately adhere the SKA protocol for RFI measurements, but with lower sensitivity. The RFI environment at Bosscha Observatory in Lembang was also measured for comparison. Within the sensitivity limit of the measurement equipment, it is found that a location called Fatumonas in the surrounding of Mount Timau in West Timor has very low level of RFI, with a total spectrum occupancy in this measured frequency range being about 1 %, mostly found at low frequency below 20 MHz. More detailed measurements as well as a strategy for a radio quiet zone must be implemented in the near future.     

Fast radio burst search: cross spectrum vs. auto spectrum method

The search for fast radio bursts(FRBs) is a hot topic in current radio astronomy studies. In this work, we carry out a single pulse search with a very long baseline interferometry(VLBI) pulsar observation data set using both auto spectrum and cross spectrum search methods. The cross spectrum method,first proposed in Liu et al., maximizes the signal power by fully utilizing the fringe phase information of the baseline cross spectrum. The auto spectrum search method is based on the popular pulsar software package PRESTO, which extracts single pulses from the auto spectrum of each station. According to our comparison, the cross spectrum method is able to enhance the signal power and therefore extract single pulses from data contaminated by high levels of radio frequency interference(RFI), which makes it possible to carry out a search for FRBs in regular VLBI observations when RFI is present.     

An agile very low frequency radio spectrum explorer

The very low frequency(VLF) regime below 30 MHz in the electromagnetic spectrum has presently been drawing global attention in radio astronomical research due to its potentially significant science outcomes exploring many unknown extragalactic sources,transients,and so on.However,the nontransparency of the Earth's ionosphere,ionospheric distortion and artificial radio frequency interference(RFI) have made it difficult to detect the VLF celestial radio emission with ground-based instruments.A straightforward solution to overcome these problems is a space-based VLF radio telescope,just like the VLF radio instruments onboard the Chang'E-4 spacecraft.But building such a space telescope would be inevitably costly and technically challenging.The alternative approach would be then a ground-based VLF radio telescope.Particularly,in the period of post 2020 when the solar and terrestrial ionospheric activities are expected to be in a 'calm' state,it will provide us a good chance to perform VLF ground-based radio observations.Anticipating such an opportunity,we built an agile VLF radio spectrum explorer co-located with the currently operational Mingantu Spectra Radio Heliograph(MUSER).The instrument includes four antennas operating in the VLF frequency range 1-70 MHz.Along with them,we employ an eight-channel analog and digital receivers to amplify,digitize and process the radio signals received by the antennas.We present in the paper this VLF radio spectrum explorer and the instrument will be useful for celestial studies of VLF radio emissions.     

L-band RFI Measurement and Mitigation at the NSRT Site

近些年，南山台址内部各类电子设备不断引入，此过程忽视了有效的设备管理及电磁防护，且台址周边无线电业务增多，以致电波环境恶化。为改善台址电波环境，采用一种准实时电波环境测量方法测量分析了台址周边瞬态信号的影响；另外，为提高微弱信号检测能力，采用便携式电磁干扰测量系统和26m射电望远镜对台址主要干扰信号特征及来源进行测量分析。依据测量和分析结果，采用屏蔽及滤波技术对望远镜观测室内部主要干扰源进行电磁防护，并针对屏蔽工程的有效性进行测量评估，结果表明，采用的电磁屏蔽措施有效。另外，提出了初步南山无线电宁静区保护办法缓解台址外部电磁干扰。     

Research on Receiver Anti-RFI of Delingha 13.7 m Telescope

In astronomical observations, the radio frequency interference (RFI) will cause pseudo spectra and reduce the reliability and validity of observational data. The RFI mitigation, which includes many technical innovations of devices and the method studies of data processing, aims at reducing the influence of RFI on the radio astronomical observation. Various efforts were made to improve the anti-RFI capability of the multi-beam receiver (Superconducting Spectroscopic Array Receiver, SSAR) of the Delingha 13.7 m telescope. The interference transmission path was analyzed. The concepts of the device RFI direct coupling coefficient and the device RFI system coupling coefficient were proposed. The proportions of interference introduced in the receiver system by the different devices were quantified, and the interference-susceptible devices in the system were located. After the anti-RFI treatment of the interference-susceptible devices, the anti-RFI capability of the receiver system is improved by 30 dB in average, and the astronomical observation efficiency of the telescope is increased by more than 10%.     

德令哈13.7m望远镜接收机抗射频干扰研究

在天文观测中,射频干扰会造成假谱,降低数据的可靠性和有效性.射频干扰消减旨在减少干扰信号对射电天文观测的影响,包含器件方面的技术革新和数据处理领域的方法研究.针对德令哈13.7 m望远镜接收机中频部分引入的射频干扰,通过优化中频器件的抗射频干扰能力,提高了接收机的整体抗射频干扰能力,以主动消除方法来减少射频干扰耦合到接收机内部.分析了接收机干扰的传输路径,提出了器件射频干扰的直接耦合系数和器件射频干扰的系统耦合系数的概念,为定位干扰敏感器件并量化干扰引入比重提供了基础.经过抗射频干扰优化后,接收机抗干扰能力改善30 dB左右,望远镜的天文观测效率提高10%以上.     

Calibrating the Pointing and Optical Parameters of the STEREO Heliospheric Imagers

The Heliospheric Imager (HI) instruments on the Solar TErrestrial RElations Observatory (STEREO) observe solar plasma as it streams out from the Sun and into the heliosphere. The telescopes point off-limb (from about 4° to 90° elongation) and so the Sun is not in the field of view. Hence, the Sun cannot be used to confirm the instrument pointing. Until now, the pointing of the instruments have been calculated using the nominal preflight instrument offsets from the STEREO spacecraft together with the spacecraft attitude data. This paper develops a new method for deriving the instrument pointing solutions, along with other optical parameters, by comparing the locations of stars identified in each HI image with the known star positions predicted from a star catalogue. The pointing and optical parameters are varied in an autonomous manner to minimise the discrepancy between the predicted and observed positions of the stars. This method is applied to all HI observations from the beginning of the mission to the end of April 2008. For the vast majority of images a good attitude solution has been obtained with a mean-squared deviation between the observed and predicted star positions of one image pixel or less. Updated values have been obtained for the instrument offsets relative to the spacecraft, and for the optical parameters of the HI cameras. With this method the HI images can be considered as “self-calibrating,” with the actual instrument offsets calculated as a byproduct. The updated pointing results and their by-products have been implemented in SolarSoft.