Sub-image data processing in Astro-WISE

Most often, astronomers are interested in a source (e.g., moving, variable, or extreme in some colour index) that lies on a few pixels of an image. However, the classical approach in astronomical data processing is the processing of the entire image or set of images even when the sole source of interest may exist on only a few pixels of one or a few images. This is because pipelines have been written and designed for instruments with fixed detector properties (e.g., image size, calibration frames, overscan regions, etc.). Furthermore, all metadata and processing parameters are based on an instrument or a detector. Accordingly, out of many thousands of images for a survey, this can lead to unnecessary processing of data that is both time-consuming and wasteful. We describe the architecture and an implementation of sub-image processing in Astro-WISE. The architecture enables a user to select, retrieve and process only the relevant pixels in an image where the source exists. We show that lineage data collected during the processing and analysis of datasets can be reused to perform selective reprocessing (at sub-image level) on datasets while the remainder of the dataset is untouched, a difficult process to automate without lineage.     

Theory of interferometric data processing

In this article I discuss methods and issues of interferometric data processing, i.e. how to compute visibilities from interferograms, correct for systematic errors, and calibrate the data. I conclude with some remarks on interferometric imaging, as it can be considered part of data processing, one step before the analysis using physical models of the target observed.     

Galilean satellite mutual occultation data processing

Results of processing seven mutual occultation lightcurves are presented. The lightcurves were obtained using the 60-inch telescope (152 cm) at Mt. Wilson to observe six J1 occulting J2 events and one J3 occulting J2 event. Using a uniformly illuminated disk model, local satellite Jovicentric longitude corrections of 675 ± 150 km, 275 ± 240 km, and 1175 ± 350 km for J1, J2, and J3, respectively, were determined. These corrections enabled the event midpoint times to be computed to ±5sec of the observed midpoint times for all seven events. These longitude corrections have been verified by Pioneer 10 and recent (1973 and 1974) conventional Jovian eclipse observations. A relative J1:J2 out-of-plane error of less than a few hundred kilometers has been indicated; however, it appears that the relative J3:J2 out-of-plane error is larger than 600 km. Deficiencies in both the uniformly illuminated disk model and Sampson's theory of the Galilean satellite motions for the reduction of mutual event data are described.     

Using Java for distributed computing in the Gaia satellite data processing

In recent years Java has matured to a stable easy-to-use language with the flexibility of an interpreter (for reflection etc.) but the performance and type checking of a compiled language. When we started using Java for astronomical applications around 1999 they were the first of their kind in astronomy. Now a great deal of astronomy software is written in Java as are many business applications. We discuss the current environment and trends concerning the language and present an actual example of scientific use of Java for high-performance distributed computing: ESA’s mission Gaia. The Gaia scanning satellite will perform a galactic census of about 1,000 million objects in our galaxy. The Gaia community has chosen to write its processing software in Java. We explore the manifold reasons for choosing Java for this large science collaboration. Gaia processing is numerically complex but highly distributable, some parts being embarrassingly parallel. We describe the Gaia processing architecture and its realisation in Java. We delve into the astrometric solution which is the most advanced and most complex part of the processing. The Gaia simulator is also written in Java and is the most mature code in the system. This has been successfully running since about 2005 on the supercomputer “Marenostrum” in Barcelona. We relate experiences of using Java on a large shared machine. Finally we discuss Java, including some of its problems, for scientific computing.     

Improvement of the numerical lunar ephemeris with laser ranging data

Analysis of lunar laser ranging data is underway at several institutions. We describe here our efforts at improving the numerical ephemeris of Moon, based on over three years' span of data. Orbit generation and correction procedures are discussed briefly. Comparisons of the new ephemeris with observations and with a widely available ephemeris are illustrated. The standard deviation of the observation residuals is 7 m.Communication presented at the conference on Lunar Dynamics and Observational Coordinate Systems held January 15–17, 1973 at the Lunar Science Institute, Houston, Tex., U.S.A.     

Tracing and using data lineage for pipeline processing in Astro-WISE

Most workflow systems that support data provenance primarily focus on tracing lineage of data. Data provenance by data lineage provides the derivation history of data including information about services and input data that contributed to the creation of a data product. We show that tracing lineage by means of full backward chaining not only enables users to share, discover and reuse the data, but also supports scientific data processing through storage, retrieval and (re)processing of digitized scientific data. In this paper, we present Astro-WISE, a distributed system for processing, analyzing and disseminating wide field imaging astronomical data. We show how Astro-WISE traces lineage of data and how it facilitates data processing, retrieval, storage and archiving. Particularly we show how it solves issues related to the changing data items typical for the scientific environment, such as physical changes in calibrations, our insight in these changes and improved methods for deriving results.     

Statistical analysis of the results of champ occultation data processing at Shanghai astronomical observatory

With the development of the CPS (Global Positioning System) it has become possible to retrieve accurate profiles of atmospheric temperature, pressure and moisture from CPS occultation data. Using the inversion module developed in Shanghai Astronomical Observatory, (SHAG), we obtained atmospheric profiles from more than 2700 CHAMP occultation events observed in the period 2002 Aug. 1–17. The retrieved profiles are compared with the data of the European Center for Medium-range Weather Forecasts (ECMWF), and their errors are analyzed. An optimal method of statistical analysis is proposed and applied. The statistical results show that GPS occultation data may contribute valuably to numerical weather forecast and long-term monitoring of the earth's climate.     

Exoplanet Transit Database. Reduction and processing of the photometric data of exoplanet transits

We demonstrate the newly developed resource for exoplanet researchers – The Exoplanet Transit Database. This database is designed to be a web application and it is open for any exoplanet observer. It came on-line in September 2008. The ETD consists of three individual sections. One serves for predictions of the transits, the second one for processing and uploading new data from the observers. We use a simple analytical model of the transit to calculate the central time of transit, its duration and the depth of the transit. These values are then plotted into the observed–computed diagrams (O–C), that represent the last part of the application.     

On the processing and analysis of the data of the CORONAS-F/SPIRIT and other solar experiments

More than 300000 solar images in the extreme ultraviolet and soft X-ray regions were obtained using two telescopes and four spectroheliometers of the CORONAS-F/SPIRIT device from August 2001 to December 2005. Methods for the processing of such data and extracting physical information are presented, taking into account the experience of processing and analysis of other space experiments on solar research. Some results on applications of the considered methods are presented.     

Statistical considerations in the analysis of solar oscillation data by the superposed epoch method

Solar Physics - Although the method of superposed epochs (Chree analysis) has been utilized for seven decades, a procedure to determine the statistical significance of the results has not been...     

Statistical properties of polarized radio continuum emission and effects of data processing

Polarized intensity and polarization angles are calculated from Stokes parameters Q and U in a nonlinear way. The statistical properties of polarized emission hold information about the structure of magnetic fields in a large range of scales, but the contributions of different stages of data processing to the statistical properties should first be understood. We use 1.4 GHz polarization data from the Effelsberg 100‐m telescope of emission in the Galactic plane, near the plane and far out of the plane. We analyze the probability distribution function and the wavelet spectrum of the original maps in Stokes parameters Q, U and corresponding PI. Then we apply absolute calibration (i.e. adding the large‐scale emission to the maps in Q and U), subtraction of polarized sources and subtraction of the positive bias in PI due to noise (“denoising”). We show how each procedure affects the statistical properties of the data. We find a complex behavior of the statistical properties for the different regions analyzed which depends largely on the intensity level of polarized emission. Absolute calibration changes the morphology of the polarized structures. The statistical properties change in a complex way: Compact sources in the field flatten the wavelet spectrum over a substantial range. Adding large‐scale emission does not change the spectral slopes in Q and U at small scales, but changes the PI spectrum in a complex way. “Denoising” significantly changes the p.d.f. of PI and raises the entire spectrum. The final spectra are flat in the Galactic plane due to magnetic structures in the ISM, but steeper at high Galactic latitude and in the anticenter. For a reliable study of the statistical properties of magnetic fields and turbulence in the ISM based on radio polarization observations, absolute calibration and source subtraction are required. (© 2007 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

An algorithm for a near real-time data processing of GPS common-view observations

Near real-time transfer of GPS common-view data is no longer a problem, but near real-time data processing of the data still calls for study, because it is not yet achieved by the usual smoothing and filtering techniques. Based on the characteristics of the GPS common-view data, a Kalman filtering algorithm is designed for estimating the time difference between two sites, while greatly reducing the observational noise. The algorithm is applied to the time difference between the National Time Service Center (NTSC) of China and the Communications Research Laboratory (CRL) of Japan (Over 2000 km apart), and to that between the CRL and the Korean Research Institute of Standards and Sciences (KRIS) (over 1000 km apart). The root mean square errors of the results obtained by the Kalman filter relative to those obtained from the Circular T of BIPM are less than 2.9 ns and 2.6 ns, in the two cases. Further, it is pointed out that, when multi-site data within a common-view network are available we can further improve the accuracy of the time comparisons by indirect observation adjustment. This statement was justified by application to the data from all three stations, i.e. NTSC, CRL, and KRIS.     

New method of analysis for TeV gamma-ray data collected by the TACTIC telescope

We describe a new fractal and wavelet based framework for analysing atmospheric Cerenkov images of extensive air showers. The methodology developed using simulated data has been tested with the data collected from Mrk421 and Crab nebula directions during 10 October 2009–16 May 2010 by the TACTIC telescope. The telescope detected flaring activity in TeV gamma-rays from Mrk421 during 15th-17th February 2010. Hillas parameter analysis of Cerenkov images showed presence of TeV γ-ray signal with statistical significance of 12.5 σ beyond 1TeV energy for the 265 hours data set. Based on a Monte-Carlo simulated database of Cerenkov images recorded by the TACTIC telescope, the wavelet dimension B₆ can be exploited as a useful parameter to segregate gamma-ray initiated events from hadron initiated events with better efficiency. We derive a preliminary value of the relevant significance for gamma initiated events using wavelet parameter B₆ for the 265 hours long observation spell of Mrk421 including the flaring episode during 15th–17th February 2010 and 66 hours data of observations on Crab nebula. Wavelet analysis approach has also been tested on Crab OFF- source data of 27 hours. Application of wavelet parameter B₆ on the recorded Cerenkov images of the TACTIC telescope reveals better results in terms of segregation efficiency and enhancement of the determined value of statistical significance as compared to the Hillas parameters approach.     

Radio frequency self-interference from a data processing centre at a radio telescope site

Apart from externally generated Radio Frequency Interference (RFI), the occurrence of self-interference is a major concern at any modern radio telescope site. Antenna servo motor controllers, data acquisition processors, and fast computing capabilities operate very close to extremely sensitive and wideband radio astronomical receivers. In this paper, we present a set of measurements of the RFI level generated by a cluster of computers that will be installed at the site of the Sardinia Radio Telescope (SRT). The measured levels are compared to Recommendation ITU-R RA.769-2, which gives the threshold levels for interference detrimental to radio astronomy observations. Our analysis shows that, with proper shielding of the noisiest devices, it will be possible to preserve the present excellent RFI conditions of the SRT site.     

The whittaker-vondrak method of data smoothing as a numerical filter

We investigate the properties of the Whittaker-Vondrak method of data smoothing as a numerical filter. We obtain the theoreticlal expression relating the mixing parameter , the period P and the frequency response A, thus verifying an empirical result found by Vondrak, and generalize the expression to other definitions of “smoothness”. Using the noise spectra before and after the filter, we give the formulae for estimating the mean error per unit weight, the mean error of a filtered value, and the autocorrelation in the filtered series.