Studies of Radio Frequency Interference Detection Methods in Microwave Radiometry DISSERTATION Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the Graduate School of The Ohio State University By Barı¸s G¨ uner, M.S., B.S. ***** The Ohio State University 2009

Dissertation Committee:

Approved by

Professor Joel T. Johnson, Adviser Professor Robert Lee Dr. Inder Jeet Gupta

Adviser Graduate Program in Electrical and Computer Engineering

c Copyright by ° Barı¸s G¨ uner 2009

ABSTRACT

Recent surveys conducted from orbiting radiometers have shown that corruption of radiometric data due to Radio Frequency Interference (RFI) may significantly impact the accuracy of the retrieved environmental data. These findings have sparked an interest in RFI detection and mitigation methods; several future microwave remote sensing satellites, like the SMAP (Soil Moisture Active and Passive) mission of NASA, plan to incorporate RFI mitigation strategies in their design. A digital radiometer with very high temporal and spectral resolution developed at OSU/ESL may be used to address this issue; versions of this radiometer working at L-band and Cband are called L-band/C-band Interference Suppressing Radiometer (LISR/CISR), respectively. Although the high resolution obtained with this radiometer makes its use in a space based system unlikely due to the stringent datarate requirements required in such systems (unless data is further integrated after mitigating RFI onboard or flagging the data), LISR/CISR sensors are very beneficial for learning about time and frequency domain characteristics of the existing RFI environment, for serving as “ground-truthing” devices for other radiometers, and for analyzing the effectiveness of pulse and cross-frequency detection methods against observed RFI sources. In this thesis, results from several groundborne and airborne radiometric campaigns performed using LISR/CISR systems are given. RFI sources observed in these experiments and RFI mitigation methods that use the high resolution obtained via LISR/CISR to remove such sources are described. Effectiveness of RFI mitigation ii

methods against the diverse RFI sources encountered in campaigns is analyzed. The ability of LISR/CISR in eliminating very weak RFI sources even on the order of natural geophysical variations is demonstrated. Comparisons with other radiometers that participated in these campaigns are given when possible. A novel method for RFI detection that uses the Shapiro-Wilk test of normality is also elucidated. Comparisons are made with another method based on the normality of thermal noise, the kurtosis detection technique, for the pulsed sinusoidal RFI case. Results prove that the Shapiro-Wilk technique is a viable alternative for RFI mitigation in actual systems. The thesis is concluded with a theoretical performance comparison against pulsed sinusoidal RFI using the three most commonly used RFI detection methods: Pulse detection, cross-frequency detection, and kurtosis detection. Particular emphasis is given to the cross-frequency detection method. It is shown that cross-frequency detection method provides good detection performance regardless of duty cycle for this important type of RFI.

iii

Dedicated to my parents...

iv

ACKNOWLEDGMENTS

I would like to thank my adviser, Professor Joel T. Johnson, for his constant help and guidance throughout my Ph.D. studies. I also would like to acknowledge Prof. Robert Lee, Dr. Inder J. Gupta, and graduate faculty representative Prof. Timothy Rhodus for participating in my Ph.D. defense committee and for reviewing this thesis. I want to thank all the ESL community as well; especially my friends (in alphabetical order) Yakup Bayram, Metin Demir, Burkay D¨onderici, Feridun G¨ unde¸s, Erdin¸c Ircı, Mustafa Kulo˘glu, G¨okhan Mumcu, Praphun Naenna, Noppasin Niamsuwan, Koray Tap, Salih Yarga, and Mehmet Emre Yavuz. Lastly, to my mom and dad, no amount of thanks would suffice.

v

VITA

April 30, 1980 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Born - Eski¸sehir, Turkey 2002 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.S. Electrical and Electronics Eng., Bilkent University, Turkey 2004 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M.S. Electrical and Electronics Eng., Bilkent University, Turkey 2004-present . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Graduate Research Associate, ElectroScience Laboratory, Electrical and Computer Eng., The Ohio State University

PUBLICATIONS Journal Publications 1. V. B. Ert¨ urk and B. G¨ uner “Analysis of finite arrays of circumferentially oriented printed dipoles on electrically large cylinders,” Microwave and Optical Technology Letters, vol. 42, no. 4, pp. 299 - 304, Aug. 2004. 2. V. B. Ert¨ urk, O. Bakır, R. G. Rojas, and B. G¨ uner “Scan blindness phenomenon in conformal finite phased arrays of printed dipoles,” IEEE Transactions on Antennas and Propagation, vol. 54, no. 6, pp. 1699 - 1708, Jun. 2006. 3. J. T. Johnson, A. J. Gasiewski, B. G¨ uner, G. A. Hampson, S. W. Ellingson, R. Krishnamachari, N. Niamsuwan, E. McIntyre, M. Klein, and V. Y. Leuski “Airborne radio frequency interference studies at C-band using a digital receiver,” IEEE Transactions on Geoscience and Remote Sensing, vol. 44, no. 7, pp. 1974 - 1985, Jul. 2006. 4. B. G¨ uner, J. T. Johnson, and N. Niamsuwan “Time and frequency blanking for radio frequency interference mitigation in microwave radiometry,” IEEE Transactions on Geoscience and Remote Sensing, vol. 45, no. 11, pp. 3672 - 3679, Nov. 2007. vi

Conference Publications 1. V. B. Ert¨ urk and B. G¨ uner, “Finite phased arrays of printed dipoles on large circular cylinders: a comparison with the planar case,” URSI EMTS International Conference on Electromagnetics Theory, vol. 2, pp. 972 - 974, May 2004, Pisa, Italy. 2. B. G¨ uner, V. B. Ert¨ urk, and O. Bakır, “A parametric analysis of finite phased arrays of printed dipoles on large circular cylinders and comparisons with the planar case,” IEEE AP-S International Symposium and USNC/URSI National Radio Science Meeting, vol. 4, pp. 4116-4119, Jun. 2004, Monterey, CA. 3. J. T. Johnson, A. Gasiewski, B. G¨ uner, M. Valerio, and M. Klein, “High altitude measurements of C-band radio frequency interference using a digital receiver,” IEEE International Conference on Geoscience and Remote Sensing, pp. 2301-2304, Jul.Aug. 2006, Denver, CO. 4. N. Niamsuwan, B. G¨ uner, and J. T. Johnson, “Observations of an ARSR system in Canton, MI with the L-band interference suppressing radiometer,” IEEE International Conference on Geoscience and Remote Sensing, pp. 2285-2288, Jul.-Aug. 2006, Denver, CO. 5. J. T. Johnson and B. G¨ uner, “Cross frequency blanking for RFI mitigation: A C-band case study,” IEEE International Conference on Geoscience and Remote Sensing, pp. 2710 - 2713, Jul. 2007, Barcelona, Spain. 6. B. G¨ uner, M. Frankford, and J. T. Johnson, “On the Shapiro-Wilk test for the detection of pulsed sinusoidal radio frequency interference,” IEEE International Conference on Geoscience and Remote Sensing, vol. 2, pp. 157 - 160, Jul. 2008, Boston, MA.

FIELDS OF STUDY Major Field: Electrical and Computer Engineering Studies in: Microwave Remote Sensing Prof. Joel T. Johnson Communications and Signal Processing Assis. Prof. Elif Uysal-Biyikoglu Mathematics Prof. U. Gerlach

vii

TABLE OF CONTENTS

Page Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ii

Dedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

iv

Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

v

Vita . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

vi

List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xii

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xiii

Chapters: 1.

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 1.2

. . . . . . .

1 7 9 10 12 12 14

RFI Observations at L-band: Canton Campaign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

16

1.3 2.

2.1

Passive Microwave Remote Sensing: A Brief Review Introduction to RFI Mitigation . . . . . . . . . . . . 1.2.1 Time-domain detection methods . . . . . . . 1.2.2 Cross-frequency detection . . . . . . . . . . . 1.2.3 Methods based on Gaussianity tests . . . . . 1.2.4 Other RFI detection methods . . . . . . . . . Outline of Thesis . . . . . . . . . . . . . . . . . . . .

Radiometer Front End and Downconversion 2.1.1 Radiometer front end . . . . . . . . 2.1.2 Downconversion stage . . . . . . . . 2.1.3 Physical properties . . . . . . . . . . 2.1.4 Radiometer state information . . . . viii

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . . . .

. . . . .

. . . . . . .

. . . . .

. . . . . . .

. . . . .

. . . . . . .

. . . . .

. . . . . . .

. . . . .

. . . . . . .

. . . . .

. . . . . . .

1

. . . . .

. . . . .

17 17 18 19 21

2.2

. . . . . . . . . . . . . . . . . . . . . . .

22 22 23 23 24 26 27 27 30 32 32 38 39 49 56 62 63 63 70 73 73 76 77

L-band Interference Suppressing Radiometer (LISR) Ground-based Observations with JPL’s Passive-Active L/S Band (PALS) Sensor . . . . .

81

2.3

2.4

2.5

2.6

2.7 3.

3.1

3.2

3.3

A Brief Description of LISR . . . . . . . . . . . 2.2.1 LISR schematic . . . . . . . . . . . . . . 2.2.2 Capture mode . . . . . . . . . . . . . . 2.2.3 APB . . . . . . . . . . . . . . . . . . . . 2.2.4 Integrating modes . . . . . . . . . . . . 2.2.5 Computer interface . . . . . . . . . . . . Hardware Issues . . . . . . . . . . . . . . . . . 2.3.1 Observed gain pattern . . . . . . . . . . 2.3.2 Reference load switch issues . . . . . . . 2.3.3 Noise diode on/off switch issues . . . . . 2.3.4 Calibration . . . . . . . . . . . . . . . . Summary of LISR Observations on June 17th . 2.4.1 Integration, APB off . . . . . . . . . . . 2.4.2 Integration, APB on . . . . . . . . . . . 2.4.3 Capture mode data . . . . . . . . . . . . 2.4.4 Post-processing . . . . . . . . . . . . . . Observations Near 14:51 and 16:03 UTC . . . . 2.5.1 Observations near 14:51 UTC . . . . . . 2.5.2 Observations near 16:03 UTC . . . . . . Laboratory Observations on June 20th and 21st 2.6.1 June 20th data . . . . . . . . . . . . . . 2.6.2 June 21st data . . . . . . . . . . . . . . Summary and Remarks . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . .

System Configuration . . . . . . . . . . . . . . . . . . . . . . . . . 82 3.1.1 PALS radiometer front end and downconversion stages . . . 82 3.1.2 PALS state timing . . . . . . . . . . . . . . . . . . . . . . . 84 3.1.3 LISR overview . . . . . . . . . . . . . . . . . . . . . . . . . 86 3.1.4 LISR computer and control interface . . . . . . . . . . . . . 88 Dataset Overview and Calibration . . . . . . . . . . . . . . . . . . 90 3.2.1 Observed gain pattern . . . . . . . . . . . . . . . . . . . . . 90 3.2.2 Noise diode delay . . . . . . . . . . . . . . . . . . . . . . . . 93 3.2.3 LISR recorded datasets . . . . . . . . . . . . . . . . . . . . 95 3.2.4 LO tuning tests and 1390 MHz interference . . . . . . . . . 97 3.2.5 PALS state classification . . . . . . . . . . . . . . . . . . . . 97 3.2.6 LISR calibration . . . . . . . . . . . . . . . . . . . . . . . . 102 LISR Overnight Observations . . . . . . . . . . . . . . . . . . . . . 103 3.3.1 Horizontally polarized brightnesses during the night of April 24th-25th . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

ix

3.3.2

3.4

3.5 4.

107 109 113 115 119 119 124 125

Airborne C-band RFI Measurements with PSR/CXI and CISR from the WB-57 aircraft . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 4.1

4.2 4.3

4.4 4.5 5.

Vertically polarized brightnesses during the night of April 24th-25th . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.3 Overnight observations on April 25th-April 26th . . . . . . 3.3.4 Overnight observations on April 27th-April 28th . . . . . . 3.3.5 Overnight observations on May 10th-May 11th . . . . . . . RFI Detection and Mitigation . . . . . . . . . . . . . . . . . . . . . 3.4.1 Properties of observed RFI . . . . . . . . . . . . . . . . . . 3.4.2 Time domain RFI mitigation . . . . . . . . . . . . . . . . . Summary and Remarks . . . . . . . . . . . . . . . . . . . . . . . .

Instrumentation . . . . . . . . . . . . . . . . . . . 4.1.1 PSR/CXI . . . . . . . . . . . . . . . . . . . 4.1.2 CISR . . . . . . . . . . . . . . . . . . . . . 4.1.3 Interface between PSR/CXI and CISR . . . 4.1.4 CISR modifications for the WB-57 aircraft . 4.1.5 ADD system . . . . . . . . . . . . . . . . . 4.1.6 Measurement process . . . . . . . . . . . . . Experiment Conditions . . . . . . . . . . . . . . . . Comparisons with PSR . . . . . . . . . . . . . . . 4.3.1 PSR scan images . . . . . . . . . . . . . . . 4.3.2 CISR observations near DFW . . . . . . . . 4.3.3 CISR observations in more rural Texas . . . 4.3.4 CISR observations over the Gulf of Mexico . Comparisons with ADD . . . . . . . . . . . . . . . Summary and Remarks . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

128 128 130 131 133 136 137 140 144 144 154 155 157 158 161

A Study of the Shapiro-Wilk Test for the detection of Pulsed Sinusoidal RFI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 5.1 5.2

5.3

5.4

Introduction . . . . . . . . . . . . . . . . . . The Shapiro-Wilk test . . . . . . . . . . . . . 5.2.1 Expressions for the ai coefficients . . . 5.2.2 Implementation in digital hardware . . 5.2.3 Quantization effects . . . . . . . . . . Simulation Procedure . . . . . . . . . . . . . 5.3.1 Signal model and notations . . . . . . 5.3.2 Cases considered . . . . . . . . . . . . 5.3.3 Computation of the kurtosis statistics Results . . . . . . . . . . . . . . . . . . . . . 5.4.1 Histograms of W . . . . . . . . . . . . x

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

166 168 169 170 171 171 173 175 176 177 177

5.4.2 5.4.3 5.5 6.

Performance Study of a Cross-Frequency Detection Algorithm for Pulsed Sinusoidal RFI in Microwave Radiometry . . . . . . . . . . . . . . . . . . 188 6.1 6.2

6.3

6.4 6.5 7.

Receiver operating characteristic curves . . . . . . . . . . . 179 Comparison of ROC curves for the Shapiro-Wilk and kurtosis tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 Summary and Remarks . . . . . . . . . . . . . . . . . . . . . . . . 185

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.1 Signal model and notations . . . . . . . . . . . . . . . . 6.2.2 Cross-frequency detection model . . . . . . . . . . . . . 6.2.3 Pulse detection model . . . . . . . . . . . . . . . . . . . 6.2.4 Kurtosis detection model . . . . . . . . . . . . . . . . . Detection Performance Results . . . . . . . . . . . . . . . . . . 6.3.1 Effects of scalloping loss . . . . . . . . . . . . . . . . . . 6.3.2 A simplified method for the calculation of scalloping loss System Temperature Estimation Issues . . . . . . . . . . . . . . Summary and Remarks . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

188 190 190 191 194 196 198 202 205 207 213

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

xi

LIST OF TABLES

Table 2.1

Page

Summary of Integration, APB off observations on June 17th. Sample numbers refer to the vertical axis of Figures 2.11 and 2.12. LO frequency is 1386.5 MHz unless otherwise indicated. H- and V-pol states are as follows: A=antenna, T=terminated, and a number indicates an amount of attenuation in dB added between the antenna output and radiometer receiver input. . . . . . . . . . . . . . . . . . . . . . . . .

40

Summary of Integration, APB on observations on June 17th. Sample numbers refer to the vertical axis of Figures 2.16 and 2.17. LO frequency is 1386.5 MHz unless otherwise indicated. H- and V-pol states are as follows: A=antenna, T=terminated, and a number indicates an amount of attenuation in dB added between the antenna output and radiometer receiver input. . . . . . . . . . . . . . . . . . . . . . . . .

50

Summary of Capture mode observations on June 17th. LO frequency is 1386.5 MHz unless otherwise indicated. H- and V-pol states are as follows: A=antenna, T=terminated, and a number indicates an amount of attenuation in dB added between the antenna output and radiometer receiver input. . . . . . . . . . . . . . . . . . . . . . . . .

57

3.1

Summary of the data recorded in the experiment . . . . . . . . . . .

96

4.1

ADD subchannel frequencies within the 75-175 MHz IF band . . . . . 136

4.2

Time history of flight on August 25th, 2005 . . . . . . . . . . . . . . 143

4.3

Statistics from PSR four sub-band interference suppression algorithm: 21600 pixels over rural Texas (18:08-18:23 UTC) . . . . . . . . . . . . 150

4.4

Statistics from PSR four sub-band interference suppression algorithm: 21600 pixels near DFW (17:54-18:08 UTC) . . . . . . . . . . . . . . . 150

2.2

2.3

xii

LIST OF FIGURES

Figure

Page

2.1

Truck, boom and horn antenna used in Canton campaign . . . . . . .

20

2.2

Block diagram of LISR . . . . . . . . . . . . . . . . . . . . . . . . . .

22

2.3

Basic operation procedure of APB . . . . . . . . . . . . . . . . . . . .

25

2.4

Normalized LISR raw data for reference load observations versus frequency for H-pol and V-pol. Black horizontal line marks approximate 3 dB point, while red vertical lines mark boundaries of protected spectrum. 28

2.5

Mean LISR raw power in 1399-1428 MHz as a function of delay after state change. (a) Antenna states (b) Reference load states . . . . . .

31

Mean LISR raw power in 1399-1428 MHz as a function of delay after state change. (a) Noise diode plus antenna states (b) Noise diode plus reference load states . . . . . . . . . . . . . . . . . . . . . . . . . . .

33

Mean LISR brightness 1397-1430 MHz for H- and V-pol observations of a controlled target; H pol observations are shown inside boxes. Horizontal lines mark expected brightnesses when target is viewed through 0, 1, 2, and 3 dB pads, respectively. . . . . . . . . . . . . . . . . . .

35

Mean LISR brightness 1406-1412 MHz for V-pol observations of a terminator (near 14:00 UTC) . . . . . . . . . . . . . . . . . . . . . . . .

36

Calibrated LISR brightnesses versus frequency during sky observations near 19:25 UTC on June 20th. . . . . . . . . . . . . . . . . . . . . . .

37

2.10 Location of the truck (circled in red) with respect to the ARSR. . . .

38

2.6

2.7

2.8

2.9

xiii

2.11 Calibrated V-pol LISR data in Integration, APB off mode, versus RF frequency. Refer to Table 2.1 for information on the vertical axis of the plot. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

41

2.12 Same as Figure 2.11, but for H-pol.

. . . . . . . . . . . . . . . . . .

42

2.13 Calibrated V-pol LISR data in Integration, APB off mode, versus RF frequency, at 1.3 msec time resolution. Data acquired near time 14:52:15 UTC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

45

2.14 Same as Figure 2.13, but for H-pol.

. . . . . . . . . . . . . . . . . .

46

2.15 Calibrated brightness 1399-1428 MHz for horizontally polarized observations including additional attenuation. A 21 dB attenuator was used at times less than 19:24 UTC, and a 41 dB attenuator was used at times greater than 19:34 UTC. Between these two times at 71 dB attenuator was used. . . . . . . . . . . . . . . . . . . . . . . . . . . .

48

2.16 Calibrated V-pol LISR data in Integration, APB on mode, versus RF frequency. Refer to Table 2.2 for information on the vertical axis of the plot. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

52

2.17 Same as Figure 2.16, but for H-pol.

. . . . . . . . . . . . . . . . . .

53

2.18 Calibrated V-pol LISR data in the band 1399-1428 MHz versus time, for both APB off and APB on modes. Text in the plot indicates the azimuth angle of the radiometer antenna, with regions between the vertical lines indicating periods of antenna rotation. . . . . . . . . .

54

2.19 Same as Figure 2.18, but for H-pol.

. . . . . . . . . . . . . . . . . .

55

2.20 Percent of samples blanked by the APB processor . . . . . . . . . . .

56

2.21 Example radar pulse observed near time 14:58:42; amplitude level of V pol data is shifted by 2500 for clarity purposes . . . . . . . . . . .

58

2.22 Thirty radar pulses obtained during a sweep over azimuth near time 18:39 UTC in horizontal polarization. The vertical axis label indicates the approximate azimuth angle of each of three sets of captures. . .

60

xiv

2.23 Maximum capture raw-data amplitudes observed (following integration to 1.28 µsec resolution) in horizontal polarization during sweeps over azimuth from 18:34 to 18:44 UTC. Reference data is interpolated in the intervals with no measurements. Refer to the text for information on the map from time to azimuthal observation angle. . . . . . . . .

61

2.24 Calibrated V-pol LISR data in Integration, APB off mode, versus RF frequency, near times 14:51:43 to 14:56:15 UTC (Samples 312-468 from Table 2.1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

65

2.25 Same as Figure 2.24, but following post-processing described in Section 2.4.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

66

2.26 Calibrated LISR data in channels similar to the UM ADD system, near times 14:51:43 to 14:56:15 UTC (Samples 312-468 from Table 2.1); results included before and after post-processing . . . . . . . . .

67

2.27 Calibrated V-pol LISR data in Integration, APB on mode, versus RF frequency, near times 14:56:43 to 14:58:18 (Samples 363-416 from Table 2.2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

68

2.28 Calibrated LISR data in channels similar to the UM ADD system, near times 14:56:43 to 14:58:18 (APB on, Samples 363-416 from Table 2.2); results included before and after post-processing . . . . . . . . . . .

69

2.29 Calibrated LISR data in channels similar to the UM ADD system, near times 16:03:41 to 16:04:14 UTC (Samples 1358-1379 from Table 2.1); results included before and after post-processing . . . . . . . . . . .

71

2.30 Calibrated LISR data in channels similar to the UM ADD system, near times 16:05:32 to 16:06:06 (APB on, Samples 613-636 from Table 2.2); results included before and after post-processing . . . . . . . . . . .

72

2.31 Calibrated H-pol LISR data in Integration, APB off mode, versus RF frequency, on June 20th . . . . . . . . . . . . . . . . . . . . . . . . .

74

2.32 Same as Figure 2.31 but in Integration, APB on mode . . . . . . . .

75

2.33 Calibrated V-pol LISR data in Integration, APB off mode, versus RF frequency, on June 21st . . . . . . . . . . . . . . . . . . . . . . . . .

78

2.34 Same as Figure 2.33 but for H-pol in Integration, APB on mode . . .

79

xv

3.1

PALS state diagram . . . . . . . . . . . . . . . . . . . . . . . . . . .

85

3.2

LISR block diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . .

86

3.3

LISR measured reference load raw power versus frequency, vertical polarization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

91

LISR measured reference load raw power versus frequency, horizontal polarization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

92

3.5

Raw power vs. time in the Ant+ND state for the capture data . . . .

94

3.6

Calibrated brightness temperatures vs. IF frequency as PALS LO tunes in horizontal polarization . . . . . . . . . . . . . . . . . . . . .

98

3.4

3.7

Average raw power levels for a single file for the night of April 27th . 100

3.8

Calibrated H-pol brightness temperature vs. RF frequency, April 25th 104

3.9

Comparison of calibrated brightnesses for PALS and LISR, April 25th 106

3.10 Calibrated V-pol brightness temperature vs. RF frequency, April 25th 108 3.11 Change in the passband properties of reference looks on April 25th . 109 3.12 Comparison of calibrated brightnesses for PALS and LISR, April 26th 110 3.13 Calibrated H-pol brightness temperature vs. RF frequency, April 26th 111 3.14 Calibrated V-pol brightness temperature vs. RF frequency, April 26th 112 3.15 Calibrated brightness temperatures vs. time for ADD, LISR and PALS on April 28th . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 3.16 Calibrated H-pol brightness temperature vs. RF frequency, April 28th 115 3.17 Calibrated V-pol brightness temperature vs. RF frequency, April 28th 116 3.18 Calibrated brightness temperatures vs. time for LISR and PALS on May 11th . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 3.19 Calibrated H-pol brightness temperature vs. RF frequency, May 11th

xvi

117

3.20 Calibrated V-pol brightness temperature vs. RF frequency, May 11th

118

3.21 Calibrated brightness temperatures vs. time for ADD, LISR and PALS sensors on April 28th between 4 and 6 am UTC time . . . . . . . . . 120 3.22 Image of pulsed interferers for V-pol, April 28th . . . . . . . . . . . . 122 3.23 Image of pulsed interferers for H-pol, April 28th . . . . . . . . . . . . 123 3.24 H-pol and V-pol total channel calibrated brightnesses before and after time domain RFI mitigation, April 28th . . . . . . . . . . . . . . . . 125 4.1

WB-57 high-altitude aircraft . . . . . . . . . . . . . . . . . . . . . . . 128

4.2

Simplified schematic of PSR/CXI . . . . . . . . . . . . . . . . . . . . 130

4.3

Simplified schematic of CISR

4.4

Photograph of the CISR enclosure installed on the WB-57 aircraft

4.5

Flight path, including nearby Texas cities (circles)

4.6

Altitude of WB-57 aircraft versus time . . . . . . . . . . . . . . . . . 142

4.7

Locations of C-band RFI sources within the JSC source database. Flight path is indicated by the thick red line. . . . . . . . . . . . . . . 145

4.8

Distribution of source center frequencies for the JSC database records plotted in Figure 4.7. Note that records centered between 5.9-6.2 GHz are described as “not included” in the database. . . . . . . . . . . . . 146

4.9

Calibrated brightnesses from PSR conical scans of the entire flight, in all four PSR main channels . . . . . . . . . . . . . . . . . . . . . . . 148

. . . . . . . . . . . . . . . . . . . . . . 132 . 134

. . . . . . . . . . 141

4.10 Average of Figure 4.9 over scan angle, versus time . . . . . . . . . . . 149 4.11 Comparison of calibrated brightness temperatures vs. time between original and mitigated data for CISR channels 8,12,16 and 20 . . . . . 153 4.12 Spectrogram images and time history of brightness temperatures for original and mitigated data over an urban landscape, CISR channel 16 156

xvii

4.13 Spectrogram images and time history of brightness temperatures for original and mitigated data over a rural landscape, CISR channel 17 . 157 4.14 Spectrogram images and time history of brightness temperatures for original and mitigated data over the Gulf of Mexico, CISR channel 8

158

4.15 Time history of calibrated brightnesses for ADD and CISR, for ADD subchannel 6 and tuned PSR channels 12 to 15 . . . . . . . . . . . . 160 4.16 Scatter plot for calibrated brightnesses of ADD vs. CISR, for ADD subchannel 6 and tuned PSR channels 12 to 15 . . . . . . . . . . . . 162 4.17 Scatter plot after the filtering on calibrated CISR data, for ADD subchannel 6 and tuned PSR channels 12 to 15 . . . . . . . . . . . . . . 163 4.18 Comparison of brightness temperature spectrograms for CISR (left) and ADD (right), tuned PSR channels 12 to 15 . . . . . . . . . . . . 164 5.1

Weight coefficients ai before and after quantization using 8-bit resolution172

5.2

Histograms (scaled to correspond to probability density functions) of W for non-quantized (upper Figure) and quantized data (G = 4, lower Figure), d ≈ 1%. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

5.3

ROC curves for d ≈ 1% . . . . . . . . . . . . . . . . . . . . . . . . . . 180

5.4

ROC curves for d = 50% . . . . . . . . . . . . . . . . . . . . . . . . . 181

5.5

ROC curves for d = 100% . . . . . . . . . . . . . . . . . . . . . . . . 182

5.6

ROC curves vs. test size N , R=2.5, d ≈ 1% . . . . . . . . . . . . . . 183

5.7

ROC curves vs. test size N , R=5, d = 50% . . . . . . . . . . . . . . . 183

5.8

Comparison of ROC curves for Shapiro-Wilk and Kurtosis tests, d ≈ 1%184

5.9

Comparison of ROC curves for Shapiro-Wilk and Kurtosis tests, d = 50%185

5.10 Comparison of ROC curves for Shapiro-Wilk and Kurtosis tests, d = 100% . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 6.1

Schematic of the cross-frequency detector . . . . . . . . . . . . . . . . 192

xviii

6.2

Schematic of the pulse detector . . . . . . . . . . . . . . . . . . . . . 195

6.3

Schematic of the kurtosis detector . . . . . . . . . . . . . . . . . . . . 197

6.4

ROC curves for the cross-frequency, pulse, and subsampled kurtosis algorithms: CW RFI, 8 channels (N = 16 for the pulse detector). R = 1, R = 2 and R = 3 cases shown for the cross-frequency case, R = 3 only for the pulse and kurtosis cases. . . . . . . . . . . . . . . 199

6.5

Same as Figure 6.4, but for RFI with 50% duty cycle. . . . . . . . . . 200

6.6

ROC curves for the cross-frequency, pulse, and subsampled kurtosis algorithms: 0.1% duty cycle, R = 1, 8 channels (N = 16 for the pulse detector). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

6.7

AUC image for the cross-frequency detector vs. RFI strength and duty cycle, four channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

6.8

AUC image comparison for the cross-frequency (upper), subsampled kurtosis (middle) and pulse (lower) algorithms vs. RFI strength and duty cycle, four channels/N = 8 . . . . . . . . . . . . . . . . . . . . . 203

6.9

AUC image comparison for the cross-frequency (upper), subsampled kurtosis (middle) and pulse (lower) algorithms vs. RFI strength and duty cycle, sixteen channels/N = 32 . . . . . . . . . . . . . . . . . . . 204

6.10 Probability of detection vs. RFI strength for a constant Pf a of 1%, CW RFI. Results when RFI is assumed to be centered in a channel (“bin centered”) are compared with the general case of random RFI frequency (“with scalloping loss”) for 8 and 16 frequency channels. . 205 6.11 AUC image comparison for the cross-frequency algorithm with (upper) and without (lower) scalloping loss vs. RFI strength and duty cycle, sixteen channels/N = 32 . . . . . . . . . . . . . . . . . . . . . . . . . 206 6.12 Probability of detection vs. RFI strength for a constant Pf a of 1%, CW RFI. Results obtained with an approximation to the scalloping loss is compared with the complete solution for 8 and 16 frequency channels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208 6.13 Mean and standard deviation of the system temperature estimate as channels are discarded in the threshold estimation procedure of the cross-frequency algorithm, for 4, 8, and 16 frequency channels. . . . 210 xix

6.14 ROC curves obtained using the analytical approximation to threshold estimation effects compared with results from Monte Carlo simulations. CW RFI having R = 3, Mdrop = 2, and for 4,8 and 16 channels. . . . 212 6.15 Results of Figure 6.10 are compared with curves including system temperature estimation effects. CW RFI . . . . . . . . . . . . . . . . . . 213

xx

CHAPTER 1

INTRODUCTION

This thesis details studies of the Radio Frequency Interference (RFI) environment and RFI detection/mitigation techniques for earth observing microwave radiometers. The aim of this chapter is introducing the reader to some fundamental concepts of microwave radiometry, providing the motivation behind this research, describing the relevant studies in the literature, and providing an outline for the rest of the thesis. The next section explains the basic theory behind passive microwave remote sensing. The benefits of microwave remote sensing are described in this section, and a history of earth observing passive microwave remote sensing systems in space is given. In Section 1.2, Radio Frequency Interference is defined. Results from surveys of RFI at microwave frequencies are given as a motivation for the research on RFI mitigation techniques. A literature overview of RFI mitigation methods in passive microwave radiometry is presented. The chapter concludes by giving a basic outline of the rest of the thesis.

1.1

Passive Microwave Remote Sensing: A Brief Review

Radiometers are passive remote sensing systems that measure the natural radiation emitted by objects [1]. If an object is in thermal equilibrium with its surroundings, it emits all the energy that it absorbs. A perfect absorber that does not reflect 1

any energy is known as a “blackbody”. If an ideal receiver of bandwidth B were to measure the power radiated by a blackbody, the received power would be equal to:

P = kT B

(1.1)

where k is Boltzmann’s constant and T is the physical temperature of the blackbody in Kelvin. On the other hand, radiated power by a non-absorbing material (i.e. a perfect conductor) would be zero. The materials in real life fall in between these two extremes; hence, the power radiated by them is bounded by 0 and kT B at a certain temperature T. Therefore, if P is the power radiated by the object and B is the bandwidth of the receiver again, an equivalent temperature called the “brightness temperature” (TB ) can be defined as:

TB =

P kB

(1.2)

If external calibration techniques are not present, the power measured by a receiver is also affected by its antenna’s radiation pattern. In that case, the ratio measured in Equation 1.2 is called the “antenna temperature” rather than the brightness temperature. The ratio of the brightness temperature of an object to its physical temperature is called the “emissivity” (e) which has a value between 0 for a non-absorbing medium and 1 for a blackbody:

e=

2

TB T

(1.3)

The emissivity of the scene observed is related to the “reflectivity” of the media. Assuming the surface observed is flat and homogeneous, the emissivity can be written in terms of the so-called “specular” reflectivity (Γsp ) as:

e(f, θ, p) = 1 − Γsp (f, θ, pol)

(1.4)

Γsp (f, θ, p) = |R(f, θ, pol)|2

(1.5)

where

In this equation, R(f, θ, pol) is the Fresnel reflection coefficient and depends on the operation frequency f , incidence angle θ and polarization pol. The Fresnel reflection coefficient is related to the dielectric constant of the medium, which is in turn affected by medium’s physical properties like its physical composition and temperature. In most cases, it is possible to establish a model of the relationship between these properties of a medium and its dielectric constant. For rough surfaces, Equation 1.4 is no longer exact. If the effect of the roughness is significant, an empirical relationship that takes the roughness of the surface into account can be developed [2]. Therefore, environmental parameters that do not directly alter the dielectric constant but modify the emissivity like wind vector over oceans can also be recovered by microwave remote sensing. As it may be evident from the previous paragraph, the main use of microwave remote sensing is in geoscientific fields like hydrology, meteorology, climatology and agriculture [1]. Radiometers are also widely used in extraterrestrial observations and have some military applications. Note that radiometric systems that observe earth from space for geoscientific purposes can further be divided into two main categories [3]: 3

• Atmospheric sounders that are used to measure vertical profiles of environmental parameters like atmospheric temperature and pressure using resonances of molecules that make up the atmosphere. • Surface sensors that operate on frequency windows where the atmospheric absorption is low; these sensors can measure surface variables like soil moisture, water-ice boundary in polar regions, sea surface temperature and salinity. Focus of this research is on the effects of radio frequency interference on passive microwave remote sensing of environmental parameters; therefore, we are primarily interested in surface sensors (in Chapters 2 and 3, sky observations will also be described). However, the principles and conclusions of this research is valid for all other radiometer types as well. Obviously, microwave remote sensing is not the only way to measure environmental parameters. For example, sea surface temperature profiles can be obtained with a “brute-force” method of direct measurements. It is also possible to make passive remote sensing measurements at other frequencies like infrared and visible portions of the spectrum. Thus, it would be beneficial to discuss the benefits of passive microwave remote sensing at this point. Possible advantages of passive remote sensing at microwave frequencies over other measurement methods are listed below [1]: • Sensitivity of brightness temperature to an environmental parameter changes with frequency. As a result, information obtained from the observed scene is not the same at microwave and infrared or visible frequencies. Even at microwave frequencies, dominant physical factor affecting emission can change with a slight change in frequency.

4

• Microwave remote sensing systems do not require a source of illumination like Sun in contrast to the observations at the visible spectrum. • Detrimental effects of clouds and rain are also much less pronounced at microwave frequencies compared to the visible and infrared parts of the spectrum. • Microwaves can penetrate vegetation or even the ground itself. • Space observations can provide accurate and up-to-date information from a great portion of the earth surface. On the other hand, if we consider the sea surface temperature example, a brute-force approach to the same measurements would require enormous amount of resources. The realization of these advantages can be dated back to the ground based measurements in the 1940s by Dicke et al. [4]. However, more than 20 years had to pass before the launch of the first passive microwave remote sensing radiometer that observed earth from space. This radiometer, launched in 1968, was onboard the Soviet Satellite Cosmos 243 and it measured atmospheric water vapor, water-ice content, sea surface temperature and ice temperature [5]. Further account of passive microwave remote sensing of earth from space upto the early 1980s can be found in Reference [3]. Scanning Multichannel Radiometer (SMMR) is another important turning point for the microwave remote sensing history. This device was operational in the first half of the 1980s [6] and it was capable of sea surface temperature and soil moisture measurements. SMMR system could perform dual (V- and H-) polarization measurements at 6.6, 10.69, 18, 21 and 37 GHz. This instrument was mounted on two different satellites: Nimbus G satellite which had a spatial resolution of 148x95 km at 6.6 GHz, and Seasat A whose resolution was 121x79 km at the same frequency.

5

Recent radiometric campaigns include Advanced Scanning Radiometer (AMSR), Advanced Scanning Radiometer for Earth Observing System (AMSR-E), and WindSat missions. The AMSR-E radiometer was a slightly modified version of the Japanese AMSR radiometer onboard the Japanese ADEOS-2 satellite [7], and it was carried on as a joint project between NASA and Japan Aerospace Exploration Agency (JAXA). The spatial resolution of the AMSR system was around 50 km [8], which doubled the resolution of the SMMR instrument. Just like SMMR, AMSR and AMSR-E sensors which were launched in 2001 and 2002, respectively, are multichannel instruments. The operating frequencies for the AMSR-E channels are 6.925, 10.65, 18.7, 23.8, 36.5 and 89 GHz; AMSR has two additional channels centered at 50.3 and 52.8 GHz in addition to the channels of AMSR. These instruments were used for the measurement of water vapor, precipitation, sea surface wind speed, sea surface temperature and soil moisture. WindSat polarimetric radiometer was developed for the U.S. Navy and the National Polar-orbiting Operational Environmental Satellite System (NPOESS) Integrated Program Office (IPO) by the Naval Research Laboratory Remote Sensing Division and the Naval Center for Space Technology [9]. WindSat was launched in 2003 and it was the first radiometer designed specifically for wind vector retrieval. However, it is also capable of sea surface temperature, precipitation, cloud water content and rain rate measurements. NASA is preparing to launch a satellite remote sensing mission to measure sea surface salinity at L-band called Aquarius [10] in 2010. European Space Agency (ESA) has planned a similar mission called Soil Moisture and Ocean Salinity (SMOS) mission [11] in which observations over land will be used to determine soil moisture and the sea observations will be used for predicting the sea surface salinity. Another mission planned is the Soil Moisture Active/Passive (SMAP) mission [12] by the Jet 6

Propulsion Laboratory of NASA. SMAP will include an L-band radar and an L-band radiometer. A conically scanning reflector antenna will be used in the device, and the goal is to have a 10 km. spatial resolution by combining the attributes of the radar and the radiometer. The planned launch date of this mission is between 2010 and 2013.

1.2

Introduction to RFI Mitigation

As described in the previous section, radiometers are built to measure natural thermal radiation. RFI in this context can be described as the radiation due to the anthropogenic emissions that radiometers involuntarily receive. RFI is additive, that is it shows as an increase in the power received by the radiometers, and as a result it causes a bias in the predictions of the environmental parameter that is being measured. Several recent works have documented the detrimental effects of radio frequency interference (RFI) on Earth observing passive microwave radiometer systems [13– 17]. In [13], analysis of RFI for interferometric radiometers was made and possible consequences to the L-band Microwave Imaging Radiometer with Aperture Synthesis (MIRAS) were discussed. Although interferometric radiometers are not the subject of our research, this work is still useful for providing an insight to the possible sources of RFI at L-band. In spite of the fact that a portion of the L-band from 1400 to 1427 MHz is reserved for passive microwave remote sensing operations, emissions from sources that operate at frequencies close to this band may cause corruption in radiometric observations if they are not properly filtered. Possible sources of RFI include L-band radars, mobile satellite services and military tactical services. Harmonics of emitters at VHF and UHF bands like broadcast satellite services, mobile 7

satellite services, meteorological satellite services and jamming might be the other sources of RFI at the protected portion of L-band. The potential contribution of RFI to the measurements of space-based radiometers due to L-band radars in terms of equivalent brightness temperatures was computed in Reference [14]. It was found that RFI due to a typical radar may contribute an equivalent brightness temperature that is as high as 10 K at the receiver of a radiometer orbiting the earth at a 675 km. altitude. It is not possible to attain the required accuracy for most environmental parameters in the presence of such a high level of RFI, and thus authors suggest the use of RFI mitigation techniques in future sensors. In [15], a survey of RFI at C-band over continental United States was made using the difference between brightness temperatures at X- (10.65 GHz) and C- band (6.925 GHz) of the AMSR-E radiometer. C-band does not have a protected portion of the spectrum while the frequencies between 10.68-10.7 GHz are protected at Xband. However, this protected portion is only a small part of the reported 100 MHz bandwidth at the X-band channel of AMSR-E. Results of the survey show significant RFI corruption, especially around urban centers. Facilities for cable TV relay and wireless communication, airport radars and manufacturing operations are mentioned as possible sources of RFI at C-band. A similar survey over United States at C- and X- bands was performed in [17], this time using data from the WindSat radiometer. This study confirmed the severity of the RFI corruption at C-band. The situation at X-band was relatively better. However, authors managed to show the presence of RFI sources at X-band using the correlation channels of the WindSat radiometer. These sources, albeit weaker than their C-band counterparts, may still cause severe accuracy issues for environmental data retrievals at X-band.

8

Results of these surveys have demonstrated the need for RFI mitigation for future passive microwave remote sensing systems. RFI differs from natural radiation in intensity, spatial variability, polarization, and spectral and distribution characteristics [15]. Using these facts, methods for the detection and mitigation of RFI [18–27] have been developed, and several radiometric systems capable of RFI detection and mitigation with varying performance against different RFI types have been implemented. Techniques used for RFI mitigation in these systems can be divided into three main groups; time-domain detection methods, cross-frequency detection methods and methods based on the Gaussianity of thermal noise. These techniques are reviewed briefly in the next few sections. A description of some other methods employed in RFI detection are given in section 1.2.4.

1.2.1

Time-domain detection methods

Time domain detection methods try to detect RFI sources concentrated in time. An example of such an RFI source is a pulsed radar. Detectors that search for energy outliers in time-domain are called “pulse” detectors (or sometimes glitch detectors) [20–23]. Detection is accomplished by comparing the power of the received fields with a threshold, which may be regarded as a conceptually simple technique. However, the fact that changes in the system temperature should be accounted for in the determination of the threshold brings some additional complexity in its implementation. Application of similar concepts for RFI detection was first investigated in radio astronomy; for example a time-domain threshold method was demonstrated in 1996 by Fridman et al. [20] while in [28] a correlator output was used to detect RFI; if RFI is detected, received signal is blanked in time-domain in this work.

9

First earth remote sensing system that is capable of pulse blanking was developed at OSU/ESL. This detector, called an Asynchronous Pulse Blanker (APB), was implemented in FPGA and it is capable of pulse blanking in real time. APB was incorporated into the digital radiometers designed at OSU/ESL called L-band/C-band Interference Suppressing Radiometer (LISR/CISR). A basic description of how APB works will be given in Chapter 2 and performance of APB against pulsed sources will be demonstrated using data from a campaign carried out in close vicinity of an air route surveillance radar (ARSR). It should be noted that in a space based system, since most satellites are not geostationary, a time-domain detector will be discriminating RFI based on its spatial variation. Such a glitch detection method is planned for use in the Aquarius radiometer and the theory of its operation can be found in [29].

1.2.2

Cross-frequency detection

Detectors that search outliers in the frequency domain are called “cross-frequency” detectors in this work. This type of detector also has its roots in radio astronomy applications; Fridman et al. describes thresholding in frequency domain in [20]. First use of cross-frequency detection for RFI mitigation in earth observations is dated to year 2002 [18]. Cross-frequency detection method employed in this work was tested using data from an airborne campaign over Southern Great Plains in 1999 obtained with NOAA (authors are now with University of Colorado) Polarimetric Scanning Radiometer C-band (PSR/C). This system had 4 analog frequency subchannels with passbands between 5.8-6.2 GHz, 6.3-6.7 GHz, 6.75-7.10 GHz and 7.15-7.50 GHz. RFI

10

detection was performed in post-processing. A spectral fit was applied to the brightness temperatures of these four channels. If fit parameters were not physically reasonable or brightness temperature of the channels were greater than physical limits, process was repeated by applying the procedure to channels within spectral fit and brightness temperature limits. When a reasonable spectral fit was obtained, brightness temperatures of the corrupted channels were replaced with a value based on this fit. On the other hand, if spectral fit could not be obtained, minimum brightness temperature among the channels was used as to replace the brightness temperatures of all the remaining channels. In the case that even the minimum brightness temperature was over the physical limits, RFI was deemed to be “uncorrectable”. Statistics of PSR data corrected with this method for different RFI environments is provided in Chapter 4.3. Although PSR algorithm demonstrated the effectiveness of a cross-frequency detection method, number of frequency channels used in this analog system were limited which restricted its efficiency. Also, with such large bandwidths it would be difficult to resolve narrowband RFI sources since these sources might only cause a weak change in brightness when averaged over a larger channel. Increasing the number of channels is expensive and would mean a significant increase in the size of such an analog device. On the other hand, it is easy to obtain a high spectral resolution in a digital system with little additional hardware complexity employing Fast Fourier Transform (FFT) methods in hardware. LISR/CISR sensor of OSU uses such technology [23]. LISR/CISR is capable of performing a 1024 point FFT operation and resolving spectrum in the order of a 100 KHz. This high resolution and large number of channels allowed us to demonstrate the effectiveness of cross-frequency detection against narrowband RFI as described in this work. 11

1.2.3

Methods based on Gaussianity tests

RFI detection can also be performed by testing the distribution of the fields received by the radiometer, which should be Gaussian for thermal noise; a deviation from the Gaussian distribution indicates the presence of RFI. Although numerous methods exist to test Gaussianity, to date only the kurtosis test of Gaussianity has been used in practical implementations [24–26]. The kurtosis test has been shown to be an effective tool against a wide variety of RFI types, but a blind spot in detecting pulsed sinusoidal interference has also been reported [24]. Later studies have shown that it is possible to remove this blind spot and to improve detection performance by subsampling in time and frequency [30–32]. Blind spot can also be avoided using the sixth moment of the data, however increase in resources and high RFI strength required make this method impractical [33]. In this thesis, Shapiro-Wilk test of Gaussianity is studied as a possible alternative for RFI detection [34,35]. Results show the possibility of designing a radiometer that use this method for RFI detection.

1.2.4

Other RFI detection methods

As mentioned before, pulse detection, cross-frequency detection and kurtosis detection methods are the main methods used for RFI detection in current and planned future radiometric systems and they form the focus of this work. However, several other RFI detection methods which are used primarily for radio astronomy applications can also be found in literature. In [18], it is mentioned that polarization diversity may be used in earth observing radiometers for RFI detection in addition to the cross-frequency detection method described in the paper. Although this method might be useful for RFI detection, 12

mitigation of RFI is not possible in this case other than throwing out the corrupted data and it has not been implemented to date in earth observing radiometers. Making radiometric observations at places or times when corruption due to RFI is low is a widely used practice. Work done on eliminating potential RFI sources by engaging stricter filtering and power requirements for emitters close to the protected portions of the spectrum should also be mentioned here [36]. Although this is not exactly an RFI mitigation method, it is obvious that if these efforts were to succeed significant improvement in corruption due to RFI will be obtained. Interferometric nulling is a RFI mitigation technique that is used in radio astronomy [20]. However, this technique requires an antenna array (such that radiation pattern is minimized in the direction of the RFI source) and it is not very useful for earth remote sensing where it is not easy to get a discrimination between the directions of data and that of the RFI source. A technique conceptually similar to interferometric nulling that is called “RFI masking” was suggested for earth observing radiometers in [37]. In this method, a global mask of RFI would be produced by the analysis of data over a long time period and this mask would be used to eliminate data obtained from locations corrupted by RFI. However, as the authors suggest, such a method would not account for new RFI locations and would be overly conservative in the sense that it eliminates data from locations where RFI is not always present (i.e intermittent). Estimating the interference waveform and subtracting from the received signal was suggested as a way to mitigate RFI in radio astronomy [38]. However, this method requires that the form of the interference to be known and hence it is not very useful for an earth observing system where RFI sources vary by type and numbers depending on the observation point. An adaptive filtering scheme is also suggested 13

for radio astronomy but this method requires the direction of the interference signal to be known and thus again is not suitable for our purposes [39].

1.3

Outline of Thesis

This thesis will basically follow the order in which the research was performed with experimental results provided first and theoretical results given in the later parts of the thesis. However, to maintain continuity between L-band campaigns, an L-band campaign conducted at Jet Propulsion Laboratory will be described in Chapter 3, although it was performed after the C-band observations narrated in Chapter 4. Chapter 2 describes observations made by LISR during a groundborne campaign at Canton, Michigan in 2005. Descriptions of the LISR system and the APB operation procedure are given. Parts of the experiment were performed in close vicinity to an ARSR, therefore the measured data was very suitable in demonstrating the performance of APB. Other than the aforementioned observations close to the ARSR, some laboratory observations where artificial RFI sources were injected to the antenna will also be described. Methods for pulse detection and cross frequency detection in post processing devised for this campaign will be explained and their success in removing RFI will be demonstrated. Chapter 3 also details groundborne observations at L-band using LISR; this time using data from a campaign at the Jet Propulsion Laboratory in Pasadena, California in 2006. Observations of the sky were made in this experiment, and the presence of RFI sources even in the protected portion of the spectrum was demonstrated using the high frequency resolution of the LISR system. It is again shown that most of the obvious RFI can be eliminated using the cross-frequency detection and the pulse

14

detection methods applied in post-processing. Intercomparisons with PALS and ADD sensors that were also present in this campaign are provided. An airborne C-band campaign performed over Texas and Gulf of Mexico is explained in Chapter 4. This campaign provided an excellent opportunity to show different RFI environments including urban and rural land observations as well as water observations over the Gulf of Mexico. Results are shown to illustrate the success achieved against both strong and weak RFI sources. Comparisons with other sensors that were present (PSR and ADD) are also provided. A novel method for RFI detection is described in Chapter 5. As explained in Section 1.2.3, tests of normality can be used for detecting RFI. While numerous tests of normality are described in the statistical literature, the kurtosis test was the only such test considered previously in radiometric systems. With this fact in mind, Shapiro-Wilk test was considered as an alternative RFI mitigation technique. Performance of the Shapiro-Wilk test against pulsed sinusoidal RFI is demonstrated in this chapter. Results prove the feasibility of employing this technique for RFI mitigation. In Chapter 6, a theoretical performance comparison is made for the three main RFI detection methods described in this introductory chapter: Pulse detection method, cross-frequency detection method, and the kurtosis method. RFI is again assumed to be pulsed sinusoidal which might model pulsed radars as well as CW sources. The focus of this chapter is on the cross-frequency detector. It is shown that the crossfrequency detector maintains a high a performance regardless of duty cycle for this type of RFI. Chapter 7 concludes this study. Contributions provided by this research is listed and a summary is provided in this chapter. 15

CHAPTER 2

RFI OBSERVATIONS AT L-BAND: CANTON CAMPAIGN

On June 16th and June 17th, 2005, a ground-based campaign involving L-band microwave radiometer observations of an ARSR in Canton, MI was conducted by researchers from the University of Michigan [24], NASA Goddard Space Flight Center [40], and The Ohio State University (OSU). The purpose of the campaign was to demonstrate the level of interference observed in L-band observations within the protected band 1400-1427 MHz, as well as to demonstrate the ability of three distinct radiometer backends at detecting and removing this interference. The campaign was conducted in close proximity to the radar location (within approximately 200 m of the radar antenna), ensuring that strong out-of-band interference would be experienced in the observations. While this configuration certainly emphasizes the effects of the radar system compared to observations from a space-based platform, the experiment was designed to demonstrate that the backends developed could successfully remove interference in both extremely strong and slight interference environments. The latter case was obtained through choice of the observation direction of the radiometer antenna, as well as the fact that the radar center frequency (reported as 1315 MHz by the station’s literature) was far out-of-band of the radiometer observations. Other properties of the radar reported by the station are a peak radiated power of 4 MW, 16

antenna gain of 35 dB, an antenna tilt angle of 2.6 degrees, a PRF of 360 Hz (or 2.78 msec between pulses), and a pulse width of 2 µsec. The campaign was continued on June 20th and 21st to include observations of artificial pulsed and continuous RFI sources in the laboratory at the University of Michigan. The June 21st data set also includes measurements of a well characterized cold load for calibration purposes. This chapter documents observations by The Ohio State University backend primarily from the June 17th dataset (the 167th day of the year 2005). The next section provides a basic description of the overall RF and downconversion systems of the radiometer; these systems were common to the backend units of the three research groups. Section 2.2 then provides a brief overview of The Ohio State University backend, LISR. Section 2.3 describes several hardware issues that were observed in processing the data, and Section 2.4 reports general properties of the observed brightnesses. Detailed examinations of a few specific datasets and the performance of LISR RFI removal post-processing algorithms are provided in Section 2.5. Section 2.6 then provides a brief overview of the data obtained from the laboratory measurements on June 20th and 21st, and final conclusions of the study are described in Section 2.7.

2.1 2.1.1

Radiometer Front End and Downconversion Radiometer front end

The L-band radiometer antenna and front end utilized in this experiment were being developed by the University of Michigan for delivery to Prof. Brian Hornbuckle at Iowa State University. The campaign provided an opportunity for the performance of the radiometer to be assessed in the field prior to delivery. Although a detailed schematic of the radiometer was not made available, a qualitative description was 17

provided. The radiometer utilizes an L-band dual polarized horn antenna of approximate dimensions 40 x 50 cm; this is not particularly high gain but sufficient to allow angular discrimination. The radiometer front end contains dual receivers so that simultaneous observations of horizontal and vertical polarization are provided. Following the antenna connection, each receiver begins with a mechanical switch to allow measurement of an internal reference load, followed by a coupler to allow injection of a noise diode signal. Both the noise diode powers (on or off) as well as the reference load switches were under control of the University of Michigan backend system (called the Advanced Digital Detector or ADD [24]). The ADD backend is capable of measuring received field moments upto their 4th order which are then used to calculate kurtosis statistic for RFI detection purposes. Kurtosis statistic will be explained in detail in Chapters 5 and 6. Following these internal calibration standards is a bandpass cavity filter from 1403.5-1423.5 MHz, reported as having 3 cavities. Next is the system low noise amplifier, followed by 2 additional bandpass cavity filters with the same cutoff frequencies, each reported as containing 4 cavities. An additional RF amplifier is then followed by a final filter containing 6 cavities and a final RF amplifier.

2.1.2

Downconversion stage

The downconversion stage consists of a mixer fed by an LO at either 1386.5 or 1440.5 MHz. This frequency plan results in the RF filter passband occupying IF frequencies 17-37 MHz. Note all image rejection in this process is provided by the RF filters. In the case of LO frequency 1386.5 MHz, the image band (lower side band) is 1351.5-1371.5 MHz, while when using LO frequency 1440.5 MHz the image band (the upper sideband) is 1457.5-1477.5 MHz. Rejection in both these bands by the RF

18

filters is assumed to be similar. The mixer is followed by additional IF amplification and a lowpass filter of 50 MHz bandwidth. With regard to the downconverter effect on the ARSR transmissions at 1315 MHz, it should be expected that the 50 MHz lowpass filter likely would result in greater attenuation when using LO frequency 1440.5 MHz, as the radar is then at IF frequency 125.5 MHz as opposed to 71.5 MHz with LO frequency 1386.5 MHz. The IF outputs for both horizontal and vertical polarizations are then power divided three ways and supplied to the three backend systems. The resulting outputs occupy the approximate bandwidth 17-37 MHz and have been amplified to provide sufficient noise power levels for further processing.

2.1.3

Physical properties

Both the radiometer front end and downconversion stages were operated in a single temperature controlled enclosure. Unfortunately means for recording the enclosure temperature were not available at the time of the campaign; it will be assumed in what follows that the reference load temperatures were maintained at the temperature set point of 305 K throughout the campaign. Any inaccuracy in this information will contribute to errors in the calibrated brightnesses presented later; however these effects are not expected to be large as the temperature control had been previously tested and found to be reasonably stable. The front end and downconversion stage enclosure was placed in close proximity to the radiometer antenna, and the entire structure was mounted on the boom of a truck from the Microwave Geophysics research group of the University of Michigan. A picture of the truck, boom and the radiometer front end can be seen in Figure 2.1. The boom included an elevation positioner for the antenna structure, so that

19

Figure 2.1: Truck, boom and horn antenna used in Canton campaign

the radiometer antenna angle could be controlled in both azimuth (through rotation of the boom) as well as elevation. IF outputs from the front end/downconversion stage enclosure were routed down the boom and into the truck roof; the three backend systems were then all located inside the truck. In addition, the LO source for the front end/downconversion stage, as well as control lines for the reference load and noise diode switches, were provided up the boom from inside the truck. The LO source utilized was a bench oscillator followed by a frequency doubler.

20

2.1.4

Radiometer state information

The experiment plan involved a periodic observation of radiometer states with a period of 1.46 seconds. The basic pattern involves a 324 msec observation of the antenna, followed by a 36 msec observation of the antenna plus the noise diodes. This pattern is repeated (i.e. 324 msec antenna, 36 msec antenna plus noise diode), then the reference load switches are thrown. The reference load is then observed for 324 msec, followed by 36 msec of the reference load plus the noise diode. This pattern is repeated again, and then the reference load switch is thrown again and the cycle repeats beginning with the antenna observations. A 6 msec delay state was also included following the command to throw the reference load switch, as well as a 1 msec delay state following the noise diode on or off commands. As stated previously, this state machine was controlled by the ADD system, and information on these states was provided to the OSU and GSFC systems through three control lines. The three control lines are labeled “N” (high when the noise diodes are on), “R” (high when the switches are set to observe the reference loads), and “I” (high when the radiometer is not in a delay state.) These were open drain outputs of the ADD, pulled up to a voltage of 3.3 volts for the other backends. While the NASA GSFC system was configured to measure continuously while recording information on these state lines, LISR was configured to trigger off the “I” line. Thus, LISR measurements were begun upon observation of a high state in the “I” line.

21

100 MSPS

V pol

ADC

ADC

50 MHz LPF (H,V), join to 100 MSPS I/Q

Asynchronous Pulse Blanker (APB)

1024 point FFT

H pol Integrate 128 FFT’s: 1.3 msec

256K FIFO National Instruments interface

H and V pol: 50 MHz@ 97.7 kHz resolution every 10.24 µsec

PC-104 Computer

Figure 2.2: Block diagram of LISR

2.2 2.2.1

A Brief Description of LISR LISR schematic

Figure 2.2 presents a block diagram of LISR as configured for this experiment. Further information on LISR can be obtained from [21, 22, 41–50]. LISR samples both the incoming H and V pol 17-37 MHz IF’s at 100 MSPS, using two 10-bit A/D converters for this purpose. The LISR A/D converters are actually clocked at 200 MSPS, but in this case one half of these samples are discarded to simplify operations at the approximate 25 MHz IF center frequency. The remaining LISR operations described in Figure 2.2 take place in a single large Altera FPGA. The first operation involves digitally low-pass filtering each of the incoming IF’s, then combining the two polarizations into a single complex datastream (16 bits in both I and Q) at 100 MSPS, occupying the band -50 to 50 MHz (typically V pol -50 to 0 MHz, H pol 0 to 50 MHz). The digital low pass filter used has a bandwidth wider than that set by the RF filters, and therefore is not expected to impact the measured data. However

22

its use allows removal of the image spectrum for each of the input polarizations so that the combination into a single 100 MSPS complex datastream is possible. Following this output is a set of processors that can be controlled by the operator in real time through an ethernet interface between the system computer and the FPGA. In the results presented here, LISR was operated in three distinct modes, labeled “Integration, APB off”, “Integration, APB on”, and “Capture” in what follows.

2.2.2

Capture mode

In “Capture” mode, the 100 MSPS complex datastream is directly passed to the control and recording computer without further processing. This extremely high temporal resolution (time sampled at 10 nsec) results in an extremely high datarate, so that only low duty cycle observations are possible. However the high temporal resolution and coherent data acquired allows detailed studies of the properties of the observed interference. It was found that 6 256K captures could be acquired and transferred to the control computer during a 324 msec antenna or reference load observation, this represents 15.73 msec observed out of 324 possible, or a duty cycle of 4.85%. Due to an accidental operator error in the field, noise diode data were not acquired in capture mode; capture data are therefore studied as raw data only.

2.2.3

APB

The other LISR modes utilize data that has passed through an asynchronous pulse blanking stage. The term “asynchronous” is used since no periodic properties of the RFI source is assumed [22]. The APB is designed to detect and blank radar pulses in real time, so that their effects can be removed without further processing of the measured data, while maintaining accurate calibration of the noise-free data. The basic operation principle of APB is illustrated in Figure 2.3. To detect pulsed 23

interferers, the APB maintains a running estimate of the mean and variance of the incoming power in the time domain (through use of an exponential averaging operation as described in [22].) Whenever a sample magnitude greater than a threshold number of standard deviations from the mean is detected, the APB blanks (sets to zero) a block of samples of length NBLANK beginning from a predetermined period before the triggering sample (shown as NWAIT in the figure), through and hopefully including any multi-path components associated with the detected pulses. APB operating parameters are adjustable and can be set by the user. In the experiments described here, the APB threshold was usually set to approximately 9 standard deviations from the mean power level, and a blanking window of 40 µsec around detected pulses was used (note 40 µsec corresponds to 6 km of radar range.) This window began 10.24 µsec prior to the detected sample. The APB provides information on the amount of blanking to further processor stages so that blanking effects on calibration can be corrected in real time. APB operates on the 100 MSPS complex datastream directly, so that the two polarizations are not separately blanked; however the effects on calibration remain accurately computed for each polarization.

2.2.4

Integrating modes

Following the APB is a length-1K complex FFT utilizing a triangular window to reduce truncation effects. This FFT operation provides an output every 10.24 µsec, with two sets of 512 bins obtained in a 50 MHz bandwidth; the first set is for horizontal and the second set for vertical polarization. This results in a spectral resolution of approximately 97 kHz, much finer than the bandwidth of many expected RFI sources. FFT outputs are then passed through detection and integration operations, with the final datastream comprised of 1024 frequency bins obtained every 1.3 msec. These

24

Power (linear units)

NBLANK

NWAIT

Threshold

0

20

40

60

80

100 120 Time (µ sec)

140

160

Figure 2.3: Basic operation procedure of APB

25

180

200

data are represented in 32 bits for each bin. Though not utilized in this experiment, the integration module is also capable of computing a max-hold operation in RFI detection applications. The distinction between “Integration, APB on” and “Integration, APB off” modes involves whether the APB blanking is turned on or off in the data input to the FFT operation. Comparing brightnesses obtained in these modes will allow the effectiveness of the APB real-time pulse removal algorithm to be examined.

2.2.5

Computer interface

Results following integration are stored in a 32 bit, 256K FIFO. FIFO data is then passed through a National Instruments interface to the system control computer. Timing tests showed that 205 spectra (i.e. FFT outputs) could be acquired and reliably transferred to the system computer within 324 msec. This represents a duty cycle of 82.9% in the integrating modes. While this duty cycle could have been increased by increasing the number of FFT’s integrated in the integration operation, it was deemed desirable to maintain a time resolution in the final dataset that was finer than the PRF period of the ARSR system. The final data obtained is approximately 0.8 MB for each antenna or reference load trigger. Noise diode observations obtained 17 spectra, representing 22.3 msec out of the 36 msec available (duty cycle 61.9%.) The final datarate if operating continuously is approximately 8.3 GB per hour. One final influence on the system datarate involves the control software. For efficiency in writing measured data to the computer hard drives, data obtained from the FIFO is first stored in a computer RAM buffer of approximately 36 MB. This buffer is written to the hard drives when full, resulting in a cessation of LISR operations

26

until the write operation is completed. In the integrating modes, the delay incurred is approximately 2.55 seconds following continuous operations for 16.45 seconds.

2.3 2.3.1

Hardware Issues Observed gain pattern

Figure 2.4 plots the mean raw power versus frequency obtained by LISR for reference load observations, normalized so that the maximum value of the frequency response is approximately 0 dB. Curves for both horizontal (blue and dashed) and vertical (black and dotted) polarizations are included. These results show a relatively smooth instrument gain pattern versus frequency, although an increase in gain with frequency is observed as well as a passband “ripple” of approximately 1 dB. Behaviors for the vertical and horizontal receivers are similar. Lines in the Figures mark the approximate gain level of -3 dB, defining a 3dB passband of around 1402 to 1425 MHz, while the gain at the protected spectrum boundaries of 1400 and 1427 MHz is found to be around 9 to 10 dB below the maximum. The patterns reach the LISR noise level of around -18 dB for a band around 1397 to 1430 MHz. Outside this region LISR is incapable of determining instrument passband properties. While this passband appears acceptable, it seems somewhat wide with regard to possible RFI effects for a spaceborne instrument. Given the possibility of RFI sources up to 1398-1399 MHz (though not observed in the data to be reported), the passband of the instrument utilized may not provide sufficient reduction of these slightly out-of-band sources. The nature of the instrument passband brings up issues related to data calibration. The majority of the LISR data to be described begin as raw data of the type in

27

Reference Load Power vs. Frequency 2 H pol V pol

0

Relative noise amplitude (dB)

−2 −4 −6 −8 −10 −12 −14 −16 −18 1390

1395

1400

1405 1410 1415 1420 1425 Frequency (MHz) with LO=1440.5 MHz

1430

1435

1440

Figure 2.4: Normalized LISR raw data for reference load observations versus frequency for H-pol and V-pol. Black horizontal line marks approximate 3 dB point, while red vertical lines mark boundaries of protected spectrum.

28

Figure 2.4, which represent 50 MHz of spectrum sampled into 512 bins. The majority of the LISR results to be illustrated perform calibration of each of these 512 bins separately; larger bandwidth channel brightnesses are then obtained by averaging the calibrated brightnesses over the desired bandwidth. However, instruments with less fine spectral resolution instead observe raw powers integrated over larger bandwidths originally. In this case, particular parts of the spectrum have been weighted according to the instrument passband before calibration is performed. For example, the instrument passband results in a total power calibration over the entire passband highly emphasizing channels within the 3 dB bandwidth of the radiometer, compared to those outside the 3dB bandwidth. In addition, the instrument gain pattern emphasizes brightnesses at higher frequencies within the passband compared to lower frequencies. These differences can make direct comparisons of calibrated data difficult. One method for addressing this issue involves first combining the LISR raw data into a larger channel before performing calibration. For example, comparison of LISR calibrated data with that obtained by the ADD system (channels approximately 3 MHz wide) can be performed directly if the exact properties of the ADD channels are utilized first in weighting the LISR spectrum. Several tests performed in this manner using estimated properties of the ADD channels (from [24]) showed only minor differences between pre- and post-calibration combination of LISR channels. For this reason, post-calibration combination of LISR channels was deemed acceptable, and is utilized in the results to be shown. However differences in brightnesses are introduced by this procedure so that precise inter-comparisons among sensors of brightnesses (i.e. beyond the level of a few K) are not possible.

29

2.3.2

Reference load switch issues

Examination of LISR data showed issues regarding transition of the antenna/reference load switch in the state machine pattern of the radiometer. Recall again that the basic switching pattern is an antenna observation, followed by antenna plus noise diode, followed by a second antenna observation, and another antenna plus noise diode, then a change of the antenna/reference load switch position to the reference load. A 6 msec delay is included in the state machine following the switch command before observations of the reference load are begun. The pattern is repeated again for reference load observations. Figure 2.5 illustrates the mean raw power in LISR channels representing 13991428 MHz as a function of additional delay introduced after the beginning of the state, relative to that with no additional delay. These data were recorded near time 14:51 UTC on June 17th in “Integration, APB off” mode, and represent means over approximately 20 seconds of observed data. Recall that within one 324 msec antenna or reference load state, LISR records 205 1.3 msec spectra, so that the power in these spectra can be investigated as a function of time from the state beginning. Figure 2.5, plot (a) represents antenna measurement data, while plot (b) represents reference load measurement data. The multiple curves in the plots are for the two polarization channels and for the two possible states: either following a noise diode on/off transition or following a transition of the antenna/reference switch. Results show all of the mean powers have only slight trends versus the additional delay, with the exception of the vertically polarized channel following transition of the antenna/reference load switch. A decrease in the power measured with time is observed, for both the antenna and reference load

30

(b) Doy 167: 14:51 UTC Reference 100.05

100

100 Mean Power Relative to No Delay (%)

Mean Power Relative to No Delay (%)

(a) Doy 167: 14:51 UTC Antenna 100.05

99.95

99.9

Ant after Ant: H Ant after Ref: H Ant after Ant: V Ant after Ref: V

99.85

99.8

99.95

99.9

Ref after Ant: H Ref after Ref: H Ref after Ant: V Ref after Ref: V

99.85

99.8

99.75 0 50 100 150 200 Delay after switch command (msec)

99.75 0 50 100 150 200 Delay after switch command (msec)

Figure 2.5: Mean LISR raw power in 1399-1428 MHz as a function of delay after state change. (a) Antenna states (b) Reference load states

31

states. Although the relative power changes are small, calibration coefficients to be described later show these trends to be on the order of at least 1 K. To avoid calibration issues introduced by this effect, all states following transition of the antenna/reference load switch are omitted in what follows for both polarizations; only antenna or reference load data following observation of a noise diode is utilized. While this reduces the data available to only 50%, the majority of the data obtained were recorded in a stable environment over long time periods, so that sufficient data remains for the goals of this investigation.

2.3.3

Noise diode on/off switch issues

A similar issue was observed with regard to noise diode observations. Due to the expected rapid response of the noise diode on/off switch, a time delay of only 1 msec was allocated following a noise diode on/off switch command. Figure 2.6 plots the mean LISR raw power in 1399-1428 MHz as a function of delay after the state beginning, relative to the power obtained with no additional delay. In this case, no significant trends are observed at long times, but the first 1.3 msec data sample shows a significantly reduced power relative to those at later time delays. Calibration coefficients show this change to be on the order of 3-4 K. For this reason, the first 1.3 msec LISR spectrum was removed in all noise diode data in what follows.

2.3.4

Calibration

Calibration of LISR measured data was performed using the reference and reference plus noise diode states to determine the radiometer gain and offset coefficients, both of which were obtained in H and V polarizations for 512 frequency channels. Means of the reference and reference plus noise data over a period of plus or minus

32

(a) Doy 167: 14:51 UTC ND

(b) Doy 167: 14:51 UTC Reference

101 Mean Power Relative to No Delay (%)

Mean Power Relative to No Delay (%)

101

100.8

100.6

100.4 ND after Ant: H 100.2

ND after Ant: V

100

99.8

100.8

100.6

100.4 ND after Ref: H 100.2

ND after Ref: V

100

0 5 10 15 20 Delay after switch command (msec)

99.8

0 5 10 15 20 Delay after switch command (msec)

Figure 2.6: Mean LISR raw power in 1399-1428 MHz as a function of delay after state change. (a) Noise diode plus antenna states (b) Noise diode plus reference load states

33

5 seconds surrounding the measurement of interest were utilized in determining the calibration coefficients. As stated previously, it was assumed that reference load observations represented a 305 K brightness, while the noise diode added an additional 128.4 K. The latter figure was obtained from calibration studies by the University of Michigan, and did not include any possible state transition delay effects. Although a detailed examination was not performed for LISR, a few qualitative tests of calibration accuracy were performed using the June 21st observations of a liquid nitrogen target. Figure 2.7 illustrates LISR mean brightnesses in the band 1397-1430 MHz during observations of the liquid nitrogen target, including extra 0, 1, 2, and 3 dB pads. Horizontal lines in the figure mark the expected brightnesses as the degree of padding is varied, and results are illustrated in both H (APB on) and V (APB off) polarizations. Results show the calibration procedure to yield absolute brightnesses accurate to within 3-4 K, with the absolute error increasing for larger brightnesses. Modifications to the noise diode and reference load assumed brightnesses could be used to address these errors, but were not pursued further. An additional indication of the calibration and receiver stability obtained is illustrated in Figure 2.8, which plots the calibrated LISR brightness in the band 1406-1412 MHz for V-pol channel observation of a terminator at times near 14:00 UTC on June 17th. Results are shown for both APB off and APB on modes. At times prior to 14:36 UTC on the 17th, the V-pol radiometer channel was connected to an external terminator rather than to the radiometer antenna. Results show a relatively stable terminator brightness near 288 K (288.099K APB off, 288.08K APB on) for this case over several minutes, with a standard deviation among 324 msec states of 0.55K APB off and 0.63K APB on. Clearly the calibration procedure utilized is producing 34

Total Brightness 1397−1430 MHz 220 210 200 190

Brightness (K)

180 170 160 150 V pol, APB off (18:00) H pol, APB on (19:00)

140 130 120 110 20

30

40

50 60 70 Minutes past UTC 18:00 or 19:00

80

90

100

Figure 2.7: Mean LISR brightness 1397-1430 MHz for H- and V-pol observations of a controlled target; H pol observations are shown inside boxes. Horizontal lines mark expected brightnesses when target is viewed through 0, 1, 2, and 3 dB pads, respectively.

35

Terminator observations Doy 167 290 APB off APB on

V pol Brightness 1406−1412 MHz (K)

289.5

289

288.5

288

287.5

287

286.5

286 27

28

29

30 31 Minutes past 14:00 UTC

32

33

34

Figure 2.8: Mean LISR brightness 1406-1412 MHz for V-pol observations of a terminator (near 14:00 UTC)

a stable means for measuring brightness temperatures, and the APB calibration correction is functioning properly. Note the APB reports 1.8% of data being blanked in the dataset shown; this blanking is due to triggering of the APB detection algorithm by large noise peaks, and is consistent with expectations for Gaussian noise given the APB parameters used. Figure 2.9 provides another test of calibration involving observations of the sky near 19:25 UTC on June 20th. During this time period, the radiometer antenna elevation angle was 62 degrees, providing near zenith measurements. Although some influence of the RFI sources could be possible, the LISR spectra shown appear RFI

36

Calibrated Sky Brightness 170, 19:25−19:26 80 H Pol V pol 70

Brightness (K)

60

50

40

30

20

10

1400

1405

1410 1415 Frequency (MHz)

1420

1425

Figure 2.9: Calibrated LISR brightnesses versus frequency during sky observations near 19:25 UTC on June 20th.

free, with the exception of narrowband sources near 1400 and 1403 MHz. Neglecting these sources, the sky brightnesses obtained appear reasonable compared to the reported values of 44.7 K from the ADD system (believed to be in H-pol). Mean brightnesses from 1405-1430 MHz from LISR are found to be 45.0 K for H-pol (blue in the Figure) and 50.0 K for V-pol (red). Again, there are several possible sources of small differences between brightnesses of LISR and the ADD system, but overall the results validate that a reasonable calibration of LISR results is being obtained.

37

Figure 2.10: Location of the truck (circled in red) with respect to the ARSR.

2.4

Summary of LISR Observations on June 17th

On June 17th, LISR obtained datasets between times 13:25 to 16:28 UTC as well as times 18:34 to 19:53 UTC. A satellite map of the experiment site can be seen in Figure 2.10. Location of the truck is circled in red. In its default position, antenna is facing north at an azimuth of angle of ≈ 160◦ with respect to the ARSR. The first period of data contains primarily observations with the radiometer antenna held at a fixed azimuthal observation angle, while the second dataset contains primarily observations during periods when the antenna azimuth angle is varied. Data in all three LISR modes was recorded during the experiment; the three modes are considered separately in the next sections.

38

2.4.1

Integration, APB off

Table 2.1 summarizes LISR observations in Integration, APB off mode. These data are originally acquired as 1024 point spectra sampled at 1.3 msec; however, to reduce the data volume, the majority of the results to be shown in this mode will be integrated over the 205 spectra in an antenna observation (approximately 269 msec). Figures 2.11 and 2.12 plot calibrated brightnesses in vertical and horizontal polarizations, respectively, obtained in this mode for all data in Table 2.1. The horizontal axis of these figures is RF frequency, mapped from the LISR IF frequencies of 0 to 50 MHz given the known LO frequency in a given observation. Although these plots do not provide detailed quantitative information on the observed brightnesses, the basic qualitative variations with observation of the sky, the external environment, and a terminator are obvious in these figures. RFI source near 1413-1415 MHz A strong RFI source in the range 1413-1415 MHz is also observed in Figures 2.11 and 2.12; the strength of this source is found to be highly correlated to the azimuth position of the antenna. Later analysis will show this source to originate from the ARSR system. The source is seen to have some apparent slight variations in its frequency throughout the experiment, and to be stronger in vertical than in horizontal polarization. While later results will be reported to indicate some possibility of saturation of the radiometer receiver when directly observing the radar (azimuth angle approximately 160 degrees), the majority of the data does not show evidence of saturation. However, measurements with a spectrum analyzer showed very high emitted powers (near -5 dBm received in a 1 MHz bandwidth at 1315 MHz), so that

39

Start Time 13:25:32 14:13:31 14:21:25 14:27:09 14:46:34 14:51:43 15:00:58 15:07:09 15:14:28 15:20:38 15:24:40 15:27:42 15:35:44 15:39:55 15:42:59 15:45:51 15:47:37 15:51:57 15:54:29 16:01:52 16:03:41 16:07:23 16:09:14 16:13:19 16:17:10 16:20:38 16:25:52 16:26:59 19:16:14 19:30:53 19:39:42 19:53:25

Stop Time 13:27:07 14:14:29 14:22:20 14:30:42 14:48:29 14:56:15 15:05:43 15:12:51 15:16:25 15:23:33 15:26:35 15:28:37 15:36:46 15:41:46 15:44:31 15:46:26 15:48:12 15:53:10 15:55:04 16:02:26 16:04:14 16:08:17 16:09:29 16:15:34 16:17:25 16:21:13 16:26:08 16:27:53 19:18:45 19:31:29 19:42:33 19:55:36

Start Sample 1 54 87 120 245 312 469 621 811 869 969 1037 1072 1120 1166 1220 1244 1268 1313 1336 1358 1380 1414 1425 1474 1485 1508 1519 1553 1643 1666 1767

Stop Sample 53 86 119 244 311 468 620 810 868 968 1036 1071 1119 1165 1219 1243 1267 1312 1335 1357 1379 1413 1424 1473 1484 1507 1518 1552 1642 1665 1766 1846

LO (MHz) 1440.5

1440.5 1440.5 1440.5 1440.5 1440.5

1440.5

1440.5

1440.5 1440.5 1440.5

1440.5

H-pol state A A A A A A A A A A A A A A A A A A A A A A A A A A A A 21 71 41 41

V-pol state T T T T A A A A A A A A A A A A A A A A A A A A A A A A A A A A

El (deg) 1 1 50 50 50 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 x 44 44 89 89 89 1 1 1 1

Az (deg) 0 0 0 0 0 0 x x 100 x 135 135 135 x x x 160 160 160 0 0 0 0 0 0 0 0 0 x x x 170

Table 2.1: Summary of Integration, APB off observations on June 17th. Sample numbers refer to the vertical axis of Figures 2.11 and 2.12. LO frequency is 1386.5 MHz unless otherwise indicated. H- and V-pol states are as follows: A=antenna, T=terminated, and a number indicates an amount of attenuation in dB added between the antenna output and radiometer receiver input.

40

V pol 300

200 250

400

Sample

600

200

800 150

1000 1200

100

1400 50

1600 1800

0

1400

1405

1410 1415 Frequency (MHz)

1420

1425

Figure 2.11: Calibrated V-pol LISR data in Integration, APB off mode, versus RF frequency. Refer to Table 2.1 for information on the vertical axis of the plot.

41

H pol 300

200 250

400

Sample

600

200

800 150

1000 1200

100

1400 50

1600 1800

0

1400

1405

1410 1415 Frequency (MHz)

1420

1425

Figure 2.12: Same as Figure 2.11, but for H-pol.

42

significant power at this frequency may still reach the receiver even after passing through the front end filters. If the ARSR transmissions at 1315 MHz were insufficiently attenuated by the radiometer front end filters, the resulting IF frequencies would be 71.5 MHz or 125.5 MHz for LO frequencies 1386.5 MHz and 1440.5 MHz, respectively. These transmissions would again be attenuated by the IF 50 MHz lowpass filter (more severely with LO 1440.5 MHz), and when sampled would eventually be mapped to RF frequency 1415 MHz. The 1-2 MHz offset observed has not been explained, but could result due to either inaccuracy of the specified LO frequency of the radiometer or inaccuracy in the reported ARSR frequency. Data will be shown in Section 2.6 that suggest that the LO frequency utilized on June 20th and 21st was highly stable. In addition, the maximum brightness observed with LO frequency 1440.5 MHz is only slightly (on the order of 10-20 K) less than that observed with LO frequency 1386.5 MHz. If the proposed explanation is accurate, this result must indicate that the IF lowpass filter utilized has only a slight change in attenuation between frequencies of 71.5 MHz and 125.5 MHz, which is quite surprising. For these reasons, the exact means by which the out-of-band ARSR transmission is observed can not be considered completely resolved. Narrowband sources near 1400, 1403, and 1420 MHz In addition to the ARSR emissions, narrowband RFI is observed near frequencies 1400 and 1403 MHz throughout the experiment, in both polarizations. These sources also are correlated (through less than the 1414 MHz source) to the radiometer antenna location. Later evidence will be shown that these sources are more continuous in nature. The 1400 MHz source occupies a bandwidth < 500 kHz, and is stronger in

43

horizontal than vertical polarization. The reported brightness in horizontal polarization exceeds 1800 K in some measurements. The 1403 MHz has similar properties and bandwidths, and achieves a maximum brightness of approximately 420 K. However the 1403 MHz source is observed less frequently than the source at 1400 MHz. Though not evident in the Figures illustrated, some evidence of narrowband interference is observed near 1420.5 MHz, particularly during periods when the radiometer antenna observed the sky at high elevation angles. This source appears to produce brightnesses up to 30 K larger than the background in a bandwidth less than 1 MHz. It is noted that both the 1400 and 1403 MHz sources are near or outside the 3 dB band edges of the radiometer front end filters. This implies that their influence is reduced in data that is integrated over the front end filter response before being calibrated. In addition, the small bandwidth of these sources greatly reduces their contribution when integrated over frequency. Nevertheless, these small sources retain large amplitudes and could contribute measurable RFI if the radiometer filter response provides insufficient attenuation. The origin of these sources is not known at present, although similar emissions have been observed in ground based experiments at both Ohio State and NASA GSFC. One possible producer of these emissions is radiation from various electronics and test equipment within the vicinity of the measurements. Sample high time resolution data Figures 2.13 and 2.14 are images of V- and H-pol data at 1.3 msec time resolution, near time 14:52:15 UTC. At this time, the radiometer antenna was directed at azimuth 0 degrees, so that received ARSR emissions were relatively weak. Although the images of Figures 2.13 and 2.14 are noisier than those obtained from longer time integrations,

44

Figure 2.13: Calibrated V-pol LISR data in Integration, APB off mode, versus RF frequency, at 1.3 msec time resolution. Data acquired near time 14:52:15 UTC.

the high time resolution allows the pulsed nature of the source near 1414 MHz to be clearly observed. Fifty LISR spectra are plotted (65 msec of data); note that the pulse power observed varies significantly within successive pulse observations. This variation can be attributed to inclusion of a triangular window prior to FFT computation within LISR; the pulse amplitude is then influenced by where it falls within a 1024 point LISR FFT “frame”. Overall these plots illustrate the pulsed nature of the 1414 MHz source.

45

Figure 2.14: Same as Figure 2.13, but for H-pol.

46

Effect of adding attenuation to H-pol Data from 19:16 to 19:55 UTC were taken with additional attenuation inserted between the horizontally polarized antenna port and the radiometer receiver input. As indicated in Table 2.1, the radiometer antenna was being rotated in azimuth throughout these measurements. Calibrated brightnesses in horizontal polarization in the band 1399-1428 MHz are plotted in Figure 2.15 versus time. Results at times less than 19:24 UTC still show significant variations, indicating the continued reception of radar pulses even with 21 dB extra attenuation. Results with 71 dB (19:2419:34) and 41 dB attenuation however do not show such variations. Overall these results demonstrate that the ARSR emissions have an extremely significant effect on brightnesses integrated over the total instrument bandwidth. Further demonstration of this fact will be shown in the next section. Summary of Integration, APB off data The results of Figure 2.11 and 2.12 show the ability of the LISR system in detecting RFI within a wideband radiometer channel. Although no real-time mitigation of the observed sources was performed in APB off mode, post-processing can be performed to remove obvious RFI sources. It is important to note that any narrowband RFI sources with amplitudes significant enough to influence the brightness of a large (i.e. 10-20 MHz) channel must appear dramatically larger than the noise background when resolved in frequency. Algorithms for detecting and removing RFI sources in APB off mode data will be described in Section 2.4.4, and applied to specific datasets in Section 2.5.

47

H pol antenna port padded, 167 19:00

Brightness Temp 1399−1428 MHz (K)

305

300

295

290

285

280 15

20

25

30 35 40 45 Minutes past 19:00 UTC

50

55

60

Figure 2.15: Calibrated brightness 1399-1428 MHz for horizontally polarized observations including additional attenuation. A 21 dB attenuator was used at times less than 19:24 UTC, and a 41 dB attenuator was used at times greater than 19:34 UTC. Between these two times at 71 dB attenuator was used.

48

2.4.2

Integration, APB on

Table 2.2 summarizes LISR observations in Integration, APB on mode. Again, the majority of the results to be shown in this mode will be integrated over the 205 spectra in an antenna observation (approximately 269 msec). Two different threshold settings were used for the APB processor. The symbol β will be used to represent the threshold number of standard deviations above the mean power at which a detection is declared by the APB. A value of β 2 = 90 was used for the observations before 16:17:45 UTC time and a β 2 = 40 value was used for observations later in the day. However, the β 2 = 40 cases were found to contain excessive blanking (on the order of 45% of the data blanked, as could have been predicted from statistical analysis), and so are omitted in the following discussions. Information on the percent of samples blanked is available from the APB processor, and will be discussed in what follows for the time periods before 16:17 UTC. Figures 2.16 and 2.17 plot calibrated brightnesses in vertical and horizontal polarizations, respectively, obtained in this mode for data in Table 2.2 with β 2 = 90. Again the basic qualitative variations with observations of the sky, environment, and terminator are obvious in these Figures. An additional variation occurs in the vertically polarized data near sample 500; in this case, the connector for the V-pol IF signal became loose, so that the V-pol data plotted in this period (samples 479 to 524) is meaningless. An obvious feature of these data compared to those in Figures 2.11 and 2.12 for most of the same antenna configurations is the dramatic reduction of the 1414 MHz source. This is consistent with expected performance of the APB algorithm against pulsed sources, and indicates that the APB approach is highly effective against the ARSR observed for real-time pulsed RFI mitigation. The sources

49

Start Time 13:27:56 14:15:30 14:23:07 14:31:50 14:49:08 14:56:43 15:17:34 15:29:13 15:37:12 15:53:41 15:55:27 16:05:32 16:17:45 16:21:50 16:24:53 16:28:15 18:48:57 18:50:57 18:53:56 19:11:27 19:15:41 19:25:38 19:26:15 19:35:53 19:52:01

Stop Time 13:30:10 14:19:03 14:25:38 14:33:45 14:49:23 14:58:19 15:19:28 15:30:28 15:38:26 15:53:56 15:56:21 16:06:07 16:18:58 16:22:05 16:25:39 16:28:50 18:49:13 18:53:25 18:57:26 19:14:00 19:15:57 19:25:53 19:30:04 19:39:03 19:52:54

Start Sample 1 76 198 285 352 363 417 479 525 568 579 613 637 682 693 715 738 750 804 926 1015 1027 1038 1174 1287

Stop Sample 75 197 284 351 362 416 478 524 567 578 612 636 681 692 714 737 749 803 925 1014 1026 1037 1173 1286 1320

LO (MHz) 1440.5

1440.5 1440.5

1440.5 1440.5

1440.5

1440.5

H-pol state A A A A A A A A A A A A A A A A A A A 21 21 71 71 41 41

V-pol state T T T T A A A A A A A A A A A A A A A A A A A A A

El (deg) 1 50 50 50 50 1 1 1 1 1 1 1 44 89 89 89 0 0 0 0 0 0 0 0 0

Az (deg) 0 0 0 0 0 0 100 135 135 160 160 0 0 0 0 0 x x x x x x x x 170

Table 2.2: Summary of Integration, APB on observations on June 17th. Sample numbers refer to the vertical axis of Figures 2.16 and 2.17. LO frequency is 1386.5 MHz unless otherwise indicated. H- and V-pol states are as follows: A=antenna, T=terminated, and a number indicates an amount of attenuation in dB added between the antenna output and radiometer receiver input.

50

near 1400 and 1403 MHz remain visible and are not strongly affected by the APB processor, indicating that these sources are of a more continuous nature. Comparison of APB on and APB off brightnesses To provide a more quantitative examination of the APB algorithm, Figures 2.18 and 2.19 plot V-pol and H-pol calibrated brightnesses in the band 1399-1428 MHz, respectively, versus time, for both APB on and APB off modes. Vertical lines in the plot mark the boundaries during which the radiometer antenna azimuth angle is varied. Data when the connector of the V pol became loose (around 15:30 UTC time) is omitted in Figure 2.18. Results show the APB on data always to be lower than that of the APB off data, with the difference increasing as 160 degrees azimuth (looking directly at the ARSR) is approached. At 160 degrees azimuth, differences on the order of 20K are observed, while the difference reduces to around 1-2 K at azimuth angle 0 degrees. The APB on data also shows much smaller variations among samples at a fixed azimuthal angle compared to the APB off results (with the exception of the period during which the vertically polarized IF signal was lost), and APB on data never shows the extremely high brightnesses obtained in some of the APB off cases (up to 350 K). These behaviors are similar between the two polarizations. Again these results, combined with the calibration studies of Section 2.3.4 show that the APB algorithm is extremely effective in this RFI environment, even with apparently “weak” pulses when integrated over the radiometer bandwidth. Figure 2.20 is a plot of the percent of samples blanked by the APB processor during the period 13:25 to 16:06 UTC. Results for both the antenna and reference load observations are included. A base level of around 1.6% of samples blanked is observed for the reference load observations throughout most of this period, consistent

51

V pol, blanker on

Sample

300

100

250

200

200

300

150

400 100

500 50

600 0

1400

1405

1410 1415 Frequency (MHz)

1420

1425

Figure 2.16: Calibrated V-pol LISR data in Integration, APB on mode, versus RF frequency. Refer to Table 2.2 for information on the vertical axis of the plot.

52

H pol, blanker on

Sample

300

100

250

200

200

300

150

400 100

500 50

600 0

1400

1405

1410 1415 Frequency (MHz)

1420

1425

Figure 2.17: Same as Figure 2.16, but for H-pol.

53

Doy 167: V pol mean brightness 1399−1428 MHz 350 APB off APB on

V pol Brightness (K)

300

0°

70°

100°

135°

160°

0°

250

200

150

100 50

60

70

80 90 100 Minutes past 14:00 UTC

110

120

Figure 2.18: Calibrated V-pol LISR data in the band 1399-1428 MHz versus time, for both APB off and APB on modes. Text in the plot indicates the azimuth angle of the radiometer antenna, with regions between the vertical lines indicating periods of antenna rotation.

54

Doy 167: H pol mean brightness 1399−1428 MHz 350 APB off APB on

H pol Brightness (K)

300

0°

70°

100°

135°

160°

0°

250

200

150

100 50

60

70

80 90 100 Minutes past 14:00 UTC

110

120

Figure 2.19: Same as Figure 2.18, but for H-pol.

with expected statistical properties of the incoming noise signal and the chosen APB parameters. The percent of antenna samples blanked is consistently higher than that of the reference load, with the exception of the period of time during which the antenna observed the sky at elevation angle 50 degrees. Following these observations, the percent of antenna samples blanked is found to increase as the radar observation direction is approached, up to around 4.5% in some cases. Given the radar’s reported duty cycle of 1.44%, combined with the basic noise power blanking of 1.6%, the obtained value near 3% for many of the antenna observations appears reasonable. Higher values indicate that a single radar pulse is resulting in multiple APB 40 µsec blanking windows, which is a possibility for the APB processor. Overall, the behavior of these statistics is consistent with expectations for the APB system.

55

APB Statistics 167 5 Antenna

4.5

Reference load Percent of samples blanked

4 3.5 3 2.5 2 1.5 1 0.5 0 0

50

100 Minutes past 13:00 UTC

150

200

Figure 2.20: Percent of samples blanked by the APB processor

One variation that has not been explained involves the increased blanking levels observed for times near 13:25 UTC (as the experiment was beginning). This might be related to an error in the tabulated value of the azimuth angle.

2.4.3

Capture mode data

In capture mode, 6 256 K captures sampled at 10 nsec (a total 15.7 msec) are acquired on each antenna observation. This high time resolution allows detailed temporal properties of the RFI sources to be observed. Note that each capture is 2.6 msec long, near the PRF rate of the radar system, so that each capture contains at 56

Start Time 13:30:49 14:19:42 14:26:09 14:58:42 15:20:07 15:30:50 15:38:44 15:56:49 15:59:34 16:06:36 16:09:50 18:34:17 18:39:04 19:44:12 19:45:03 19:46:07 19:50:20

Stop Time 13:32:54 14:20:12 14:26:33 14:59:03 15:20:19 15:31:34 15:39:00 15:57:15 15:59:40 16:06:53 16:10:04 18:38:12 18:43:30 19:44:32 19:45:35 19:47:09 19:51:23

LO (MHz) 1440.5 1440.5

1440.5

1440.5

H-pol state A A A A A A A A A A A A A 41 41 41 41

V-pol state T T T A A A A A A A A A A A A A A

El (deg) 1 50 50 1 1 1 1 1 1 1 44? 0 0 0 0 0 0

Az (deg) 0 0 0 0 100 135 135 160 0 0 0 x x 170 170 170 170

Comment

Radar@18:35:53 Radar@18:39:55

Table 2.3: Summary of Capture mode observations on June 17th. LO frequency is 1386.5 MHz unless otherwise indicated. H- and V-pol states are as follows: A=antenna, T=terminated, and a number indicates an amount of attenuation in dB added between the antenna output and radiometer receiver input.

most one radar pulse, and in some cases, no radar pulses. Table 2.3 summarizes the capture data measured on June 17th. A variety of means exist for examining the capture data, including a detailed pulse detection study of the type described in [41]. Here only a few basic pulse properties are illustrated. Capture mode data is presented as the raw LISR outputs, no calibration is applied. Figure 2.21 plots an example pulse observed near time 14:58:42 in both horizontal and vertical polarizations. During this time, the radiometer antenna is directed at azimuth angle 0 degrees, so that radar pulses observed are weak compared to those

57

observed with the antenna directed toward the radar. The horizontal axis in the plot is time in microseconds; only a portion of the 2.6 msec capture is plotted. The pulse observed is significantly higher in amplitude than the surrounding noise power, and slightly stronger in vertical than in horizontal polarization. The pulse width is observed to be near 2 µ sec, as reported in the ARSR literature, although other cases show longer pulses due to possible multi-path effects.

Pulse near doy 167 14:58:42 UTC 5000 H pol V pol

4500 4000

Pulse amplitude

3500 3000 2500 2000 1500 1000 500 0 0

2

4

6

8

10 12 Time (µ sec)

14

16

18

20

Figure 2.21: Example radar pulse observed near time 14:58:42; amplitude level of V pol data is shifted by 2500 for clarity purposes

Several radar pulses are illustrated Figure 2.22, which plots a set of captures on the vertical axis along with time in µsec on the horizontal axis. The raw LISR data here is plotted in decibels, and the captures utilized were obtained during a scan of

58

the radiometer antenna in azimuth near time 18:39 UTC. The vertical axis label of this plot explains that captures 1 through 12 were obtained near azimuth angle -90 degrees, while captures 13 through 19 were obtained near azimuth 160 degrees, and captures 20 through 30 were obtained near azimuth 0 degrees. These three cases contain moderate, strong, and weak radar pulses, respectively. The figure arranges the pulses in time by attempting to place the maximum pulse amplitude observed at time zero; this results in some shifting of the pulses between captures, but still allows comparison of these cases. Results show the expected near 2 µsec pulse width in the moderate and weak RFI cases, but a significant increase in the pulse width to 3-4 µsec during direct observation of the radar. Further analysis of these data shows clear evidence of receiver saturation during times in which the radiometer antenna was directed toward the ARSR location. An strong correlation of the pulse amplitudes observed with the radiometer antenna azimuth angle is also clearly present. Spectral analysis of these pulses shows their spectrum to be centered near 1414 MHz. To allow analysis of a large number of capture files, a reduction process was utilized in which capture power was integrated to a time resolution 1.28 µsec and stored. Figure 2.23 is a plot of the maximum raw data value observed following this power integration for a set of captures taken from times 18:34 to 18:44 UTC. During this period, the antenna azimuthal angle was increased in a continuous sweeping process from directly North at 18:34 through a full rotation then to 60 degrees West of North at 18:37:30. The antenna was directed toward the radar at times near 18:36. The antenna azimuth remained fixed toward the 60 degrees west of North from times 18:37:30 to 18:38:50. The rotation was reversed at 18:38:50, reaching the radar at times near 18:40 and paused at 18:42 again at 60 degrees West of North. 59

H pol Time domain pulses near 18:39 UTC: Amplitude in dB

(1:12)−90 Az, (13:19) 160 Az, (20:30) 0 Az

80

75

5

70

10 65

15

60

55

20

50

25 45

30 −5

40

−4

−3

−2

−1 0 1 Time (µ sec)

2

3

4

5

Figure 2.22: Thirty radar pulses obtained during a sweep over azimuth near time 18:39 UTC in horizontal polarization. The vertical axis label indicates the approximate azimuth angle of each of three sets of captures.

60

Maximum capture values obtained from the horizontally polarized antenna observations show an obvious correlation with the azimuth sweeping process described, along with an obvious absence of these effects in the reference load captures. Note the occasional small capture maxima obtained occur when a 2.6 msec capture fails to contain a radar pulse. The broad maxima versus time in the antenna data again indicate saturation effects in the receiver.

Doy 167 H pol 100 Antenna Reference

Capture Raw Date Maximum (dB)

95

90

85

80

75

70

65

60 34

35

36

37

38 39 40 Minutes past UTC 18:00

41

42

43

44

Figure 2.23: Maximum capture raw-data amplitudes observed (following integration to 1.28 µsec resolution) in horizontal polarization during sweeps over azimuth from 18:34 to 18:44 UTC. Reference data is interpolated in the intervals with no measurements. Refer to the text for information on the map from time to azimuthal observation angle.

Polarimetric processing of the capture data is potentially possible, given the coherent recorded simultaneous horizontally and vertically polarized data. A simple 61

observation of polarization correlations among the capture data showed the expected noise-like behavior when pulses were not present, along with dramatic increases during pulse observations.

2.4.4

Post-processing

Although the APB algorithm has been shown to be effective against the primary interference source in this dataset (the ARSR), other more continuous sources were also observed that are not affected by the APB. However the high spectral resolution of LISR still allows cross-frequency mitigation algorithms to be developed for removal of these sources in post-processing. Such algorithms along with additional pulsed source removal algorithms can be applied to both APB off and APB on data in an attempt to discern the degree of RFI remaining within data acquired in these modes. Two primary algorithms were developed for this purpose. The first algorithm is primarily for the removal of pulsed-type sources, while the second is for CW interferers. The former algorithm utilizes the raw LISR data, while the latter utilized calibrated data, both of which are obtained as 205 1024 point spectra (each representing 1.3 msec) for each antenna observation. The pulsed source algorithm examines the maximum and mean values for the 205 raw LISR data points in each frequency bin. The max to mean ratio is computed and used as the detection statistic; each antenna observation thus produces 512 ratio values for a given antenna observation and polarization. The 512 ratio values are sorted, and the mean and standard deviation of the lower 85% of these values are computed. Bins for which the ratio value is greater than 5 standard deviations from its mean are then deemed corrupted throughout the antenna observation. Calibrated brightnesses corresponding to the corrupted bins as well as the brightnesses within 4

62

bins of a corrupted bin are all replaced with the mean brightness of the points deemed RFI-free. Due to the emphasis on time variations of the max over mean ratio, this algorithm is suited for detecting and removing pulsed sources. The cross-frequency algorithm is applied following the pulsed source algorithm. In this algorithm, the 512 point LISR brightnesses at 269 msec time resolution are integrated over frequency to a 3 MHz spectral resolution. An interpolated version of this lower frequency resolution curve is then generated at the 512 point spectral resolution. The detection statistic utilized is obtained by subtracting the low frequency resolution curve from the original LISR brightnesses. Again the mean and standard deviation of the low 85% values of this statistic are computed, and RFI detection is declared when the LISR brightness exceeds the low frequency resolution curve by 3 standard deviations. Bins deemed as containing RFI have their brightnesses as well as those of the surrounding 4 bins replaced by the mean brightness of the RFI-free channels. Due to the emphasis of this algorithm on detecting variations in frequency, this is a narrowband source detection algorithm. Codes for both these algorithms were developed and applied to the LISR observed data for this campaign. Results from this process will be illustrated in the following detailed analysis of two specific datasets.

2.5 2.5.1

Observations Near 14:51 and 16:03 UTC Observations near 14:51 UTC

At times between 14:50 and 15:00 UTC, the radiometer antenna was directed to the North (away from the radar) at elevation angle 1 degree. The LO frequency was set to 1440.5 MHz. This configuration represents a fairly weak RFI environment, due

63

to the small antenna gain toward the radar. Data was acquired in both APB off and APB on modes during this period of time. Figure 2.24 is an image of vertically polarized brightnesses in APB off mode from times 14:51 to 14:56. Though this is a somewhat weak RFI environment, ARSR transmissions (1414 MHz) are obvious in the figure, as well as narrowband interference near 1400 MHz. Horizontally polarized images are similar and are not shown. Figure 2.25 illustrates the same data as in Figure 2.24, but following application of the post-processing RFI removal techniques described in the previous section. The processing algorithms have clearly removed both the 1414 and 1400 MHz sources, replacing them by the mean over all brightnesses deemed acceptable. While this process produces some distortion of the image, the overall effect is to remove RFI while retaining a reasonable mean brightness over the radiometer passband. Figure 2.26 compares mean channel brightnesses in both horizontal and vertical polarizations for these images before and after the RFI removal processing. The channels considered correspond roughly to those of the UM ADD system, although a simple rectangular weighting over a 3 MHz bandwidth is utilized here rather than a particular filter shape. Results show the algorithm to produce a dramatic reduction in channel brightnesses near the 1414 MHz source (around 10 K in the fifth channel), while showing only small corrections in other channels. Note the narrow bandwidth of the 1400 MHz source results in this source having almost no effect even on the first modeled channel. Figure 2.27 is an image of calibrated V-pol brightnesses in APB on mode for the period 14:56:43 to 14:58:18. Results show the APB algorithm again to be effective in removing ARSR emissions; the 1400 MHz source however is not affected, suggesting again that this source has a more continuous nature. 64

Vertical APB off 167−145142 240

20 220

40 200

60 Sample

180

80

160

100

140

120

120

140

100

1400

1405

1410 1415 Frequency (MHz)

1420

1425

80

Figure 2.24: Calibrated V-pol LISR data in Integration, APB off mode, versus RF frequency, near times 14:51:43 to 14:56:15 UTC (Samples 312-468 from Table 2.1)

65

Vertical RFI removed APB off 167−145142 240 20 220 40 200

Sample

60

180

80

160

100

140

120

120

140

100

1400

1405

1410 1415 Frequency (MHz)

1420

1425

80

Figure 2.25: Same as Figure 2.24, but following post-processing described in Section 2.4.4

66

Horizontal, APB off 167−145142 Mean brightness (K)

165 Original Automatically Corrected

160 155 150 145 1402.95

1405.98

1408.96 1411.94 1414.96 1417.94 UMich Radiometer Channel Center Freq (MHz)

1420.92

1423.95

1420.92

1423.95

Vertical, APB off 167−145142 Mean brightness (K)

190 185 180 175 170 1402.95

1405.98

1408.96 1411.94 1414.96 1417.94 UMich Radiometer Channel Center Freq (MHz)

Figure 2.26: Calibrated LISR data in channels similar to the UM ADD system, near times 14:51:43 to 14:56:15 UTC (Samples 312-468 from Table 2.1); results included before and after post-processing

67

Vertical APB on 167−145642 240

5 10

220

15

200

20 Sample

180

25 30

160

35

140

40 120

45 100

50 1400

1405

1410 1415 Frequency (MHz)

1420

1425

80

Figure 2.27: Calibrated V-pol LISR data in Integration, APB on mode, versus RF frequency, near times 14:56:43 to 14:58:18 (Samples 363-416 from Table 2.2)

68

Horizontal, APB on 167−145642 Mean brightness (K)

154

Original Automatically Corrected

152 150 148 146 1402.95

1405.98

1408.96 1411.94 1414.96 1417.94 UMich Radiometer Channel Center Freq (MHz)

1420.92

1423.95

1420.92

1423.95

Vertical, APB on 167−145642 Mean brightness (K)

178

176

174

172 1402.95

1405.98

1408.96 1411.94 1414.96 1417.94 UMich Radiometer Channel Center Freq (MHz)

Figure 2.28: Calibrated LISR data in channels similar to the UM ADD system, near times 14:56:43 to 14:58:18 (APB on, Samples 363-416 from Table 2.2); results included before and after post-processing

69

Figure 2.28 compares mean channel brightnesses in APB on mode before and after RFI removal post-processing. In this case, brightnesses remain within 0.5K before and after post-processing, indicating that the APB algorithm has removed detectable ARSR emissions. The small changes observed are likely due to false detections in postprocessing. A comparison of brightnesses in Figures 2.26 and 2.28 shows variation of up to 3-4 K between corrected APB off and APB on data. The source of these changes is not known, but because the measurements are not simultaneous, it is possible that variations in the environmental brightness at this level could occur over this time span, as well as possible errors introduced by temperature variations in the receiver.

2.5.2

Observations near 16:03 UTC

A similar dataset is available from times 16:03 to 16:07 UTC; at this time the antenna configuration was identical to that at times near 14:50. Again both APB off and on data are available, and images are similar to those shown in the previous section. Figure 2.29 is an plot of mean brightnesses before and after post-processing for the APB off mode, while Figure 2.30 is the corresponding plot with the APB on. In this dataset, the ARSR emissions are weaker than those at 14:51, perhaps due to a slight change in the radiometer azimuth angle. Results show the post-processing algorithm again to be effective at removing ARSR emissions near channel 5, although in this case the maximum correction is only approximately 2 K. Results in APB on mode show a much smaller effect of post-processing, again indicating that the APB is removing ARSR pulses even when their net effect on channel brightnesses is weak. Comparison of corrected brightnesses in Figures 2.29 and 2.30 shows them to be within 1-2 K, a reasonable difference given the time delay between the two observations.

70

Horizontal, APB off 167−160339 Mean brightness (K)

156 Original Automatically Corrected

154 152 150 148 1402.95

1405.98

1408.96 1411.94 1414.96 1417.94 UMich Radiometer Channel Center Freq (MHz)

1420.92

1423.95

1420.92

1423.95

Vertical, APB off 167−160339 Mean brightness (K)

180 179 178 177 176 175 1402.95

1405.98

1408.96 1411.94 1414.96 1417.94 UMich Radiometer Channel Center Freq (MHz)

Figure 2.29: Calibrated LISR data in channels similar to the UM ADD system, near times 16:03:41 to 16:04:14 UTC (Samples 1358-1379 from Table 2.1); results included before and after post-processing

71

Horizontal, APB on 167−160532 Mean brightness (K)

156 154

Original Automatically Corrected

152 150 148 1402.95

1405.98

1408.96 1411.94 1414.96 1417.94 UMich Radiometer Channel Center Freq (MHz)

1420.92

1423.95

1420.92

1423.95

Vertical, APB on 167−160532 Mean brightness (K)

182 180 178 176 174 1402.95

1405.98

1408.96 1411.94 1414.96 1417.94 UMich Radiometer Channel Center Freq (MHz)

Figure 2.30: Calibrated LISR data in channels similar to the UM ADD system, near times 16:05:32 to 16:06:06 (APB on, Samples 613-636 from Table 2.2); results included before and after post-processing

72

Overall these results quantitatively demonstrate LISR capabilities at detecting and removing RFI through the use of high temporal and spectral resolution information. The APB is also quantitatively demonstrated to remove pulsed sources so that further post-processing yields no pulsed source detection, even in apparently weak RFI environments.

2.6 2.6.1

Laboratory Observations on June 20th and 21st June 20th data

Observations on June 20th involved removal of the radiometer antenna from the truck boom and placing it on the ground in an upward viewing geometry (near elevation angle 62 degrees). Artificial RFI was introduced into the experiment by placing a small log-periodic dipole antenna on the horn antenna aperture; this antenna was used to radiate power produced by a bench source. Source emissions were centered at either 1398 MHz or 1412 MHz, and were varied to either CW or pulsed type transmissions. Pulses were set to use a PRF of either 360 or 720 Hz, along with pulse widths of either 2 or 4 µsec. Source output power was varied over a range of 35 dB, from -20 dBm to -55 dBm. Figures 2.31 and 2.32 are images are H-pol brightnesses from June 20th, in APB off and APB on modes, respectively. A detailed log of the map from sample number in the figures to source properties is not listed here, but the presence of RFI of varying intensity at 1398 and 1412 MHz is obvious in Figure 2.31. In addition, narrowband emissions at 1400 and 1403 MHz are again observed, though these were likely not produced by the RFI generator, but rather are sourced by emissions from other electronic equipment. The APB on image in Figure 2.32 shows a dramatic reduction in RFI; the sources observed near 1412 MHz occur during the period of CW operation 73

Doy 170 H pol APB off 200

100

180

200

160

300 140

400 Sample

120

500 100

600 700

80

800

60

900

40

1000

20

1100 1400

1405

1410 1415 1420 Frequency (MHz)

1425

1430

0

Figure 2.31: Calibrated H-pol LISR data in Integration, APB off mode, versus RF frequency, on June 20th

74

Doy 170 APB on H pol 200

180

100 160

140

200 Sample

120

300

100

80

400 60

40

500

20

600 1400

1405

1410 1415 1420 Frequency (MHz)

1425

1430

Figure 2.32: Same as Figure 2.31 but in Integration, APB on mode

75

0

of the RFI generator only (samples 70 to 125), while pulsed 1412 MHz emissions of varying power levels were included during samples 126 to 259 and 470 to 620. Periods when the 1398 MHz source is observed correspond to periods when the RFI source power was set to lower values: due to this low power level as well as the fact that the 1398 MHz source is reduced by the radiometer front end filters, these pulses were of insufficient amplitude to be detected by the APB algorithm, and therefore remain in the data. However, data between the two periods of 1398 MHz interference in Figure 2.32 (i.e. samples around 320-420) contain 1398 MHz interference at a power level sufficient to trigger the APB detector. Again the APB algorithm does not affect the 1400 and 1403 MHz emissions. While examination of the APB on mode brightness integrated over the radiometer bandwidth as RFI source properties are varied is of interest, such comparisons are hindered by the fact that the RFI source generator apparently also produced a lowlevel, broadband noise output that varied with the generator power level. Analysis of capture datasets from June 20th and 21st were performed to provide evidence of this fact. Therefore no further analysis of this dataset is reported here.

2.6.2

June 21st data

Data on June 21st was acquired inside the laboratory, and involved observations of a well characterized liquid nitrogen target, with additional RFI introduced through a coupler into the radiometer datastream. These measurements were conducted only in a single polarization, with the other receiver polarization input being terminated. Again RFI source properties were varied throughout these observations in a manner similar to that used on June 20th. In addition, pads of 0, 1, 2, or 3 dB were introduced onto the calibration target input in order to produce well-characterized changes in

76

the calibration target brightness. Mean brightnesses observed by LISR in this process were reported in Section 2.3.4. Figure 2.33 is an image of V-pol APB off brightnesses during the time period 18:15 to 19:26 UTC. Variations in the mean brightness are observed throughout this plot as pads are added or removed from the calibration target. RFI emissions at 1398 and 1412 MHz are also obvious, although the period from sample 452 to 518 is specified as including 1412 MHz source emissions at power level -83.5 dBm. Note apparent sources at frequencies less than 1398 MHz or larger than 1427 MHz are due to calibration problems for these far out-of-band frequencies when the LO is set to 1440.5 MHz. Figure 2.34 is an image of H-pol APB on brightnesses during the time period 19:33 to 20:30 UTC. Note the APB mode was varied only at time 19:30 so that APB off and on mode data are not available for observations with identical source settings and polarizations. Results show all 1412 MHz emissions to be removed, while 1398 MHz emissions remain only in cases where the source power level was set to -63 dBm so that pulse powers when filtered by the front end filters of the radiometer were insufficient to trigger the APB detector.

2.7

Summary and Remarks

Results of Canton campaign show the capabilities of the LISR system at detecting and removing pulsed interference in real-time while maintaining calibration of pulsefree noise, as well as the RFI detection and removal that can be accomplished in post-processing based on LISR’s high temporal and spectral resolution. Analysis of the capture data also allows detailed properties of ARSR emissions to be examined. It was shown that the APB algorithm is sufficiently conservative so that data is removed 77

Doy 171, V pol APB off 100 200

Sample

300 400 500 600 700 800 1390 1395 1400 1405 1410 1415 1420 1425 1430 1435 Frequency (MHz)

Figure 2.33: Calibrated V-pol LISR data in Integration, APB off mode, versus RF frequency, on June 21st

78

Doy 171, H pol APB on 350 50 300

100 150

250

Sample

200 200

250 300

150

350 100

400 450

50 500 550 1400

1405

1410 1415 1420 Frequency (MHz)

1425

1430

0

Figure 2.34: Same as Figure 2.33 but for H-pol in Integration, APB on mode

79

only when obviously corrupted, and hence is suitable for implementation in a space based radiometer. Algorithms utilized here in post-processing for RFI removal were the starting point for the algorithms used in campaigns that will be described in the next few sections, as well as the simplified algorithm used in theoretical comparisons.

80

CHAPTER 3

L-BAND INTERFERENCE SUPPRESSING RADIOMETER (LISR) GROUND-BASED OBSERVATIONS WITH JPL’S PASSIVE-ACTIVE L/S BAND (PALS) SENSOR

Between April 24th and May 12th, 2006, a ground-based measurement campaign was conducted at the Jet Propulsion Laboratory (JPL) (Pasadena, California) to demonstrate advanced radio frequency interference (RFI) mitigation methods for Lband microwave radiometry. Backend RFI mitigating systems included in the campaign were the L-Band Interference Suppressing Radiometer of The Ohio State University , ADD of the University of Michigan, and the Double-Detector system developed by NASA Goddard Space Flight Center as was the case in Canton campaign. These three backend systems simultaneously observed IF signals provided from the L-band radiometer of the Passive/Active L-S band sensor (PALS) of JPL [51,52]. The goals of the campaign included identification and mitigation (if possible) of RFI observed by PALS as well as intercomparison of the results from the multiple backends in order to assess the RFI mitigation performance of each. This chapter describes results obtained by the LISR backend along with comparisons of brightness temperatures with other sensors when available. The next section describes the basic components of the PALS radiometer front-end, modifications to the LISR backend made for this campaign and the modes of observation used for 81

LISR. An overview of the dataset and its basic properties, along with a description of the method used for state identification and calibration, is provided in Section 3.2. Section 3.3 then summarizes LISR observations on specific dates of the campaign, while Section 3.4 describes properties of the RFI observed and RFI mitigation techniques developed for these sources. A final analysis and overall conclusions for this campaign follow in Section 3.5.

3.1 3.1.1

System Configuration PALS radiometer front end and downconversion stages

The Passive/Active L/S-band Sensor ( [51,52]) of JPL has been deployed in several remote sensing field campaigns in the past, and has provided important information regarding the use of combined radar/radiometer instruments for observing several geophysical quantities while emphasizing soil moisture and sea surface salinity measurements in particular. Only the radiometer portion of PALS, which operates at L-band, was used in the campaign described here. The PALS configuration for the campaign involved zenith looking sky observations using a dual-polarized L-band patch array antenna located on the roof of building 168 at JPL, similar to the configuration described in [51]. Given the expected slow variation with time of the observed sky brightness, any RFI varying rapidly in time should be readily discernible. The relevant portion of the PALS radiometer front end has dual channels for the two polarizations; a hybrid coupler is also included to allow measurement of the third Stokes parameter, but these data were not recorded by any of the backend systems, except that of PALS. For each channel, a portion of the front end following the antenna is located on the roof; this portion includes a Dicke switch to allow 82

observation of a temperature controlled reference load, a coupler to allow injection of a known noise brightness, a bandpass filter centered at 1415 MHz with a 3 dB bandwidth of 60 MHz, and a low noise amplifier with a gain of 36 dB. The number of poles of the bandpass filter was not provided, but it assumed to be a modest number in order to minimize the impact on system noise figure. The 1415 ± 30 MHz output of this front end is then fed down cables approximately 100 ft long from the roof into a laboratory in building 168. In the laboratory, the RF signals are again bandpass filtered using a 1413 ± 37.5 MHz filter (number of poles again not provided), passed through an amplifier with 31 dB gain, and then mixed with an LO at 1210 MHz to obtain an IF signal occupying an approximate bandwidth of 205±30 MHz. Note that 67 dB of gain neglecting cable and other component losses is applied to the nominal band 1415 ± 30 MHz prior to the mixer; a review of the mixer specifications [53] shows 1 dB compression when the total input power reaches +7 dBm. IF signals are then passed through a 300 MHz lowpass filter followed by a final band-defining filter of 200 ± 10 MHz. After an additional 40 dB of gain and a second 300 MHz low pass filter, a portion of the IF signal (now nominally 200 ± 10 MHz) is coupled out for use by the backend systems. The remainder of the IF signal passes into the PALS backend system, which performs measurements of the first three brightness Stokes’ parameters. Note the use of the 1210 MHz LO frequency results in the IF band representing the RF frequencies 1400 − 1420 MHz at the 3 dB point, while the protected portion of L-band ranges from 1400 to 1427 MHz. The double-detector backend system of NASA GSFC directly observed the PALS IF provided, but both ADD and LISR used an additional downconversion (LO 173 MHz) and gain stage to center the IF signal at 27±10 MHz. Component details 83

on the second stage downconverter are not available. The final input to the LISR backend is two 27 ± 10 MHz IF cables containing the horizontally (H) and vertically (V) polarized radiometer channels as in the Canton campaign. For calibration purposes, JPL reported that the effective additional brightness provided by the noise diode was approximately 500 K. Measured front end temperature information was not provided, but JPL personnel stated that the reference load physical temperature could be reasonably estimated to be 33 C during overnight hours. These estimates were used in obtaining the LISR calibrations to be reported. Note JPL local time during the campaign was 7 hours behind UTC; sunset occurred around 1:30 UTC.

3.1.2

PALS state timing

The PALS front-end state was controlled by PALS during the campaign; the basic timing cycle consecutively observed the antenna+noise diode (Ant+ND), antenna, and reference loads for 4.2 msec each. Each 4.2 msec interval was further divided into 12 350 µsec periods, within which the first 300 µsec were used for radiometer measurements while the following 50 µsec were not used. These 50 µsec are used by the radar when PALS is performing active/passive measurements; during these times the radiometer observes the reference load in all states. Figure 3.1 provides plots of the radiometer state control lines versus time in order to illustrate the basic timing cycle. The “R-line” plot illustrates the state of the reference load Dicke switch, with high values indicating that the reference load is observed. The “N-line” plot corresponds to the noise diode on/off state, with a high value indicating that the noise diode is on. Finally a high value on the “I-line” represents that radiometer measurements should be performed; the low values of the

84

R−Line

PALS State Diagram

1

0

2000

4000

I−Line

8000

10000

12000

REF

ANT

ANT+ND 1

0

N−Line

6000

2000

4000

2000

4000

6000

8000

10000

12000

6000 8000 Time (µ secs)

10000

12000

1

0

Figure 3.1: PALS state diagram

85

Figure 3.2: LISR block diagram

“I-line” mark the 50 µsec excluded intervals during each 350 µstate interval. Note that JPL also recommends that data from the first 350 µsec interval within a 4.2 msec state be discarded.

3.1.3

LISR overview

Figure 3.2 is a block diagram of LISR as used in this campaign. The basic configuration is very similar to that described in Chapter 2 for the Canton campaign, and will not be repeated here. However, several modifications were required as it will be readily explained.

86

As in Canton campaign, LISR was operated in three distinct modes: “Integration, APB off”, “Integration, APB on”, and “Capture” modes. LISR capture mode in this campaign was designed to acquire 28K samples (286.72 µsec) during a 300 µsec antenna or reference observation. Due to the large amount of data recorded and the need to transfer recorded data to the control computer, a duty cycle of approximately 5% was achieved for capture mode observations (i.e. one 286.72 µsec observation per 18 350 µsec intervals.) However, the acquired data remains useful due to its high temporal resolution and coherency, which allows detailed studies of the temporal properties of the observed interference. With the exception of Section 3.2.2, results from LISR capture mode observations are not presented in detail in this work. However, conclusions obtained from the capture mode data are consistent with those to be described in what follows. For the integrating modes in this campaign (i.e “Integration, APB on” and “Integration, APB off” modes), the basic FPGA output data unit was an integration of 3 FFT outputs; this data unit consists of the power in 1024 frequency channels integrated over a time period of 30.72 µsec and reported in 16 bits. These data units are obtained every 40.96 µsec from the FPGA, because an additional 10.24 µsec is needed to write the data unit out of the FPGA. Seven of these data units therefore can be recorded within the 300 µsec radiometer integration period. This report focuses on the “Integration, APB off” data, and all following data in an integrating mode should be assumed to be with “Integration, APB off”. Although the “Integration, APB on” mode has been shown to be highly effective against pulsed interference in Chapter 2, an analysis of the “Integration, APB on” PALS campaign data shows little advantage over “Integration, APB off,” due to the configuration of

87

blanker parameters which were not optimized for the relatively rapid switching of radiometer states utilized.

3.1.4

LISR computer and control interface

The LISR FPGA is free running and not synchronized to any external time control. FPGA output data units are passed to a “capture board” containing a smaller FPGA as well as a 16 bit, 256 K FIFO that serves as a buffer to a data-recording and control computer. In past campaigns including Canton campaign, external time information such as “trigger” signals that tell LISR when to begin a measurement were typically passed into the control computer, which then through a program running on the control computer could reset the buffer FIFO’s to begin recording data following a trigger signal. This process makes clear that LISR was designed primarily for radiometer systems operating with integration periods on the order of milliseconds. In such systems, an increased level of software system control is acceptable, because time delays introduced by software components remain negligible compared to the overall radiometer time scales. In this campaign however, timing intervals as short as 50 µsec required modification of several LISR computer and control aspects in order to achieve reasonable measurements. To avoid delays introduced by the software triggering described above, LISR was modified to input the trigger line directly into the capture board, allowing the trigger signal to reset the FIFO’s directly. Software on the control computer was revised to communicate with the capture board so that FIFO resets are allowed only when the computer is ready. This modification allowed successful hardware triggering of LISR in the campaign.

88

FIFO output data is passed to the control computer through a National Instruments high speed digital I/O card. An additional system timing issue was introduced by total latencies and transfer times on the order of 1.75 msec associated with this card; such delays are encountered following every 350 µsec period observed due to the absence of synchronization control of the LISR main FPGA. Modifications to LISR were implemented to reduce the impact of these latencies. FPGA data units were modified from the previous width of 32 bits to 16 bits; this change allowed only a single FIFO to be read instead of two FIFO’s (there are 2 FIFO’s actually on the capture board) read in the Canton campaign. The LISR FPGA was modified appropriately, including a saturation of the data unit output if 16 bits were found insufficient. While the data recorded during a 300 µsec radiometer observation consists of 7 1024 point spectra each integrated for 30.72 µsec, the free running nature of the LISR FPGA also requires that the first of these spectra be discarded, since the precise time at which the first integration began relative to the trigger signal is not known. Therefore the final LISR data unit consisted of 6 30.72 µsec spectra, or 184.32 µsec observed out of 300 µsec available. Given the required 1.75 msec to transfer this data, the final duty cycle of LISR in the integrating modes is around 10%, and the final data output for a single trigger is recorded approximately every 2.1 msec. Following the recording of 8743 of these data units in approximately 18.6 seconds, the LISR computer paused to write data to its internal hard drive. To reduce data storage requirements on this hard drive, the average of the 6 obtained spectra in each trigger was recorded; the final dataset consists of 512 point spectra in each of H and V pols representing the bands 1383 to 1433 MHz at 184.32 µsec time resolution. During the experiment, LISR alternated between recording files successively in “Integration, 89

APB off” and “Integration, APB on” modes, with every twentieth file in “Capture” mode. Files were approximately 17 MB in all modes; the final data rate was around 2.6 GB per hour.

3.2 3.2.1

Dataset Overview and Calibration Observed gain pattern

To illustrate basic spectral properties of the PALS IF signal, Figures 3.3 and 3.4 plot the mean raw power (in dB) versus frequency obtained by LISR for reference load observations in vertical and horizontal polarizations, respectively. Data is normalized so that the maximum value of the frequency response is 0 dB. Averages are taken over an entire file (around 18.6 seconds real time, 0.54 seconds reference load integration time) for two example files acquired within 13 minutes of each other during the April 24th observations. Lines in the Figures mark the approximate gain level of -3 dB, and confirm a 3 dB passband of about 1400 (one of the protected spectrum boundaries) to 1420 MHz. The gain at the protected spectrum boundary 1427 MHz is found to be around 13-16 dB below the maximum. The figures show relatively similar but non-identical behaviors for vertical and horizontal polarizations, due to the varying frequency responses of these separate receiver channels. Passband plots for the two files show slight differences, indicating the potential for change in the passband characteristics with time. Such changes are observed in calibrated data as well, and will be described further in Section 3.3.2. While this passband appears acceptable, it seems somewhat wide with regard to possible RFI effects for a spaceborne instrument. Given the possibility of RFI sources up to 1398-1399 MHz, the passband of the instrument utilized may not provide sufficient reduction of these slightly out-of-band sources. It is recommended in future 90

Normalized Average Raw Power in Reference Observations for Vertical Polarization 5

data_114_222946_000002_00 data_114_224200_000002_00

Normalized Power in dB

0

−5

−10

−15

−20

1395

1400

1405

1410 1415 Frequency (MHz)

1420

1425

Figure 3.3: LISR measured reference load raw power versus frequency, vertical polarization

91

Normalized Average Raw Power in Reference Observations for Horizontal Polarization 5

data_114_222946_000002_00 data_114_224200_000002_00

Normalized Power in dB

0

−5

−10

−15

−20

1395

1400

1405

1410 1415 Frequency (MHz)

1420

1425

Figure 3.4: LISR measured reference load raw power versus frequency, horizontal polarization

92

deployment of the PALS system at a minimum that an LO frequency of 1213 MHz be used, in order to place the observed passband more within the boundaries of the protected spectrum. A narrower second RF passband filter or filters could also be advantageous, although such filters would require a large number of sections.

3.2.2

Noise diode delay

One radiometer state issue that appeared in analysis of LISR capture mode data involves apparent transients in the noise diode turn on response. Figure 3.5 plots the horizontally polarized raw power vs. time obtained from LISR capture mode observations averaged over all Ant+ND state observations from a single file obtained on April 27th. Plots for vertical polarization (not shown) show nearly identical behaviors. A settling time of about 100 µsec following the trigger signal (time zero in LISR capture mode observations) is observed from the plots. There is another transient state beginning around 260 µsec, but the 286.72 µsec capture duration does not allow analysis of the turn off response for the full 300 µsec interval. This behavior was observed throughout the experiment for Ant+ND states, and remains relatively consistent for individual Ant+ND triggers. Tests performed following the JPL campaign showed that capture mode delays inherent in the LISR system were less than 1 µsec following a trigger signal. Data acquired by GSFC at 2 µsec time resolution (and using the 200 MHz IF of PALS directly) also showed a similar transient response. It appears that this delay is related to a transient response within PALS, although JPL personnel have indicated that the source of this transient is unknown at that time. As a result of this behavior, the noise diode effective temperature reported for PALS, while still allowing accurate PALS calibration, is likely to be inaccurate for systems not integrating over precisely the time period used by PALS, including LISR

93

6

x 10

Averaged Raw Power over Time for File data_117_234605_000105_02

1.6

1.5

1.4

Raw Power

1.3

1.2

1.1

1

0.9

0.8

0.7 50

100

150 Time (µ sec.)

200

250

Figure 3.5: Raw power vs. time in the Ant+ND state for the capture data

94

due to the use of only 184.32 out of 300 µsec available. Such inaccuracies will create biases in final calibrated brightness temperatures. Therefore the ability to quantitatively compare brightnesses among the multiple systems within the campaign is limited. Qualitative comparisons remain possible however, and can still provide a good indication of RFI detection and mitigation effectiveness.

3.2.3

LISR recorded datasets

The campaign began on Monday, April 24th (local time), with overnight PALS observations performed (local time, beginning day) on the 24th, 25th, 26th, and 27th. The campaign was then halted due to other commitments by campaign personnel, and resumed for overnight observations on May 8th, 9th, 10th, and 11th. Table 3.1 summarizes datasets recorded by LISR during the campaign. While several small datasets were often recorded during the day, and are included here for completeness, the larger datasets were primarily acquired during overnight observations. The best overnight datasets acquired by LISR occurred on the 24th, 25th, and 27th, with a smaller overnight dataset obtained during the early morning of May 11th. At other times, LISR encountered either long-term saturation or missing data problems. The latter were later found to be associated with a loose wire on one of the ADC card power supply inputs; unfortunately correction of this problem was exacerbated by the fact that OSU personnel were unable to be present at JPL during the second campaign week. The source of the long-term saturation problem is not known; while this could be caused by the presence of RFI, by changes in overall system gain, or intermittent LISR changes, it is difficult to reach any definitive conclusion.

95

Day (UTC) April 24th April 24th April 24th April 24th April 24th-April 25th April 25th April 25th April 25th April 25th-April 26th April 26th April 26th April 27th April 27th April 27th April 27th April 27th April 27th-April 28th May 10th May 11th May 11th May 11th May 11th May 11th May 11th May 11th

Start time 22:07:01 22:28:59 22:41:13 22:07:01 22:59:21 19:06:04 19:23:44 19:25:19 22:58:27 22:07:01 22:23:40 17:13:31 17:19:41 18:10:31 18:40:24 21:23:55 22:39:49 22:24:13 03:46:50 05:01:09 05:18:22 06:01:48 10:39:40 11:58:50 15:18:27

End Time 22:08:13 22:30:31 22:42:23 22:08:13 14:45:15 19:07:18 19:24:53 19:26:51 14:56:52 22:18:59 22:35:50 17:13:50 17:22:35 18:18:10 18:41:14 21:50:01 15:21:57 22:24:36 03:47:13 05:01:32 05:58:49 06:28:45 10:41:55 13:23:19 15:25:05

Description

Overnight Run LO tests LO tests Overnight Run

LO tests LO tests LO tests Overnight Run Part of overnight Part of overnight Part of overnight Part of overnight Part of overnight Part of overnight Part of overnight Part of overnight

data data data data data data data data

Table 3.1: Summary of the data recorded in the experiment

96

3.2.4

LO tuning tests and 1390 MHz interference

The files marked “LO tests” in Table 3.1 were measured during periods in which either the PALS LO of 1210 MHz or the second stage downconverter LO of 173 MHz were varied. Figure 3.6 illustrates calibrated brightnesses (calibration procedure described in the next sections) in horizontal polarization for the 18:10:31 UTC observation on April 27th. The horizontal axis is the IF frequency, while the vertical axis is UTC time in decimal hours. During this measurement, the PALS LO was tuned from its original value of 1210 MHz, so that the IF frequencies of any external RFI sources are changed. Note a strong source of brightness around IF frequency 17 MHz in the Figure whose frequency is changed as the LO is changed; this source is likely to be RFI at a frequency of 1400 MHz, as will be more apparent in the next Section. Movement of the PALS passband through varying portions of the IF band-defining filter is also apparent as the LO is tuned. However, a source of large brightness around 7 MHz IF frequency is also observed that is not affected by changes in the PALS LO. This source appears to be an IF source, either at 180 MHz in the PALS IF band or at 7 MHz following the second downconversion. Analysis of a dataset in which the second stage LO was tuned seems to indicate that this source is more likely to be at 180 MHz. This source in future plots of brightness versus RF frequency will be seen at a frequency of 1390 MHz. Because this source is outside the passband of the radiometer, it produces only a small contribution to overall brightness when total power within the radiometer passband is calibrated.

3.2.5

PALS state classification

PALS provided state line outputs to the three backend observing systems for triggering and state classification purposes. However the state lines provided were found 97

Brightness vs. Frequency as LO Frequency Changes 250

18.19

240

18.2

230

Decimal UTC Time

18.21

220

18.22

210

18.23

200

18.24

190

18.25

180

18.26

170

18.27

160

18.28

150

18.29

140

5

10

15

20 25 30 IF Frequency (MHz)

35

40

Figure 3.6: Calibrated brightness temperatures vs. IF frequency as PALS LO tunes in horizontal polarization

98

to be relatively noisy, resulting in errors in real-time state classification. A method for classifying PALS states in post-processing was developed to eliminate these problems for LISR. While such a process could be developed based on comparisons with timing information from the other backend systems, the post-processing method described below has been found to be reasonably accurate while requiring no additional information other than the LISR data itself. Figure 3.7 plots LISR raw powers integrated over frequency for “integration, APB off” mode using triggers obtained from a single file on the night of April 27th. The horizontal axis is the trigger number (i.e. time), while the vertical axis is the raw power averaged out throughout the 1024 point spectrum. It can be seen that power levels cluster into three main groups; the group with the highest power level contains primarily Ant+ND observations, the middle group contains primarily reference load observations, and the lowest power level group contains antenna observations. A simple thresholding procedure was used as a first stage detector of Ant+ND observations. For example in Figure 3.7, an integrated raw power value of 1050 can be set as a threshold, and triggers with raw power levels above this deemed to be in the Ant+ND state. A specific threshold level was found to be applicable throughout an overnight observation, so that thresholds were determined empirically by looking at a small number of such power plots for each day where data was collected. A drawback of this method is the fact that an antenna observation can be confused with the other states if RFI is present. Bias caused by improper triggering due to noise on the trigger line may also affect the results. To overcome these problems, a twostage algorithm was developed that utilizes the fact that the Ant+ND observations are separated by a relatively constant time interval, as can be seen in Figure 3.1. The algorithm is summarized as follows: 99

File: data_118_043503_000868_00 2000 1800 1600

Raw Power

1400 1200 1000 800 600 400 200 0

1000

2000

3000

4000 5000 Trigger Number

6000

7000

8000

Figure 3.7: Average raw power levels for a single file for the night of April 27th

100

• Saturated or all-zero (missing data) spectra are discarded in all cases. A file is discarded entirely if more than 2/3 of the data in the file are saturated or missing. • A threshold is applied on the frequency averaged raw power for each remaining trigger to determine the cases likely to be Ant+ND. • Time differences between states classified as Ant+ND are examined; states are retained as Ant+ND if time differences between Ant+ND states are less than 4.2 msec or between 8.4-16.8 msec (note that average time delay between each observation is around 2.1 msec so a longer time delay is highly unlikely). • Time center of Ant+ND states are estimated by taking the average of the times of the triggers closer than 4.2 msec. If a second trigger is not present within 4.2 msec of the first one, this trigger is not used in further classifications since there is too much uncertainty in determining the time center of the state. • If a trigger is closer than 1.8 msec to an Ant+ND state center time, it is classified as an Ant+ND observation, else if the time difference is between 2.3 and 6.1 msec after the center of And+ND states it is classified as an Antenna observation, and if it is between 2.3 to 5.3 msec before the center of an Ant+ND state it is classified as a Reference observation. These time differences are based on the PALS state timing shown in Figure 3.1, with the triggers which are close to state transitions (and hence more ambiguous) discarded. • If the resulting number of Ant+ND, Antenna, and Reference triggers found in a file are each above 1000, calibration is proceeded. Otherwise this file is discarded. 101

This algorithm was found to provide satisfactory state classifications; less than 1% of the triggers in a given file were left as uncertain (and therefore discarded.)

3.2.6

LISR calibration

Gain and offset parameters for LISR calibration were determined for each frequency sub-channel using reference load, antenna, and Ant+ND data integrated in time over an entire file (around 18.6 seconds real time, 0.54 seconds antenna observation time.) Calibrated brightness spectra were then computed for each 184.32 µsec data period. To reduce data management requirements over longer observation periods and also to reduce brightness standard deviations, a second stage integration was performed over 64 triggers (around 11.8 msec antenna observation time, 409 msec real time) and stored as a “level one” data. A further integration over an entire file (18.6 seconds real time, 0.54 seconds antenna observation time) was performed in plots over time periods on the order of hours. The calibrated brightness temperature obtained through this procedure is an estimate of the antenna temperature, which should include a roughly 10 K sky brightness at L-band but also contributions from any surrounding structures observed in the antenna sidelobes as well as contributions from losses in the antenna or any components prior to the location of the internal calibration loads. Brightnesses reported for PALS have apparently removed many of these contributions through an additional calibration step not utilized for LISR. Note that, as mentioned in Canton campaign, other backends in this campaign (including PALS) with less spectral resolution are calibrated using the raw powers integrated over larger bandwidths initially, which emphasizes the contributions of stronger portions of the passband while in LISR, the average of the data over larger bandwidths is taken after the calibration when making a comparison with other

102

sensors. Results from the Canton campaign have shown the resulting error not to be substantial, however very precise comparisons with the other sensors (beyond the level of a few K) are affected. These differences are expected to be smaller than those associated with the noise diode transient issue discussed in Section 3.2.2.

3.3 3.3.1

LISR Overnight Observations Horizontally polarized brightnesses during the night of April 24th-25th

The campaign began on April 24th; during the daytime hours, LISR was tested and found to be functioning without problem. PALS, LISR, and the double detector system recorded datasets during the overnight period. Figure 3.8 is a “spectrogram” of calibrated brightnesses in horizontal polarization for the overnight period starting at midnight (UTC) April 25th. The horizontal axis is RF frequency while the vertical axis is the UTC time in hours. Individual brightness spectra illustrated in the figure are integrated over an entire LISR file (around 18.6 seconds real time, around 0.54 seconds of antenna observing time). The spectrogram of Figure 3.8 illustrates the high spectral resolution of LISR, and is very useful for identifying RFI or other emissions that are localized in frequency. The long integration time of Figure 3.8 as well as the large amount of data plotted reduces the figure’s utility in identifying time-localized interference. Note brightnesses over the full spectral range of LISR’s observations (1383-1433 MHz) are shown, even though the instrument passband is nominally 1400-1420 MHz. Several emissions localized in frequency are observed in Figure 3.8, including apparent hydrogen-line emissions at 1420.4 MHz, the IF RFI source mentioned in Section 3.2.3 at 1390 MHz, and narrowband sources near 1397.5 and 1400 MHz.

103

th

April 25 , H pol, APB off 0

250 240

2 230

4

220

UTC (hours)

210

6 200

8

190 180

10 170

12

160 150

14 140

1390

1400 1410 Frequency (MHz)

1420

1430

Figure 3.8: Calibrated H-pol brightness temperature vs. RF frequency, April 25th

104

Classification of the 1420.4 MHz source as natural hydrogen line emissions appears reasonable given the increased level of these emissions as the galactic plane comes into view during later parts of the image. The contributions of specific frequency localized sources to the entire channel brightness can be estimated by comparing brightnesses integrated over the receiver passband including or excluding these sources. Using this procedure, the contribution of the hydrogen line to the total channel brightness is a maximum of 0.2-0.3K while that of the 1400 MHz source is around 0.1 K. Several instances of apparent time-localized interference are also apparent in the image, particularly at times from 3 to 9 AM UTC. However, during these periods, a substantial portion of the data was saturated and was removed. Further discussions of time domain properties of LISR observed brightnesses and comparisons with PALS data are provided in Section 3.4. To illustrate general trends of brightness versus time, Figure 3.9 plots LISR total channel brightnesses versus time, including both horizontal and vertical polarizations. LISR channel brightnesses are averaged both in time (to the level of one LISR file) and in frequency over the channel 1402.5-1417 MHz. PALS brightnesses for the same period (each point corresponding to 4 seconds of observing time) are also included on the plot; due to the previously discussed differences in calibration between different sensors, brightness temperatures are shifted for illustration purposes, as indicated in the Figure legends. PALS brightnesses are linearly interpolated across the boundaries of periods of time for which LISR data is missing (see for example around 9 AM UTC.) The comparison shows overall similar trends in general brightness trends with time, including relatively higher brightness temperatures near midnight UTC (which corresponds to 5 pm local time when the sun was still up) and at the end of the experiment as the galactic center enters the antenna pattern. PALS brightnesses 105

April 25th, Horizontal, APB off

Brightness (K)

196

LISR+7K PALS+175 K

194 192 190 188 186 0

5

10

15

UTC (hours) th

April 25 , Vertical, APB off

Brightness (K)

190 LISR−2K PALS+172 K

188 186 184 182 0

5

10

15

UTC (hours)

Figure 3.9: Comparison of calibrated brightnesses for PALS and LISR, April 25th

show much larger apparent RFI contributions than those of LISR however, likely due to LISR’s averaging over a longer real time period than that used for PALS, so that time localized interference is reduced, as well as LISR’s lower overall duty cycle. LISR nevertheless captures many of the high brightness features observed by PALS. It is interesting to note that LISR also obtains some brightnesses substantially less than those of the mean brightness, as well as an abrupt change in the vertical channel brightness around time 3 AM UTC. The source of these effects is discussed in the next section.

106

3.3.2

Vertically polarized brightnesses during the night of April 24th-25th

Figure 3.10 is a spectrogram of the vertically polarized brightness analogous to Figure 3.8 for horizontal polarization. While many properties of Figure 3.10 are similar to those of Figure 3.8, a clear difference is observed in an abrupt change in observed brightness spectral properties beginning at 2:52 AM UTC. This change in spectral properties occurs at the same time as the abrupt change in vertically polarized total channel brightnesses in Figure 3.9. It is interesting to note in Figure 3.9 that the dramatic change in brightness spectral properties observed in the spectrogram of Figure 3.10 results in only small changes (less than 2 K) in total channel brightness. An additional investigation as to the source of these spectral variations was performed by examining spectral properties of the reference load before and after 3 AM UTC. Figure 3.11 plots reference load raw power versus frequency for horizontal and vertical polarizations averaged over Figure 3.10 before and after 3 AM UTC. Figure 3.11 shows that the change in vertically polarized brightnesses follows an associated change in reference load raw power versus frequency, with an oscillating pattern versus frequency occurring in the latter case after 2:52 AM. This change is not observed in horizontal polarization. The source of this change in observed power spectral properties is not known; possible explanations include an additional VSWR effect introduced into the system somehow, a phenomenon associated with receiver saturation in vertical (but not horizontal) polarization, issues in either the first or second stage downconverters, or problems with LISR associated with one of the ADC cards. Similar changes in brightness spectral properties are observed (but not as clearly) in other overnight datasets. A possible receiver saturation effect due to contributions 107

th

April 25 , V pol, APB off 0

250 240

2 230

4

220

UTC (hours)

210

6 200

8

190 180

10 170

12

160 150

14 140

1390

1400 1410 Frequency (MHz)

1420

1430

Figure 3.10: Calibrated V-pol brightness temperature vs. RF frequency, April 25th

108

Change in the Passband Properties for Vertical Polarization (Reference Looks) 800 Before 3 am After 3 am UTC Raw Power

600 400 200 0 0

5

10

15

20 25 30 IF Frequency (MHz)

35

40

45

50

Change in the Passband Properties for Horizontal Polarization (Reference Looks) 1000 Before 3 am After 3 am UTC

Raw Power

800 600 400 200 0 0

5

10

15

20 25 30 IF Frequency (MHz)

35

40

45

50

Figure 3.11: Change in the passband properties of reference looks on April 25th

from either in- or out-of-band sources does not seem impossible due to the high gain and limited filtering prior to the PALS mixer discussed in Section 3.1.1, but again conclusions in this regard are not conclusive. Further discussion of general RFI properties as well as mitigation algorithms for the observed RFI is provided in Section 3.4.

3.3.3

Overnight observations on April 25th-April 26th

The campaign continued on April 25th, and all three backend systems obtained data during the overnight observations. Figure 3.12 compares LISR total channel brightnesses versus time with those obtained from PALS; behaviors similar to those in Figure 3.9 are observed. Note LISR saturation occurred frequently following 6 AM

109

April 26th, Horizontal, APB off

Brightness (K)

196

LISR+12K PALS+177 K

194 192 190 188 186 0

5

10

15

UTC (hours) th

April 26 , Vertical, APB off

Brightness (K)

190 LISR−4K PALS+172 K

188 186 184 182 0

5

10

15

UTC (hours)

Figure 3.12: Comparison of calibrated brightnesses for PALS and LISR, April 26th

UTC. Calibrated brightness spectrograms are illustrated in Figures 3.13 and 3.14 for horizontal and vertical polarizations, respectively, at times prior to the onset of frequent saturation at 6 AM UTC. Brightness general spectral properties are similar to those observed in Figures 3.8 and 3.10, including the abrupt change in vertically polarized brightness spectral properties around time 2:48 UTC. Narrowband sources remain similar to those seen on the night of April 24th-25th, although the presence of a narrowband source at 1395 MHz in vertical polarization is more apparent in Figure 3.14. Apparent time localized RFI in Figure 3.13 shows some evidence of spectral localization simultaneously near the low (around 1395-1400 MHz) and high (around 1418-1423 MHz) portions of the band; properties of time localized RFI are discussed further in Section 3.4. 110

th

April 26 , H pol, APB off 250

0.5

240

1

230

1.5

220

UTC (hours)

2

210

2.5 200

3 190

3.5 180

4

170

4.5 5

160

5.5

150

1390

1400 1410 Frequency (MHz)

1420

1430

140

Figure 3.13: Calibrated H-pol brightness temperature vs. RF frequency, April 26th

111

th

April 26 , V pol, APB off 250

0.5

240

1

230

1.5

220

UTC (hours)

2

210

2.5 200

3 190

3.5 180

4

170

4.5 5

160

5.5

150

1390

1400 1410 Frequency (MHz)

1420

1430

140

Figure 3.14: Calibrated V-pol brightness temperature vs. RF frequency, April 26th

112

3.3.4

Overnight observations on April 27th-April 28th

LISR data on the third campaign day encountered frequent saturation and is not discussed further here. On the fourth campaign day, all three backends acquired data during the overnight observations. Data from the ADD was made available and is included in the following discussion. Figure 3.15 plots total channel LISR brightness temperatures versus time, as well as the calibrated brightness temperatures of ADD and PALS for both horizontal and vertical polarizations. Brightnesses are shifted by the constants indicated in the Figure legend in order to make the curves distinguishable. General brightness properties are similar to those observed on earlier campaign days, and results from the three systems show similar general behaviors. No major saturation or missing data periods were encountered by LISR during the overnight measurement. Abrupt changes in the vertically polarized LISR total channel brightness are also absent in Figure 3.15. However, a closer examination of the horizontally polarized data did show variations in spectral properties of the calibrated brightness over short (order of 10-20 seconds) intervals of time that are not obvious in Figure 3.15. These variations occurred approximately every 10-11 minutes for times prior to approximately 4:15 AM UTC; again the source of these variations is not known, and they are only weakly correlated to changes in the total channel averaged brightness shown in Figure 3.15. Brightness spectrograms for horizontal and vertical polarization are provided in Figures 3.16 and 3.17, respectively. Narrowband sources are again present at 1397 and 1400 MHz for H-pol, and 1395 and 1400 MHz for V-pol, as well as the IF interference at 1390 MHz for both polarizations (but stronger in H-pol.) The hydrogen line is again visible with an increased brightness near noon UTC (5 am local time). Variations

113

April 28th, Horizontal, APB off 198 ADD LISR+5K PALS+164 K

Brightness (K)

194 190 186 182 178 174 0

2

4

6

8 UTC (hours)

10

12

14

th

April 28 , Vertical, APB off 202 ADD LISR−3.5K PALS+168 K

Brightness (K)

198 194 190 186 182 178 0

2

4

6

8 UTC (hours)

10

12

14

Figure 3.15: Calibrated brightness temperatures vs. time for ADD, LISR and PALS on April 28th

114

th

April 28 , H pol, APB off 250

2

240 4

230 220

UTC (hours)

6

210 200

8 190 180 10 170 160

12

150 14 1385

1390

1395

1400

1405 1410 1415 Frequency (MHz)

1420

1425

1430

140

Figure 3.16: Calibrated H-pol brightness temperature vs. RF frequency, April 28th

in brightness spectral properties in horizontal polarization are not obvious in Figure 3.16 due to the low time resolution shown.

3.3.5

Overnight observations on May 10th-May 11th

After April 28th, the campaign was halted and restarted on May 8th. As discussed in Section 3.2.3, LISR encountered missing data issues during the overnight datasets of May 8th, 9th, and 11th; these measurements are not discussed further. Although data was acquired during the overnight of May 10th-11th, frequent saturation was again encountered; the largest portions of the final dataset available consist of observations from 5:18-5:50 and 12-13:15 UTC on May 11th. Figure 3.18 compares LISR total channel brightnesses versus time with those obtained from PALS for these time periods. Properties of LISR observed brightnesses are qualitatively similar to those measured by PALS during these periods, although the different duty cycles of the 115

th

April 28 , V pol, APB off 250

2

240 4

230 220

UTC (hours)

6

210 200

8 190 180 10 170 160

12

150 14 1385

1390

1395

1400

1405 1410 1415 Frequency (MHz)

1420

1425

140

1430

Figure 3.17: Calibrated V-pol brightness temperature vs. RF frequency, April 28th th

th

May 11 , Horizontal, APB off

May 11 , Vertical, APB off

196

202 LISR PALS+175 K

Brightness (K)

194 192

198

190

196

188

194

186

192

184

190

182 5.2

5.4

5.6

5.8

LISR PALS+179 K

200

188 5.2

6

5.4

5.6

5.8

6

202 LISR PALS+175 K

Brightness (K)

195

LISR PALS+179 K

200 198 196

190

194 192

185

190 12

12.5 13 UTC (hours)

13.5

188

12

12.5 13 UTC (hours)

13.5

Figure 3.18: Calibrated brightness temperatures vs. time for LISR and PALS on May 11th 116

th

May 11 , H pol, APB off 240

UTC (hours)

5.4 220

5.5 200

5.6 180

5.7 160

5.8 140

1390

1400

1410

1420

1430

240

UTC (hours)

12.2

220

12.4 12.6

200

12.8

180

13

160

13.2

140

1390

1400 1410 Frequency (MHz)

1420

1430

Figure 3.19: Calibrated H-pol brightness temperature vs. RF frequency, May 11th

two instruments result in high brightness features that are similar but not completely correlated. Spectrograms for H- and V-polarized data are shown in Figures 3.19 and 3.20, respectively, for the times periods 5:18-5:50 and 12-13:15 UTC. Since the data plotted for this day is shorter than the other days, only a 64 sample integration (≈ 11.5 msec antenna observation time) was used. Overall brightness properties versus frequency remain similar to those of previous days, although in this case, the 1395 MHz V-pol and 1397 MHz H-pol sources are not as apparent. A new narrowband interferer at 1398 MHz also is present for V-pol.

117

May 11th, V pol, APB off 240

UTC (hours)

5.4 220

5.5 200

5.6 180

5.7 160

5.8 140

1390

1400

1410

1420

1430

240

UTC (hours)

12.2

220

12.4 12.6

200

12.8

180

13

160

13.2

140

1390

1400 1410 Frequency (MHz)

1420

1430

Figure 3.20: Calibrated V-pol brightness temperature vs. RF frequency, May 11th

118

3.4 3.4.1

RFI Detection and Mitigation Properties of observed RFI

The previous Section showed the presence of several persistent narrowband RFI sources (for example at 1395, 1397, and 1400 MHz) throughout the campaign, as well as an IF source that appears as 1390 MHz and apparent natural hydrogen line emissions at 1420.4 MHz. LISR’s high spectral resolution allows these sources to be discarded easily, simply by computing total channel brightnesses over regions of the spectrum that do not contain these sources. The discussions of the previous Section showed that the contributions of these sources to total channel brightnesses was typically on the order of a few tenths of a Kelvin. These contributions would also be reduced for instruments (such as PALS) that integrate over frequency before calibration due to the fact that these sources encounter reduced gain in the instrument passband. The emitters of the RFI at 1395 to 1400 MHz are unknown, but, as described in Chapter 2, they were encountered in Canton campaign as well and some of these emissions may be associated with radiation from monitors, computers, or other common electronic devices. Larger time-localized changes in brightness were also observed throughout the campaign. An example of these time-localized changes is highlighted in Figure 3.21, which plots PALS, ADD, and LISR total channel brightnesses versus time from the April 27th-28th overnight observation (as in Figure 3.15) for the shorter time period 4:00 to 6:00 UTC. Periodic interference is observed by all three sensors, with the RFI level and other properties closely correlated among sensors. An examination of LISR data at the highest time resolution of 184.32 µsec was performed during the periods of observed RFI. Several examples of “pulsed” time

119

April 28th, Horizontal, APB off 198 ADD LISR+5K PALS+164 K

Brightness (K)

194 190 186 182 178 174 0

10

20

30

40

50 60 70 Minutes past 4:00 UTC

80

90

100

110

120

th

April 28 , Vertical, APB off 202 ADD LISR−3.5K PALS+168 K

Brightness (K)

198 194 190 186 182 178 0

10

20

30

40

50 60 70 Minutes past 4:00 UTC

80

90

100

110

120

Figure 3.21: Calibrated brightness temperatures vs. time for ADD, LISR and PALS sensors on April 28th between 4 and 6 am UTC time

120

RFI were observed, with the total channel brightness during a pulsed 184.32 µsec observation period substantially exceeding that of surrounding periods. RFI spectral properties during a 184.32 µsec observation period are somewhat difficult to discern due to the relatively short integration time. In many cases, there are no obvious spectral signatures observed for a specific pulse, but rather simply an increase in overall power level throughout the band, suggesting either a very wide band interference source, corruption of the antenna measurement by the reference load or noise diode internal calibration loads, or possible saturation of the receiver due to in or out of band interference. If pulse-like interference without an obvious spectral signature is removed, the remaining pulsed signatures can be examined for the spectral information they contain. An example of such an examination is provided in Figures 3.22 and 3.23, which plot V- and H-pol spectra, respectively, acquired at 184.32 µsec time resolution for pulse-like events during the time period 1:54-4:27 UTC on April 28th. In these Figures, V-pol “pulse” spectra having their highest mean brightness located between 1402.5-1405.3 MHz have been plotted, while the H-pol image retains pulses with the highest mean brightness in the band 1399.5-1402.4 MHz. These criteria were found to retain a large set of pulses having apparent spectral signatures. The color scale utilized is logarithmic in order to capture high brightness events. Results show evidence of broadband interference in the band 1400-1410 MHz for V-pol, and in the band 1400-1404 MHz in H-pol. The latter case also indicates a correlation between high brightnesses in the frequency range 1400-1404 MHz and those in the range 1412-1420 MHz. The sources shown are clearly of a “wideband” nature given than bandwidths substantially larger than 1 MHz are occupied.

121

th

Image of Detected Pulses in V pol, April 28

600 50

500

100 Trigger Number

400 150 300

200 250 300

200

350

1400

1405

1410 Frequency

1415

1420

Figure 3.22: Image of pulsed interferers for V-pol, April 28th

122

Image of Detected Pulses in H pol, April 28

th

600 50

500

100

Trigger Number

400 150 200

300

250 300 200 350 400 1400

1405

1410 Frequency

1415

1420

Figure 3.23: Image of pulsed interferers for H-pol, April 28th

123

3.4.2

Time domain RFI mitigation

Although it is difficult to conclusively identify the RFI sources observed by PALS, given the broadband spectral properties discussed in the last section, it is nevertheless clear that the observed RFI occurs on short time scales, likely shorter than the 184.32 µsec integration period of LISR. In these situations, a simple time-domain blanking strategy can be highly effective at mitigating RFI. A simple algorithm was developed based on blanking data at 184.32 µsec time resolution before the 64 point integration of the calibration process described in Section 3.2.6. The algorithm executes as follows: 1. Compute the total channel brightness for all antenna triggers in a file; sort these brightnesses and find the mean and standard deviation of the lower 90%. 2. Declare antenna observations with total channel brightnesses more than 5 standard deviations above the mean to be pulses, and set their contributions to zero in the 64 point integration. 3. Integrate the data over 64 antenna observations; scale result to account for presence of blanked pulses A similar second-stage blanking process was applied at 11.5 msec time resolution when integrating brightnesses over an entire file, as shown in the majority of the plots illustrated in this report. Figure 3.24 displays sample LISR total channel brightnesses before and after this mitigation procedure for horizontally and vertically polarized data during the time period 08:30 to 12:00 UTC on April 28th. Although a few examples are observed where apparent low-level interference remains after the mitigation, results in general 124

April 28

th

180 LISR Mitigated

B

H−pol T (K)

179 178 177 176 175 30

50

70

90

110 130 150 170 Minutes after 8:00 UTC

190

210

230

50

70

90

110 130 150 170 Minutes after 8:00 UTC

190

210

230

193

191

B

V−pol T (K)

192

190 189 188 187 30

Figure 3.24: H-pol and V-pol total channel calibrated brightnesses before and after time domain RFI mitigation, April 28th

show this simple mitigation algorithm to be effective in removing pulsed RFI from the dataset, due to the pulsed nature of the RFI encountered.

3.5

Summary and Remarks

This chapter has described LISR observations with JPL’s PALS in a ground-based measurement campaign conducted April 24th-May 11th, 2006. A description of modifications made to LISR for this campaign compared to previous deployments and its

125

observation modes during the campaign were provided. A review of LISR measurements showed that LISR results were reasonably consistent with those obtained from PALS in terms of observed RFI properties. LISR’s high spectral resolution indicated the presence of several narrowband interference sources throughout the campaign, primarily in the region 1395-1400 MHz, although time-varying contributions from hydrogen line emissions were also observed. The dominant RFI sources were of a pulsed type; examination of these pulses in the time domain showed them to be generally of a wideband nature, with some evidence of frequency localization in the region 1400-1410 MHz as well as 1412-1420 MHz in some cases. The source of these emissions is not known but simple pulse and crossfrequency detection strategies were found to be highly effective against the observed RFI. Other questions regarding the accuracy of the results include the observed significant variations in brightness spectral properties, as well as system calibration given the noise diode transient issue in the PALS system. Even given these unresolved problems, the LISR dataset represents a highly useful tool for examining RFI properties, particularly in the spectral domain, for this campaign.

126

CHAPTER 4

AIRBORNE C-BAND RFI MEASUREMENTS WITH PSR/CXI AND CISR FROM THE WB-57 AIRCRAFT

In this chapter observations of C-band RFI made from NASA’s WB-57 highaltitude aircraft using the PSR/CXI system of NOAA/ETL and the CISR digital backend of OSU/ESL is documented. Figure 4.1 shows a picture of the WB-57 aircraft used in this campaign. The C-band Agile Digital Detector (ADD) system of the University of Michigan [24] was also included in the campaign. ADD system did not record RF frequency channel information which led to a difficulty in analyzing the data from this instrument; since a comparison of ADD data with that of the CISR data was desired, an automatic algorithm is developed to sort out this problem. Resulting comparisons using this algorithm will also be described in this chapter. The observations described were performed during a test flight on Aug. 25th, 2005 beginning at Ellington Field, Houston, TX and including overflights of Dallas and San Antonio as well as a flight segment over the Gulf of Mexico. This campaign will be denoted as “WB-57 campaign” in the rest of this work. Only the vertically polarized C-band measurements are discussed here. The PSR/CXI instrument also recorded horizontally polarized and polarimetric C-band channels, as well as X-band brightnesses in multiple frequency sub-bands, but these were not sampled by CISR during the flight. 127

Figure 4.1: WB-57 high-altitude aircraft

Note that a similar earlier airborne experiment that involved CISR and the PSR/CXI frontend was performed on October 8, 2004 over Wallops Flight Facility (WFF) and was reported in [16]. Unfortunately, in that campaign, a reasonable calibration of CISR could not be obtained for most of the flight. Since WB-57 campaign is similar to WFF campaign in terms of hardware used and processing techniques applied for RFI detection, and results from the WB-57 campaign allowed a much better comparison between different sensors, WFF campaign will not be described further in this thesis.

4.1 4.1.1

Instrumentation PSR/CXI

The PSR/CX instrument of NOAA/ETL has been deployed in several previous airborne remote sensing campaigns [18], and provides well calibrated brightness measurements along with a variety of possible scanning patterns during flight operations. 128

The majority of the data to be presented were obtained from conically scanned observations (i.e. the antenna is rotated in azimuth at a fixed speed) at a fixed incidence angle of 55 degrees from nadir (when in level flight). One portion of the dataset near the end of the flight includes fixed scan angle sky observations during steep rolls of the aircraft for calibration verification purposes. The antenna 3 dB beamwidth (two-sided) is approximately 10 degrees for all observations. The PSR/CX instrument includes four C-band sub-channels, with respective 3 dB bandpass frequency ranges of 5.8-6.2, 6.3-6.7, 6.75-7.1, and 7.15-7.5 GHz. These sub-channels can provide some measure of RFI mitigation, but remain large analog channels (∼400 MHz) compared to the bandwidth of likely RFI sources. The PSR/CXI instrument utilized in this flight is a modification of PSR/CX that includes a downconverter module so that tuned observations can be made throughout C-band in a narrower bandwidth than the 400 MHz of the main channels. Figure 4.2 is a simplified schematic of the PSR/CXI, and shows that following initial filtering and front-end amplification, a portion of the antenna power is coupled to the downconverter section. The local oscillator (LO) utilized is capable of tuning from 5.4 to 7.5 GHz, uses a 125 MHz IF center frequency, and provides > 20 dB rejection of the upper RF side band. The IF signal is filtered to both 10 and 100 MHz bandwidths, and both of these bandwidths are passed through a logarithmic amplifier/detector, power integrated, and recorded. The 100 MHz bandwidth signal is also passed through a linear amplifier; a portion of this coherent signal is coupled to the CISR instrument in the cabin rack of the P-3, while the remainder is detected, power integrated, and recorded by PSR/CXI (PSR/CXI post-amplifier section not shown in Figure 4.2). Tuned channel observations recorded by the PSR are reported elsewhere [54], and are not considered further here. 129

To PSR/C direct detection channels (5.8−6.2, 6.3−6.7, 6.75−7.1, 7.15−7.50 GHz)

Antenna 5.75− 7.55 GHz

LNA

125 MHz IF

To PSR Computer

120− 130 MHz

5.75− 7.55 GHz 75− 175 MHz 5.4−7.5 GHz Trigger pulse

To digital receiver

Figure 4.2: Simplified schematic of PSR/CXI

For the CISR data to be reported, the LO was tuned so that 100 MHz channel observations were made at 5.5-5.6, 5.6-5.7, ..., 7.6-7.7 GHz; this is a set of 22 channels. The LO was swept continuously throughout the flight, with each channel being observed for approximately 37 msec; a complete sweep of channels required approximately 814 msec. The PSR/CXI LO is a YiG-tuned device, and therefore is subject to hysteresis effects as well as temperature sensitivity. For this reason, the accuracy of the RFI frequencies should be taken as ≈ ±2 MHz throughout this study. Although this is acceptable accuracy for radiometric studies, possible inclusion of an LO with improved tuning accuracy and stability deserves consideration for future flights of PSR/CXI.

4.1.2

CISR

A simplified schematic of the CISR instrument is provided in Figure 4.3. It can be seen that CISR is similar in design to LISR, but with some major modifications. The CISR digital receiver backend measures the incoming 100 MHz bandwidth through 130

the use of two 200 MSPS A/D converters, each of which samples the band 125175 MHz. A “channel-selection” is therefore required to upconvert the 75-125 MHz portion of the PSR/CXI IF signal to 125-175 MHz. Following A/D conversion, the operation of the system is basically same with LISR; CISR instrument can perform an APB operation to suppress temporally localized interference, as well as a 1024 point FFT operation, followed by optional power integration or max-hold computations. “Integration, APB off” and “Capture” modes were the modes utilized in WB-57 campaign. In “Integration, APB off’ mode, an 1024 point FFT operation is performed, followed by power computation and an integration over a 1.3 msec time period (same as Canton experiment in Chapter 2). The output in this mode is then a spectrum of the power in 1024 ∼100 kHz bandwidth sub-channels within the tuned 100 MHz spectrometer channel. The “capture” mode again refers to the direct recording of the sampled 100 MHz channel at 10 nsec temporal resolution with no further processing. CISR’s asynchronous pulse blanking (APB) algorithm was not used in this campaign due to the relative infrequency of pulsed interferers at C-band, as well as a desire to optimize calibration of the “Integration, APB off” data. Results of Chapter 2 have clearly demonstrated the effectiveness of the APB algorithm at mitigating pulsed source interference while maintaining accurate brightness temperature measurements.

4.1.3

Interface between PSR/CXI and CISR

Because it is the PSR/CXI data acquisition computer that controls oscillator tuning in the downconverter, the CISR and PSR/CXI computers must be interfaced. To make this interface as simple as possible, a simple 1-bit “trigger” signal was used. This

131

Trigger in

IF/Trigger

Computer

from PSR

Digital data

PCI− DIO32HS

Processor control

Atten control Channel selection

Digital Receiver ADC

75−125 MHz

125−175 MHz

Clock 250 MHz

DIF/ APB

FFT

SDP

Capture

6 dB ADC 125−175 MHz

Figure 4.3: Simplified schematic of CISR

TTL-level pulse is sent from the PSR/CXI control computer to the CISR computer whenever a tuning operation has been completed and measurements should begin. In addition, both computers include highly accurate on-board clocks, synchronized through the IRIG-B standard, so that knowledge of the oscillator frequency for a particular CISR measurement is obtainable by matching up recorded trigger pulse times in post-processing of a joint data set. In the data considered here, unambiguous matchups of the PSR/CXI and CISR measurements were possible utilizing this process. Offsets in the two “timestamps” were within 3 msec for the majority of the recorded data. One issue in this time synchronization involves observations of the internal noise diodes of the PSR/CXI system. These noise diodes are useful for verifying or improving system calibration stability between measurements of the external hot and cold targets embedded in the PSR/CXI scanhead. The simple 1-bit trigger interface however does not allow specification of the noise diode switch state during a measurement. 132

To address this issue, a plan was developed wherein the noise diode measurements were to be performed within a specified time interval following the trigger signal. However, the time accuracy of the PSR/CXI instrument states was insufficient to ensure an accurate noise diode measurement by CISR at each opportunity.

4.1.4

CISR modifications for the WB-57 aircraft

A few modifications to the CISR hardware were performed for this campaign to improve survivability in the high altitude environment. The CISR hardware was located in a unpressurized portion of the WB-57 aircraft, and therefore exposed to the ambient (though inside the aircraft) air temperature and pressure at altitude. Figure 4.4 is a photograph of the CISR enclosure when installed within the “foreward racks” in the WB-57 aircraft. Atmospheric pressure issues The low atmospheric pressure of approximately 0.1 atm at altitude (62500 ft) complicates heat dissipation issues within the CISR enclosure, and also requires use of modified hard drive systems for recording observed data. Though solid state hard drives are desirable in such cases, the high data rate of CISR makes use of faster magnetic drives preferable. In this campaign, the CISR operating system and source executable were placed on a 4 GB solid state drive, while data was stored on an 80 GB sealed magnetic hard drive. In an earlier deployment of CISR on-board the WB-57 aircraft (April 2005), the latter drives failed due to improper sealing by the vendor. However these problems were corrected by the vendor prior to August 2005, and no further issues were encountered.

133

Figure 4.4: Photograph of the CISR enclosure installed on the WB-57 aircraft

Temperature control CISR was further modified to include simple temperature control systems. A set of heaters capable of providing 175 W of heating were included, under the control of a thermostat within the CISR enclosure which turned the heaters on when the temperature of the thermostat declined below 10 C. A second thermostat was also included at an alternate location within the enclosure to override the heater on state if temperatures greater than 50 C were encountered at a second thermostat. Thermostats were also placed within the enclosure to turn CISR electronics off when the internal temperature decreased below 0 C or above 50 C. These thermostats were simple on/off bimetal thermostats, and their status was not monitored or recorded. The CISR enclosure included substantial thermal insulation due to the low ambient 134

air temperature at altitude. Given this insulation and the existing heating provided by the CISR electronics, it is not believed that the heating system was utilized during flight. The reduced amount of convective cooling available at low atmospheric pressure increases the possibility of localized “hot-spots” within the CISR enclosure, even though many portions of the enclosure may be cold. The only enclosure temperature information recorded during the flight was obtained from a temperature, humidity, and pressure monitoring card within the CISR PC-104 computer. Data from this sensor indicated local temperatures approaching but not exceeding 62 C. While further consideration of CISR thermal transfer could possibly reduce the likelihood of these high internal temperatures, rated operational temperatures for CISR electronic components exceed 62 C, and no apparent problems were observed. Condensation A final modification was made in an attempt to reduce the amount of condensation within the CISR enclosure as the aircraft returns from altitude. Though the enclosure was not hermetically sealed, and in fact becomes an explosive hazard if pressurized relative to the ambient environment, attempts were made to keep the enclosure reasonably air tight with the exception of a single air exchange opening. This opening was routed through a tube desiccator, so that air taken into the enclosure upon descent would contain a reduced moisture content. The PSR and University of Michigan systems utilized a similar approach to reduce condensation problems. Though quantitative information on the performance of this system was not recorded, no damage due to condensation within the CISR enclosure was encountered.

135

ADD Subband: Subband #1 Subband #2 Subband #3 Subband #4 Subband #5 Subband #6 Subband #7 Subband #8 Fullband

3dB 122.00 127.25 132.50 137.75 143.00 148.25 153.50 158.75 122.00

band edges - 127.25 MHz - 132.50 MHz - 137.75 MHz - 143.00 MHz - 148.25 MHz - 153.50 MHz - 158.75 MHz - 164.00 MHz - 164.00 MHz

Table 4.1: ADD subchannel frequencies within the 75-175 MHz IF band

4.1.5

ADD system

Since comparisons with the ADD system will also be provided, a brief description of ADD as configured for this campaign is given here. ADD backend system, like CISR system, observed IF signals provided by PSR. The ADD system further filtered the IF data into 8 5.25 MHz ADD subchannels, with results from all ADD subchannels recorded simultaneously and continuously at 1.2 msec time resolution. The 3 dB points of the ADD subchannel IF filters are shown in Table 4.1. The ADD also has a fullband channel with band edges at 122 and 164 MHz. Since the PSR computer controlled IF tuning throughout the experiment, information on the current channel state was not recorded by the ADD system with the hope that channel identification for the ADD data could be obtained by reconciling recorded time information between the ADD and PSR systems. However, time synchronization between the instruments was found insufficiently accurate to achieve this goal. Therefore, an alternative method was required for determining the PSR channel corresponding to a given ADD observation.

136

Researchers at University of Michigan could only identify channel information of a limited portion of the flight by using manual techniques. Since it was our goal to compare the two sensors, an automatic method was developed at OSU to classify ADD channel information. This method will not be described here since a thorough description will likely disrupt the flow of this study. Once the ADD channels have been identified, it is possible to calibrate the data. An initial comparison of calibrated ADD brightnesses with those recorded by CISR is provided in Section 4.4. The ADD dataset also allows the signal’s kurtosis to be computed and examined; however, this data is not presented in this study since exact RFI detection procedure using kurtosis statistic was not made available by University of Michigan.

4.1.6

Measurement process

PSR observation and spot Properties As stated previously, the PSR/CXI LO was continuously tuned through the 22 bands of interest throughout flight operations, requiring approximately 814 msec to complete a sweep. When in conical scan mode, the PSR/CXI antenna rotation period was typically 40 seconds, so that 49.1 sweeps were performed per antenna “scan” (i.e. rotation). PSR conical scan mode observations were performed in a total of 137 antenna rotations during the flight. At the typical flight altitude of 62500 ft, and using the nominal PSR antenna 3 dB beamwidth (two-sided) of 10 degrees, the 3 dB footprint observed by the antenna is approximately 10.3 km in diameter along track. At the typical flight speed of 205 m/sec, the 40 sec scan rate results in an along track sampling distance of 8.2 km, slightly less than the 3 dB footprint size. The cross-track 3 dB footprint is approximately 4.75 km in diameter; this dimension represents approximately 1/36

137

of the circumference of the cross-track scan. Therefore each independent cross-track footprint is observed for more than a second during the scanning process, allowing more than a full sweep of the 22 tuned channels within each footprint. As the PSR antenna rotates these 4.75 km by 10.3 km spots are swept in a circle along the Earth surface of approximate diameter 64.3 km (between 3 dB spot boundaries.) Thus, data from one PSR rotation represents observations comparable in spatial dimensions to a typical C-band satellite footprint, although the total area observed is only approximately half of the complete 64.3 km diameter circle. Based on these properties, it appear reasonable to assume that RFI effects in single pixel satellite observations are roughly comparable to those observed in a complete PSR antenna rotation. CISR operational process CISR was configured to observe in “average, APB off” mode for 16 consecutive sweeps of the tuned channels (around 13 seconds), followed by “capture” mode observations for 2 consecutive sweeps (around 1.63 seconds). CISR then paused for approximately 4.6 seconds to write the recorded data to its internal hard drives. This results in an approximate 19 second periodic pattern, completed approximately twice per PSR antenna rotation. In “Integration, APB off” mode, a base temporal resolution of 1.3 msec was utilized, with 12 1024 point spectra obtained in 15.75 msec following each PSR/CXI trigger. In the capture mode, a 128K capture (1.3 msec) was recorded for each PSR/CXI trigger. The resulting CISR data rate is approximately 9.2 GB/hr; although this is certainly unacceptable for end-user radiometer applications, the goal of recording as much information on the observed RFI as possible results in the high data rate for this study.

138

Calibration The PSR main channel data to be reported was calibrated using the external hot and cold loads of the PSR scan head; this procedure is expected to produce brightnesses typically accurate to within 1-2 K for the obtained 20 msec observations. Cal load observations were performed following 8 rotations of the PSR antenna for the majority of the flight. Examination of the PSR main channel data following calibration showed the expected 1-2 K standard deviations for PSR main channels 2 through 4, but an increased standard deviation of around 4 K in PSR main channel 1. This appears to have resulted due to a decreased system gain in main channel 1. However it is possible to reduce these standard deviations further by performing an additional integration over scan angle. This is because the standard PSR observation occurs every 37 msec, which represents only 0.33 degrees of scan angle, compared to the 3 dB spot of 10 degrees. PSR scan angle images to be illustrated have therefore undergone an additional smoothing by taking an average over the 9 surrounding pixels; this reduces the standard deviation by a factor of 3. The low gain of channel one also reduces the accuracy of its absolute calibration relative to the other channels. This offset among channels plays a role in decreasing the sensitivity that can be achieved by the PSR four channel RFI mitigation algorithm. Calibration of the tuned PSR/CXI and CISR channels is also available through this process, and the slower scan rate and corresponding increased cal load observation time result in greatly improved CISR calibration compared to [16]. However, as with PSR main channel 1, calibration of CISR data below 6.2 GHz and above 7.5 GHz remain problematic due to the low apparent gain of the PSR front end in these regions. For this reason, calibrated data will be illustrated for CISR channels between

139

6.2 GHz-7.5 GHz. No additional scan angle integration of these data are performed due to the tuning process. Note that some evidence of corruption of the PSR main channel calibration procedure due to strong RFI during cal-target observations is observed in the campaign, although the majority of the cal load observations appear to be RFI-free.

4.2

Experiment Conditions

Table one provides information on the flight plan. Figure 4.5 illustrates the geographical region observed; note major cities including Houston, Dallas-Fort Worth (DFW), and San Antonio, are marked with circles. Figure 4.6 is a plot of the WB-57 altitude versus time. As can be determined from Table 1, the flight plan included takeoff from Ellington Field (Houston area) at 17:14 along a West-Northwesterly heading, followed by a turn to the North near Krebsville (a small town) at 17:32. Ascent to a flight altitude near 62000 ft was completed by 17:51 as the DFW area was approached. The flight path then included a turn to the South over DFW around 18:00, followed by observations over more rural Texas locations until an overflight of San Antonio around 18:30. A turn to the Southeast over San Antonio then led to an overflight of the Gulf of Mexico beginning at 18:49. The flight plan then included a short circular flight at roll angle 30 degrees beginning at 18:54, followed by steeper rolls to allow sky observations beginning at 18:59. These rolls were completed by 19:06, and all observations stopped at 19:11. The aircraft returned to Ellington field at 19:42. PSR and CISR were both powered on before takeoff, and both acquired data until 19:11. However the analysis reported here focuses on conically scanned data obtained after flight altitude was reached (17:51) and before the circle flights were 140

Flight path between 17:18:11 and 19:09:51 o

34 N

18:0 −

0

Fort Worth

Dallas o

32 N

−

−

− Austin

30 N

−

0

18:3

Houston

Krebsville

−

o

−1 7

:3

0

− San Antonio

− o

−

28 N

:0

0

− 19

Latitude (°)

Waco

o

26 N o 102 W

o

100 W

o

98 W Longitude (°)

o

96 W

o

94 W

Figure 4.5: Flight path, including nearby Texas cities (circles)

141

Altitude of Plane vs. Time 65

60

55

Altitude (kfeet)

50

45

40

35

30

25

20

15

17:29

17:40

17:51

18:02

18:14 18:25 Time

18:36

18:47

Figure 4.6: Altitude of WB-57 aircraft versus time

142

18:58

19:09

Time (UTC) Event 17:14 Takeoff from Ellington Field 17:17 Start PSR conical scans 17:23 Altitude 25 kft, ADD on 17:32 Turn Northward over Krebsville 17:51 Altitude 62 kft 17:59 Over DFW 18:30 Over San Antonio 18:49 Over Gulf of Mexico 18:52 Begin descent from flight altitude 18:54 Begin circle flight 18:59 Stop PSR conical scans 19:06 Finish cal rolls 19:11 PSR off 19:42 Land at Ellington Field

Comments

Depart Houston vicinity Heading 280 degrees to 354 degrees Approaching DFW Turn toward San Antonio Turn toward coast

roll 30 deg Short rolls up to 60 deg

Table 4.2: Time history of flight on August 25th, 2005

begun (18:54). The ADD system was powered on after reaching altitude 25,000 ft (17:23) and observed until all systems were powered off at 19:11. The dataset contains observations in a variety of RFI environments, including urban, rural, and water scenes. Images of PSR/CXI conically scanned data to be shown will be presented in terms of time and scan angle; these should not be taken as geographic images given possible variations in the aircraft heading, pitch, roll, and altitude. RFI source information from the JSC database A review of the NPOESS RFI source database obtained from the Joint Spectrum Center (JSC) was performed to assist in preliminary interpretation of the datasets to be described in the next sections. The JSC database utilized contains data not updated since 1999, and therefore is subject to significant errors as well as source omissions. Within the frequency ranges of interest, the JSC database is described 143

as including only 5.5-5.9 GHz and 6.2-7 GHz, and the database also omits classified source information. Nevertheless, it is this dataset that has been utilized by the IPO in performing RFI simulation studies, so that information on the accuracy of this particular database is relevant. The dataset also provides some degree of information on the spatial distribution of RFI sources. Figure 4.7 plots RFI source locations in the frequency range of interest taken from apparently valid records in the JSC database. The flight path is indicated by the thick red line, and the Houston, DFW, and San Antonio areas are marked by the large circles. The strong correlation of the source density with urban locations is clear. The portion of the flight encountering the smallest source density is found between the DFW and San Antonio areas. Note that some off-coast sources are also included in the database. A histogram of center frequencies for the sources included in Figure 4.7 is provided in Figure 4.8. Overall the distribution is fairly uniform from 6.2-7 GHz, with the exception of the region 6.4-6.6 GHz, which contains a smaller number of sources. Note again that sources centered from 5.9-6.2 GHz are described as “not included” in the database, so that the histogram should not be taken as accurate in this region. Database information also reveals that many of the allocations below 5.9 GHz are associated with radar or other pulsed sources, while those above 5.9 GHz are primarily communication systems.

4.3 4.3.1

Comparisons with PSR PSR scan images

The WB-57 campaign represents a very dynamic RFI environment, with differing RFI sources observed as the tuned channel is swept through C-band and as the 144

36oN

34oN

32oN

30oN

o

28 N

102oW

100oW

98oW

96oW

94oW

Figure 4.7: Locations of C-band RFI sources within the JSC source database. Flight path is indicated by the thick red line.

145

450 400

Number of Records

350 300 250 200 150 100 50 0 5400

5600

5800

6000 6200 6400 6600 Center Frequency (MHz)

6800

7000

Figure 4.8: Distribution of source center frequencies for the JSC database records plotted in Figure 4.7. Note that records centered between 5.9-6.2 GHz are described as “not included” in the database.

146

aircraft location changes. As seen in Figure 4.5, the flight path included major urban centers, rural areas, and the Gulf of Mexico, all of which involve differing RFI properties. Figure 4.9 plots PSR scan angle images for the foreward part of the PSR conical scan, in all four PSR “main” channels. Obvious RFI is common in these images, with brightnesses up to 10700 K, 2994 K, 2442 K, and 2660 K observed in channels one through four, respectively. Of these channels, channel 1 generally shows the largest degree of RFI corruption, and channel 4 the least, although obvious RFI remains present in channel 4. Increase RFI associated with the Houston (prior to 17:25), DFW (around 18:00), and San Antonio (18:30) portions of the flight is apparent. The decreased brightness associated with observations over the Gulf of Mexico is also obvious in the later portions of the images. Figure 4.10 plots the average of the images in Figure 4.9 over scan angle, and shows behaviors consistent with those discussed for Figure 4.9. As described in [16, 18] and also in Chapter 1, it is possible to apply the PSR four sub-channel RFI mitigation algorithm to these images in order to remove RFI corruption. Use of the algorithm in this case eliminates the vast majority of the obvious RFI. However it is difficult to determine the amount of low-level RFI remaining following this process. A particular problem results when the algorithm determines that three or more channels contain RFI, so that the validity of the remaining channel is difficult to determine. Tables 4.3 and 4.4 provide summary statistics from the PSR RFI mitigation algorithm for the time periods 18:08-18:23 (more rural locations) and 17:54-18:08 (near DFW). Statistics are provided in terms of the percent of 21600 pixels determined to have a specified “interference level”. A “clean” classification results for a channel only if it is determined to be RFI free and RFI was detected in

147

Figure 4.9: Calibrated brightnesses from PSR conical scans of the entire flight, in all four PSR main channels

148

PSR Average Brightness vs. Time 1000 Ch1 Ch2 Ch3 Ch4

900

800

Brightness (K)

700

600

500

400

300

200 20

30

40

50

60 70 80 Minutes past 17:00 UTC

90

100

110

120

Figure 4.10: Average of Figure 4.9 over scan angle, versus time

less than 3 of the remaining channels. The classifications “1 channel” and “2 channel” for a particular channel indicate that that channel was determined to contain RFI when only one or two of the four PSR channels were deemed corrupted. Cases with “3 channels” however result in a classification for all PSR channels, due to the limitations of the algorithm with regard to assessing the remaining channel in this case. While the percentages shown are influenced by PSR calibration issues and by the particular algorithmic parameters utilized, overall the results clearly indicate a significant RFI problem both in the DFW (53.1% classified as “3 channel”) and rural

149

PSR freq (GHz) 5.8-6.2 6.3-6.7 6.75-7.1 Clean % 39.6 41.8 49.1 1 Channel % 13.0 11.4 3.0 2 Channel % 19.3 18.8 19.7 3 Channel % 27.0 27.0 27.0 Algorithm failed % 1.3 1.3 1.3

7.15-7.5 65.2 0.6 5.9 27.0 1.3

Table 4.3: Statistics from PSR four sub-band interference suppression algorithm: 21600 pixels over rural Texas (18:08-18:23 UTC)

PSR freq (GHz) 5.8-6.2 6.3-6.7 6.75-7.1 Clean % 12.5 30.8 30.6 1 Channel % 10.7 3.6 1.5 2 Channel % 22.6 11.5 13.8 3 Channel % 53.1 53.1 53.1 Algorithm failed % 1.1 1.1 1.1

7.15-7.5 44.1 0.1 1.7 53.1 1.1

Table 4.4: Statistics from PSR four sub-band interference suppression algorithm: 21600 pixels near DFW (17:54-18:08 UTC)

150

(27.0% percent “3 channel’) observations. Other statistics show a decreased, but nonnegligible, presence of RFI in the highest frequency PSR channel, consistent with the scan images of Figure 4.9. Examination of CISR data can be used to help to reveal further the properties of the RFI encountered, its distribution in frequency, and its temporal properties. Since naturally emitted thermal noise is expected to vary slowly with frequency, and since CISR has a very high spectral resolution, cross frequency mitigation using CISR data should be very effective against RFI that is localized in frequency. A cross-frequency mitigation technique slightly modified from the ones used in the L-band campaigns was applied to calibrated data. For a particular observation, we have brightness temperatures in 1024 frequency bins of approximately 97 kHz each as a result of the FFT operation. The algorithm operates on a single set of 1024 frequency bins averaged to 15.75 msec time resolution, as follows: 1. First, an acceptable brightness temperature range is set depending on the scene observed. Frequency bins with brightnesses outside the acceptable range are marked as corrupted. For land observations, the acceptable range was set as 200 K to 400 K. 2. The standard deviation and mean (both over frequency) of brightnesses are found for the lowest 85% (in terms of brightness) of the remaining frequency bins (to avoid bias of the mean and standard deviation by RFI). Another threshold test is then applied: frequency bins that are more than 4 standard deviations from the mean over frequency are marked as corrupted. Neighboring frequency bins within 4 frequency bins of the corrupted bins are marked as well.

151

3. Brightness temperatures of corrupted bins are replaced with the mean of the remaining frequency bins. This algorithm is relatively simple compared to other cross-frequency algorithms that have been developed for a smaller number of channels [18], [16]. However it will be shown to be successful in removing much of the observed RFI in what follows. Simplicity in the algorithm is desirable in order to make future implementation in digital hardware feasible. An algorithm very similar to the one described here will be used to analyze the performance of cross-frequency detection algorithm theoretically in Chapter 6. While it is again possible ideally to predict the false-alarm rate of this algorithm, the consideration of neighboring frequency bins in the cross-frequency blanking process complicates the analysis. A Monte Carlo analysis however showed the false alarm rate for the specified algorithm to be ≈ 2%. These false alarms do not impact obtained mean brightnesses due to the replacement of detected bin brightnesses with the average of the remaining brightnesses. As an example of the algorithm’s performance, Figure 4.11 plots a comparison of unmitigated and mitigated brightness temperatures versus UTC time for CISR channels 8, 12, 16, and 20 during a one-hour portion of the flight (all over land). These total channel brightnesses were calculated by taking the average of calibrated brightnesses in the 1024 CISR frequency bins corresponding to a 100 MHz bandwidth. Channel 8 represents observations from 6.2 to 6.3 GHz, channel 12 is 6.6 GHz to 6.7 GHz, channel 16 is 7 to 7.1 GHz, and channel 20 is 7.4 to 7.5 GHz. The brightness temperature points shown are averaged over observations in the specified channel during a 40 sec. time period; this is equal to one period of the antenna’s conical scan. Time periods with no points in the figures correspond to times when the antenna was 152

CISR Channel: 8

CISR Channel: 12

500 450

500 CISR CISR Mitigated

450 400

350

350

300

300

K

400

CISR CISR Mitigated

250 17:29:52

18:00:27

18:31:03

250 17:29:53

CISR Channel: 16

18:31:04

CISR Channel: 20

500 450

18:00:28

500 CISR CISR Mitigated

450 400

350

350

300

300

K

400

CISR CISR Mitigated

250 17:29:53

18:00:29 UTC Time

18:31:04

250 17:29:53

18:00:29 UTC Time

18:31:04

Figure 4.11: Comparison of calibrated brightness temperatures vs. time between original and mitigated data for CISR channels 8,12,16 and 20

153

not in conical scanning mode or when it observed external hot or cold load targets for calibration purposes. All four of these channels (as well as the CISR channels not plotted) show significant RFI corruption, with brightnesses averaged over 40 seconds exceeding 400 K in some cases, and the level of RFI corruption shows a significant variation over time. In measurements around 18:00 UTC when the aircraft was over the Dallas-Fort Worth area, all channels show large brightness temperatures. The higher frequency channels show less RFI corruption compared to the lower frequency channels, which is in accordance with the expected higher source densities at lower frequencies. However, all channels show at least 50 K of RFI contributions at some instant during this interval. Results following application of the cross-frequency mitigation procedure show a greatly reduced impact of RFI. One interesting observation in general is that, although all 22 CISR channels were mitigated separately, mitigated brightnesses are very similar in each channel as should be expected for thermal noise measurements. Variations of the mitigated temperatures with time also are consistent with expectations for observations over geophysical landscapes. To make a more detailed analysis, CISR data from three different portions of the flight will be inspected: one over DFW, one in rural Texas, and one over the Gulf of Mexico.

4.3.2

CISR observations near DFW

A detailed time history of CISR data for channel 16 (7-7.1 GHz) (without any averaging of the observed data over time) is presented in the lower half of Figure 4.12 for an approximately 90 sec. portion of the flight over the Dallas-Fort Worth area. As this is an urban scene, it is not surprising that a high degree of RFI activity

154

with an apparent period of ≈40 sec. (the antenna rotation time) can be observed. The mitigation algorithm significantly reduces this RFI. The results also show that portions of the time history without obvious RFI have similar brightnesses before and after the mitigation algorithm. However, RFI mitigations as large as 215 K are observed in some cases. Spectrogram images of brightnesses for the original and mitigated data for the same time period are also shown in Figure 4.12 (two upper plots). The horizontal axis is UTC time and the vertical axis is frequency in MHz for these images. The source of the periodic brightness increases in the time domain plot can be seen clearly here, including apparent sources near 7010, 7040, 7060, 7070, and 7090 MHz. The mitigated data image shows that the algorithm developed removes these contributions. It is possible to match the sources observed in this overpass with licensed sources from the FCC as well as the JSC database; the FCC’s license locator tool can be used for this purpose [55]. RFI sources that are easily identifiable for this purpose include two TDWR Doppler-Weather Radar systems located near the DFW airport; these systems are pulsed radars operating between 5600 and 5650 MHz, and appear to have been captured in the CISR dataset.

4.3.3

CISR observations in more rural Texas

To demonstrate cross frequency mitigation performance for weak RFI environments, a 45 sec. observation over a rural area between Fort Worth and San Antonio is used. A comparison of original and mitigated CISR data for channel 17 (7.1 GHz7.2 GHz) is provided in Figure 4.13, again with the time history of total channel brightnesses in the lower plot and brightness spectrograms in the upper plots. Narrowband interferers around 7107, 7135, 7145 and 7157 MHz are observed in original

155

Calibrated Brightness (K°)

Mitigated Brightness (K°)

7000

7000 400

7020

400 7020

MHz

350 7040

350 7040

300 7060

300 7060

250 7080

250 7080

200 7100 17:55:30

17:56:16 UTC Time

200 7100 17:55:30

17:57:03

17:56:16 UTC Time

17:57:03

CISR Channel: 16 700

CISR CISR Mitigated

600

K

500 400 300 200 17:55:30

17:56:16 UTC Time

17:57:03

Figure 4.12: Spectrogram images and time history of brightness temperatures for original and mitigated data over an urban landscape, CISR channel 16

brightness image, with 7145 MHz having the largest amplitude; the mitigated image indicates removal of these sources. The time history plot demonstrates that the contributions of these detected RFI sources to the brightness of the 100 MHz channel are in the range 1-3 K, and comparable to the estimated radiometer brightness standard deviation of 1.5 K. Such low level RFI is very difficult for a traditional radiometer to detect because it is within the range of the instrument sensitivity as well the range of expected geophysical brightnesses; the high spectral resolution of CISR however allows these narrowband but large amplitude sources (i.e. amplitudes more than four standard deviations from the mean brightness over frequency) to be readily detected and removed. 156

Calibrated Brightness (K°)

Mitigated Brightness (K°)

7100

450

7100

450

7120

400

7120

400

MHz

350 7140

350 7140

300 7160

300 7160

250 7180 7200 18:10:06

18:10:28 UTC Time

18:10:51

250

200

7180

150

7200 18:10:06

200 18:10:28 UTC Time

18:10:51

150

CISR Channel: 17 310

K

305

300

CISR

295

CISR Mitigated 290 18:10:06

18:10:28 UTC Time

18:10:51

Figure 4.13: Spectrogram images and time history of brightness temperatures for original and mitigated data over a rural landscape, CISR channel 17

4.3.4

CISR observations over the Gulf of Mexico

An example of observations in CISR channel 8 over the Gulf of Mexico is provided in Figure 4.14. The plot of total channel brightness versus time (lower plot) is not completely free of RFI even in these at-sea observations. For this case, sources near 6204 MHz and 6282 MHz are successfully mitigated, as shown in the spectrogram images in the upper portion of the figure. Although the contribution of these RFI sources to the total channel brightness is smaller than in the case of observations in the Dallas-Fort Worth area, brightness increases near 45 K can still be observed.

157

Calibrated Brightness (K°) 6200 6220

MHz

Mitigated Brightness (K°) 300

250

6240

6200

300

6220

250

6240 200

6260 150

6280 6300 18:51:34

200 6260

18:52:18 UTC Time

18:53:02

100

150

6280 6300 18:51:34

18:52:18 UTC Time

100

18:53:02

CISR Channel: 8 250

K

CISR CISR Mitigated

200

150 18:51:34

18:52:18 UTC Time

18:53:02

Figure 4.14: Spectrogram images and time history of brightness temperatures for original and mitigated data over the Gulf of Mexico, CISR channel 8

4.4

Comparisons with ADD

As mentioned before, an automatic algorithm that is based on timing constraints and known PSR passband characteristics is developed at OSU to recover channel information for the ADD data. However, this algorithm is quite complicated and a complete description would disrupt the focus of this thesis. In this section, a comparison of the resulting ADD data with that obtained by the CISR system will be provided. Agreement between the general characteristics of the two datasets is a proof of success for the ADD channel identification algorithm.

158

One issue in comparing calibrated CISR and ADD brightnesses involves the fact that the two systems observe antenna brightnesses over slightly different time intervals. CISR observed the antenna only for ≈ 15.75 msec out of the approximately 20 msec total antenna observation, while the ADD observed for a larger portion of this interval. In both cases, the brightnesses to be shown are averages over the entire antenna observing time available. Although some differences between observed brightnesses may result due to these differences, it is expected that the differences would be largest for pulsed-type interferers, which are unusual at C-band over 5.9 GHz. In addition, CISR did not make measurements continuously because approximately 2 sec. was required to write data to the CISR internal hard drive following an 11 second measurement. In the results to be shown, ADD data is not plotted during periods in which CISR did not make observations. Figure 4.15 plots time histories of calibrated brightness temperatures in ADD subchannel 6 (i.e. 148.25 to 153.25 MHz IF frequency) and CISR. Results from channels 12 through 15 are shown here. The CISR results shown are an average of calibrated brightness temperatures over the frequency bins corresponding to the ADD subchannel 6 3-dB bandwidth. The results show highly correlated observations in general, although significant differences are observed in some cases. The general similarity of the datasets nevertheless verifies the procedure used for determining the ADD data RF frequency. Scatter plots of CISR vs. ADD brightness temperatures for the same time period are shown in Figure 4.16. Again the high degree of correlation between the two datasets is evident in these results. Appreciable differences between the two often show larger ADD brightnesses rather than larger CISR brightnesses. One possible explanation for this difference is the fact that CISR results were averaged only over 159

ADD (blue solid) and CISR (red cross) brightness temp., PSR ch.:12, ADD subch.:6

K

400 200 56

58

60

62 PSR channel:13

64

66

68

56

58

60

62 PSR channel:14

64

66

68

56

58

60

62 PSR channel:15

64

66

68

56

58

60 62 64 Minutes past 17:00 UTC

66

68

K

400 200

K

400 200

K

400 200

Figure 4.15: Time history of calibrated brightnesses for ADD and CISR, for ADD subchannel 6 and tuned PSR channels 12 to 15

160

the 3 dB passband of ADD subchannel 6, whereas the true ADD subchannel 6 has a non-ideal passband and may therefore include RFI contributions from outside the 3 dB passband. To investigate this possibility, the CISR data was re-averaged over Kaiser-Bessel windows with β = 3.2, order 27, and the 3 dB cutoff frequencies of ADD subchannel 6. Figure 4.17 illustrates the new scatter plots obtained via this procedure. The results show that many of the outlier points in the previous scatter plots have been eliminated. Further improvements in these results would be obtained if a more precise description of the ADD subchannel filters were incorporated. Another comparison of ADD and CISR calibrated brightnesses for PSR channels 12 through 15 is provided in Figure 4.18. These spectrogram images have frequency on the horizontal axis and time on the vertical axis. Data corresponding to ADD subchannels 1 and 2 are not shown in the spectrogram due to the low IF gain of CISR in the middle of the 75-175 MHz IF bandwidth,. The results show that CISR and ADD locate very similar RFI sources, again verifying the ADD frequency determination procedure.

4.5

Summary and Remarks

An examination of CISR and PSR data obtained from the August 25th, 2005 test flight on board the WB-57 aircraft has been provided in this Chapter. The approximately 125 minutes of joint data contains observations of a variety of scenes, including strong and weak RFI environments. The data clearly show the possibility of significant corruption of measured brightnesses throughout C-band, with the higher frequencies typically observing less but still appreciable corruption. It was shown that

161

ADD

Scatter plot of calibrated ADD and CISR brightness temperatures, PSR channel:12, ADD subchannel:6 1000 500 100

200

300

400

500 600 PSR channel:13

700

800

900

1000

200

300

400

500 600 PSR channel:14

700

800

900

1000

200

300

400

500 600 PSR channel:15

700

800

900

1000

200

300

400

700

800

900

1000

ADD

1000 500 100

ADD

1000 500 100

ADD

1000 500 100

500

600 CISR

Figure 4.16: Scatter plot for calibrated brightnesses of ADD vs. CISR, for ADD subchannel 6 and tuned PSR channels 12 to 15

162

Scatter plot after filtering, PSR channel:12, ADD subchannel:6 ADD

1000 500 100

200

300

400

500 600 PSR channel:13

700

800

900

1000

200

300

400

500 600 PSR channel:14

700

800

900

1000

200

300

400

500 600 PSR channel:15

700

800

900

1000

200

300

400

700

800

900

1000

ADD

1000 500 100

ADD

1000 500 100

ADD

1000 500 100

500

600 CISR

Figure 4.17: Scatter plot after the filtering on calibrated CISR data, for ADD subchannel 6 and tuned PSR channels 12 to 15

163

Calibrated ADD Brightness

UTC, ch:12

Calibrated CISR brightness 400 17.95 18 18.05 18.1

300 200

UTC, ch:13

6620

UTC, ch:14

6640

300 200 6730

300 200 6830

17.95 18 18.05 18.1

300 200

200

6740 400 300 200

6820

300

6730

17.95 18 18.05 18.1

6840 400

6640 400

6720 400

6820

200 6630

17.95 18 18.05 18.1

6740

17.95 18 18.05 18.1

300

6620 400

17.95 18 18.05 18.1 6720

UTC, ch:15

6630

400

17.95 18 18.05 18.1

6830

6840 400

17.95 18 18.05 18.1

6920 6930 6940 Frequency (MHz)

300 200 6920 6930 6940 Frequency (MHz)

Figure 4.18: Comparison of brightness temperature spectrograms for CISR (left) and ADD (right), tuned PSR channels 12 to 15

164

a cross-frequency algorithm combined with CISR’s high frequency resolution can be used to remove most of the obvious RFI. A cross-comparison of the CISR dataset with observations from the University of Michigan’s C-band Agile Digital Detector is also performed; results were similar and verified the channel sorting algorithm developed for the ADD data.

165

CHAPTER 5

A STUDY OF THE SHAPIRO-WILK TEST FOR THE DETECTION OF PULSED SINUSOIDAL RFI

5.1

Introduction

It was mentioned in Chapter 1 that “tests for normality” can be used to test the degree to which measured fields appear to be Gaussian random variables, as expected for the thermal noise measured by a radiometer. These tests are more general (“omnibus”) compared to pulse detection and cross-frequency detection methods which are effective against RFI sources that are temporally or spectrally “narrow”, respectively. Several such tests for normality of a data sample have been studied extensively in the statistical literature (e.g. [34, 56–65]). Among tests for normality, only the kurtosis test [56, 57] has been demonstrated to date for radiometer RFI detection with the ADD radiometer which was described to some extent in previous Chapters [24–27, 31, 66]. Studies to date have focused on pulsed-sinusoidal RFI due to the potential impact of radar interference on L-band radiometers, and the results of [24,25], [26,27,31,66] show the kurtosis test to provide high sensitivity to low duty cycle pulsed-sinusoidal sources as well as some sensitivity to continuous sinusoidal RFI. However, as mentioned in Section 1.2.3, a “blind spot” for pulses having fifty percent duty cycle was also shown for which the algorithm is 166

insensitive to RFI, and reduced detection performance occurs for a broad range of duty cycles around fifty percent. In general, the kurtosis algorithm is insensitive to RFI sources that produce kurtosis values identical to those of a Gaussian random variable. It is possible to alleviate this problem by using higher order moments or time subsampling as described in Section 1.2.3, but these methods require additional resources or post-processing of the data. This chapter documents an examination of an alternate test for normality called the Shapiro-Wilk test, and compares its performance with that achieved by the kurtosis test for pulsed and continuous sinusoidal RFI. The Shapiro-Wilk test was first proposed in 1965 [58], and has been shown to be capable of detecting non-normality for a wide variety of statistical distributions, including those with Gaussian kurtosis values [59, 60]. It has been recommended as a powerful omnibus test of normality [61]. An initial study of the Shapiro-Wilk test’s performance in detecting pulsed sinusoidal RFI in comparison with two similar omnibus normality tests, those of Shapiro-Francia [62] and Chen-Shapiro [63], was provided in [34]. The results of this study generally showed that performance among these three tests is similar, and all three tests are based on order statistics of Gaussian distributions so that only the Shapiro-Wilk test is examined here. The discussion to follow considers only pulsed and continuous sinusoidal RFI sources; other RFI types may produce distinct conclusions regarding performance. In Section 5.2, a brief review of the Shapiro-Wilk test is provided. Given the interest in performing the test in digital hardware on-board a radiometer system, test implementation is a key issue that is considered. Section 5.3 describes the Monte Carlo simulation procedure used to examine test performance. Computation of kurtosis statistics is described next in Section 5.3.3. Section 5.4 presents results obtained 167

from Monte Carlo simulations of the Shapiro-Wilk test. Results are compared with those achieved by the kurtosis method, and the influence of a test size parameter is also explored. Final remarks are presented in Section 5.5.

5.2

The Shapiro-Wilk test

In the following it is assumed that a microwave radiometer measures N samples of a received field, written as y(i) for i = 1 to N . Such samples are assumed to be separated by the Nyquist sample rate of the receiver so that no correlation among samples is present in the absence of RFI. The Shapiro-Wilk test is based on a correlation of sample “order statistics” with those of a normal distribution. The use of order statistics implies that the data sample must be sorted, with the sorted sample written in vector form as y0 = (y1 , ..., yN ) in increasing values; here the prime

0

is used to note the transpose of a vector. The

Shapiro-Wilk test statistic W is defined as: ³P W = PN

N i=1

i=1 (yi

where

m1 =

ai y i

´2

− m1 )2

N 1 X y(i) N i=1

(5.1)

(5.2)

is the sample mean. W can be interpreted as a ratio of two estimates of the variance of the sample, with the estimate in the numerator holding only if the sample is drawn from a normal distribution since coefficients ai are calculated by linear regression to the expected values of standard normal order statistics. It can be shown that W is bounded by 0 and 1, and that the expected value of W converges to 1 for Gaussian input data as the sample size is increased. The expected value of W becomes smaller as the input signal becomes non-Gaussian. 168

5.2.1

Expressions for the ai coefficients

The vector of coefficients a0 = (a1 , ..., aN ) is normalized so that a · a0 = 1 and is also antisymmetric, so that a1 = −aN , a2 = −aN −1 , etc. Analytical expressions for the coefficients ai are provided in [58], but the original forms are somewhat computationally complex. Instead empirical forms for these coefficients have been determined in terms of u = N −1/2 as [67]: aN = −2.706056u5 + 4.434685u4 − 2.071190u3 −0.147981u2 + 0.221157u + cN (5.3) aN −1 = −3.582633u5 + 5.682633u4 − 1.752461u3 −0.293762u2 + 0.042981u + cN −1 (5.4) ai = ²−1/2 m ˜i

(5.5)

with the final equation holding for i = 3 to N − 2. In equation (5.5), m ˜ 0 = (m ˜ 1 , ..., m ˜ N ) is a vector of expected values for the order statistics of the standard normal distribution, approximated by [68, 69]: m ˜ i = Φ−1 {(i − 3/8)/(N + 1/4)}

(5.6)

where Φ−1 denotes the inverse of the standard normal distribution function. Also in equation (5.5), m ˜0 ·m ˜ − 2m ˜ 2N − 2m ˜ 2N −1 ²= 1 − 2a2N − 2a2N −1

(5.7)

Finally, the ci values in equations (5.3) and (5.4) are determined from vector c0 = (c1 , ..., cN ) by c = m/( ˜ m ˜ 0 · m) ˜ 1/2 169

(5.8)

5.2.2

Implementation in digital hardware

Computation of the W statistic requires evaluation of the sample moments up to second order (for the denominator), and also a weighted sum of the sorted sample (for the numerator). It is assumed that floating point operations are not desired for an on-board digital processor, so that the numerator and denominator are recorded for computation of W in post-processing. The required sorting operation for the numerator can be accomplished either through the use of existing sort algorithms for a fixed sample size [70] or through a histogramming procedure in which counters are used to record the number of occurrences of each possible sample value. It is assumed here that a sorting operation is used, and that the value of N is limited to a maximum of 4096 by hardware constraints. If it is desired to compute W for datasets larger than 4096 samples, it is assumed that a larger sample of size Q = IN is split into I sub-sets of size N , for which the square root of the numerator and the denominator are separately computed. The I values for the sub-sets are then averaged, the numerator term is squared, and the result divided by the averaged denominator to obtain the W estimate for the entire sample. The impact of N in this process will be examined in Section 5.4.2. The coefficients ai are fixed for a given N , so that computation of the W numerator is similar to a standard filter computation once the data sort operation is completed. While these steps are somewhat more complicated in terms of hardware implementation than purely moment based tests, overall the Shapiro-Wilk test appears to be reasonably well suited for implementation in hardware.

170

5.2.3

Quantization effects

When the yi data are rounded to integer values as in a fixed point digital processor, Sheppard’s correction to the variance estimate is applied by subtracting (N − 1)/12 from the value in the denominator [71]. An examination of quantization impact on the coefficients ai was also performed. Figure 5.1 plots a comparison of coefficients before and after quantization using 8-bits for N = 4096. The anti-symmetric nature of the ai coefficients is apparent in the Figure, along with the increasing nature of the coefficients at large offsets from the array center. The lower plot of Figure 5.1 focuses on the region near the center of the coefficient array, and reveals the quantization more clearly. In the Monte Carlo simulations of Shapiro-Wilk test performance to be presented in Section 5.4, tests showed that the ai coefficients for N = 4096 could be quantized in as few as 8 bits without a significant loss in test performance, and eight bit quantizations for these coefficients are used in all the results to be shown. The impact of quantization of the yi data is further examined in Section 5.4.1.

5.3

Simulation Procedure

Because the W test statistic for normally distributed input data does not follow a standard probability density function (pdf), the original presentation of the ShapiroWilk test [58] included empirical tables of this pdf for sample sizes up to 50. Later, an approach was derived to transform W into a statistic that is approximately normal in the RFI-free case [67, 72, 73] so that prediction of expected false-alarm rates is possible. However, tests on the simulation results described in Section 5.4 showed this transformation to be very sensitive to quantization effects, and the transformation is not used for this reason. Instead a Monte Carlo simulation approach is applied; the probability of false alarm and probability of detection are determined by counting 171

(a) 0.06

Coefficient

0.04 0.02 0 −0.02 Before quantization After quantization

−0.04 −0.06

500

1000

1500

2500

3000

3500

4000

(b)

−3

1.5

2000 Index

x 10

Coefficient

1 0.5 0 −0.5 Before quantization After quantization

−1 −1.5

1950

2000

2050 Index

2100

2150

Figure 5.1: Weight coefficients ai before and after quantization using 8-bit resolution

172

the number of realizations declared as RFI through a test on W in cases with no RFI present and with RFI present, respectively. Monte Carlo simulations utilized 30000 realizations in all the results to be shown, and tests with larger numbers of realizations showed negligible impact on the curves illustrated.

5.3.1

Signal model and notations

In a previous modeling study of the kurtosis test [26], received fields when pulsed sinusoidal RFI is present were modeled as uncorrelated Gaussian random variables with zero mean and standard deviation σ plus sinusoidal RFI with a specified amplitude and a uniformly distributed random phase. Although such an approach can be acceptable when calculating statistics for large samples containing many pulses, it does not realistically describe radars and other pulsed sources when resolved at higher time resolutions. Another signal model suggested in [74] is instead used in this study (which was in turn adapted from [75]), and is reviewed here. In this model, two different time scales are used. The first scale, N , denotes the number of samples on which the denominator and the square root of the numerator of the Shapiro-Wilk test are computed (the N in equation (5.1)). The longer scale consists of Q = IN samples, and represents the radiometer integration period in samples. As described previously, I numerator and denominator values are obtained from the Shapiro-Wilk computations during an integration period, and then averaged and used to compute the W value for the entire sample. For illustration, the sampling period of the digital radiometer system is assumed to be 16 nsec, and the integration period Q is taken as 32768 samples so that the total radiometer integration period is 524.288 µsec. The Shapiro-Wilk test length N

173

is set to 4096 samples typically (i.e. 65.536 µsec); the effect of varying N is examined in Section 5.4.2. For the case of pulsed sinusoidal RFI, sampled received fields can be written as [74]: xi [n] = A cos(2πf0 [(i − i0 )N + n] + φ)I(n, i) + wi [n] n = 0, 1, · · · , N − 1, i = 0, 1, · · · , I − 1

(5.9)

where wi [n] refers to the independent identically distributed (i.i.d.) Gaussian measurements, with zero mean and standard deviation σ. The function I(n, i) =

½

1 i0 N + Ns ≤ iN + n < i0 N + Ns + Np 0 otherwise

(5.10)

is an “indicator” function that locates RFI containing samples. Here, i0 is the pulse arrival “frame”, A is the amplitude, f0 is the frequency, and φ is the phase of a single sinusoidal pulse. It is assumed that if there are multiple pulses (i.e with arrival frames i0 , i1 , · · · ) within the Q sized sample, they have identical amplitudes A and durations Np , but the frequency f0 , phase φ, and arrival sample Ns are chosen independently. These choices result in a duty cycle, d, of d=

Npulse Np IN

(5.11)

where Npulse is an integer. As in [26], sine wave amplitudes in what follows are described in terms of the ratio R of the average “signal-to-noise” power ratio (dA2 / (2σ 2 )) normalized by a factor √ proportional to the uncertainty in the radiometer power estimate (σ 2 / N I). This definition gives dA2 √ R = NI s2 A =

2R √ d NI

174

(5.12) (5.13)

The parameter R thus describes the RFI power contribution to the integrated power in units proportional to the brightness standard deviation (i.e. NEDT). Typically RFI having R > 10 would be readily detectable without additional detection procedures, so the primary interest is in detecting RFI having R < 10.

5.3.2

Cases considered

Monte Carlo simulations were performed for fixed pulse amplitudes (R), duration (Np ), and number of pulses (Npulse ) in a frame of 32768 total samples. Pulse frequencies were uniformly distributed from 0 < f0 < 1/2 and independent from pulse-to-pulse. Phases φ were also selected uniformly from 0 to 2π radians, and pulse arrival frames i0 and arrival sample Ns were also selected uniformly within the constraints required by the specified number of pulses and pulse durations (i.e. pulses were not allowed to overlap in time.) The simulations performed were chosen to obtain a basic understanding of test performance in semi-realistic RFI situations. First, a case having ten pulses of length Np = 32 (0.512 µsec) within the integration period of 524.288 µsec was considered to model very short pulse radar-like emissions at a very high pulse repetition frequency; the resulting duty cycle is approximately 1%, and simulations were performed for R values of 0 (false alarm case), 2.5, 5, and 10. Second, a case having a single pulse of length Np = 16384 (i.e. 256.14 µsec) was used to obtain a 50% duty cycle which is of interest due to the blind spot encountered by kurtosis at this duty cycle. Such longer radar pulses are also produced by many radar systems, as evidenced by the data reported in [66]. R values used for the 50% case were identical to those in the 1% duty cycle case. Finally, a case with a 100 percent duty cycle was used to

175

model continuous RFI sources; in this case much larger R values of 0, 30, · · · 120 were required, as will be discussed in Section 5.4.

5.3.3

Computation of the kurtosis statistics

Monte Carlo simulations are also used to compute the kurtosis statistics whenever a performance comparison between the Shapiro-Wilk and the kurtosis tests is given. To compute kurtosis statistics, second and fourth order central moments of the received fields were calculated; Received fields used were same for both Shapiro-Wilk and the kurtosis tests and they were simulated in the manner described in Section 5.3.1. Note that ith central moment, mi , of a signal x is defined as:

E D mi = (x− < x >)i

(5.14)

where “” denotes the expected value operation. In practice, moments around the origin (denoted as µi ) are used to calculate central moments. Obviously, µ1 = m1 while second and fourth order central moments are given as [26]:

m2 = µ2 − m21

(5.15)

m4 = µ4 − 4µ3 µ1 + 6µ2 m21 − 3m41

(5.16)

The kurtosis statistic was computed using the entire dataset of IN samples (i.e. no sub-sampling.) Incorporating temporal subsampling into the kurtosis test may produce different results [31, 66]; however, for the approximations in the calculation of kurtosis test to hold at least 10000 samples is recommended ( [26] whereas number of samples used for SW test is 4096, therefore such an approach is not considered here. After second and fourth order central moments are calculated, kurtosis statistic (denoted as Kn here) is found as: 176

Kn =

m4 m22

(5.17)

This definition of kurtosis does not include quantization effects; Sheppard’s correction is used to eliminate quantization effects in the kurtosis calculations as well [26].

5.4 5.4.1

Results Histograms of W

The Monte Carlo simulation produces a set of W values for the cases described in Section 5.3.2, which include both a false alarm simulation R = 0 and simulations when RFI is present. Histograms of the W values obtained for the d ≈ 1% case are presented in Figure 5.2. The upper plot (no-quantization of yi data) shows the concentration of W near 1 in the RFI free case, as expected, although the mean of the pdf when R = 0 is not exactly 1 [58] due to the finite sample size utilized in the test. Non-Gaussian behavior is indicated in the Shapiro-Wilk test when the test statistic value becomes significantly less than one; values of W less than a threshold t are therefore declared as containing RFI. This basic behavior of W is apparent in the Figure, where although the histograms of R = 0 and R = 2.5 are nearly identical, the histogram means become smaller with increasing R. A larger variance of the test statistic is also observed in this case as R is increased. Histograms are also shown in the lower plot of Figure 5.2 when quantization effects of the yi data are included. These data are generated by assuming that the RFI free noise has a standard deviation σ = G and that the data are then rounded to integer values. Larger values of G therefore should lead to reduced quantization effects, although clipping would eventually become a problem for very large G values. A rule of thumb to avoid clipping is to select the number of bits for the RFI-free 177

case such that signals with amplitudes at least 6 times G do not cause clipping. In practice, the value of G is determined by the ratio of the observed thermal noise voltage standard deviation to the bit width of the analog to digital converter used in the radiometer. The results show negligible differences between the histograms illustrated when quantization with G = 4 is considered. Quantization effects can become significant for G < 4; however such effects are not considered further as G ≥ 4 is a reasonable expectation for many radiometer systems. Duty Cycle (d) ≈ 1%, No rounding

PDF of W

10000 R=0 R=2.5 R=5 R=10

1000

100

10 0.996

0.9965

0.997

0.9975

0.998

0.9985

0.999

0.9995

1

1.0005

W Rounded, G = 4

PDF of W

10000 R=0 R=2.5 R=5 R=10

1000

100

10 0.996

0.9965

0.997

0.9975

0.998

0.9985

0.999

0.9995

1

1.0005

W

Figure 5.2: Histograms (scaled to correspond to probability density functions) of W for non-quantized (upper Figure) and quantized data (G = 4, lower Figure), d ≈ 1%.

178

5.4.2

Receiver operating characteristic curves

Given the Monte Carlo set of W values, the probability of false alarm and probability of detection as a function of the threshold t in the test W < t can be computed. Receiver operating characteristic (ROC) curves plotting the probability of detection versus the probability of false alarm as the threshold t is varied can then be produced. Figure 5.3 illustrates the obtained ROC curve for the d ≈ 1% case. The results show that the test has poor performance at detecting RFI with R = 2.5, but improved performance for R = 5. A further examination of quantization effects is also provided by including the corresponding ROC curves using G values of 2, 4, and 8. The results show a modest impact of quantization when G = 2 that becomes negligible for G ≥ 4. Equation (5.13) shows that, for a fixed R value, the RFI sine wave amplitude A is decreased as d is increased. This fact suggests that test performance may degrade for higher duty cycle interference. To examine this issue, Figures 5.4 and 5.5 plot the corresponding results for d = 50% and d = 100%, respectively. For the d = 100% case in Figure 5.5, performance is relatively poor even with an R value of 60; other tests such as Cross-frequency blanking whose practical implementation is described in [23] as well as previous chapters of this work, would be preferable for continuous sinusoidal interference. This will indeed be proven in Chapter 6. It is interesting to note however that the d = 50% case of Figure 5.4 shows performance comparable to that achieved for d = 1%. This is due to the fact that detection is not only a function of d and R, but also of the relationship between the pulse duration (Np ) and the local test size N . Simulations using a large number of shorter pulses to achieve a 50% duty cycle showed that much higher R values were required to obtain good test performance. Both Figures 5.4 and 5.5 again show quantization of the yi data to

179

have minimal impacts when G ≥ 4, although the impact is somewhat larger than in the d ≈ 1% case. Duty Cycle (d) ≈ 1% 1 0.9 0.8 0.7

Pd

0.6 0.5 0.4

No rounding, R=2.5 Rounded, G=2, R=2.5 Rounded, G=4, R=2.5 Rounded, G=8, R=2.5 No rounding, R=5 Rounded, G=2, R=5 Rounded, G=4, R=5 Rounded, G=8, R=5

0.3 0.2 0.1 0 0

0.1

0.2

0.3

0.4

0.5 Pfa

0.6

0.7

0.8

0.9

1

Figure 5.3: ROC curves for d ≈ 1%

To examine further the impact of N , ROC curves were computed for fixed R values as the sample size N was varied from 64 to 4096 (keeping the total integration period N I constant at 32768 samples). The results are shown in Figures 5.6 and 5.7 for the d ≈ 1% (R = 2.5) and d = 50% (R = 5) cases, respectively. For d ≈ 1%, the best detection performance is obtained when N = 64, and performance degrades as N increases. This is not surprising since the pulses simulated in this case have length Np = 32, so that the individual N = 64 sub-tests are more closely matched to the duration of the RFI pulses. The d = 50% cases instead shows improved performance

180

Duty Cycle (d) = 50% 1 0.9 0.8 0.7

Pd

0.6 0.5 No rounding, R=2.5 Rounded, G=2, R=2.5 Rounded, G=4, R=2.5 Rounded, G=8, R=2.5 No rounding, R=5 Rounded, G=2, R=5 Rounded, G=4, R=5 Rounded, G=8, R=5

0.4 0.3 0.2 0.1 0 0

0.1

0.2

0.3

0.4

0.5 P

0.6

0.7

0.8

0.9

1

fa

Figure 5.4: ROC curves for d = 50%

as N becomes larger, again as might be expected since the RFI pulse in this case has a duration of 16384 samples. These results show that it is desirable to choose the sample size N to be matched to the duration of expected RFI pulses if this duration is known a-priori. In the absence of such information, the parameter N should be chosen as a trade-off between implementation complexity and test performance given any knowledge of a potential range of RFI pulse lengths. The results of the next section were computed using N = 4096 to represent the latter case.

5.4.3

Comparison of ROC curves for the Shapiro-Wilk and kurtosis tests

Figures 5.8, 5.9, and 5.10 compare ROC curves achieved by the Shapiro-Wilk and kurtosis tests for the d ≈ 1, 50, and 100 percent cases, respectively. ROC curves for 181

Duty Cycle (d) = 100% 1 0.9 0.8 0.7

Pd

0.6 0.5 0.4

No rounding, R=30 Rounded, G=2, R=30 Rounded, G=4, R=30 Rounded, G=8, R=30 No rounding, R=60 Rounded, G=2, R=60 Rounded, G=4, R=60 Rounded, G=8, R=60

0.3 0.2 0.1 0 0

0.1

0.2

0.3

0.4

0.5 P

0.6

0.7

0.8

0.9

1

fa

Figure 5.5: ROC curves for d = 100%

the kurtosis test were obtained from a Monte Carlo procedure under the same signal model as that used for the Shapiro-Wilk analysis as explained in Section 5.3.3. Results in Figure 5.8 show both tests to go from very poor performance to near perfect performance as R is varied from 2.5 to 10. In between these values, the kurtosis test achieves the better performance. The performance of the Shapiro-Wilk test can be improved to exceed that of the full sample kurtosis test if N is selected through an a-priori knowledge of the RFI pulse length, although kurtosis test performance could also be improved if temporal sub-sampling were incorporated. The situation is distinctly different in Figure 5.9 for duty cycle 50 percent, due to the “blind spot” of the kurtosis test in this case. The Shapiro-Wilk test retains sensitivity (although the results may be different for other choices of N and Np as 182

Duty cycle (d) ≈ 1%, R = 2.5

1

0.8

Pd

0.6

0.4

N=64 N=256 N=1024 N=4096

0.2

0 0

0.1

0.2

0.3

0.4

0.5 P

0.6

0.7

0.8

0.9

1

fa

Figure 5.6: ROC curves vs. test size N , R=2.5, d ≈ 1% Duty cycle (d) = 50%, R = 5

1

0.8

Pd

0.6

0.4

N=64 N=256 N=1024 N=4096

0.2

0 0

0.1

0.2

0.3

0.4

0.5 Pfa

0.6

0.7

0.8

0.9

Figure 5.7: ROC curves vs. test size N , R=5, d = 50% 183

1

explained in Section 5.4.1), while the kurtosis test is insensitive to the presence of RFI. Conclusions for the 100 percent duty cycle case in Figure 5.10 are similar to those in the one percent duty cycle case in that both tests go from poor to near perfect performance over a similar range of R values. Again the kurtosis test achieves better performance over this range. However, the required R value for good performance is quite high (up to 120) which suggests that the performance of both tests would likely be exceeded by other approaches or by incorporating multiple frequency channels into the radiometer system. In Chapter 6, cross-frequency detection will be shown to be a viable alternative for the detection of CW signals.

Duty Cycle (d) ≈ 1%

1

0.8

Pd

0.6

0.4 SW, R=2.5 Kurtosis, R=2.5 SW, R=5 Kurtosis, R=5 SW, R=10 Kurtosis, R=10

0.2

0 0

0.1

0.2

0.3

0.4

0.5 Pfa

0.6

0.7

0.8

0.9

1

Figure 5.8: Comparison of ROC curves for Shapiro-Wilk and Kurtosis tests, d ≈ 1%

184

Duty Cycle (d) = 50%

1

0.8

Pd

0.6

0.4 SW, R=2.5 Kurtosis, R=2.5 SW, R=5 Kurtosis, R=5 SW, R=10 Kurtosis, R=10

0.2

0 0

0.1

0.2

0.3

0.4

0.5 P

0.6

0.7

0.8

0.9

1

fa

Figure 5.9: Comparison of ROC curves for Shapiro-Wilk and Kurtosis tests, d = 50%

5.5

Summary and Remarks

The performance of the Shapiro-Wilk test for normality was analyzed in detecting pulsed sinusoidal RFI. Results showed that the test can be successful in detecting pulsed sinusoidal RFI, particularly for duty cycles of 50% or less, and simulations not reported here over a larger range of duty cycles showed similar performance to those illustrated in Section 5.4. A comparison of results with those achieved by the kurtosis test showed that the two tests produce qualitatively similar results, with the kurtosis test generally achieving better performance in most cases. However, the Shapiro-Wilk test was shown not to suffer from the “blind spot” encountered in the kurtosis test at duty cycle 50 percent. It was also shown that Shapiro-Wilk test performance can be improved if a-priori expectations regarding RFI pulse lengths are available. 185

Duty Cycle (d) = 100%

1

0.8

Pd

0.6

0.4 SW, R=30 Kurtosis, R=30 SW, R=60 Kurtosis, R=60 SW, R=120 Kurtosis, R=120

0.2

0 0

0.1

0.2

0.3

0.4

0.5 Pfa

0.6

0.7

0.8

0.9

1

Figure 5.10: Comparison of ROC curves for Shapiro-Wilk and Kurtosis tests, d = 100%

186

Discussions of implementation of the Shapiro-Wilk test in hardware along with the effects of the quantization were also provided. Simulations showed that performance loss due to quantization is not significant if the radiometer thermal noise has a voltage standard deviation of approximately four times the analog to digital converter bit width. Implementation appears feasible, so that experimental tests should be possible in the future.

187

CHAPTER 6

PERFORMANCE STUDY OF A CROSS-FREQUENCY DETECTION ALGORITHM FOR PULSED SINUSOIDAL RFI IN MICROWAVE RADIOMETRY

6.1

Introduction

In previous chapters, several earth observing radiometric systems including LISR/CISR, ADD and PSR/CX that is capable of RFI detection have been described. Three main methods used in these radiometers for RFI detection purposes were time-domain detection, cross-frequency detection and kurtosis detection methods. A central question for future radiometric systems is the selection of a detection method (or methods) that results in the best performance for a specific application. Such a selection of course depends on the RFI environment that is to be observed, but present knowledge of the RFI environment remains limited. It has been mentioned in Chapter 5 that one RFI type that has been considered in previous studies is pulsed sinusoidal RFI, due to the ability of this type to describe radar-like emissions (low duty cycles) as well as continuous (i.e. very narrowband or “CW”) sinusoidal interference. Comparative studies of several algorithms for pulsed sinusoidal RFI have been performed [32, 66, 74]. However, the performance of cross-frequency detection algorithms was not analyzed in the past. Cross-frequency detection can be performed for 188

radiometers having multiple frequency channels with only modest additional requirements on hardware or datarate, and also can be used together with other methods such as kurtosis algorithms. Therefore, it is of interest for future systems to determine the detection performance of the cross-frequency approach, particularly in cases where other detection methods perform poorly. In this chapter, a theoretical performance model for cross-frequency detection of pulsed sinusoidal RFI is reported with the motives outlined above. Results for pulse-detection and subsampled kurtosis detection methods are also provided for comparison purposes. In a practical implementation of the cross-frequency detector, the detection threshold should vary with respect to the brightness temperature of the observed scene in order to maintain approximately constant false alarm and detection rates. A method for estimating this threshold and the impact of this estimation on detection performance is also studied. Notations and the signal model used in this Chapter is very similar to the one used in Chapter 5 for the analysis of the ShapiroWilk test; they will be briefly described in next section. Performance models used for each of the three detectors considered will also be described. Results comparing the performance of these detectors for different duty cycles and RFI strengths are then presented in Section 6.3. Special attention is given to the CW case where the pulse and subsampled kurtosis algorithms perform poorly. Effects of the system temperature estimation on the cross-frequency detector are studied in Section 6.4, and final remarks are provided in Section 6.5.

189

6.2 6.2.1

Formulation Signal model and notations

The signal model employed in Chapter 5 (which was adapted from [74]) for the RFI detection performance analysis of the Shapiro-Wilk test of normality is also used in this chapter when analyzing the cross-frequency approach. This model is based on a discretized representation of the fields observed by the radiometer, assuming sampling at the Nyquist rate of the observed bandwidth. However, a simplified signal model from [32] is used when comparing with the pulse and kurtosis methods because modeling kurtosis algorithm performance is more difficult under the signal model of [74]. The signal model is not reviewed further here save for the meaning of parameters like I and N ; however, the simplifications of [32] will be explained when the subsampled kurtosis method is discussed in Section 6.2.4. In the model of Section 5.3.1, a radiometer integration period of Q samples is divided into a set of I sub-sampling periods of length N each such that Q = IN . For the cross-frequency detector, N corresponds to the length of an FFT operation (in samples) used to produce the multiple frequency channels examined by the detector. For illustration, Q is chosen as 768000 samples in this Chapter, and N values of 8, 16 and 32 are considered. For a radiometer sampling, for example, at 20 nsec (i.e. the Nyquist rate of a 50 MHz bandwidth), the value of Q used results in a 15.36 msec radiometer integration period. It was mentioned in Section 5.3.1 that it is of interest to represent the amplitude of the RFI sine waves in terms of the uncertainty in the radiometer power estimate q (which for an integration of Q samples is equal to Tsys Q2 where Tsys = σ 2 is the radiometer system temperature).

190

For this reason, RFI amplitudes were described in terms of the exact ratio of the average signal power (dA2 /2) normalized by the uncertainty in the radiometer power estimate in a journal paper we submitted that was based on this study [76]. Note that this ratio, which I will denote here as R0 , is equal to:

dA2 R0 = 2Tsys There is an extra factor of

√

r

Q 2

(6.1)

2 in the definition of R0 compared to R given in

Equation 5.12 (ratio R was also defined as such in the journal paper we submitted on the Shapiro-Wilk test for RFI detection; reference [35]). In this study, to be consistent with the notation of the previous chapter, signal strength will be given in terms of R √ as defined by Equation 5.12. Thus, reader should keep in mind the absence of 2 if he/she were to compare the results of this chapter and the related journal paper [76]. There is not any real importance of using one definition of RFI strength over another as long as the definition employed is clearly explained. It should be mentioned again that RFI having R larger than approximately 10 would likely be detectable in the original radiometer measurements; cases of interest therefore have R < 10.

6.2.2

Cross-frequency detection model

A schematic of the cross-frequency detection algorithm considered here is shown in Figure 6.1. Incoming time domain field samples xi [n] are passed through a nonoverlapping N point FFT operation to obtain FFT outputs Xi [k]. The power in these FFT outputs (|Xi [k]|2 ) is then computed; this quantity is called a power spectrum. A total of I power spectra is obtained in the radiometer integration period of Q samples; these I spectra are averaged to obtain the average power spectrum Y [k]. The cross frequency detector chooses the maximum over k (as described below) of 191

Y [k] and declares detection if the maximum exceeds a threshold value Tcross . This threshold is related to the expected scene brightness temperature if it is assumed that the detector is applied after calibration of the radiometer measurements. If the scene temperature is not known a-priori (as is the case in radiometry), methods for estimating the threshold can be applied as described in Section 6.4.

Figure 6.1: Schematic of the cross-frequency detector

In the absence of RFI, the FFT operation produces N complex Gaussian random variables for Xi [k]. N/2 − 1 of these N values are positive frequencies that have corresponding conjugate negative frequency points; these negative frequencies are discarded. The square of the amplitude of each of these N/2 − 1 positive frequency components is a chi-squared random variable with two degrees of freedom. The FFT operation also produces DC (k = 0) and Nyquist (k = N/2) frequency outputs that are purely real. For consistency with the positive frequency points, the power in the DC and Nyquist frequency outputs is averaged to obtain an additional chi-squared random variable with two degrees of freedom. As a result, N/2 such random variables are obtained for each frame; the algorithm is described as using

192

N/2 “channels” henceforth. The averaging over I spectra then produces N/2 chisquared random variables with 2I degrees of freedom (Y [k]). If these N/2 random variables are independent, which is the case in the absence of RFI when a rectangular FFT “window” is used, the probability that the maximum of the Y [k]’s is less than Tcross can be calculated by multiplying the probabilities that each of the N/2 random variables is less than Tcross . The resulting quantity then allows computation of the detector false alarm rate as a function of Tcross . When RFI is present, the detector computes the maximum of N/2 non-central chi-squared random variables with 2I degrees of freedom. The non-centrality parameters of these random variables can be obtained by computing the Fourier transform of a specified pulsed source. These random variables remain independent so that the maximum operation can be computed as in the false-alarm case. An average probability of detection for a specified threshold can then be obtained by numerically averaging the probability of detection over the frequency, phase, arrival sample, and other properties of the RFI signal. This process includes the influence of “scalloping” loss in the FFT (i.e. the RFI pulse frequency is not aligned with an FFT bin) as well as partial filling of FFT N-point “frames” by an RFI pulse. In comparison with the cross-frequency detection algorithms applied in postprocessing to LISR/CISR data that were described in the previous chapters, this algorithm have two main differences: • Passband response of the radiometer system is not taken into account. • Spectral slope of the natural thermal radiation is not considered. In other words, all frequency bins are assumed to have the same mean. System passband response is device dependent; however, it can be said that to consider the 193

spectrum to be relatively flat where the variations in the passband response is small in magnitude compared to the difference between the threshold and the assumed mean level is a reasonable assumption. Spectral slope can be incorporated to a crossfrequency detection algorithm, such an algorithm is used for RFI mitigation in the PSR/CX radiometer [18] as described in Chapter 4; but this effect is also not very important if the total bandwidth considered is not on the order of GHz, and obviously PSR/CX cross-frequency model is not suitable for a theoretical study or a real time implementation of the cross-frequency method in hardware. It should also be pointed out that in LISR/CISR systems a 1024 point FFT operation is applied to the received fields. Although this high number of frequency channels provide a great opportunity for the identification and mitigation of RFI, such a high number of frequency channels is unlikely in a radiometer located in a satellite where energy and datarate is at a premium. Hence, number of frequency channels used in this chapter is limited by 32 as mentioned in Section 6.2.1.

6.2.3

Pulse detection model

A schematic of the pulse detection model considered in this chapter is shown in Figure 6.2. In this model, it is assumed that the square of time domain measured fields xi [n] are summed over N samples. This sum, denoted as P Di in the schematic, is a chi-squared random variable with N degrees of freedom when there is no RFI. When RFI is present, a non-central chi-squared random variable with N degrees of freedom is obtained instead. The non-centrality parameter for this case can again be computed for specified RFI pulse properties. During a radiometer integration period of Q = IN samples, I such random variables are attained, and the detection statistic is the maximum of these N random variables. The probabilities of false

194

alarm and detection for a specified threshold are calculated as the probability that the test statistic is greater than the threshold (TP U LSE ) in the absence and presence of RFI, respectively.

Figure 6.2: Schematic of the pulse detector

This algorithm is fairly similar to the APB algorithm described in Chapter 2.2.3. Primary difference between the two pulse detection models is the fact that the APB system is built for RFI “mitigation”, while the sole purpose of this model is pulse detection. As a result, while the APB algorithm assumed that there could be several pulses within an integration period and eliminated them in real time, in this particular model data from the whole radiometer integration period of Q is classified as either “corrupt” or ”clean” depending on the outcome of the test. Although this may seem more restrictive compared to the APB algorithm at first, we might remember that blanking window of APB had a certain width (denoted as NBLAN K ) and this width was comparable to the expected pulse widths of the RFI sources. Therefore, it might be concluded that if radiometer integration length Q is comparable to NBLAN K , mitigation performance of the two algorithms should not be very different.

195

6.2.4

Kurtosis detection model

A schematic of the “sub-sampled” kurtosis detector (different from Chapter 5 in the sense that it allows multiple frequency channels and time subsampling) is shown in Figure 6.3. In this case, N/2 frequency sub-channels are assumed to be obtained by filtering in hardware rather than through an FFT operation. In the absence of RFI, received fields in each channel are independent Gaussian random variables having a variance that is reduced from that of the total channel (σ 2 ) by a factor of N/2. Additionally, each of the kurtosis sub-channels originally has 2Q/N time samples to maintain the Nyquist rate. This set of samples can further be split into a set of time sub-samples, shown as tss in the schematic, if desired. The kurtosis estimator Kn is computed for each time and frequency sub-sample by dividing the fourth central moment of the received field (m4 ) by the field’s second central moment (m2 ) squared as given by Equation 5.17. For a sufficiently large number of field samples in the computation, Kn can be modeled as a Gaussian random variable with known mean and variance values as given in [26] (note that quantization effects were assumed to be negligible in these formulas). In the presence of RFI, the mean kurtosis can either become larger or smaller than its mean value in the absence of RFI (which is three), hence a two-sided test is used as shown in the schematic. The overall probabilities of detection and false alarm are then computed as the probability that the kurtosis values of at least one of the time/frequency subsamples exceeds the threshold of the two-sided test. The process used here is based on that described in [32], in which a simplified version of the pulsed sinusoidal RFI signal model is used. In this model, pulsed RFI is assumed to occur in only one of the frequency sub-samples (i.e. no scalloping effects.)

196

Figure 6.3: Schematic of the kurtosis detector

It is also assumed that RFI pulses, when present, begin at the beginning of the observations, and only one RFI pulse is present in each radiometer integration period (i.e. Npulse is 1). To provide a fair comparison among the detectors, this simplified signal model is used in the results shown whenever the three detection methods are compared. See [32] for additional information on the kurtosis detector performance model. This signal model also results in considerable savings of computational time, since a numerical integration is required to model the complete signal model with the scalloping loss where at least 2N frequency points and 20 phase points are required to obtain accurate results (in comparison with Monte Carlo simulations). The results to be shown use four time sub-samples for the kurtosis algorithm throughout. Other numbers of time sub-samples were also examined (including no time sub-sampling) and only modest changes in the results, particularly for higher duty cycle interference, were observed.

197

6.3

Detection Performance Results

Detector performance is presented using receiver operating characteristic (ROC) curves. A ROC curve is a plot of the probability of detection (Pd ) vs. the probability of false alarm (Pf a ) as the threshold is changed. A good detector achieves a high probability of detection with a low false alarm rate. ROC curves of the cross-frequency, pulse, and kurtosis detectors are plotted for the case of CW interference (i.e. 100 percent duty cycle) in Figure 6.4 with an 8 channel radiometer (or N = 16 for the pulse detector). ROC curves corresponding to R = 1, R = 2 and R = 3 interference power levels are shown for the cross-frequency detector, and for R = 3 only for the pulse and kurtosis detectors. ROC curves for the pulse and kurtosis detectors are almost a straight line indicating that these detectors are insensitive to CW RFI at this power level. In contrast, the cross-frequency detector obtains good performance at R = 3. In Figure 6.5, simulations of Figure 6.4 are repeated with the assumption of a pulsed sinusoidal RFI source with 50% duty cycle. Performance of the cross-frequency detector is virtually same with the CW RFI case. This is due to the fact that, for fixed R, the non-centrality parameters in the cross-frequency algorithm depend primarily on the total RFI power contained within a radiometer measurement, regardless of the duty cycle. Pulse and kurtosis detectors remain insensitive for this duty cycle. Figure 6.6 compares ROC curves for all three detectors for an RFI strength of 1 and a duty cycle of 0.1%. Cross-frequency performance remains the same but this time the other two detectors perform better with the kurtosis detector obtaining the best performance.

198

ROC curve comparison, 8 channels, d=1

1

0.8

Pd

0.6

0.4 Cross−freq R=1 Cross−freq R=2 Cross−freq R=3 Pulse detection R=3 Sub−sampled Kurtosis R=3

0.2

0 0

0.1

0.2

0.3

0.4

0.5 Pfa

0.6

0.7

0.8

0.9

1

Figure 6.4: ROC curves for the cross-frequency, pulse, and subsampled kurtosis algorithms: CW RFI, 8 channels (N = 16 for the pulse detector). R = 1, R = 2 and R = 3 cases shown for the cross-frequency case, R = 3 only for the pulse and kurtosis cases.

Although previous three ROC plots gave an idea of the performance of these three detectors versus RFI strength and duty cycle, a more complete description of the detection performance is desired. Following [32], a single “area under the curve” (AUC) metric can be used to achieve this goal; this quantity corresponds to the area under the ROC curve. As seen in Figure 6.4, in the worst case the probability of detection is equal to the probability of false alarm, yielding an area under the ROC curve of 0.5. The best case produces perfect detection for any false alarm rate, yielding an area under the ROC curve of 1. The AUC is defined such that the AUC in the worst case is equal to 0 and equal to unity in the best case; this definition

199

ROC curve comparison, 8 channels, d=0.5

1

0.8

Pd

0.6

0.4 Cross−freq R=1 Cross−freq R=2 Cross−freq R=3 Pulse detection R=3 Sub−sampled Kurtosis R=3

0.2

0 0

0.1

0.2

0.3

0.4

0.5 Pfa

0.6

0.7

0.8

0.9

1

Figure 6.5: Same as Figure 6.4, but for RFI with 50% duty cycle.

requires subtracting 0.5 from the true area under the ROC curve and multiplying the result by 2. Figure 6.7 presents an image of AUC values for the cross-frequency detector using four frequency channels (i.e. N = 8). The horizontal axis of the image is the ratio R that describes the RFI strength, and the vertical axis is the duty cycle of pulsed sinusoidal interference on a logarithmic scale. The logarithmic color scale is defined so that “blue” areas indicate AUC values near unity while “red” areas correspond to low AUC values where the detector is ineffective. The image makes clear that the performance of the cross-frequency algorithm depends only weakly on the RFI duty cycle. The results show that it is possible to obtain good detection performance (AUC ≈ 0.95 or more) for R ≥ 2.6 at all duty cycle values. 200

ROC curve comparison, 8 channels, d=0.001

1

0.8

Pd

0.6

0.4

Cross−freq R=1 Pulse detection R=1 Sub−sampled Kurtosis R=1

0.2

0 0

0.1

0.2

0.3

0.4

0.5 Pfa

0.6

0.7

0.8

0.9

1

Figure 6.6: ROC curves for the cross-frequency, pulse, and subsampled kurtosis algorithms: 0.1% duty cycle, R = 1, 8 channels (N = 16 for the pulse detector).

Results from the cross-frequency detector (upper) are compared with those of the subsampled kurtosis (middle) and pulse (lower) detectors in Figure 6.8 ( four frequency subchannels and four time subsamples for the kurtosis detector and an N = 8 sample integration for the pulse detector.) Although the pulse and kurtosis methods provide a performance improvement for RFI with low duty cycles compared to the cross-frequency detector, they become insensitive to RFI at higher duty cycles. Kurtosis only becomes advantageous compared to cross-frequency detector when duty cycle gets less than 1% while pulse detector, as it might be expected, performs the best for extremely low duty cycles (less than 0.05%). Similar conclusions are obtained as the number of time sub-samples in the kurtosis algorithm is varied.

201

Figure 6.7: AUC image for the cross-frequency detector vs. RFI strength and duty cycle, four channels

Figure 6.9 provides a similar AUC image comparison for a 16 channel radiometer (N = 32). The results show improved detection performance for the cross-frequency algorithm compared to N = 8, such that good performance is achieved at R ≥ 1.5, while the performance of the sub-sampled kurtosis method is also improved compared to N = 8 but remains insensitive to higher duty cycle RFI. A modest increase in performance of the pulse detector can be observed as well due to the increase in integration time.

6.3.1

Effects of scalloping loss

The results presented to this point have used the signal model of [32], which assumes that the RFI is centered in a single frequency channel. To quantify the 202

Figure 6.8: AUC image comparison for the cross-frequency (upper), subsampled kurtosis (middle) and pulse (lower) algorithms vs. RFI strength and duty cycle, four channels/N = 8

impact of non-centered RFI frequencies (i.e. scalloping loss), plots of the probability of detection versus the RFI strength are provided in Figure 6.10 for a constant Pf a of 1%; note the inverted and logarithmic nature of the vertical axis in the Figure. Curves using the signal model of [32] and of Section 6.2.1 are compared for 8 and 16 frequency channels in the case of continuous sinusoidal interference. The results show detector performance to degrade when scalloping loss is included. For example, a 99 percent probability of detection is not achieved until R ≈ 3.3 with 16 channels when scalloping loss is considered, versus R ≈ 2 neglecting scalloping loss. These variations are significant, but the subsampled kurtosis algorithm would also be affected in a

203

Figure 6.9: AUC image comparison for the cross-frequency (upper), subsampled kurtosis (middle) and pulse (lower) algorithms vs. RFI strength and duty cycle, sixteen channels/N = 32

similar manner when non-centered frequencies are considered, so that the relationship between the algorithms is not expected to be significantly impacted. As a further inspection of scalloping loss effects, AUC images of the cross-frequency detection algorithm with (upper) and without (lower) scalloping loss are provided in Figure 6.11 for sixteen channels. It can again be seen that scalloping loss causes a degradation in performance: the RFI strength needed for a good detection performance (AU C ≈ 0.95) increases from R ≈ 1.5 to R ≈ 2.2. However, good performance is still achieved, and the behavior of the cross-frequency detector’s performance versus duty cycle and RFI strength is very similar in both cases. In general, the cross frequency algorithm appears to be very useful for detecting pulsed sinusoidal RFI 204

Pfa=1% 0

P

d

8 channels, with scalloping loss 8 channels, bin−centered 16 channels, with scalloping loss 16 channels, bin−centered

0.9

0.99 0

0.5

1

1.5

2 2.5 RFI Strength in R

3

3.5

4

4.5

Figure 6.10: Probability of detection vs. RFI strength for a constant Pf a of 1%, CW RFI. Results when RFI is assumed to be centered in a channel (“bin centered”) are compared with the general case of random RFI frequency (“with scalloping loss”) for 8 and 16 frequency channels.

with high duty cycles, and use of such an algorithm (in addition to other approaches) in any radiometer system having multiple frequency channels is recommended.

6.3.2

A simplified method for the calculation of scalloping loss

While the approach described in Section 6.2.2 for the calculation of scalloping loss is general, the required numerical integrations can be computationally expensive (although reasonable for desktop level computing resources in time scales of minutes to hours.) Later analyses showed that a simplified approach without the complete

205

Figure 6.11: AUC image comparison for the cross-frequency algorithm with (upper) and without (lower) scalloping loss vs. RFI strength and duty cycle, sixteen channels/N = 32

averaging procedure could yield predictions practically identical to those of the complete computation. This approach models the RFI as appearing in only two of the FFT outputs Y [k] while the remainder are RFI free. The impact of scalloping loss is considered by modeling the two corrupted frequencies as non-central chi-squared random variables with 2I degrees of freedom; the non-centrality parameter is determined by pulsed RFI properties, and performance is averaged over a splitting of the RFI power across two adjacent channels using weights determined by a channel filter model. The channel filter used was that of the rectangular FFT window. It is possible that other channel filters (or FFT windows) may also be describable in the same way. However, note that when a window other than rectangular is used in 206

performing the FFT, the frequency outputs Xi [k] become correlated even in the RFI free case, making the analysis much more difficult. A comparison of this approximation and a complete solution of scalloping loss is given in Figure 6.12. This figure is a plot of detection probability vs. RFI strength for 8 and 16 frequency channels. The difference between the approximation and complete solution is negligible; thus, it could be concluded that this approximation can be helpful to describe the effect of scalloping in similar simulations where power is divided between frequency channels and computational efficiency is required. However, as mentioned above, computational time required for the complete solution was reasonable of the purposes of this study and this approximation was not employed in the results shown.

6.4

System Temperature Estimation Issues

The previous results have assumed a fixed threshold Tcross is used in the crossfrequency detection algorithm, corresponding to a fixed variance of the radiometer thermal noise. In reality, the variance of the radiometer observed thermal noise can vary significantly with the geophysical scene observed, so that some estimate of the current thermal noise variance is required in order to maintain a known relationship between the threshold value and the detector probability of false alarm. One method for estimating this variance involves taking the mean of the observed brightnesses versus frequency, excluding a specified number of the largest brightnesses in the computation of the mean. Detection can then be declared if any of the frequency channels exceed some function of this system temperature estimate by a specified threshold.

207

Pfa=1% 0

P

d

8 channels, approximation 8 channels, complete solution 16 channels, approximation 16 channels, complete solution

0.9

0.99 0

0.5

1

1.5

2 2.5 3 RFI Strength in R

3.5

4

4.5

5

Figure 6.12: Probability of detection vs. RFI strength for a constant Pf a of 1%, CW RFI. Results obtained with an approximation to the scalloping loss is compared with the complete solution for 8 and 16 frequency channels.

To model the estimation process, consider a radiometer having M channels (M = N/2 in the notation used) so that the cross frequency blanker considers M independent chi-squared random variables with 2I degrees of freedom in the RFI free case. The estimation procedure requires sorting these measurements, so that M sorted measurements are obtained, and the largest Mdrop values are excluded in the computation of the system temperature estimate. One quantity of interest is the probability density function of the system temperature estimate, which is the mean of the smallest M − Mdrop of the chi-squared random variables. This pdf in theory could be obtained using properties of order statistics, but in practice is very difficult to evaluate since the computation involves 208

an M − Mdrop dimensional space. To simplify the computation, it is assumed that the pdf of the system temperature estimate can be modeled as a Gaussian random variable; this is likely to be reasonable so long as 2I and M − Mdrop are not small. Once this approximation is applied, knowledge is required only of the mean and variance of the system temperature estimate. This mean and variance can be obtained by combining means and covariances of the lower M − Mdrop order statistics; the required means and covariances of these order statistics can be obtained using the one and two point order statistic pdfs in [77]. It is also assumed that the pdf of the system temperature estimate so obtained is applicable both in the presence and absence of RFI; this is equivalent to assuming that the RFI lies entirely within the discarded Mdrop largest channels. As an illustration of this procedure, Figure 6.13 plots the mean and standard deviation of the system temperature estimate with respect to the number of channels discarded for the 4, 8, and 16 channel cases. Here it is assumed that the mean value of Tsys is 590 K and its standard deviation is 0.95 K (corresponding to Q = 768000 from Section 6.2.1) so that the results shown are in units of Kelvin. The negative bias in the mean system temperature estimate is less than 1.5 K if 2 channels are discarded even for a four channel system, and in addition this bias in the mean can be corrected since it is fixed given the number of channels discarded. The dependence of the standard deviation of the system temperature estimate on the number of channels discarded is also modest, confirming that the effect of discarding channels in the system temperature estimation should be small, especially as the number of channels becomes larger. To incorporate the system temperature estimation process into the detector performance model, the difference between the maximum brightness temperature (TM AX ) 209

3

Change in mean/standard deviation (Kelvin)

2 1 0 −1 −2 −3 −4

Change in mean, 4 channels Change in mean, 8 channels Change in mean, 16 channels Change in std, 4 channels Change in std, 8 channels Change in std, 16 channels

−5 −6 −7

0

2

4

6 8 10 Number of channels discarded

12

14

16

Figure 6.13: Mean and standard deviation of the system temperature estimate as channels are discarded in the threshold estimation procedure of the cross-frequency algorithm, for 4, 8, and 16 frequency channels.

and the mean of the smaller M − Mdrop frequency channels (TM EAN ) can be taken to be a single random variable and compared with a fixed threshold. However, to model this detector analytically, the joint pdf of TM AX and TM EAN must be known, but this quantity is difficult to compute. One possible approach involves approximating TM AX and TM EAN as independent, but since TM AX is always greater than TM EAN , this assumption is problematic. An alternate approach creates a new random variable TN = cTM AX + TM EAN .

210

(6.2)

and defines c so that the correlation between TN and TM AX is zero: c=

−cov(TM AX , TM EAN ) var(TM AX )

(6.3)

Here, cov and var represent the covariance and variance operations respectively. The new detection rule is to compare the difference between TM AX and TN to a threshold: P (TM AX − TN > ∆TCROSS )

(6.4)

where P denotes the probability and ∆TCROSS is a fixed threshold. TM AX and TN are now assumed to be independent, so that the pdf of TM AX − TN can be obtained analytically. Note that while TN and TM AX are uncorrelated, they are clearly not independent, however the assumption that TN and TM AX are independent appears to be less restrictive than assuming that TM EAN and TM AX are independent. Using the assumption of independence, the pdf of the difference random variable can be evaluated as a convolution between the original pdf of the maximum brightness temperature and a scaled version of the Gaussian pdf of the system temperature estimate. The resulting random variable is then compared with a scaled version of the original threshold. To assess these approximations, Figure 6.14 compares ROC curves obtained with this method to Monte Carlo simulations that include system temperature estimation effects without approximation. CW RFI having R = 3 is considered, and the estimation procedure discards the 2 channels with the highest brightnesses. Comparisons are shown for 4, 8, and 16 channels, and 16384 realizations were used in the Monte Carlo simulation. The comparisons show that the accuracy of the approximation improves as the number of channels increases, and that the approximation is reasonable for 8 channels and very good for 16 channels. Other simulations also confirmed that 211

results from this procedure can be used without a significant loss in accuracy if the number of frequency channels is greater than or equal to 8.

R=3, 2 dropped channels 1 0.9 0.8 0.7

Pd

0.6 0.5 0.4 Approximation, 4 ch. Approximation, 8 ch. Approximation, 16 ch. Monte Carlo, 4 ch. Monte Carlo, 8 ch. Monte Carlo, 16 ch.

0.3 0.2 0.1 0 0

0.1

0.2

0.3

0.4

0.5 Pfa

0.6

0.7

0.8

0.9

1

Figure 6.14: ROC curves obtained using the analytical approximation to threshold estimation effects compared with results from Monte Carlo simulations. CW RFI having R = 3, Mdrop = 2, and for 4,8 and 16 channels.

To show the effect of the system temperature estimation on detection performance, the probability of detection versus CW RFI strength is plotted in Figure 6.15 for Pf a = 1% when the system temperature is estimated with 2 channels discarded in the 8 and 16 channel cases using the approximate model. For comparison, results from Figure 6.10 when the system temperature is exactly known are also included. These results confirm that estimation of the system temperature causes only a modest degradation in detection performance for high duty cycle sinusoidal interference.

212

Pfa=1% 0

P

d

8 channels, no est. effects 8 channels, with est. effects 16 channels, no est. effects 16 channels, with est. effects

0.9

0.99 0

0.5

1

1.5

2 2.5 3 RFI Strength in R

3.5

4

4.5

5

Figure 6.15: Results of Figure 6.10 are compared with curves including system temperature estimation effects. CW RFI

6.5

Summary and Remarks

The performance of a cross-frequency detection algorithm was analyzed for pulsed sinusoidal RFI and compared with that achieved by the pulse and sub-sampled kurtosis algorithms in this chapter. It was shown that the performance of the crossfrequency detector improves as the number of frequency channels increases, and that its performance is only weakly sensitive to the RFI duty cycle at a fixed R RFI power level. A reasonable sensitivity to pulsed sinusoidal RFI is achieved even with only four radiometer channels. The performance of the pulse and sub-sampled kurtosis methods was found to exceed that of the cross-frequency algorithm for low duty cycle

213

pulsed RFI, but the cross-frequency method offers significant performance improvements over these methods for higher duty cycle pulses including CW interference. A method for modeling the impact of system temperature estimation effects on detector performance was also provided, and it was shown that system temperature estimation is expected to result in only a modest degradation for pulsed sinusoidal RFI sources. A simple way to model scalloping effects was discussed as well. It should be noted that, since pulsed sinusoidal RFI sources are localized in frequency by definition, it is not surprising that the cross-frequency detector performs very well against this class of RFI. Because this class of RFI, especially the CW case which represents any very narrowband emission, is expected to be encountered in Earth observations, use of cross-frequency detection algorithms appears to be warranted in future Earth observing radiometer systems. Such use is compatible with other algorithms such as the sub-sampled kurtosis or pulse methods that can provide enhanced performance for low duty cycle pulses and/or other RFI source types.

214

CHAPTER 7

CONCLUSIONS

Several surveys on the severity of RFI at microwave frequencies have been published in the last ten years. Results suggested that RFI might cause a significant degradation in the accuracy of the microwave radiometric observations. This thesis documented studies of the RFI environment and of RFI mitigation methods with the purpose of reducing the detrimental effects of the RFI problem. Contributions of this research can be listed as follows: • Using LISR and CISR digital radiometers that possess high spectral and temporal resolution, time and frequency properties of the existing RFI sources at L- and C-band were investigated. • Joint airborne and groundborne campaigns were conducted with other microwave remote sensing groups to intercompare the measurements of different sensors. LISR/CISR have served as a ground-truthing device for the other radiometers participating in the campaigns. These experiments were devised in part to test radiometric designs that are in consideration for future space based microwave remote sensing missions like the Soil Moisture Active and Passive (SMAP) mission of NASA. As a consequence, results of the campaigns have provided a reference for the science teams of these radiometric missions. 215

• Effectiveness of the Asynchronous Pulse Blanker (APB) system in a practical setting was demonstrated with observations in the vicinity of an air route surveillance radar. • Pulse detection and cross-frequency detection algorithms that make use of the high temporal and frequency resolution of the LISR/CISR sensors were developed and applied to the data obtained from the aforementioned campaigns in post-processing. Considerable success was achieved in mitigating real world RFI even against sources on the order of natural geophysical variations. Although a system similar to LISR/CISR that operates in space and downlinks data at such a high resolution is unlikely, proficiency of these methods in mitigating RFI as well as the successful implementation of a real time RFI mitigation system in APB showed that it might be possible to have real time RFI mitigation onboard the satellite and to downlink the clean data after further integration. Downlinking a single flag that shows the presence/absence of RFI in a radiometer integration period was suggested as another alternative since it might not be desirable to throw data out automatically. • Shapiro-Wilk test of normality was proposed as a novel RFI mitigation technique. Results were compared to the kurtosis technique for the case of pulsed sinusoidal RFI. It was shown that the Shapiro-Wilk test did not suffer from the blind spot observed by the kurtosis algorithm. However, both the kurtosis and the Shapiro-Wilk tests performed poorly against CW RFI. The Shapiro-Wilk test was found to be suitable for implementation in a radiometric system. Considering the more omnibus character of this test cited in the statistical literature

216

compared to the kurtosis test, such an implementation of the Shapiro-Wilk test in hardware might be a worthwhile endeavour. • A theoretical comparison of detection performance was given between crossfrequency detection, pulse detection and kurtosis detection methods for the case of pulsed sinusoidal RFI. Emphasis was on the cross-frequency detection for this study and a very simple cross-frequency detection model that is suitable for implementation in hardware was devised. It was shown that the cross-frequency algorithm maintains a high (and almost constant) level of performance for this type of RFI regardless of duty cycle while the other methods performed poorly for the high duty cycle or CW RFI cases. Due to the abundance of narrowband RFI (as also evidenced by experimental results of this thesis) and the relative ease of implementing the cross-frequency algorithm in a multiband radiometer, it was suggested that the cross-frequency algorithm should be considered in radiometers with RFI suppression capability; possibly as an addition to a more general method like kurtosis. A summary of the thesis is as follows: In Chapter 1, a very brief overview of the microwave radiometry is given. The severity of the RFI problem was described and strategies used in literature for RFI mitigation were introduced. Chapter 2 described an L-band groundborne campaign conducted in Canton, Michigan. A review of the front-end and LISR back-ends were provided and solutions of several practical problems encountered in this campaign were discussed. The capabilities of the LISR system at detecting and removing pulsed interference in real-time were demonstrated. As a result of these capabilities, LISR system was able to continue measurements even in the presence of very high levels of RFI. More 217

complicated algorithms that make use of the high spectral and temporal resolution of LISR were also developed to mitigate RFI in post-processing . Another L-band expedition was discussed in Chapter 3. This campaign was performed at the Jet Propulsion Laboratory (JPL) of NASA in Pasadena, California. The durations of the observational states (e.g. antenna state) were very short in this experiment which necessitated changes in the LISR design. Practical problems encountered (like the noise in the state trigger channel) and their solutions were described. Presence of RFI sources in the protected portion of the spectrum at Lband was shown using the high spectral resolution of the LISR system. These RFI sources were successfully removed using pulse detection and cross-frequency detection techniques in post-processing. Comparisons of LISR data with those measured by Passive/Active L-S band (PALS) sensor of JPL and Agile Digital Detector (ADD) sensor of University of Michigan were made. All three sensors showed periodic interference with RFI level closely correlated among sensors. An airborne campaign at C-band was described in Chapter 4. CISR backend was mounted to the WB-57 high altitude aircraft of NASA in this experiment. Polarimetric Scanning Radiometer (PSR) of University of Colorado provided the front-end for the CISR and the ADD backends. Comparisons between the measurements of these three sensors were provided. Flight path of this expedition included overflights of major Texas cities, observations of the rural parts of Texas, and the observations of the Gulf of Mexico. This variety in the observed RFI environments allowed a demonstration of the CISR ’s capabilites in mitigating RFI with diverse amplitude levels as well as distinct temporal and spectral characteristics. In Chapter 5, possible use of the Shapiro-Wilk test as a detector of RFI was analyzed. An overview of the Shapiro-Wilk test was given. As mentioned above, 218

detection performance was compared with that of the kurtosis test for the detection of pulsed sinusoidal RFI. Feasibility of the Shapiro-Wilk test’s implementation in hardware was discussed. Results suggested that the Shapiro-Wilk test is a viable detector alternative for the RFI suppressing radiometer systems. Chapter 6 provided detection performance comparisons between cross-frequency detection, pulse detection, and kurtosis detection methods for the case of pulsed sinusoidal RFI. Particular detection models used in this study were described. The probabilities of detction and false alarm for each detector were given mathematically for this particular type of RFI. As mentioned among the list of contributions, it was shown that the cross-frequency method can sucessfully detect pulsed sinusoidal RFI regardless of duty cycle. Future research on this field should include further field experiments to better understand the existing RFI environment and to quantify the effectiveness of RFI detection strategies against different RFI sources. Geolocating RFI sources and trying to obtain a match with the known source databases might provide additional information that might be useful in removing RFI from earth remote sensing observations. Tests of radiometers employing novel RFI detection strategies will likely continue in the future. Shapiro-Wilk test or the simple cross-frequency blanking model described in this thesis are candidates for implementation in hardware. Theoretical studies on RFI detection techniques should also continue. Studies so far have focused on the case of pulsed sinusoidal RFI. Simulations using models of different types of RFI sources would enhance our knowledge about the capabilities and weaknesses of different RFI detection strategies. In conclusion, it can be said that although the history of the satellite microwave remote sensing dates back to the late 1960s, the ramifications of RFI on the accuracy 219

of the obtained measurements have only been understood in this decade. There was a significant need for learning more about the existing RFI environment, for developing simple and effective techniques for RFI detection, and for analyzing and comparing the performance of these techniques against different types of RFI sources. The studies described here contributed to the development of this relatively new research area and provided a reference for future planned microwave satellite radiometry missions.

220

BIBLIOGRAPHY

[1] F. T. Ulaby, R. K. Moore, and A. K. Fung. The advanced theory of statistics, Vol. 1. Artech House, Norwood, MA, 1981. [2] J. R. Wang and B. J. Choudhury. Remote sensing of soil moisture content over bare field at 1.4 GHz frequency. Journal of geophysical research, 86(C6):5277– 5282, June 1981. [3] E. G. Njoku. Passive microwave remote sensing of the earth from space-a review. Proceedings of the IEEE, 70(7):728–750, July 1982. [4] R. H. Dicke, R. Beringer, R. L. Kyhl, and A. B. Vane. Atmospheric absorption measurements with a microwave radiometer. Phy. Rev., 70(5-6):340, September 1946. [5] A. E. Basharinov, A. S. Gurvich, S. T. Yegorov, A. A. Kurskaya, D. T. Matveyev, and A. M. Shutko. The results of microwave sounding of the earth’s surface according to experimental data from the satellite Cosmos 243. In Space Research XI, Berlin, Germany, 1971. [6] P. Gloersen and F. T. Barath. A scanning multichannel microwave radiometer for Nimbus-G and Seasat-A. IEEE Journal of Ocean Engineering, 2:172–178, 1977. [7] AMSR/AMSR-E home page. index_ e. htm .

In http: // sharaku. eorc. jaxa. jp/ AMSR/

[8] A. Shibata, K. Imaoka, M. Kachi, and Y. Fujimoto. AMSR/AMSR-E sea surface temperature algorithm development. In IEEE Geoscience and Remote Sensing Symposium, volume 1, pages 672–673, July 2003. [9] WindSat homepage. In http: // www. nrl. navy. mil/ WindSat/ index. php . [10] D. Le Vine, C. Koblinsky, and F. Pellerano. The measurement of salinity from space: sensor concept. In IEEE Geoscience and Remote Sensing Symposium, volume 3, pages 1010–1012, July 2001.

221

[11] J. Font, G. S. E. Lagerloef, D. Le Vine, and A. Camps. Open issues for the soil moisture and ocean salinity (SMOS) satellite mission salinity retrieval. In Proceedings of SMOS Campaigns Workshop, pages ESA SP–525: 7–13, 2003. [12] SMAP website. In http: // smap. jpl. nasa. gov/ . [13] A. J. Camps, I. Corbella, F. Torres, J. Bara, and J. Capdevila. RF interference analysis in aperture synthesis interferometric radiometers: application to L-Band MIRAS instrument. IEEE Transactions on Geoscience and Remote Sensing, 38(2):942–950, March 2000. [14] D. M. LeVine. ESTAR experience with RFI at L-band and implications for future passive microwave remote sensing from space. In IEEE Geoscience and Remote Sensing Symposium, volume 3, pages 847–849, 2002. [15] L. Li, E. G. Njoku, E. Im, P. S. Chang, and K. M. St.Germain. A preliminary survey of radio-frequency interference over the US in Aqua AMSR-E data. IEEE Transactions on Geoscience and Remote Sensing, 42:380–390, 2004. [16] J. T. Johnson, A. J. Gasiewski, B. G¨ uner, G. A. Hampson, S. W. Ellingson, R. Krishnamachari, N. Niamsuwan, E. McIntyre, M. Klein, and V. Leuski. Airborne radio frequency interference studies at C-band using a digital receiver. IEEE Transactions on Geoscience and Remote Sensing, 44(7):1974–1985, July 2006. [17] S. W. Ellingson and J. T. Johnson. A polarimetric survey of radio frequency interference in C- and X-bands in the continental United States using WindSAT radiometry. IEEE Transactions on Geoscience and Remote Sensing, 44:540–548, 2006. [18] A. J. Gasiewski, M. Klein, A. Yevgrafov, and V. Leuski. Interference mitigation in passive microwave radiometry. In IEEE Geoscience and Remote Sensing Symposium, volume 3, pages 1682–1684, June 2002. [19] M. Basseville and I. V. Nikiforov. Detection of abrupt changes: theory and applications. Prentice-Hall, Englewood Cliffs, NJ, 1993. [20] P. A. Fridman and W. A. Baan. RFI mitigation methods in radio astronomy. Astronomy and Astrophysics, 378:327–344, 2001. [21] G. A. Hampson, S. W. Ellingson, and J. T. Johnson. Design and demonstration of an interference suppressing microwave radiometer. In IEEE Aerospace Conference, volume 2, pages 993–999, March 2004. [22] N. Niamsuwan, J. T. Johnson, and S. W. Ellingson. Examination of a simple pulse blanking technique for RFI mitigation. Radio Science, 40, June 2005. 222

[23] B. G¨ uner, J. T. Johnson, and N. Niamsuwan. Time and frequency blanking for radio frequency interference mitigation in microwave radiometry. IEEE Transactions on Geoscience and Remote Sensing, 45(11):3672–3679, November 2007. [24] C. S. Ruf, S. M. Gross, and S. Misra. RFI detection and mitigation for microwave radiometry with an agile digital detector. IEEE Transactions on Geoscience and Remote Sensing, 44(3):694–706, March 2006. [25] S. Misra, S. S. Kristensen, S. S. Sobjaerg, and N. Skou. CoSMOS: Performance of kurtosis algorithm for radio frequency interference detection and mitigation. In IEEE Geoscience and Remote Sensing Symposium, number 23-28, pages 2714– 2717, July 2007. [26] R. D. De Roo, S. Misra, and C. S. Ruf. Sensitivity of the kurtosis statistics as a detector of pulsed sinusoidal RFI. IEEE Transactions on Geoscience and Remote Sensing, 45(7):1938–1946, July 2007. [27] J. R. Piepmeier, P. Mohammed, and J. Knuble. A Double Detector for RFI Mitigation in Microwave Radiometers. IEEE Transactions on Geoscience and Remote Sensing, 46(2):458–465, February 2008. [28] R. Weber, C. Faye, F. Biraud, and J. Dansou. Spectral detector for interference time blanking using quantized correlator. Astronomy and Astrophysics Supp., 126:161–167, 1997. [29] C. S. Ruf and S. Misra. Detection of radio frequency interference with the Aquarius radiometer. In IEEE International Geoscience and Remote Sensing Symposium, pages 2722–2725, July 2007. [30] G. M. Nita, D. E. Gary, Z. Liu, G. J. Hurford, and S. M. White. Radio frequency interference excision using spectral-domain statistics. The Publications of the Astronomical Society of the Pacific, 119(857), July 2007. [31] S. Misra and C. Ruf. Comparison of pulsed sinusoidal RFI detection algorithms using time and frequency sub-sampling. In IEEE Geoscience and Remote Sensing Symposium, 2008. [32] S. Misra, P. N. Mohammed, B. G¨ uner, C. Ruf, J. R. Piepmeier, and J. T. Johnson. Microwave radiometer radio frequency interference detection and mitigation algorithms: a comparative study. submitted to IEEE Transactions on Geoscience and Remote Sensing, 2008. [33] R. DeRoo and S. Misra. Effectiveness of the sixth moment to eliminate a kurtosis blind spot in the detection of interference in a radiometer. In IEEE Geoscience and Remote Sensing Symposium, 2008.

223

[34] B. G¨ uner, M. Frankford, and J. T. Johnson. Comparison of three omnibus normality tests for the detection of pulsed sinusoidal RFI sources. In URSI General Assembly, 2008. [35] M. T. Frankford B. G¨ uner and J. T. Johnson. A study of the Shapiro-Wilk test for the detection of pulsed sinusoidal radio frequency interference. IEEE Transactions on Geoscience and Remote Sensing, accepted for publication, 2009. [36] National Research Council of the National Academies. Handbook of frequency allocations and spectrum protection for scientific uses. The National Academies Press, Washington DC, 2007. [37] E. G. Njoku, P. Ashcroft, T. K. Chan, and L. Li. Global survey and statistics of radio-frequency interference in AMSR-E land observations. IEEE Transactions on Geoscience and Remote Sensing, 43(5):938–947, May 2005. [38] S. W. Ellingson, J. D. Bunton, and J. F. Bell. Removal of the GLON ASS C/A signal from OH spectral line observations using a parametric modeling technique. The Astrophysical Journal Supplement Series, 135:87–93, July 2001. [39] C. Barnbaum and R. F. Bradley. A new approach to interference excision in radio astronomy: real-time adaptive cancellation. The Astronomical Journal, 115:2598–2614, 1998. [40] P. Mohammed, J. Knuble, and J. Piepmeier. Analog radio-frequency interference (RFI) suppression system for microwave radiometers. In American Geophysical Union 2006 Fall Meeting, 2006. [41] S. W. Ellingson and G. A. Hampson. Mitigation of radar interference in L-band radio astronomy. The Astrophysical Journal Supplement Series, 147(1):167–176, 2003. [42] S. W. Ellingson, G. A. Hampson, and J. T. Johnson. Characterization of L-band RFI and implications for mitigation techniques. In IEEE Geoscience and Remote Sensing Symposium, volume 3, pages 1745–1747, Toulouse, France, July 2003. [43] S. W. Ellingson, G. A. Hampson, and J. T. Johnson. Design of an L-band microwave radiometer with active mitigation of interference. In IEEE Geoscience and Remote Sensing Symposium, volume 3, pages 1751–1753, July 2003. [44] G. A. Hampson, S. W. Ellingson, and J. T. Johnson. Design of an L-band microwave radiometer with active mitigation of interference. In IEEE Antennas and Propagation/URSI Symposium, 2003. [45] S. W. Ellingson, G. A. Hampson, and J. T. Johnson. Design of an L-band microwave radiometer with active mitigation of interference. In NASA Earth Science Technology Conference, 2003. 224

[46] J. T. Johnson, G. A. Hampson, and S. W. Ellingson. Design and demonstration of an interference suppressing microwave radiometer. In IEEE Geoscience and Remote Sensing Symposium, volume 3, pages 1683–1686, September 2004. [47] N. Niamsuwan, J. T. Johnson, and S. W. Ellingson. Examination of a simple pulse blanking technique for RFI mitigation. In Workshop in Mitigation of Radio Frequency Interference in Radio Astronomy, 2004. [48] J. T. Johnson, A. J. Gasiewski, G. A. Hampson, S. W. Ellingson, R. Krishnamarachi, and M. Klein. Airborne radio frequency interference studies at C-band using a digital receiver. In IEEE Geoscience and Remote Sensing Symposium, volume 3, pages 1687–1690, September 2004. [49] N. Niamsuwan and J. T. Johnson. Sky observations at L-band using an interference suppressing radiometer. In IEEE Geoscience and Remote Sensing Symposium, volume 8, pages 5562–5565, July 2005. [50] S. W. Ellingson, G. A. Hampson, and J. T. Johnson. Digital receiver with interference suppression for microwave radiometry. In NASA Tech Brief GSC14776-1, February 2005. [51] S. Yueh, W. Wilson, E. Njoku, S. Dinardo, D. Hunter, Y. Rahmat-Samii, K. S. Kona, and M. Manteghi. Compact, lightweight dual-frequency microstrip antenna feed for future soil moisture and sea surface salinity missions. In NASA Earth-Sun System Technology Conference 2006, 2006. [52] E. Njoku, W. Wilson, S. Yueh, S. Dinardo, T. Jackson, and V. Lakshmi. PALS observations of soil moisture in SMEX02. In IEEE Geoscience and Remote Sensing Symposium, 2003. [53] MA-COM MD-169 mixer datasheet. DataSheets/ MD_ MDC_ MDS-169. pdf .

In

http: // www. macom. com/

[54] E. McIntyre, A. J. Gasiewski, M. Klein, V. Leuski, B. L. Weber, and V. Irisov. Experimental results from a spectral scanning technique for passive remote sensing of soil moisture. In U. S. National Radio Science Meeting, 2006. [55] Federal Communications Commission Universal Licensing System search page. In http: // wireless2. fcc. gov/ UlsApp/ UlsSearch/ searchLicense. jsp . [56] R. Fisher. The moments of the distribution for normal samples of measures of departures from normality. Proc. Royal Soc. A, 130:16–28, 1930. [57] E. Pearson. A further development of tests for normality. Biometrika, 22:239– 249, 1930.

225

[58] S. S. Shapiro and M. B. Wilk. An analysis of variance test for normality (complete samples). Biometrika, 52:591–611, 1965. [59] S. S. Shapiro, M. B. Wilk, and H. J. Chen. A comparative study of various tests for normality. Journal of the American Statistical Association, 63:1343–1372, 1968. [60] E. Seier. Comparison of tests for univariate normality. Interstat Stat. Journal, 2002. [61] F. F. Gan and K. J. Koehler. Goodness of Fit Tests Based on P-P Probability Plots. Technometrics, 32:289–303, 1990. [62] S. S. Shapiro and R. S. Francia. Approximate analysis of variance test for normality. J. Am. Statist. Assoc., 67:215–225, 1972. [63] L. Chen and S. S. Shapiro. An Alternative test for normality based on normalized spacings. J. of Stat. Comput. and Sim., 53:269–287, 1995. [64] C. Jarque and A. Bera. Efficient tests for nonlinearity, heteroscedasticity, and serial independence of regression residuals. Economics Letters, 6:255–259, 1980. [65] A. Bera and C. Jarque. A test for normality of observations and regression residuals. Int’l. Stat. Rev., 55:163–172, 1987. [66] P. N. Mohammed, J. Knuble, and J. Piepmeier. Pulse and kurtosis detection of radio-frequency interference (RFI): an experimental comparison. In IEEE Geoscience and Remote Sensing Symposium, 2008. [67] J. P. Royston. Approximating the Shapiro-Wilk W-test for non-normality. Statistics and Computing, 2:117–119, 1992. [68] G. Blom. Statistical estimates and transformed Beta-variables. John Wiley and Sons, New York, 1958. [69] H. L. Harter. Expected values of normal order statistics. Biometrika, 48:151–165, 1961. [70] J. Harkins, T. El-Ghazawi, E. El-Araby, and M. Huang. Performance and analysis of sorting algorithms on the SRC 6 reconfigurable computer. IEEE Field Programmable Technology Con. Proc., 48:295–296, 2005. [71] J. P. Royston. Correcting the Shapiro-Wilk W for ties. Journal of Computation and Simulation, 31:237–249, 1989. [72] J. P. Royston. An extension of Shapiro and Wilk’s W test for normality to large samples. Applied Statistics, 31:115–124, 1982. 226

[73] J. P. Royston. A toolkit for testing for non-normality in complete and censored samples. The Statistician, 42:37–43, 1993. [74] J. T. Johnson and L. C. Potter. Performance study of algorithms for detecting pulsed sinusoidal interference in microwave radiometry. IEEE Transactions on Geoscience and Remote Sensing, 47(2):628–636, February 2009. [75] S. M. Kay. Fundamentals of Statistical Signal Processing: Volume II, Detection Theory. Prentice Hall, Upper Saddle Creek, NJ, 1998. [76] B. G¨ uner and J. T. Johnson. Performance study of a cross-frequency detection algorithm for pulsed sinusoidal RFI in microwave radiometry. submitted to IEEE Transactions on Geoscience and Remote Sensing, 2009. [77] M. G. Kendall and A. Stuart. The advanced theory of statistics. Hafner, New York, 1969.

227