[OpenMadrigal-developers] Header and Catalog records for new Millstone ISR data
William Rideout
brideout at haystack.mit.edu
Wed Jul 7 17:10:52 EDT 2004
Before we send any of our recent data to the Cedar database, we need to add
header and catalog records to our data. As Larisa pointed out at the integrated
data workshop at Cedar, this metadata can be very important to users, so I'd
like to make an effort to get as much information in as possible.
Previously these headers were generated by filing in templates, with (I assume)
the ability to add experiment specific comments. Attached below is an example
header and catalog template for a standard experiment. Please let me know if
any parts of the template are out-of-date. Also, please let me know if an
different template is appropriate for other types of data.
Since our experiments tend to run in certain modes (extra wide coverage, local
10 position, etc), I'd like to get concise descriptions of these modes and
incorporate these descriptions.
Also important to header and catalog records are descriptions of the
non-standard parameters measured. The Millstone-specific parameters in the new
files that I can't find described in the old headers are:
3351 Millstone Hill radar operatn par 52
3352 Millstone Hill radar operatn par 53
3353 Millstone Hill radar operatn par 54
3321 Ti, Tr correlation coefficient
3322 Ti, Ph correlation coefficient
3323 Ti, Co correlation coefficient
3324 Tr, Ph correlation coefficient
So if someone could write a one paragraph description of these, I'd appreciate it.
*****************CATALOG template********************
KRECC 2001 Catalogue Record, Version 1
KINSTE 30 Millstone Hill - MISA Steerable/ Zenith Fixed Antennas
MODEXP 0 Millstone Hill Incoherent Scatter Radar Data
CMODEXP More details are available from: http://www.haystack.edu/
C
TIMCY 5 minutes
ALT1 93 km. Lowest altitude measured
ALT2 1171 km. Highest altitude measured
GGLAT1 42 degrees. Lowest geographic latitude measured
GGLAT2 43 degrees. Highest geographic latitude measured
GGLON1 289 degrees. Westmost geographic longitude measured
GGLON2 289 degrees. Eastmost geographic longitude measured
PL1 40 Shortest radar pulse length
PL2 2000 Longest radar pulse length
IBYRE 2000 Beginning year
IBDTE 105 Beginning month and day
IBHME 1315 Beginning UT hour and minute
IBCSE 1000 Beginning centisecond
IEYRE 2000 Ending year
IEDTE 105 Ending month and day
IEHME 2041 Ending UT hour and minute
IECSE 1600 Ending centisecond
C
CPURP Incoherent scatter observations of the ionosphere
C
CIREM See: http://www.haystack.edu/
CSREM See: http://www.haystack.edu/
C
CPI J. Holt
CPREPDAT Wed Apr 24 14:36:43 EDT 2002
*****************HEADER template********************
KRECH 3002 Header Record, Version 2
KINST 3 30 Millstone Hill - MISA Steerable / Zenith Fixed Antennas
KINDAT 4 3408 Basic Parameter Set via INSCAL Version 8.1
C
CKINDAT STANDARD MILLSTONE HILL PROCESSING METHOD USING INSCAL.
CKINDAT
CKINDAT INSCAL analyzes incoherent scatter autocorrelation functions (acfs)
CKINDAT to determine ionospheric plasma parameters.
CKINDAT
CKINDAT The acfs are formed from the measured lag-products using a trapezoidal
CKINDAT summation rule. A multidimensional non-linear least squares fit to each
CKINDAT acf is then performed to compute estimates of the plasma parameters.
CKINDAT Parameter error bars are computed by assuming that chi square is 1.0.
CKINDAT Analysis parameters for this experiment are summarized below. More
CKINDAT details, including the actual INSCAL input parameters, output listing
CKINDAT and error messages are avaialable from http://www.haystack.edu.
CKINDAT
CKINDAT 1. The search parameters for 90-130 [km] were:
CKINDAT
CKINDAT Ion Temperature, ACF Normalization Factor, Collision Frequency
CKINDAT and Ion Drift Velocity
CKINDAT
CKINDAT Assumed or model parameters for 90-130 [km]:
CKINDAT
CKINDAT Temperature Ratio = 1.0
CKINDAT
CKINDAT Density (Before temperature correction) =
CKINDAT Calfac(i) * [S/N](i) * Systmp * Range(i)**2 / Xmtr_power
CKINDAT where Calfac = radar calibration factor
CKINDAT S/N = signal to noise ratio
CKINDAT Systmp = system temperature (K)
CKINDAT Range = range (km)
CKINDAT Xmtr_power = transmitter peak power (MW)
CKINDAT i = range index
CKINDAT
CKINDAT n(H+)/Ne = 0.0
CKINDAT
CKINDAT n(mass 31)/Ne: if Altitude < 120 km then = 1.0
CKINDAT else = 1. - 2./(1. + SQRT(1.+8.*EXP(ZZ2)))
CKINDAT where:
CKINDAT ZZ2 = MIN(-(Altitude-180.)/H, 50.)
CKINDAT H = 10. - 6.*EXP(ZZ1)
CKINDAT ZZ1 = MIN(-(Altitude-120.)/40., 50.)
CKINDAT
CKINDAT The measurements were not sensitive to the H+ Drift velocity
CKINDAT
CKINDAT 2. The search parameters for 130-400 [km] were:
CKINDAT
CKINDAT Ion Temperature, ACF Normalization Factor, Temperature Ratio,
CKINDAT and Ion Drift Velocity
CKINDAT
CKINDAT Assumed or model parameters for 130-400 [km]:
CKINDAT
CKINDAT Collision Frequency = 0.0
CKINDAT
CKINDAT Density (Before temperature correction) =
CKINDAT Calfac(i) * [S/N](i) * Systmp * Range(i)**2 / Xmtr_power
CKINDAT where Calfac = radar calibration factor
CKINDAT S/N = signal to noise ratio
CKINDAT Systmp = system temperature (K)
CKINDAT Range = range (km)
CKINDAT Xmtr_power = transmitter peak power (MW)
CKINDAT i = range index
CKINDAT
CKINDAT n(H+)/Ne = 0.0
CKINDAT n(mass 31)/Ne: if Altitude < 120 km then = 1.0
CKINDAT else = 1. - 2./(1. + SQRT(1.+8.*EXP(ZZ2)))
CKINDAT where:
CKINDAT ZZ2 = MIN(-(Altitude-180.)/H, 50.)
CKINDAT H = 10. - 6.*EXP(ZZ1)
CKINDAT ZZ1 = MIN(-(Altitude-120.)/40., 50.)
CKINDAT
CKINDAT The measurements were not sensitive to the H+ Drift velocity
CKINDAT
CKINDAT The search parameters for 400-1200 [km] were:
CKINDAT
CKINDAT Ion Temperature, Temperature Ratio, n(H+)/Ne, ACF Normalization
CKINDAT Factor, and Ion Drift Velocity
CKINDAT
CKINDAT Assumed or model parameters 400-1200 [km]:
CKINDAT
CKINDAT Collision Frequency = 0.0
CKINDAT
CKINDAT Density (Before temperature correction) =
CKINDAT Calfac(i) * [S/N](i) * Systmp * Range(i)**2 / Xmtr_power
CKINDAT where Calfac = radar calibration factor
CKINDAT S/N = signal to noise ratio
CKINDAT Systmp = system temperature (K)
CKINDAT Range = range (km)
CKINDAT Xmtr_power = transmitter peak power (MW)
CKINDAT i = range index
CKINDAT
CKINDAT n(mass 31)/Ne = 1. - 2./(1. + SQRT(1.+8.*EXP(ZZ2)))
CKINDAT where:
CKINDAT ZZ2 = MIN(-(Altitude-180.)/H, 50.)
CKINDAT H = 10. - 6.*EXP(ZZ1)
CKINDAT ZZ1 = MIN(-(Altitude-120.)/40., 50.)
CKINDAT
CKINDAT The measurements were not sensitive to the H+ Drift velocity
CKINDAT
CKINDAT Chirp correction:
CKINDAT
CKINDAT A chirp correction has been applied to the line of sight velocities to
CKINDAT compensate for a frequency offset produced in the UHF transmitter.
CKINDAT This is calculated from the applied dc voltage and current in the
CKINDAT transmitter klystron which are measured continuously, and from the
CKINDAT known pulse length, interpulse period, and pulse rise and fall times.
CKINDAT For a given pulse length this chirp correction normally varies little
CKINDAT over the course of an experiment, and in practice a single chirp
CKINDAT correction is usually applied for each pulse length for the whole
CKINDAT experiment. Typical values of the chirp correction vary from 10-20
CKINDAT m/s.
CKINDAT
CKINDAT Density Calibration:
CKINDAT
CKINDAT The densities have been calibrated using the Umass Lowell Digisonde.
CKINDAT Millstone Hill Incoherent Scatter electron densities are calculated by
CKINDAT inserting into the radar equation a calibration factor relating radar
CKINDAT signal temperature to electron density. This calibration constant is
CKINDAT determined by direct comparison of high elevation measurements of
CKINDAT signal temperature from the F-region peak with local digisonde
CKINDAT measurements of peak electron density.
C
CHIST See http://www.haystack.edu/
CHIST A complete history of the analysis of this experiment is maintained
CHIST at the above Web site. In some cases, an updated analysis of this
CHIST experiment may be found there.
C
IBYRT 2000 Beginning year
IBDTT 105 Beginning month and day
IBHMT 1315 Beginning UT hour and minute
IBCST 1000 Beginning centisecond
IEYRT 2000 Ending year
IEDTT 105 Ending month and day
IEHMT 2041 Ending UT hour and minute
IECST 1600 Ending centisecond
LPROL 13 16 Length of prologue in data records
JPAR 14 11 Number of single-valued parameters
MPAR 15 24 Number of multiple-values parameters
NROW 16 17 Number of entries for multiple valued parameter
C 1D Parameters:
KODS(1) 17 132 Beginning azimuth (0=geog N,90=east) 1.0e-02 deg
KODS(2) 18 133 Ending azimuth (0=geog N,90=east) 1.0e-02 deg
KODS(3) 19 142 Beginning elevation angle 1.0e-02 deg
KODS(4) 20 143 Ending elevation angle 1.0e-02 deg
KODS(5) 21 402 Pulse length 1.0e-06 sec
KODS(6) 22 482 System temperature 1.0e+00 K
KODS(7) 23 483 Additional increment to system temp 1.0e-04 K
KODS(8) 24 486 Peak power 1.0e+00 kW
KODS(9) 25 490 Transmitted frequency 1.0e+05 Hz
KODS(10) 26 3318 D.P. Power Normalization constant 1.0e-03 N/A
KODS(11) 27 3319 Additional increment to D.P. Power NrmK 1.0e-07 N/A
C 2D Parameters:
KODM(1) 39 -3350 Error in Line of sight Doppler Vlos (po 1.0e+00 m/s
KODM(2) 40 -3313 Error in ACF Normalization Factor 1.0e-03 N/A
KODM(3) 41 -710 Error in Ion-neutral collision frequenc 1.0e+00 s-1
KODM(4) 42 -690 Error in Comp - (ions with mol wt 28 to 1.0e-03 N/A
KODM(5) 43 -660 Error in Composition - [H+]/Ne 1.0e-03 N/A
KODM(6) 44 -580 Error in Line of sight ion velocity (po 1.0e+00 m/s
KODM(7) 45 -570 Error in Temperature ratio (Te/Ti) 1.0e-03 N/A
KODM(8) 46 -550 Error in Ion temperature (Ti) 1.0e+00 K
KODM(9) 47 -505 Error in Log10(uncorrected electron den 1.0e-03 lg(m-3)
KODM(10) 48 120 Range 1.0e+00 km
KODM(11) 49 121 Additional increment to range 1.0e-01 m
KODM(12) 50 411 Signal to noise ratio 1.0e-03 N/A
KODM(13) 51 420 Reduced-chi square of fit 1.0e-03 N/A
KODM(14) 52 430 Goodness of fit 1.0e+00 N/A
KODM(15) 53 461 Millstone Hill data quality code 1 1.0e+00 N/A
KODM(16) 54 505 Log10(uncorrected electron density) 1.0e-03 lg(m-3)
KODM(17) 55 550 Ion temperature (Ti) 1.0e+00 K
KODM(18) 56 570 Temperature ratio (Te/Ti) 1.0e-03 N/A
KODM(19) 57 580 Line of sight ion velocity (pos = away) 1.0e+00 m/s
KODM(20) 58 660 Composition - [H+]/Ne 1.0e-03 N/A
KODM(21) 59 690 Comp - (ions with mol wt 28 to 32)/Ne 1.0e-03 N/A
KODM(22) 60 710 Ion-neutral collision frequency 1.0e+00 s-1
KODM(23) 61 3313 ACF Normalization Factor 1.0e-03 N/A
KODM(24) 62 3350 Line of sight Doppler Vlos (pos = away) 1.0e+00 m/s
C
C 420 Reduced-chi square of fit
C If all statistical assumptions are correct, the expectation value of
C chi-square is one. In fact, it tends to be much smaller when the
C signal-to-noise ratio is large. This is probably due to the large
C correlations between the lag products in this case, which are not
C taken into account in the fit.
C
C 430 Goodness of fit
C This is 1000 times the root mean square deviation of the fit from
C the the measured autocorrelation function (ACF). Since the ACF is
C normalized to 1.0, values of ~1000 indicate ~100% deviations, values
C of ~100 indicate ~10% deviations and values ~10 indicate ~1%
C deviations.
C
C 461 Millstone Hill data quality code 1
C The data quality parameter is generated by an algorithm which
C attempts to detect the presence of either a satellite echo or
C radio frequency interference (RFI), resulting in a 0 for clean
C spectra and a 1 for contaminated spectra. The algorithm's
C thresholds are deliberately set to avoid false detections on
C genuine incoherent scatter signals, and therefore data
C contaminated by weaker satellite or RFI signals will not always
C be flagged. In particular, the algorithm will miss
C satellites/RFI at altitudes with significant heavy ion (mass >
C O+) fractions, or for altitudes with temperatures < 300 K. This
C parameter should not be used as the sole data quality flag.
C
C -505 Uncertainty in Log10(uncorrected electron density)
C This is computed from the statistical uncertainty of the fit ACF at
C zero lag. In conformity with the CEDAR standard, it is the logarithm
C of the uncertainty, not the uncertainty of the logarithm. If the fit
C fails, the density itself is still stored in the data record, and
C the uncertainty is missing. This statistical uncertainty is
C normally much smaller than the larger uncertainty in the density
C calibration, which is ~20%.
C
C 3350 Line of sight Doppler Vlos (pos = away)
C This is a separate non-linear least squares calculation of the
C Doppler velocity, which, unlike parameter 580, does not assume an
C incoherent scatter spectrum. This may be particularly useful in
C coherent echo studies.
C
CAREM Analysis included a correction for possible spectral asymmetry
CANALYST J. M. Holt
CANDATE Mon Jan 10 10:50:52 2000
Bill
--
Bill Rideout
MIT Haystack Observatory
Email: brideout at haystack.mit.edu
Phone: 781 981-5624
More information about the OpenMadrigal-developers
mailing list