[OpenMadrigal-developers] Re: Modifications to fof2 loading program

[William Rideout]

The new versions of these two loading programs are in CVS and on 
hyperion at /opt/madrigal/bin.  To see usage, just type:

./updateFof2 (old format)

or

./updateFof2_2 (new format)

The prgrams also have some extra error checking:

1. If you try to update the wrong Madrigal file (that is, you use 
updateFof2 to update the new format, or visa-versa), it will print an 
error and abort.

2. If the input file contains dates that are out of order, it will print 
an error and abort.

The program updateFof2_2 is quite slow (~45 minutes) since its updating 
nearly 100K records, and written in a scripting language. It does print 
"% done" messages to reassure the user it still running.  It could be 
faster in C, but unless Glenn complains (and/or gives me a bribe), I'll 
leave it in tcl.

Glenn - let me know if there are any problems.

Bill


William Rideout wrote:
> As requested by Glen,  I'm planning to make changes to the fof2 loading 
> programs.  As a review, there are at present two fof2 loading programs, 
> that take a text file Glen creates and adds that new fof2 data to the 
> Madrigal fof2 file.  One supports the old, non-standard madrigal fof2 
> file, which contains time parameters in the data itself, and the other 
> supports the present fof2 file.  The old fof2 file is maintained only 
> because its format is hardcoded into the Fortran madlib, and Larisa 
> still uses that library in her fof2 analysis.
> 
> With the present loading programs, all added data must be after the last 
> existing data in the Madrigal file.  This means that the only way to 
> insert data in-between existing data is to go back to an old version of 
> the fof2 file, and reload all the data after the last record in the old 
> file.
> 
> With the new version, data can be added at any point in the fof2 file. 
> If the data already exists in the madrigal fof2, it will be overwritten 
> with new data from the text file.
> 
> Note that in the new version the entire day will be overwritten, so to 
> change data for a given day, the entire day worth of new data must be in 
> the text file.  That means to change one fof2 value, you must put all 48 
> values in the test file.  Also, the new data in the text file must be 
> chronological (okay, this last requirement is to make it easier to code, 
> but I think its not unreasonable).  Out of order data in the input text 
> file will cause an exception.
> 
> Please let me know if you have any comments.
> 
> Bill
> 


-- 
Bill Rideout
MIT Haystack Observatory
Email: brideout at haystack.mit.edu
Phone: 781 981-5624