[OpenMadrigal-developers] Re: suggest a new isprint flag

[Shunrong Zhang]

Bill,

I'm struggling with the madrigal global search results.

John and I think the global search very useful for creating models. I am
working on the high-latitude convection model. As John suggested, I'll
directly read the global search result, instead to scan invidual database
files.

There are some issues relevant to the output format from the search.  It
returns in an email a link to the result. This report file very often is
very large (>200 Mb, in my test run for 1980), and is in html. Due to its
size, I decide to fetch it by going directly to the httpd directory and
copying it to my local directory. Mixted with flat data, this file
contains also html scripts and header info (e.g., Instruments, Kinds of
Data, Experiment Name, Parameters, etc) of each experiment (which I
mentioned in a prior message cited below). However, for this purpose of
modeling work, we always prefer a plain data file.  If this report file
were small, we could edit it to keep only the data part.

To provide a flat data file that is friendly to further analysis, maybe
there are at least 2 options. In the first option, modify isprint to
include a flag so that no info other than plain data is generated; the
search report in plain text file rather than html. The second option is,
provide a converter that scans the report file generated from the current
version of global search and remove html script and other unnecessary
parts to return a flat data file. 

In short, I think the plain text format is more useful for systematical
analyses.

Shunrong


On Thu, 18 Sep 2003, Shunrong Zhang wrote:

> 
> Hi,
> 
> The current madrigal isprint lists data with a header like 
> 
> Filters used:
> Filter 1:
>         GDALT
>         Range 1: Lower = 90, upper = 610
>      DAYNO         UT      GDALT      GDLAT       GLON        NEL
> 
> even though we set header off. This is useful especially for single
> experiment, but may become annoying for, such as global search. For global
> search, this info should occur at most once and not in between two
> experiments. Manually editing of the output to remove that extra info
> sometimes is not easy when the outputs are very large as a result of
> global search (e.g., the A/C data for the whole 30-day run are about 200
> Mb). So I suggest a new isprint flag that allows for controlling that part
> of data table head.
> 
> Shunrong
> 
>