Volume Software Interface Specification (SIS) for Cassini Radar Instrument Team Data Products
California Institute of Technology
Table of Contents
Document Log .............................................................................................................. ii
1 INTRODUCTION ..................................................................................................... 1
1.1 Purpose and Scope ................................................................................. 1
1.2 Applicable Documents ............................................................................. 1
2 ARCHIVE VOLUME GENERATION ....................................................................... 2
2.1 Data Transfer and Validation Method ................................................... 2
2.2 Data Volume Estimates ............................................................................ 2
2.3 Interface Media Characteristics ............................................................. 3
2.4 Backup and Duplicates ........................................................................... 3
3 ARCHIVE VOLUME CONTENTS ........................................................................... 4
3.1 Root Directory Contents ......................................................................... 4
3.2 INDEX Directory Contents ...................................................................... 5
3.3 DOCUMENT Directory Contents ........................................................... 6
3.4 CATALOG Directory Contents ............................................................... 7
3.5 DATA Directory Contents ........................................................................ 7
3.6 CALIB Directory Contents .................................................................... 10
3.7 EXTRAS Directory Contents ................................................................ 10
4 ARCHIVE VOLUME FORMAT .............................................................................. 12
4.1 Document File Format ........................................................................... 12
4.2 Tabular File Format ................................................................................ 12
4.3 PDS Label Format .................................................................................. 13
4.4 Catalog File Format ................................................................................ 14
4.5 Science Data File Format ...................................................................... 14
This Software Interface Specification (SIS) describes the formats, contents, and generation of the archive volumes for data produced by the Cassini Radar Instrument Team including the Burst Ordered Data Products (BODP) and the Basic Image Data Records (BIDR). Observations in the Saturn system will target Titan, Saturn, the Saturnian ring systems, and various icy satellites.
Observations of targets outside the Saturn system will also be included in the data volume. For example, radiometer observations of Jupiter, the sun, and various microwave sources will be included as well as radiometer and scatterometer observations of the Earth.
The specifications in this document apply to all data products produced by the Cassini Radar Instrument Team at the Jet Propulsion Laboratory.
This SIS is intended to provide enough information to enable users to understand the way in which the data products are organized within the volume data set. It is intended to be used in conjunction with the Cassini Radar Burst Ordered Data Products SIS D-27891 that describes the internal format of the BODP data files and the Cassini Basic Image Data Record SIS D-27889, which similarly documents the BIDR files. The users for whom this SIS is intended are software developers, engineers, and scientists interested in accessing and using either of the two data product types.
This SIS is responsive to the following Cassini Mission Documents:
1. Project Data Management Plan, JPL PD 699-061
2. Cassini/Huygens Program Archive Plan for Science Data, JPL PD 699-068
This SIS is also consistent with the following Planetary Data System documents:
3. Planetary Data System Data Preparation Workbook, February 1, 1995, Version 3.1, JPL D-7669, Part 1
4. Planetary Data System Data Standards Reference, June 15, 2001, Version 3.4, JPL D-7669, Part 2
Finally, this SIS is consistent with the contract negotiated between the Cassini Project and the Cassini Radar Team Leader (TL) in which data products and documentation are explicitly defined as deliverable products.
The Cassini Radar Team will produce the BODP and BIDR Data Archive Collection. BODP files will be produced for each observation of each body. BIDR files will be produced for each SAR observation of Titan. A data volume product will be created for each observation of each body during the tour of the Saturnian system (from June 1, 2004 to the end of the mission) plus relevant observations made during cruise (before June 1, 2004). During active mission operations intermediate science data products will be made available to Cassini Radar Team members for their analysis and evaluation as they are produced. The data products will be updated several times based on improved pointing and ephemeris information. At the end of the data preparation and evaluation phase, a data volume as specified in this SIS will be created and sent to the PDS for archiving. Before production of the first data volume product, a sample volume will be distributed to the peer review panel for their evaluation. The peer panel, with representation from the Cassini Radar Team, PDS, Cassini Science Archive Working Group (SAWG) and the general science community will review and validate the products and volume for scientific integrity and compliance with PDS standards. The data products will be evaluated against the design as defined in the product and volume SIS's. The software and procedure used to create the data volume will be configuration controlled to insure that future products continue to conform to all specifications.
The following table summarizes product sizes for the Cassini Radar Instrument Team Data Products. These include the BIDR files and the four different formats of BODP files. The various types are described in detail in section 3.5 (DATA Directory Contents) and in the BODP Data SIS. BIDR files and BODP files of each format have fixed record lengths that can be read from the PDS label attached at the beginning of the file or from the file's detached label where applicable. The approximate record sizes for each file type are also listed in Table 1 along with the nominal data volume per pass (for 10 hour Titan passes) and some notes briefly describing each product type. For the exact record sizes see the BODP or BIDR data SIS. The total data volume for the mission is estimated to be 160 GBytes for 20 Titan passes plus a much smaller amount from Saturn, rings, and icy satellite observations. This estimate is probably high because the radar will not be obtaining measurements for the full ten hours for every Titan pass. On the other hand, in the event of an extended mission we may acquire more passes.
A single copy of the data product volumes will be delivered to the PDS Imaging Node. The Imaging Node will be responsible for developing the methods for distributing the data products to the user community. Copies of the preliminary (sample) volumes will be provided electronically to the peer review panel for their evaluation.
Table 1: Data Volume Estimates
Data products will be delivered to the PDS electronically. Deliveries will be tracked using the Cassini Archive Tracking System (CATS).
The Cassini Radar Instrument Team will keep backup copies of the data throughout the duration of the Cassini Mission.
This section describes the directories and contents of the Cassini Radar Instrument Team Data Product volumes, including the file names, file contents, and file types. The ancillary files described herein appear on each volume in the Cassini Radar Instrument Team Data Product volume series.
The following table shows the contents of the Root directory.
Table 2: Contents of Root Directory
The following table summarizes files located in the INDEX directory. A single index table is provided for all Cassini Radar Instrument Team Products. Data set ID is one of the fields in the index table.
Table 3: Contents of INDEX Directory
Index table for Cassini Radar Instrument Team Data Products. |
|
This section describes the fields in INDEX.TAB. This file is included to provide the user with a way of quickly producing a list of data files from desired ranges of search criteria such as time, target, latitude, or longitude. See below for descriptions of each column in the index table.
The name of each BODP or BIDR data file in the data volume.
The directory in which the file can be found. This entry should always be `data/XXXX' where XXXX is the four-letter acronym for the data set (SBDR, LBDR, ABDR or BIDR).
A text string that identifies the type of data in the file. The valid strings are listed in Table 1.
UTC (YYYY-DOYTHH:MM:SS.sss) formatted time at start of data acquisition for data in the file.
UTC formatted time at end of data acquisition for data in the file.
Identifier for target of observation (TITAN, SATURN, etc.).
Minimum latitude in target body frame for data in file. For targets for which this is not applicable the default value of -1000 is used.
Maximum latitude in target body frame for data in file. For targets for which this is not applicable the default value of -1000 is used.
Westernmost longitude in target body frame for data in file. For target for which this is not applicable, the default value of -1000 is used. Valid values are between 0 and 360.
Easternmost longitude in target body frame for data in file. For target for which this is not applicable the default value of -1000 is used. Valid values are between 0 and 360.
5 letter ASCII string ("RIGHT", "LEFT", or "BOTH") indicating the side of the spacecraft groundtrack to which the antenna is pointed for data acquired within each data product (file). The "BOTH" option will typically be used for BODP files. The SAR images stored in the BIDR files are always acquired on only one side of the ground track for each Titan pass. This value also indicates from which side the SAR image is illuminated. If the spacecraft images to the left of its ground track, the image will be illuminated from the (viewer's) left side, and, conversely, if the spacecraft looks to the right, the illumination will come from the right in the image file. The direction of illumination is critical to interpretation of features in the image. For unusual SAR passes in which measurements are obtained on both sides of the ground track, BOTH will be used. The transition across nadir and resultant data gap should make such passes easily identifiable.
UTC (YYYY-DOYTHH:MM:SS.sss) formatted time of creation of the product file.
11 character string that uniquely identifies the archive volume. The format is CORADR_XXXX where XXXX is a four-digit number unique across all Cassini Radar archive volumes.
The following table lists files located in the DOCUMENT Directory.
Table 4: Contents of DOCUMENT Directory
PDF and HTML versions of Burst Ordered Data Products SIS and detached label file documenting them. |
|
PDF and HTML versions of Basic Image Data Record SIS and detached label file documenting them. |
|
The files in the CATALOG directory provide a top-level understanding of the Cassini Mission, spacecraft, instruments, and data sets in the form of completed PDS templates. The files are produced or collected by the Cassini Radar Team. The files in this directory are coordinated with the PDS engineer. The following table lists files in the CATALOG Directory.
Table 5: Contents of CATALOG Directory
The DATA directory has five sub-directories SBDR, LBDR, ABDR, ASUM, and BIDR, which together contain all the data products for the volume. Each directory contains:
· The data files for the corresponding type of product. ABDR, LBDR, and BIDR files are compressed with the "zip" utility so that an entire volume will fit on one 4.7 Gbyte DVD. Although we plan to submit our volumes electronically, PDS requires that each volume fit on a single media.
· Detached labels for each of the compressed ABDR, LBDR, and BIDR files and for each of the ABDR-SUMMARY products. Each of the compressed ABDR, LBDR, and BIDR file labels duplicates the information in the label attached to the uncompressed file. In this manner, a user can determine whether or not they need a particular data file without unzipping it. The detached label also contains all the information necessary to acquire the necessary zip and unzip software and has a keyword REQUIRED_STORAGE_BYTES that specifies the size in bytes of the uncompressed file.
· For the SBDR, LBDR, and ABDR directories, a format file that is referenced by the label files (i.e., SBDR/SBDR.FMT).
Files are named using the product ID and an extension specifying the PDS file type. The product ID is the string following the PRODUCT_ID keyword in the attached PDS label. Uncompressed BIDR files are of the format aabcdeefggg_Dhhh_Tiii_Vnn.IMG, where "aabcdeefggg_Dhhh_Tiii_Vnn" is the product ID and ".IMG" is the extension for PDS image files. Similarly the SBDR, uncompressed LBDR, and uncompressed ABDR files are named xxxx_yy_Dzzz_Vnn.TAB, where "xxxx_yy_Dzzz_Vnn" is the product ID and ".TAB" is the extension for a PDS Table file. The ABDR-SUMMARY files are named ABDR_SUMMARY_yy_Dzzz_Vnn.CSV, where "ABDR_SUMMARY_yy_Dzzz_Vnn" is the product ID and ".CSV" is the extension for a PDS Comma-Separated Value file. All compressed files have the extension ".ZIP".
The BIDR product ID aabcdeefggg_Dhhh_TiiiSjj_Vnn is defined as follows:
F = Primary dataset (e.g., incidence angle corrected sigma0) in 32-bit floating-point format (Linear scale values, not in dB.)
B = Primary dataset in unsigned byte format. (Values converted to dB then normalized to fit in [0, 255] range.)
S = Sigma0 with noise subtraction but without incidence angle correction in 32-bit floating point format. (Linear scale values, not in dB.)
U = Sigma0 without noise subtraction or incidence angle correction in 32-bit floating point format. (Linear scale values, not in dB.)
E = Incidence angle map, floating point values in degrees.
T = Latitude map, floating point values in degrees.
N = Longitude map, floating point values in degrees.
M = Beam mask map, 8 bit values.
L = Number of looks map, 32 bit integer values.
Q= Oblique Cylindrical (this is the only projection used for BIDRs)
B = 2 pixels/degree
ee = Absolute value of latitude at center of file, rounded to nearest degree
f = Hemisphere of center of file
ggg = West longitude at center of file, rounded to nearest degree
hhh = Three digit mission unique Cassini radar data take number from which data are included.
iii = Three digit Titan flyby identifier, e.g., for flyby TA iii=00A and for T7, iii=007, etc.
jj = Two digit segment number when there are several pieces of a swath for the same flyby. Segment 01 is usually the main imaging segment. Other segments include turns from/to altimetry to/from SAR, and high altitude imaging (HiSAR; of lower resolution).
The BODP product ID xxxx_yy_Dzzz_[Pm_]Vnn is defined as follows:
xxxx = The acronym for the data set LBDR, SBDR, or ABDR, or ABDR_SUMMARY for the ABDR-SUMMARY data set
yy = The radar mode of the data in the file represented as a two digit decimal integer between 00 and 15. This value represents a 4-bit binary flagging scheme
Bit 0 (LSB) = 1 means radiometer only mode data is present in the file.
Bit 1 = 1 means scatterometer mode data is present in the file
Bit 2 = 1 means altimeter mode data is present in the file.
Bit 3 = 1 means SAR mode data is present in the file.
It is important to note that scatterometer measurements are obtained in all modes except radiometer only, not just in scatterometer mode. Similarly, radiometer measurements are obtained in all modes without exception.
zzz = 3 digit radar observation counter (data take number, DTN). One observation corresponds to a single up-linked radar command sequence. For example, a Titan fly-by is one observation.
[Pm_] = an optional piece of the LBDR (only) identifier where m = 1 or 2 for the two parts of an LBDR that exceeds the 2 GB file size limit. If the LBDR is less than 2 GB so that only 1 file is needed, this part of the identifier is omitted.
nn = 2 digit version number
The CALIB directory contains two subdirectories BEAMPAT and CONFIG. These directories contain ancillary information that specifies how the Cassini Radar data was calibrated and processed. The following table lists files located in the CALIB Directory.
Table 6: Contents of CALIB Directory
Directory containing antenna gain pattern files for each beam and documentation for those files. |
|
Directory containing processor configuration files and documentation for those files. |
The BEAMPAT directory contains estimates of the antenna gain pattern for each of the five Cassini Radar beams. The beam pattern file name convention is BEAMX_VNN.PAT, where X is a beam number between 1 and 5 and NN is a two-digit version number.
The CONFIG directory contains a processor configuration file for each Cassini Radar observation. This file contains keyword and value pairs (Processor keywords are not PDS keywords.) which were used in running the ground processing software used to generate the BODP and BIDR files for that observation. Each processor configuration file is named "PCF_zzzz_Vnn.CFG" where zzzz is the four-digit radar observation counter and nn is a two-digit version number.
Files in the BEAMPAT and CONFIG directories will be not be needed by most users of Cassini Radar Instrument Team data. For those who desire to use them label files BEAMX_VNN.LBL and PCF_zzzz_Vnn.LBL are available as documentation in the appropriate directories.
This directory contains files that do not belong anywhere else on the volume but are included to provide extra and perhaps convenient information. The contents of the directory may vary from volume to volume. It contains at least one file , EXTRINFO.TXT, that describes the contents of the directory. Usually it also contains a memo describing the command sequence design employed for the observation and a Cassini Radar Transition (CRT) file. Both of these files document the times of radar mode transitions.
The memo is in Portable Document Format (PDF). The CRT file is an ASCII table suitable as an input for custom analysis software. The information provided by these files is also available in the SBDR, ABDR, and LBDR files themselves on a burst by burst (record by record) basis.
CRT file names are of the form CRT_zzz_Vnn.TAB where zzz is the three-digit radar observation counter, and nn is a two-digit version number. A small text file (TITAN_FLYBY_ID_TABLE.TXT) that relates the observation counter to the common Titan pass designations (T##) is included in volumes with SAR observations of Titan.
The CRT files consist of four tab-delimited columns of ASCII text. The first column is a UTC time tag in yyyy-doyThh:mm:ss.sss format. The second column is a two digit decimal integer denoting the type of transition. The third column is a 15-character (spaces are padded at the end) ASCII transition type tag. The fourth column is a 10-character unique record identification number. Rows are in temporal order.
The following table maps transition type integers to tags. The tags denote the beginning and ending times of raster scans (scatterometer and radiometer modes only) as well as the ending and beginning times for each radar mode. For completeness, the end time of one mode and the beginning time of the next mode are both listed. Unless there is a gap between the two modes both times will be identical and will correspond to the starting time of the first burst in the second mode. Recall that each record in a BODP data file contains a UTC time tag. For a given mode or scan the UTC time tags of the bursts within that mode or scan are all greater than or equal (>=) the starting time and less than or equal (<=) the ending time in the CRT file. All bursts outside of the range are not included in that scan or mode. Record identification number on the other hand will be different for the two rows. The mode ending row will have the last record identification number for the old mode. The mode staring row will contain the first record identification number for the new mode. It is important to note that scatterometer data (i.e., normalized radar backscatter cross section) will be available during altimetry and SAR modes and that radiometer data will be available during all modes.
Table 7: Transition Number Map for CRT Files
Geometrical data such as spacecraft position, velocity and attitude, are reported in the inertial target-centered J2000 coordinate frame as well as in a target body fixed (TBF) rotating frame. Measurement locations are reported in the TBF frame as planetodetic surface coordinates (latitude and longitude). The PDS Navigation and Ancillary Information Facility (NAIF) definitions are used for the frames. The TBF frame for each target is defined in the NAIF planetary kernel file (PCK) for the Saturnian system. This file will be updated during tour as Cassini observations improve knowledge of the states of bodies in the Saturnian system. In particular, the spin state of Titan has been updated based on an initial analysis of the Radar images (B.W. Stiles et al, 2008, Ref. 16). In order to fully document the coordinates used, the PCK file used in the processing is included in the EXTRAS directory. As of this writing, the solution from Stiles et al is being used for data takes Ta through T30 for which it was fit, while a simpler long-term solution from NAIF is used for data takes after T30.
For convenience, the set of angles used to transform coordinates from the inertial frame to the target body fixed (TBF) frame is also included in the BODP files (see Intermediate Data Segment, e.g., pole_right_ascension, pole_declination, target_rotation_rate, target_rotation_angle).
This section describes the format of files contained in the Cassini Radar Instrument Team Archive Volumes. Data that comprise the Archive will be formatted in accordance with the Planetary Data System specifications [Planetary Science Data Dictionary, 1996; PDS Data Preparation Workbook, 1995; PDS Standards Reference 1995].
Document files with the .TXT suffix exist in the Root, CALIB, CATALOG, DOCUMENT, EXTRAS, INDEX, and SOFTWARE directories. They are ASCII files with embedded PDS labels with line lengths that do not exceed 80 characters. Document files have variable-length lines with an end-of-line designator consisting of a carriage-return character followed by a line-feed character, thus allowing the files to be read by the MacOS, Microsoft Windows, and Unix operating systems.
A Tabular file (INDEX.TAB) exists in the INDEX directory. This file is an ASCII file formatted for direct reading into many database management systems on various computers. Commas separate all fields, and character fields are enclosed in double quotation marks ("). Character fields are left justified and padded on the right with spaces to keep quotation marks in the same columns of successive records. Number fields are right justified and padded with spaces on the left to keep fixed-width fields. The records are of fixed length, and the last two bytes of each record contain the ASCII carriage-return and line-feed character. This allows a table to be treated as a fixed-length record file on computers that support this file type and as a text file with embedded line delimiters on those that don't.
A detached PDS label describes INDEX.TAB. The PDS label file name is the same as the tabular file but with the extension ".LBL".
All data files in the Cassini Radar Instrument Team Archive Collection have PDS labels [Planetary Science Data Dictionary, 1996; PDS Standards Reference, 1995]. A PDS label, whether embedded or detached from its associated file, provides descriptive information about the associated file. The PDS label is an object-oriented structure consisting of sets of "keyword = value'' declarations. The object to which the label refers (e.g. IMAGE, TABLE, etc.) denotes the form:
^object = location
in which the caret character (^, also called a pointer in this context) indicates where to find the object. In an embedded label, the location is an integer representing the starting record number of the object (the first record in the file is record 1). In a detached label, the location denotes the name of the file containing the object, along with the starting record or byte number, if there is more than one object in the file. For example:
^IMAGE = ("F01.IMG", 1025 <BYTES>)
indicates that the IMAGE object begins at byte 1025 of the file F01.IMG, in the same directory as the detached label file. Below is a list of the possible formats for the ^object definition.
^object = ("[dirlist]filename.ext", n)
^object = ("filename.ext", n<BYTES>)
^object = ("[dirlist]filename.ext", n<BYTES>)
n is the starting record or byte number of the object, counting from the beginning of the file (record 1, byte 1);
<BYTES> indicates that the number given is in units of bytes;
filename is the up to 24-character, alphanumeric upper-case file name, ext is the 3 character upper-case file extension;
[dirlist] is a period-delimited path-list of parent directories, in upper case, that specifies the object file directory (used only when the object is not in the same directory as the label file). The list begins at the directory level below the root directory of the CD-ROM. [dirlist] may be omitted when the object being described is located in the same directory as the detached label.
Detached label files have variable-length lines with an end-of-line designator consisting of a carriage-return character followed by a line-feed character. This allows the files to be read by the MacOS, Microsoft Windows, and Unix operating Systems.
Catalog files (with suffix .CAT) exist in the Root and CATALOG directories. They are formatted in an object-oriented structure consisting of sets of "keyword = value" declarations.
The Cassini SBDR, LBDR, and ABDR BODP science data files are formatted using the "table object" structure of the PDS standards. Each row of the table consists of a data record comprising radar and ancillary data for a single burst. Rows occur in time order. A column represents a particular data field with a specific meaning and data type (i.e. integer, real, etc.). Real values are either 32-bit or 64-bit IEEE floating-point values with little endian byte order (PC_REAL). The Cassini ABDR-SUMMARY science data files are formatted using the "spreadsheet object" PDS structure. Each record in the file corresponds to a burst in the corresponding ABDR science data file and contains a set of parameters derived for that burst. As for the other BODP products, rows occur in time order. Each field in a record represents a particular parameter with a specific meaning and data type. For more information about the format and content of the BODP files, see the Cassini Radar Burst Ordered Data Product SIS, JPL D-27891.
The Cassini BIDR science data files are formatted using the "image object" structure of the PDS standards. The size and coordinate system of the image is defined in the attached PDS label. Images pixels may have several different meanings and data types. For more information about the format and content of the BIDR files, see the Cassini Radar Basic Image Data Record SIS, JPL D-27889.