M2020 Derived Imaging Data Archive Peer Review

Dec 21, 2020 — Feb 15, 2021

Review Comments Due Jan 29, 2021

Reminder: M2020 Landing Day is February 18, 2021

PURPOSE & OBJECTIVES

For the Mars 2020 Perseverance Rover Mission (M2020), almost all of the instruments acquire imaging data for either primary or secondary/context purposes — M2020 is festooned with imagers.  The data from which will be processed through the JPL Instrument Data System (IDS) pipeline to create operational image products for each of the instrument teams which, in turn, also produce their own derived data.  This peer review (the RDR PR) will therefore be an all-encompassing look at the entire scope and scale of derived products generated from these operational imaging data.

This is a lot of data.

Prior to this RDR PR, a raw data peer review (the EDR PR) uncovered issues to be addressed.  Therefore, this RDR PR has had changes implemented to address most of those issues.  Updated versions of the raw products are included and may be re-reviewed.

As M2020 is still on approach to Mars, the test data used for this review will portray imagery acquired from testbed/laboratory environments and, as such, may be irrelevant in any scientific sense.  Nevertheless, the main goal of this review is to ensure that the data ultimately submitted for archival will be meet several important objectives:

1. That the data are scientifically useful — This is, by far, the most important objective
2. That the metadata correctly and comprehensively describe the data — a close 2nd
3. That the supporting documentation is complete and accurate — a close 3rd
4. That the archive wholly complies with the PDS4 standards — much less urgent

Not all review members will be expected to comment with respect to all the above objectives.  The group, as a whole, will accomplish this.

Ultimately, your review of these archives will consist of a written set of comments, clarifications and discrepancies that, if implemented and/or corrected, would improve the archive.  These written statements need to be provided to the Review Coordinator by the “Comments Due” date in the Schedule (below).

GETTING STARTED

This is the way...

Though this peer review covers a lot of data, there is a methodical path through it.  The first, and most important, thing to look at is probably the camera SIS, found here:

https://pds-imaging.jpl.nasa.gov/reviews/mars2020/mars2020_mission/document_camera/Mars2020_Camera_SIS.pdf

The SIS itself is huge, you are probably better off skimming it and honing in on sections of interest rather than trying to read the whole thing.  Note that it has been completely reorganized and largely rewritten since the EDR PR, so if you participated in that review you should revisit the document.

Of particular note are the label tables described in Appendix B, they describe in detail all of the label items.  Start with this one:

https://pds-imaging.jpl.nasa.gov/reviews/mars2020/mars2020_mission/document_camera/Mars2020_Camera_SIS_Labels_sort_pds.html

(There’s also a PDF version, but the hyperlinks likely work better in the above HTML.)

Then it’s a matter of looking for the data you’re most interested in.  Pick a camera type, choose the right bundle from the list below, go to the directory starting with “data”, and drill down through the sol numbers to find the data of interest.  The XML label can be viewed directly in most Web browsers.  Cross-reference interesting items against the label table.

The browse directory contains viewable images.  However, Marsviewer (described below) is recommended to access the bulk of the image data.

Providing comments:

Reviewers are welcome to provide comments in any preferred format via email to the Review Coordinators. The below Google Sheet is available for use, but ultimately not required.  Each reviewer has a named tab within this Sheet for their own use.

https://docs.google.com/spreadsheets/d/1zlOmca83Ft_RGmQwuPVBOrWCoo9Cej5Ie7mh2FZjwNo/edit?usp=sharing 

This sheet will ultimately be used to adjudicate reviewer comments and convey the Data Provider’s response to each of these comments.  So, however review comments are delivered to the Review Coordinators, they will be distilled and tracked here.

DATA CONTENTS

Organization, structure, and description:

There are 5 image archive “bundles” being reviewed here (see the Bundle SIS).  They are:

1. mars2020_ecam: Navcam, Hazcam, EDLcam, CacheCam, LVScam

https://pds-imaging.jpl.nasa.gov/reviews/mars2020/mars2020_ecam

2. mars2020_mastcamz: Mastcam-Z

https://pds-imaging.jpl.nasa.gov/reviews/mars2020/mars2020_mastcamz

3. mars2020_helicam: Helicopter cameras

https://pds-imaging.jpl.nasa.gov/reviews/mars2020/mars2020_helicam

4. mars2020_imgops: SHERLOC, PIXL, and SuperCam cameras

https://pds-imaging.jpl.nasa.gov/reviews/mars2020/mars2020_imgops

5. mars2020_meda: SkyCam

https://pds-imaging.jpl.nasa.gov/reviews/mars2020/mars2020_meda

As described in the Bundle SIS, the first four will ultimately be archived at the Imaging Node, with mars2020_imgops data being included by reference at the Geosciences Node within the specific instrument bundles.  The last is archived at the Atmospheres Node, and is a partial bundle containing only the SkyCam; it will be combined with the rest of the MEDA non-imaging data in a single, unified bundle prior to release.

There is one additional bundle that is important to this review:

6. mars2020_mission: unified bundle for all calibration, documentation, and other miscellaneous information

https://pds-imaging.jpl.nasa.gov/reviews/mars2020/mars2020_mission

which will be maintained at the Geosciences Node.  A portion of this bundle is included with this review, containing the files that are relevant to the camera data and are a primary, critical part of this review.  However, the bundle itself is not complete - feel free to point out issues but keep that in mind during the review.

Data pedigree

The data being reviewed here is largely from Operational Readiness Test (ORT) 11, from mid-November, 2020.  This was a “surface nominal” test, which exercised most of the capabilities of the rover (except mobility) and most of the instruments.  It consists of sols 00113-00118.  This is a lot of data, on the order of 500 GB.  But as this rover generates a lot of data, especially a lot of derived data, we felt it was important to present an example of the scope and scale of data users should expect, and to exercise the generation tools.  We certainly do not expect anyone to look at all of the data; pick and choose what makes sense to you and your interests.

Note that due to some last-minute fixes, terrain meshes are only included for sol 00114, even though they were in more sols for the actual ORT.

In addition to the ORT-11 data, updated EDRs from the EDR Peer Review are included for instruments or modes that were not represented in the ORT.  The EDR data files are the same as for the EDR PR, but the labels have been regenerated.  There are no derived products generated from these EDRs.  The sols with this EDR-only data, and the reason for inclusion, are:

• 00000 - helicopter images
• 00008 - RMI zstack and JPEG-compressed
• 00011 - Rear hazcam
• 00022 - B-side fhaz, separate R/G/B files from nav (like in cruise)
• 00071 - cachecam
• 00074 - cachecam
• 00157 - cachecam
• 00194 - zcam video
• 00231 - Watson video and zstack
• 02865 - LVS camera (new, not in EDR PR, includes FDRs)

See the liens list for caveats regarding this non-ORT-11 data.

Finally, there are a few extra example files here: extra_examples which contains examples of files that illustrate issues in the liens list that are fixed in software but could not be applied to the entire data set.  They are proper products with labels, but are not in any bundle or directory structures.

DATA ACCESS

Overview of access/interaction methods:

1. Possible interactive methods for perusing the imaging archives:

1. Directly interacting with the web site: you can simply go spelunking through the archive structures by clicking on the links.  This method is useful for deciphering the archive bundle structure.  It may also be the simplest way to examine the PDS4 labels.  Use the URL’s in the Overview for each bundle.
2. Browse image collections: each of the archive bundles includes a browse collection that contains representations of the data in easily rendered formats.  This only contains data for actual images though; many derived products like XYZ images are not represented in the browse collection.
3. Use the Marsviewer application (as described below): Marsviewer allows the interactive perusal and investigation of the data in more detail.

2. Some additional documents are provided outside the archives that may be helpful as part of your review.  These documents are not typically delivered within the archive bundles, but are necessary for making the archive bundles correct and useful.  Normally these will be delivered to PDS directly, but not all of them have been officially published yet.  If you wish to do your own PDS validation, you will need to use the supplied versions.  These are:

1. The Data Dictionaries that describe and enforce the schema for the metadata, and
2. The context products that are used to associate the metadata with certain objects (e.g., the planet Mars, the spacecraft Perseverance, the various instruments, etc…)

        All, however, are appropriate for your review.  They can respectively be found in:

https://pds-imaging.jpl.nasa.gov/reviews/mars2020/support/LDD/

https://pds-imaging.jpl.nasa.gov/reviews/mars2020/support/context/

Interactive perusal via Marsviewer

Marsviewer is an interactive, Java-based tool that understands the relationships between the various data products, their organization within the archive, and how to present the data in a visual manner.  Most importantly, it provides visualization for the non-image data types, such as XYZs and surface normals, that are stored in image format.  To get started, simply perform the following:

Windows:

Download the executable from the Peer Review site here:

https://pds-imaging.jpl.nasa.gov/reviews/mars2020/support/marsviewer/marsviewer-install.zip

Once downloaded, double-click the .zip file to reveal the marsviewer-install.exe installation executable.

Run the installation wizard, specifying the location to which the application suite will be installed on your computer.  The default installation location is %PROGFILES%\JPL\Marsviewer\. (where %PROGFILES% should be replaced with the suitable ‘Program Files’ directory of your system).

After installation, you should see a ‘Marsviewer’ icon on your Windows Desktop.  Double-click the application icon to launch the tool.

Mac OS:

Download the DMG image from the Peer Review site here:

https://pds-imaging.jpl.nasa.gov/reviews/mars2020/support/marsviewer/Marsviewer.dmg

Once downloaded, the DMG file should be expanded. If not, then double-click on the file to expand it

Select and drop the Marsviewer (for search and display) and Jadeviewer (image viewer only) apps into your Applications folder. Marsviewer is used for browsing Mars images that reside on PDS servers. Jadeviewer is used for looking at a specific image data product on your computer.

Double-click the application to launch it.  If you get a security violation when doing that, try ctrl-click on the icon and then select Open.  It should present you with a dialog asking if you really want to open it.  Say yes.  This should only be needed once; a simple double-click should be sufficient for future invocations.

Setting Up Marsviewer (both OS’s):

When Marsviewer starts, it presents you with a dialog box from which to select a “File Finder”.  There are preset values for each of the instrument groups, which should be straightforward  The one confusing one is ECAM.  Select “PDS M2020 ECAM - PR” for most uses.  This gives you access to the reassembled FDR and RDR products.  Selecting “PDS M2020 ECAM Tiles - PR” will give you access to the original tiles, but not the reassembled products.

File/New File Finder will allow you to change the file finder preset to a different type.

The server hosting this data for the PR is slow, so be patient during loading - especially with very large ECAM images.

For each of the file finders, data is available on the following sols:

LIENS/ERRATA/NOTES/COMMENTS

The Mars2020 image processing system is still under development; not everything is finished yet.  There are thus aspects of the archives submitted for this review that are incomplete, incorrect, or missing.  The known issues are listed below:

1. Some data types are NOT included in this review.  The following types of data are not included in this review, and will be picked up at a later time:

• EDLCAM Microphone data
• Movie files from EDL cameras, including single images extracted from the movies
• PLACES, Target, Science Intent, and CRUST databases

2. A number of label groups are missing from the ODL and XML labels.  They did not get properly transferred from the VICAR labels, but they are correct in the VICAR labels.  Unfortunately this was discovered too late to fix for the PR, although the fix is already in place in code.  This affects both the ODL/xml labels, and the keyword table (SIS appendix C).  A handful of examples including these labels will be provided outside the bundle structure (see below).  The affected label groups are:

• DRILL_ARTICULATION_STATE
• HELI_G_COORDINATE_SYSTEM
• HELI_M_COORDINATE_SYSTEM
• HELI_S1_COORDINATE_SYSTEM
• LED_ILLUMINATION_PARMS
• PIXL_BASE_COORDINATE_SYSTEM
• PIXL_SENSOR_COORDINATE_SYSTEM
• ROW_SUM_REQUEST_PARMS
• SCAM_MINI_HEADER
• SCS_ARTICULATION_STATE
• SHA_ARTICULATION_STATE
• SLI_ILLUMINATION_PARMS
• TURRET_COORDINATE_SYSTEM
• VIDEO_REQUEST_PARMS
• ZSTACK_REQUEST_PARMS

Example products with these labels present are available in extra_examples.  They are outside the bundle structure because the fixes were made too late to rerun all the data.  Nevertheless, the examples show what the above items should look like for the rest of the data set.

3. EDLcam data is outside the bundle.  It should be part of the mars2020_ecam bundle but got missed.  Sample EDLcam data is in the extra_examples directory, filenames starting with E.  They should be in mars2020_ecam/data/sol/00116/ids/edr/edl/ .
4. The data comes from a testbed environment.  The ORT-11 data specifically uses test/engineering versions of instruments, which are close to flight but not exactly the same.
5. Labels are not completely finalized from the Project.  Expect minor additions as they continue to be fleshed out.  The intent is to not re-review minor additions or adjustments, but major changes may require a re-review.
6. The mars2020_mission bundle is incomplete.  Look at the files in it, and the secondary pointers to it from the other bundles, but not the bundle structure itself, as it will be maintained by the Geosciences node.
7. Data from the Mastcam-Z team is not available.  Sample data was not available in time for this review.  Mastcam-Z data processed by IDS is present, but not data from the Mastcam-Z team.  Such data should conform to the SIS, however.
8. There are a few TBD’s in the Camera SIS document.  Reviewers should think about what’s missing that is NOT marked TBD.
9. The pipeline flow chart is preliminary.  The flowchart referred to by Appendix D (in a separate document named Mars2020_Camera_SIS_Pipeline_Flow.pdf) is a working version with many development notes attached.  It will be cleaned up for the final delivery.
10. Calibration data is far from complete, what's there is just a snapshot of what we have now to show how it will look.  It includes a lot of testbed files that may or may not appear in the final calibration collection.
11. Issues with the label table (Appendix C of SIS).  A bug in the table generator was uncovered too late to fix for the PR.  It manifests as follows.  For PDS shorthand identifiers (<namespace>:<keyword> in the top-left of the leftmost cell in each row) whose XPaths have different parent classes one step up from the leaf:

• If each XPath is supposed to map to a different VICAR keyword and/or valid values list, the VICAR keys and valid values list may be incorrect/incomplete.
• If each XPath maps to the same VICAR keyword and valid values list, the row in the table is correct

An example of this behavior can be seen with img:active_flag.  Its two XPaths in the table are:
/Product_Observational/Observation_Area/Discipline_Area/Imaging/Radiometric_Correction/active_flag
/Product_Observational/Observation_Area/Discipline_Area/Imaging/Flat_Field_Correction/active_flag
Note that the active_flag key has two different parents: Radiometric_Correction and Flag_Field_Correction.  The corresponding VICAR keys are:
Radiometric_Correction.img.active_flag -> DERIVED_IMAGE_PARMS.RADIOMETRIC_CORRECTION_TYPE
Flat_Field_Correction.img.active_flag -> OBSERVATION_REQUEST_PARMS.FLAT_FIELD_CORRECTION_FLAG
Yet the VICAR keys displayed are:
Shutter_Subtraction.img.shutter_subtraction_mode -> INSTRUMENT_STATE_PARMS.SHUTTER_CORRECTION_MODE
 Shutter_Subtraction.img.active_flag -> OBSERVATION_REQUEST_PARMS.SHUTTER_CORRECTION_MODE
This bug will be fixed in the next release of the table generation software.

12. Portions of the Ops Keyword Table (Appendix B) will be deprecated.  Ignore the entries in the Ops Keyword Table (Appendix B) for Archive Attribute, X-path, and Reference (they're all marked TBD).  With the inclusion of the other keyword tables, these entries are no longer needed.  They just haven't been deleted from the document yet.
13. The documents for the review are PDF rather than PDF/A.  That will be fixed for the actual delivery.
14. The <Header> class for the VICAR label may be incorrect.  It points to offset 0 instead of the start of the VICAR label; that is a known issue that is being fixed.  The fix will also add a second <Header> class to point to the ODL label, if it exists.  The extra_examples files show this.

Some additional liens which apply ONLY to the non-ORT-11 data (outside sols 113-118):

15. Not all aspects of the data are flightlike.  The data from outside of ORT-11 is a hodgepodge of images from ATLO and various testbeds, across a wide range of flight software versions, therefore not all aspects are flightlike. For example, very few of the images have enough information to successfully use SPICE, so things like solar azimuth and elevation may not appear in most images.  Some images have invalid or inconsistent data in their headers.  Only a few instruments have camera models.  This lack of consistent test data is a Project issue which is beyond the scope of discussion in this forum.  As a result, reviewers should look more at the format of the images and labels for this data rather than the specific content (e.g. what the label names are, not so much the values).
16. Much of the non-ORT-11 data comes from testbed runs which use the DOY form of the filename, with a SOL containing an alpha first character.  For the purposes of this PR, these files are treated like a sol by just zeroing out the first character.  Thus a sol in the filename of "D074", which should be Day-Of-Year 74 in 2020 per the SIS, is treated as sol 0074 in this PR, and thus appears in the sol/00074 directory.
17. All products are labeled version 00.  Normally versioning starts at 01. However, versioning is a function of the IDS pipeline, which was not used for the creation of these products, so they are all version 00 as assigned by the EDL generation program.
18. Most RDR/FDR files are not available for the non-ORT-11 data (with the exception of LCAM which contains FDRs).

SCHEDULE

REVIEWERS