PDS_VERSION_ID = PDS3 RECORD_TYPE = STREAM OBJECT = TEXT PUBLICATION_DATE = 2013-05-20 INTERCHANGE_FORMAT = ASCII NOTE = "N/A" END_OBJECT = TEXT END MSL MARS HAND LENS IMAGER (MAHLI) EXPERIMENT DATA RECORD (EDR) AND REDUCED DATA RECORD (RDR) ARCHIVE Table of Contents 1. - INTRODUCTION 2. - FILE FORMATS 3. - VOLUME CONTENTS, DIRECTORY, AND FILE NAMING CONVENTIONS 4. - CONTACTS FOR MORE INFORMATION 5. - ACKNOWLEDGEMENTS 1. INTRODUCTION This volume contains the ancillary data files that support the MAHLI EDR and RDR archive. These files include an index file ('EDRINDEX.TAB and RDRINDEX.TAB') that tabulates the contents of the volume and gives useful ancillary information, and documentation files. For more information on the contents and organization of the volume set refer to the 'Volume Contents, Directory, and File naming conventions' section of this document. The MAHLI EDR and RDR archive is compiled for the National Aeronautics and Space Administration (NASA) by Malin Space Science Systems personnel under the direction of Dr. Kenneth S. Edgett, MAHLI Team Leader. Leslie Lipkaman, Daniel Krysak, and Robert Zimdar are responsible for its compilation. The image files are organized according to NASA's Planetary Data System (PDS) standards. An image file is organized as a PDS labeled file containing an "image object". 2. FILE FORMATS 2.1 Document File Format Document files with the .TXT suffix exist in the Root, Index, Calib, Catalog, Document, and Software directories. They are ASCII files, which may have embedded PDS labels. Lines in a .TXT file end with a carriage return character (ASCII 13) and a line feed character (ASCII 10). This allows the files to be readable under various operating systems. Documents in the DOCUMENT directory may contain formatting and figures that cannot be rendered as ASCII text. In such a case, a PDF (Portable Document Format) is used which is an open standard of Adobe Systems Incorporated that is frequently used for distributing documents. Adobe offers a free program, Acrobat Reader, for viewing PDF files. 2.2 Tabular File Format Tabular files (.TAB suffix) exist in the Index directory. Tabular files are ASCII files formatted for direct reading into many database management systems on various computers. All fields are separated by commas while character fields are enclosed in double quotation marks ("). (Character fields are padded with spaces to keep quotation marks in the same columns of successive records.) Character fields are left justified, and numeric fields are right justified. The "start byte" and "bytes" values listed in the labels do not include the commas between fields or the quotation marks surrounding character fields. The records are of fixed length, and the last two bytes of each record contain the ASCII carriage return and line feed characters. This allows a table to be treated as a fixed length record file on computers that support this file type and as a text file with embedded line delimiters on those that don't. All tabular files are described by PDS labels, either embedded at the beginning of the file or detached. If detached, the PDS label file has the same name as the data file it describes, with the extension .LBL; for example, the file EDRINDEX.TAB is accompanied by the detached label file EDRINDEX.LBL in the same directory. 2.3. PDS Label Format All data files in the archive have PDS labels, either attached at the beginning of the file or detached in a separate file. For examples of PDS labels for different types of data products, see the Mast Camera (Mastcam), Mars Hand Lens Imager (MAHLI), and Mars Descent Imager (MARDI) Experiment Data Record (EDR) and Reduced Data Record (RDR) PDS Data Products Software Interface Specification, JPL D-75410. A PDS label, whether attached or detached from its associated file, provides descriptive information about the associated file. The PDS label is an object-oriented structure consisting of sets of 'keyword= value' declarations. The object to which the label refers (e.g. IMAGE, TABLE, etc.) is denoted by a statement of the form: ^object = location in which the carat character (^, also called a pointer in this context) indicates where to find the object. In an embedded label, the location is an integer representing the starting record number of the object (the first record in the file is record 1). In a detached label, the location denotes the name of the file containing the object, along with the starting record or byte number, if there is more than one object in the file. For example: ^HEADER = ("F01.IMG",1) ^IMAGE = ("F01.IMG",1025 ) indicates that the IMAGE object begins at byte 1025 of the file F01.IMG, in the same directory as the detached label file. ^MINIHEADER_TABLE = ("F01.DAT",0 ) ^DAT_TABLE = ( "F01.DAT", 64 ) indicates that the file begins with the mini-header which precedes the original binary data packet produced by the instrument and that the binary packet is 64 bytes from the first byte of the file Lines of text in detached labels end with a carriage return character (ASCII 13) and a line feed character (ASCII 10). This allows the files to be readable under various operating systems. 2.4. Software File Format The software tool "build_all" is a UNIX C-shell script. All other source code files are in the C programming language conforming to the standard ISO/IEC 9899:1999. The software tool "dat2img" is provided in a pre-compiled executable for the Linux and Darwin operating systems. The supported Linux platform is Linux kernel 2.6.18 x86_64 (Red Hat Enterprise Linux release 5) and is compiled as an ELF 64-bit LSB executable. The supported Darwin platform is Darwin kernel 11.4.0 i386 (Apple OS X 10.7) and is compiled as a Mach-O 64 bit executable. 2.5. Catalog File Format Catalog files (suffix .CAT) exist in the Root and Catalog directories. They are text files formatted in an object-oriented structure consisting of sets of 'keyword=value' declarations. 2.6. Science Data File Formats For information about the format and content of the data products, see the Mast Camera (Mastcam), Mars Hand Lens Imager (MAHLI), and Mars Descent Imager (MARDI) Experiment Data Record (EDR) and Reduced Data Record (RDR) PDS Data Products Software Interface Specification, JPL D-75410. 3. VOLUME CONTENTS, DIRECTORY, AND FILE NAMING CONVENTION The files in this volume are organized starting at the root or 'parent' directory. Below the parent directory is a directory tree containing data, documentation, and index files. Directory names are indicated by brackets (<...>), upper-case letters indicate an actual directory or file name, and lower-case letters indicate the general form of a set of directory, software programs, or file names. Top-level Directory ------------------- AAREADME.TXT The file you are currently reading. ERRATA.TXT This text file contains a listing of comments and updates concerning the archive volume. VOLDESC.CAT This text file contains a description of the volume contents as a PDS catalog object. It is a required file on PDS archive volumes. CALIB Directory ------------------ The CALIB directory contains flat field files as well as decompanding tables 0 through 32 applicable to MAHLI. Additionally, a summary of camera calibration can be found in MSL_MMM_CAL.TXT. See CALINFO.TXT for a description of the files in the CALIB directory. CATALOG Directory ----------------- The files in the Catalog directory provide a top-level understanding of the mission, spacecraft, instruments, and data set. The files in this directory become part of the PDS Catalog to provide background information for the user searching for data. Their format and contents are further specified in the PDS Standards Reference. See the file CATINFO.TXT for details. DATA Directory -------------- The DATA directory contains version 3.0 of MAHLI CODMAC Level 2 EDRs and version 3.0 of MAHLI CODMAC Level 4 RDRs for SURFACE data for releases 0001-0004. PRE_ATLO, ATLO, and CRUISE data can be found in release 0001; these products are version 3.0 MAHLI CODMAC Level 2 EDRs only. Prior to release 0004, the format of the .LBL file was changed for EDR products. In addition, several keywords in the .LBL file were improved for both EDRs and RDRs. Previous keywords that were N/A or NULL now contain actual data; other keyword's valid values have been updated to be more accurate. Formerly released data from releases 0001-0004 have been modified to reflect changes to the .LBL format as well as keyword improvements. To make it clear which products have been updated, all products in releases 0001-0004, regardless if a version 2.0 existed or not, have been versioned to 3.0 (PRODUCT_VERSION_ID = V3.0). Keywords in the .LBL that have changed: MSL:REQUEST_ID PRODUCT_CREATION_TIME PRODUCT_VERSION_ID RELEASE_ID ROVER_MOTION_COUNTER OBJECT = COMPRESSED_FILE (EDR only): FILE_NAME, RECORD_TYPE, FILE_RECORDS, ENCODING_TYPE, INTERCHANGE_FORMAT, UNCOMPRESSED_FILE_NAME, REQUIRED_STORAGE_BYTES, ^MINIHEADER_TABLE, DESCRIPTION OBJECT = UNCOMPRESSED_FILE (EDR only): FILE_NAME, RECORD_TYPE (same keyword, different location), FILE_RECORDS, RECORD_BYTES (same keyword, different location) OBJECT = IMAGE (EDR only): LINES, LINE_SAMPLES, SAMPLE_TYPE, SAMPLE_BITS, BANDS, FIRST_LINE, FIRST_LINE_SAMPLE GROUP = PDS_HISTORY_PARMS: SOFTWARE_VERSION_ID GROUP = GEOMETRIC_CAMERA_MODEL_PARMS: MODEL_COMPONENT_1, MODEL_COMPONENT_2, MODEL_COMPONENT_3, MODEL_COMPONENT_4, MODEL_COMPONENT_5 (EDR and RDR DRCX and DRXX only), MODEL_COMPONENT_6 (EDR and RDR DRCX and DRXX only), REFERENCE_COORD_SYSTEM_INDEX GROUP = ROVER_COORDINATE_SYSTEM_PARMS, GROUP = RSM_COORDINATE_SYSTEM_PARMS, GROUP = ARM_COORDINATE_SYSTEM_PARMS, GROUP = SITE_COORDINATE_SYSTEM_PARMS: COORDINATE_SYSTEM_INDEX, ORIGIN_OFFSET_VECTOR, ORIGIN_ROTATION_QUATERNION GROUP = RSM_ARTICULATION_STATE_PARMS, GROUP = CHASSIS_ARTICULATION_STATE_PARMS, GROUP = HGA_ARTICULATION_STATE_PARMS: ARTICULATION_DEVICE_ANGLE, ARTICULATION_DEVICE_MODE GROUP = ARM_ARTICULATION_STATE_PARMS: ARTICULATION_DEVICE_ANGLE, ARTICULATION_DEVICE_MODE, ARTICULATION_DEVICE_TEMP, CONTACT_SENSOR_STATE_NAME, CONTACT_SENSOR_STATE, ARTICULATION_DEV_VECTOR, ARTICULATION_DEV_INSTRUMENT_ID GROUP = OBSERVATION_REQUEST_PARMS: RATIONALE_DESC GROUP = IMAGE_REQUEST_PARMS: FIRST_LINE, FIRST_LINE_SAMPLE, LINES, LINE_SAMPLES, EXPOSURE_TYPE, EXPOSURE_DURATION, INST_CMPRS_MODE, INST_CMPRS_NAME, INST_CMPRS_QUALITY, AUTO_EXPOSURE_PERCENT, AUTO_EXPOSURE_PIXEL_FRACTION, MAX_AUTO_EXPOS_ITERATION_COUNT, MSL:INVERSE_LUT_FILE_NAME GROUP = VIDEO_REQUEST_PARMS: MSL:COMMANDED_VIDEO_FRAMES, INTERFRAME_DELAY GROUP = INSTRUMENT_STATE_PARMS: HORIZONTAL_FOV, VERTICAL_FOV, INSTRUMENT_TEMPERATURE, MSL:INSTRUMENT_TEMPERATURE_STATUS GROUP = VIDEO_PARMS: MSL:GOP_LENGTH GROUP = DERIVED_IMAGE_PARMS: MSL:MINIMUM_FOCUS_DISTANCE, MSL: BEST_FOCUS_DISTANCE, MSL: MAXIMUM_FOCUS_DISTANCE, FIXED_INSTRUMENT_AZIMUTH, FIXED_INSTRUMENT_ELEVATION, SOLAR_AZIMUTH, SOLAR_ELEVATION GROUP = PROCESSING_PARMS: DARK_LEVEL_CORRECTION, SHUTTER_EFFECT_CORRECTION_FLAG (RDR only), FLAT_FIELD_CORRECTION_FLAG OBJECT = IMAGE (RDR .LBL only): INVALID_CONSTANT (RDR DRLX and DRXX only), MISSING_CONSTANT (RDR only) OBJECT = MINIHEADER_TABLE: RECORD_TYPE, FILE_RECORDS OBJECT = BIT_COLUMN: NAME = CCD_STATE: DESCRIPTION, NAME = CLKDIV2: DESCRIPTION, NAME = CLKDIV1: DESCRIPTION Keywords in the .LBL that have been removed: ^DAT_TABLE (EDR only) OBJECT = DAT_TABLE (EDR only) For releases 0005 onwards, the directory contains version 1.0 of MAHLI CODMAC Level 2 EDRs and version 1.0 of MAHLI CODMAC Level 4 RDRs for SURFACE data. See MSL_MMM_EDR_RDR_DPSIS.PDF in the DOCUMENT directory for a description of the different levels of data. DOCUMENT Directory ------------------ The DOCUMENT directory contains documentation to help the user understand and use the archive data. See DOCINFO.TXT for a description of the files in the DOCUMENT directory. INDEX Directory --------------- The INDEX directory contains PDS index files for this archive. An index file is an ASCII table with each record (or line) in the table containing information about a single data product in the archive. Files in the Index directory are provided to help the user locate products on the archive volume. See the file INDXINFO.TXT for additional details. SOFTWARE Directory --------------- The SOFTWARE directory contains MMM_DAT2IMG.ZIP which includes the source code for dat2img and a compiled version for each of the supported operating systems. dat2img allows the user to decode MMM EDR .DAT files. See SOFTINFO.TXT for details. DIRECTORY/FILE -------------- | -- AAREADME.TXT | -- | -- CALINFO.TXT | -- DECOMPAND{0-32}.TXT | -- FLAT_MH_0.IMG | -- MSL_MMM_CAL.TXT | -- | -- CATINFO.TXT | -- MAHLI_EDR_IMG_DS.CAT | -- MAHLI_EDR_VID_DS.CAT | -- MAHLI_EDR_Z_DS.CAT | -- MAHLI_INST.CAT | -- MAHLI_PERSON.CAT | -- MAHLI_RDR_IMG_DS.CAT | -- MAHLI_RDR_VID_DS.CAT | -- MAHLI_RDR_Z_DS.CAT | -- MAHLI_REF.CAT | -- MSL_INSTHOST.CAT | -- MSL_MISSION.CAT | -- MSL_REF.CAT | -- SOFTWARE.CAT | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- DOCINFO.TXT | -- GEOMETRIC_CM.TXT | -- MAHLI_TECH_REPT_0001.LBL | -- MAHLI_TECH_REPT_0001.PDF | -- MSL_MASTCAM_MAHLI_FOCUS_MERGE_PRODUCTS.LBL | -- MSL_MASTCAM_MAHLI_FOCUS_MERGE_PRODUCTS.PDF | -- MSL_MMM_EDR_RDR_ARCHSIS.LBL | -- MSL_MMM_EDR_RDR_ARCHSIS.PDF | -- MSL_MMM_EDR_RDR_DPSIS.LBL | -- MSL_MMM_EDR_RDR_DPSIS.PDF | -- ERRATA.TXT | -- | -- EDRINDEX.LBL | -- EDRINDEX.TAB | -- EDR_CMDX.LBL | -- EDR_CMDX.TAB | -- INDXINFO.TXT | -- RDRINDEX.LBL | -- RDRINDEX.TAB | -- RDR_CMDX.LBL | -- RDR_CMDX.TAB | -- | -- | -- MMM_DAT2IMG.TXT | -- HUFFMAN_TABLE.TXT | -- SOFTINFO.TXT | -- | -- MMM_DAT2IMG.LBL | -- MMM_DAT2IMG.ZIP | -- OSXBIN | -- DAT2IMG | -- DAT2IMG.LBL | -- RH5BIN | -- DAT2IMG | -- DAT2IMG.LBL | -- VOLDESC.CAT 4. CONTACTS FOR MORE INFORMATION For details regarding generation of the MAHLI EDR & RDR archive: ---------------------------------------------------------------- Dr. Kenneth S. Edgett Malin Space Science Systems PO Box 910148 San Diego CA 92191 mmmdata@msss.com For overall content and design of the MAHLI EDR & RDR archive as presented on this volume: --------------------------------------------------------------- Leslie Lipkaman, Daniel Krysak & Robert Zimdar Malin Space Science Systems PO Box 910148 San Diego CA 92191 mmmdata@msss.com 5. ACKNOWLEDGEMENTS We wish to thank the following people at MSSS for their invaluable contributions to the production and review of these volumes: Michael Caplinger, Craig Hardgrove, David E. Harker, Paul Herrera, Elsa Jensen, Gillian Krezoski, Elaina McCartney, S. McNair, Brian Nixon, Liliya Posiolova, Jennifer Sandoval, Kimberly Supulver, Jason Van Beek, Tessa Van Beek, and Megan Kennedy Wu.