Qimera Metadata Extraction Process

Qimera - Metadata Extraction Process

When data is loaded into Qimera, a metadata extraction process runs to intelligent strip information out of the file to optimize the Dynamic Workflow engine of Qimera thus reducing the amount of work you have to do to process your data.  The metadata extraction has some variability based on the file type and the basics are listed below.  When a file is loaded into Qimera, it is given a unique identifier and the filename and identifier are used to create a specific folder inside the project ~/Metadata folder.  For example, if the first file loaded is named 0000_20140321_034136_ICEBEAM.all, the associated folder will be 0000_20140321_034136_ICEBEAM_00001.  If the next line is named 0001_20140321_035137_ICEBEAM.all, its folder will be named 0001_20140321_035137_ICEBEAM_00002 and so on.  WARNING: The contents of these folders are not meant to be edited by the user and could have unintended consequences!  So please don't move, delete, rename or otherwise edit any of these files.

Raw Sonar Files

The following are the tasks performed on raw sonar files during the metadata extraction process. 

File Indexing 

All raw sonar files supported by Qimera are packetized in nature and proceed linearly in time.  Qimera creates an index file that allows for fast non-linear access to every distinct packet type inside the file.  Qimera's optimized I/O engine maximizes the read performance of raw sonar files on all platforms (Windows, Mac, Linux).

Coordinate System Extraction

When loading QINSy (.db) files, the intended project coordinate system will be extracted and if the file is the first in the project and Auto was selected during project creation, Qimera will set the project coordinate system to this coordinate system.  Qimera will also selected the input navigation datum information to construct the coordinate system of the incoming navigation.  For other file formats, if the Auto option was selected, Qimera will choose the nearest UTM zone as the project coordinate system upon import of the first raw sonar file into the project.  This data is persisted in the project.xml file.

Vessel Information Extraction

Vessel file information such as sensor linear and angular offsets is extracted and stored as .vessel files in the line metadata folder.  If the vessel name is new, it will be added to the overall vessel list in the Qimera project.

Computation Settings Extraction

Certain raw sonar file types may have redundant time series data streams.  The computation settings stores the choices made by the acquisition system as to which of these are the primary, secondary, etc. sources to use during processing.  These file types are stored as.echo_##files in the line metadata folder.  The "##" in the file name is the system id.

Bathymetry Start/End Time Extraction

Qimera stores the bathymetry first and last packet times in the project.xml file.

Sound Velocity Profile Extraction

Qimera extracts and stores all sound velocity profiles in a generic binary format with the .bsvp_## extension in the line metadata folder.  The "##" is the SVP number in the file

Time Series Data Extraction

Qimera extracts the following time series data to the listed file extensions when available.  This data is extracted and stored at full resolution in the line metadata folder.  The "##" in each file name is the system id.

  • Position (.position_##)
  • Position Standard Deviation (.positionsd_##)
  • Motion (.motion_##)
  • Motion Standard Deviation (.motionsd_##)
  • Heading (.heading_##)
  • Heading Standard Deviation (.headingsd_##)
  • Height (.height_##)
  • Height Standard Deviation (.heightsd_##)
  • Surface Sound Speed (.soundspeed_##)
  • Pressure (.pressure_##)
  • Depth (.depth_##)
  • Tide (.tide_##)
  • Dynamic Squat (.squat_##)
Processing Settings Extraction

This file will maintain all the processing settings that the user selects when for the file using the Processing Settings dialog. It is created during metadata extraction and intelligent options are made based on the metadata extracted from the file.

Time Series Solution Creation

Since Qimera supports multiply redundant sources of data for the same time series attribute, the solution files (timeseriesname.sol)  maintains the priority order of the particular attribute.

QPD Matching

A description of this process can be found here.

WCD Matching

A description of this process can be found here.

Track Line Creation

If Qimera was able to match a QPD file to the raw sonar file (.db), it will create a track line based on the port-side sonar transducer location.  If a QPD does not exist, the track line is created from the primary position data time series.  The track line object is in the Fledermaus SD (.sd) format.

Binary Bathymetry Conversion

When Qimera loads raw sonar data in ASCII format such as that found in Hypack (.hsx) files, it will automatically convert the bathymetry data to an internal binary format for faster multi-core processing during sound velocity correction.

Processed Point Files

Processed Point Files extract a .metadata.xml file which contains information required by Qimera to work with the file.  Qimera will also create a track line or extents object in the Fledermaus SD (.sd) format. 

Navigation Files

Navigation files follow a similar process as raw sonar files.  The following steps are performed and described above.

  • File Indexing
  • Time Series Data Extraction
  • Track Line Creation

Tide Files

Tide files have the Time Series Data Extraction step described above performed on their data.

SVP Files

SVP Files are imported and automatically converted to the Qimera .bsvp format. 

  

Return to: Dynamic Workflow

Return to: Qimera Table of Contents