ESA Earth Home Missions Data Products Resources Applications
    24-Jul-2014
EO Data Access
How to Apply
How to Access
Index
MIPAS Data Formats Products
Records
2 MDSR per MDS 1 forward sweep 1 reverse sweep
2 MDSRs per MDS 1 forward sweep 1 reverse sweep
LOS calibration GADS
Spectral Lines MDS
P T Retrieval MW ADS
VMR Retrieval Parameters GADS
P t Retrieval GADS
Framework Parameters GADS
Processing Parameters GADS
Inverse LOS VCM matrices MDS
General GADS
Occupation matrices for vmr#1 retrieval MDS
MDS2 -- 1 mdsr forward sweep 1 mdsr reverse
Occupation matrices for p T retrieval MDS
General GADS
Priority of p T retrieval occupation matices
P T occupation matrices ADS
MIP_NLE_2P SPH
Summary Quality ADS
Instrument and Processing Parameters ADS
Microwindows occupation matrices for p T and trace gas retrievals
Scan information MDS
Level 2 product SPH
MDS1 -- 1 mdsr forward sweep 1 mdsr reverse sweep
H2O Target Species MDS
P T and Height Correction Profiles MDS
Continuum Contribution and Radiance Offset MDS
Structure ADS
Summary Quality ADS
Residual Spectra mean values and standard deviation data ADS
PCD Information of Individual Scans ADS
Instrument and Processing Parameters ADS
Microwindows Occupation Matrices ADS
Scan Information MDS
1 MDSR per MDS
Scan Geolocation ADS
Mipas Level 1B SPH
Calibrated Spectra MDS
Structure ADS
Summary Quality ADS
Offset Calibration ADS
Scan Information ADS
Geolocation ADS (LADS)
Gain Calibration ADS #2
Gain Calibration ADS #1
Level 0 SPH
DSD#1 for MDS containing VMR retrieval microwindows data
DSD for MDS containing p T retrieval microwindows data
VMR #1 retrieval microwindows ADS
P T retrieval microwindows ADS
1 MDSR per MDS
VMR profiles MDS (same format as for MIP_IG2_AX)
Temperature profiles MDS (same format as for MIP_IG2_AX)
Pressure profile MDS (same format as for MIP_IG2_AX)
P T continuum profiles MDS (same format as for MIP_IG2_AX)
GADS General (same format as for MIP_IG2_AX)
Level 0 MDSR
Values of unknown parameters MDS
Computed spectra MDS
Jacobian matrices MDS
General data
Data depending on occupation matrix location ADS
Microwindow grouping data ADS
LUTs for p T retrieval microwindows MDS
GADS General
P T retrieval microwindows ADS
ILS Calibration GADS
Auxilliary Products
MIP_MW1_AX: Level 1B Microwindow dictionary
MIP_IG2_AX: Initial Guess Profile data
MIP_FM2_AX: Forward Calculation Results
MIP_CS2_AX: Cross Sections Lookup Table
MIP_CS1_AX: MIPAS ILS and Spectral calibration
MIP_CO1_AX: MIPAS offset validation
MIP_CL1_AX: Line of sight calibration
MIP_CG1_AX: MIPAS Gain calibration
MIP_SP2_AX: Spectroscopic data
MIP_PS2_AX: Level 2 Processing Parameters
MIP_PS1_AX: Level 1B Processing Parameters
MIP_PI2_AX: A Priori Pointing Information
MIP_OM2_AX: Microwindow Occupation Matrix
MIP_MW2_AX: Level 2 Microwindows data
MIP_CA1_AX: Instrument characterization data
Level 0 Products
MIP_RW__0P: MIPAS Raw Data and SPE Self Test Mode
MIP_NL__0P: MIPAS Nominal Level 0
MIP_LS__0P: MIPAS Line of Sight (LOS) Level 0
Level 1 Products
MIP_NL__1P: MIPAS Geolocated and Calibrated Spectra
Level 2 Products
MIP_NLE_2P: MIPAS Extracted Temperature , Pressure and Atmospheric Constituents Profiles
MIP_NL__2P: MIPAS Temperature , Pressure and Atmospheric Constituents Profiles
Glossary
References
Glossaries of technical terms
Level 2 processing
Pointing
Miscellaneous hardware and optical terms
Spectrometry and radiometry
Data Processing
Alphabetical index of technical terms
Frequently Asked Questions
The MIPAS Instrument
Inflight performance verification
Instrument characteristics and performances
Preflight characteristics and expected performances
Subsystem description
Payload description and position on the platform
MIPAS Products and Algorithms
Data handling cookbook
Characterisation and calibration
Calibration
Latency, throughput and data volume
Auxiliary products
Level 2
Instrument specific topics
Algorithms and products
Level 2 products and algorithms
Products
The retrieval modules
Computation of cross-sections
Level 1b products and algorithms
Products
Calculate ILS Retrieval function
Level 1a intermediary products and algorithms
Product evolution history
Definition and convention
MIPAS Products User Guide
Image gallery
Further reading
How to use MIPAS data?
Summary of applications and products
Peculiarities of MIPAS
Geophysical coverage
Principles of measurement
Scientific background
MIPAS Product Handbook
Services
Site Map
Frequently asked questions
Glossary
Credits
Terms of use
Contact us
Search


 
 
 


2.4.4 Level 2 products and algorithms

2.4.4.1 Algorithms

2.4.4.1.1 Introduction

The middle infrared emission spectra measured by MIPAS (output of Level 1b processor 2.4.3.2. ) contain features of most atmospheric constituents. Therefore, a series of spectra measured in the limb-scan configuration can be processed to determine the volume mixing ratio (VMR) profiles of numerous atmospheric trace species. Since middle infrared emission spectra are strongly sensitive to temperature, and in general limb observations are strongly affected by the observation geometry (that, in Level 2 processing is usually identified by the value of pressure at tangent altitudes, 'tangent pressure'), a correct interpretation and analysis of the observed spectra for the retrieval of the atmospheric constituents requires a good knowledge of these quantities, which have to be determined for each limb scan sequence.
The retrieval of pressure and temperature (p, T), as well as the VMR of five high priority species, namely O3, H2O, HNO3, CH4, N2O and NO2 will be routinely performed in near real time (NRT). The retrieval of these parameters from calibrated spectra (provided by Level 1b processor 2.4.3.2. ) is indicated as NRT Level 2 processing.
The requirement for NRT analysis is very demanding because of both the time constraints (short delay between measurement and processing, and computing time shorter than measurement time) and the need for a validated algorithm capable of producing accurate and reliable results in an automated operative mode.
The main functional components of the Level 2 processor are:

The pre-processor:
is the software that manages the environment in which the retrieval modules are run. Typical tasks of this code are: selection of the observations to be processed in the subsequent retrievals, preparation of inputs (including all auxiliary data) needed by the retrieval modules, calculation of the quantities that must be calculated only once for processed scan, preparation and formatting of the output files (Level 2 products).
Go here 2.4.4.1.2. for further details on the Level 2 pre-processor.

The retrieval modules:
Satrting from the inputs prepared by the pre-processor (i.e. from selected intervals of the calibrated and apodized spectra calculated in Level 1b, a set of auxiliary data and processing setup parameters) these modules perform p,T and VMR retievals.
The retrieval modules have been implemented (in the ENVISAT Payload Data Segment) by industry on the basis of the algorithms defined in the frame of a scientific study. In this study, a "scientific" version of the retrieval code has been developed, optimized for the requirements of speed and accuracy. This code is called Optimized Retrieval Model (ORM) and includes p, T and VMR retrieval components. The objective of the study was to develop physical and mathematical optimisations of a baseline retrieval scheme and to validate them against a set of test scenarios. A summary description of the algorithms implemented in the retrieval modules is reported here 2.4.4.1.3.1. . Full details of these algorithms are reported both in the Level 2 Algorithm Theoretical Baseline Document (ATBD) 2.4.4.1.1.1. and in Ridolfi M.and CO Ref. [1.58 ] .
 

2.4.4.1.1.1 Level 2 Algorithms Theoretical Baseline Document (ATBD)

Click here to download a .pdf file (1.9 Mb) containing the MIPAS Level 2 ATBD.


2.4.4.1.2 The Level 2 pre-processor

Tasks performed by the the Level 2 Framework Processor
The overall loop structure for the processing of one product is All tasks except the p,T retrieval and the VMR retrievals are performed by the framework processor modules, while the retrieval modules are collected in the retrieval component library (RCL). 
Framework Processor Flowdiagram
Figure 2.19 Framework Processor Flowdiagram

Preprocessing
Preprocessing consists of the following steps:
Preprocessor Flowdiagram
Figure 2.20 Preprocessor Flowdiagram

Check Health of Level 1b Data The purpose of this function is to determine the spectral data of each scan, that may be used for the retrievals.

All information on quality of MIPAS measurement data is contained in the Level 1b product. In a first step the quality indicator (PCD) contained in the  Measurement Data Structure (MDS) of the Level 1b product is evaluated for each measured tangent altitude (sweep) of the scan. If health checking proofs that spectral data related to a particular tangent altitude (sweep) is corrupted, a logical flag will be set equal to "false" indicating that those spectral data are not used during p,T retrieval and VMR retrievals for this scan. Corruption of spectral data may concern a single spectral band or all spectral bands (in total five bands).

Beside quality indicators (PCD), the MDS-DSR contains information on uncorrected / remained spikes in interferograms. This information is also evaluated in order to identify spectral bands corrupted by spectral artifacts caused by possible spiking (e.g. by cosmic radiation) occurred during recording of  interferograms onboard the instrument. Spectral bands corrupted by spikes are also flagged by setting a logical variable equal to "false" so that no spectral data related to those spectral bands will be extracted for p,T and VMR retrieval.

Output of this function are two logical vectors which are evaluated by the function  "Selection of Microwindows".
 

Selection of Microwindows The purpose of this function is to select a optimized set of  microwindows 2.4.4.1.3.1.   for each retrieval and each scan.
 
The information on a variety of spectral intervals valid for p,T and VMR retrieval, called microwindows (MW´s), is stored in the microwindow database 2.4.4.4. . A dedicated set of MW´s to be used for each retrieval and scan is called a MW a occupation matrix. Those latitude dependent occupation matrices are stored in the occupation matrix database 2.4.4.4. (MIP_OM2_AX) which may be replaced occasionally to take into account seasonal effects or different measurement scenarios. The matrix elements of each occupation matrix identify which MW (identified by the columns) at which sweep (identified by rows) shall be selected and thus the matrix elements of an occupation matrix uniquely identify the spectral data points to be extracted from the Level 1b product.

The algorithm will select the first occupation matrix in the file which :

  • is valid for the scans latitude
  • has the correct number of sweeps
  • is valid for the altitudes of sweeps
  • and does not make use of corrupted data (see  Check Health of Level 1b Data ).
If  no valid occupation matrix is found the corresponding retrieval of the scan will be skipped. If no valid occupation matrix for the p,T retrieval is found , all retrievals of the scan will be skipped.
Table 2.1 Example of an Occupation Matrix
Sweep\Microwindow PT__0169 PT__0175 PT_0218
Sweep 0  FALSE TRUE TRUE
Sweep 1 FALSE TRUE TRUE
Sweep 2 FALSE TRUE TRUE
Sweep 3 TRUE TRUE TRUE
Sweep 4 TRUE FALSE TRUE
Sweep 5 FALSE FALSE TRUE
Sweep 6 FALSE FALSE TRUE
Sweep 7 TRUE TRUE TRUE
Sweep 8 TRUE TRUE TRUE
Sweep 9 TRUE TRUE TRUE
Sweep 10 TRUE TRUE FALSE
Sweep 12 TRUE TRUE FALSE
Sweep 13 TRUE TRUE FALSE
Sweep 14 FALSE TRUE FALSE
Sweep 15  FALSE TRUE FALSE

To each occupation matrix a logical retrieval vector is associated. This vector defines the altitudes of the retrieved profile. Usually this is foreseen to be identical to the altitudes of the sweeps, but  in general case it may be only a subset.
 
 

Note: The occupation matrix database references the used microwindows by name (label). The information defining the microwindows themselves is contained in the microwindow database. Therefore the occupation matrix database and microwindow database are closely linked, and it is essential to select a consistent pair of them for each processing run.
Extraction of Spectral Data from the Level 1b Product This function extracts for each sweep the spectral data points related to the cut-off wavenumbers (first and last wavenumber) of the microwindows plus some additional data points which are needed to perform resampling (if necessary) and apodisation of spectral data.

The maximum resolution of the MIPAS instrument is the inverse of the maximum path difference of the interferometer ( 1/40 cm = 0.025 cm-1).  The instrument can also be operated with a smaller optical path difference, which will allow to increase the possible number of sweeps within a scan. Spectral resolution will be lower in this case. On the other hand the retrieval modules need to receive the spectral data always on a fixed (PS2 setting) grid called the "general coarse wavenumber grid". Therefore  resampling of level 1b spectral data is necessary if the spectral grid of input spectra deviates from the general coarse wavenumber grid. Nominally, the spectral grid of level 1b spectra and the general coarse wavenumber grid are identical, resampling is expected being a non routinely performed operation.

If the spectral grid of Level 1b spectra deviates from the general coarse wavenumber grid the following two cases have to be considered:

  • Input spectra are given on a spectral grid which is a multiple integer of the general coarse wavenumber grid. In this case input spectra simply need to be undersampled, i.e. one sample is taken out of N samples.
  • Input spectra are given on a spectral grid which is not a multiple integer of the general coarse wavenumber grid. In this case spectral interpolation of Level 1b input spectra to the general coarse wavenumber grid is performed using an apodised sinc interpolation function.
Apodisation of  Selected Spectral Data All observed spectral data is the result of the convolution of the instrument line shape with the atmospheric spectra entering the instrument. The instrument line shape (ILS) is a function looking similar  to a  sinc eq. 5.3 -function, hence has sidelobes  with significant impact on the convolution result. As a consequence the forward model of the retrievals would have to consider spectral lines, which are many wavenumbers away from the selected microwindows. To avoid this, the influence of the sidelobes is suppressed by a convolution of the observations with an apodisation function. The resulting apodised observation is identical to the convolution of the atmospheric spectra entering the instrument with the apodised instrument line shape AILS.
Example of ILS and AILS
Figure 2.21 Example of ILS and AILS

To summarize :

The spectra as found in the level 1b product are :

Observed spectra = Atmospheric Spectra x ILS eq 2.3
Apodisation leads to:
Apodised Spectra =  (  Atmospheric Spectra x ILS ) x Apodisation Function eq 2.4


During the retrievals least square fit the simulated apodised spectra are computed according to:

Simulated Apodised Spectra =  Simulated Atmospheric Spectra x (ILS  x Apodisation Function) eq 2.5
where ( ILS x Apodisation Function)  is called the  AILS.
 
   Compute Apodised Instrument Lineshape for each Microwindow As explained in the section  "Apodisation of Selected Spectral Data" the retrieval modules will have to convolute the simulated atmospheric spectra with the apodised instrument lineshape AILS.  The AILS is computed by the preprocessor using auxiliary data from the PS2 file. The exact shape of AILS is a function of the wavenumber. The change of the AILS shape within the small range of wavenumbers within one microwindow can be neglected, while the change from one microwindow the next has to be considered. Therefore one AILS is computed for each microwindow using the microwindows central wavenumber.
The AILS is computed in two steps:
  • Compute the ILS
  • Convolute the ILS with the Apodisation Function
where the apodisation function must be identical to the one use for apodisation of the spectral data. The MIPAS level 2 processor uses the Norton Beer function that purpose (see  Norton R. H., R.Beer Ref. [1.54 ] ).
  Compute Variance Covariance Matrix of the Observations The least square fit performed during the retrievals will take into account the accuracy of the measurements, allowing bigger deviations between simulation and observation where variance is big and small deviations where the variance is small. The standard deviation of each measurement is defined by the NESR , which is reported in the level 1b product as function of wavenumber. The measurements at different wavenumbers are assumed to be independent, but the processing step of Apodisation of Selected Spectral Data    introduces a correlation between the apodised spectral data points used by the retrieval. Another correlation is added in the case that the measurement have to interpolated to the nominal processing grid (see Extraction of Spectral Data from the Level 1b Product). Therefore the Variance Covariance Matrix of the Observations  VCMobs has non zero off-diagonal elements. Since the preprocessing does not introduce correlations between spectral data of different microwindows or altitudes the VCM of the observations holds non zero values only in small quadric sub matrices along the main diagonal. Each sub-matrix is the VCM of one microwindow at a certain altitude. Each of this sub matrices is computed by preprocessor and passed to the retrieval modules.
 
 
 
Note: The VCM of the observations is not the VCM reported in the level 2 product. The level 2 product contains the variance and covariance data of the retrieved quantities.

2.4.4.1.2.1 The Initial Guess Processor

The first iteration of each retrieval starts with profiles called "initial guess". During the retrieval these profiles will be iteratively updated such that the deviation from simulated spectra to observed spectra is minimized.
The purpose of the initial guess processor is to provide an optimized set of initial guess profiles in order to reduce the number of iterations needed during the retrieval. Since the runtime of the retrieval is directly proportional to the number of iterations, it is of major importance to find a  good initial guess.

Please note that for each retrieval only a subset of the initial guess profiles in varied, while the others are fixed. In the p,T retrieval the VMR profiles will not be modified, but used. In each VMR retrieval only one VMR species is retrieved, while p,T, altitude and the other VMR profiles a are kept fixed. The p,T retrieval will be the first retrieval for each scan, the order of the other VMR retrievals can be defined by PS2 settings (see also  here 2.4.4.1.3.1. ).
For the details of the initial guess processor see  Framework-DPM section.

Input  to the Initial Guess Profiles

The initial guess processor uses several sources of profiles to compute the best initial guess set of profiles for each retrieval. They are listed and explained in this section.
IG2 file
Contains climatological profiles ( p,T,VMR and continua ). Profiles are different for different latitude bands (altitude range: 0-120 km). They are generated from long term climatological observations. Profiles from this source will be called IG2 profiles in the following.
ECMWF files
Contain profiles for geopotential, temperature, relative humidity and ozone on a fixed pressure grid. Geopotential can be converted to altitude (altitude range:~ 0-25.5 km), relative humidity to H20-VMR. ECMWF profiles depend on latitude and longitude. Each ECMWF profile may be flagged invalid independently of the other profiles.  ECMWF profiles are delivered by the European Center of Midrange Weather Forecast every 6 hours, each profile type in a separate file. Profiles from this source will be called ECMWF  profiles in the following.
FM2 (Forward Model File)
The idea of the Forward Model File is, that for a given initial guess from IG2 file  and given settings from PS2 auxiliary file the simulated spectra computed in the first iteration of  p,T retrieval are determined. Therefore they can be pre-computed off-line.

The FM2 file contains precomputed  spectra, the jacobian 2.4.4.1.3.3.2.   and microwindow grouping for a standard p,T retrieval scenario (nominal occupation matrix). Furthermore it contains the full  IG2  data used to generate the FM2 file. Spectra and jacobian 2.4.4.1.3.3.2.   are computed for different altitudes in this file. When the FM2 is used, interpolation to the current altitudes is performed. FM2 information is latitude dependent. Most probably, FM2 usage saves the first iteration.  FM2 files contain a reference to the PS2 settings -file used, when the FM2 file was generated. If this file does not agree with the PS2 file used in the current  retrieval, FM2 usage will be rejected. Whenever a FM2 file is selected to be used by the workorder file of MIPAS level 2 processor,   IG2  information must be taken from this file. Therefore FM2 file may be thought of  as an enhanced  IG2 file . The processing of one product may  use either  IG2 file  or FM2 file. For details of FM2 usage see also Framework-DPM.

Dataflow of Initial Guess Processor
Figure 2.22 Dataflow of Initial Guess Processor

Retrieved Profiles
The most recent information available on the pressure, temperature and VMR profiles are the profiles retrieved by the MIPAS level 2 processor itself. Therefore the initial guess processor is able to make use of its own retrieval results.
Retrieved profiles may originate from the same scan (but a preceding retrieval) as the current retrieval or from any preceding processed scan of the current product. Because of the usage of the retrieved profiles, the Initial Guess Processor must not be part of the preprocessing, but part of the processing main loop.

Initial Guess Processor Algorithm

The initial guess processor strategy is that the source of initial guess should be as near in time and space (latitude/longitude) as possible to the currently processed scan. Therefore  retrieved profiles are preferred to  ECMWF profiles, and ECMWF  profiles are preferred to  IG2  profiles. Only in cases of p,T retrieval with no  retrieved profiles and no ECMWF data available precomputed spectra (FM2) shall be used.
Merging of Profiles
ECMWF profiles do not cover the full altitude range needed and have to be extended with scaled IG2 data. This operation will be called "merging".
A Priori Profiles:
If  ECMWF data is  used "a priori profiles" are the result of merging IG2 profiles with ECMWF data. If only IG2 profiles are used, "a priori profiles" are identical to IG2 profiles.
Usage of Retrieved Profiles
When retrieved profiles are available three cases have to be considered:
  • An initial guess for pressure, temperature and altitude is needed for VMR retrieval.

  • In nominal processing this shall always be the result of the p,T retrieval of the current scan. The retrieved p,T and altitude profile will be used without change. If p,T retrieval fails the VMR retrievals of the current scan shall be skipped.
  • An initial guess for a VMR profile is needed for VMR or p,T retrieval.

  • The initial guess will be computed as optimum estimate from retrieved profile and a priori profile. Optimum estimate is a weighted  averaging using the VCM of the retrieved profile , and a VCM computed from PS2 settings for the a priori profile.
  • An initial guess is needed for pressure, temperature and altitude for a p,T retrieval.

  • In this case temperature profile will be computed by optimum estimate. For pressure no computations are performed, because pressure will be used as the free parameter to define the grid. Altitudes will be computed  using  hydrostatic equilibrium  and one altitude value from a priori profile.
Initial Guess for Continuum Profiles
Initial guess continuum profiles will always be taken from IG2 (FM2), because :
-  ECMWF data does not contain information for continua
-  Usage of retrieved continua is difficult, because of their strong altitude dependency and the fact of necessary exponential (= linear with pressure) extrapolation/interpolation for "holes" in occupation matrix and "holes" caused by micro-window grouping.

Go back to  Mipas level 2 processing  introduction 2.4.4.

Keywords: ESA European Space Agency - Agence spatiale europeenne, observation de la terre, earth observation, satellite remote sensing, teledetection, geophysique, altimetrie, radar, chimique atmospherique, geophysics, altimetry, radar, atmospheric chemistry