ESA Earth Home Missions Data Products Resources Applications
    24-Jul-2014
EO Data Access
How to Apply
How to Access
Index
MIPAS Data Formats Products
Records
2 MDSR per MDS 1 forward sweep 1 reverse sweep
2 MDSRs per MDS 1 forward sweep 1 reverse sweep
LOS calibration GADS
Spectral Lines MDS
P T Retrieval MW ADS
VMR Retrieval Parameters GADS
P t Retrieval GADS
Framework Parameters GADS
Processing Parameters GADS
Inverse LOS VCM matrices MDS
General GADS
Occupation matrices for vmr#1 retrieval MDS
MDS2 -- 1 mdsr forward sweep 1 mdsr reverse
Occupation matrices for p T retrieval MDS
General GADS
Priority of p T retrieval occupation matices
P T occupation matrices ADS
MIP_NLE_2P SPH
Summary Quality ADS
Instrument and Processing Parameters ADS
Microwindows occupation matrices for p T and trace gas retrievals
Scan information MDS
Level 2 product SPH
MDS1 -- 1 mdsr forward sweep 1 mdsr reverse sweep
H2O Target Species MDS
P T and Height Correction Profiles MDS
Continuum Contribution and Radiance Offset MDS
Structure ADS
Summary Quality ADS
Residual Spectra mean values and standard deviation data ADS
PCD Information of Individual Scans ADS
Instrument and Processing Parameters ADS
Microwindows Occupation Matrices ADS
Scan Information MDS
1 MDSR per MDS
Scan Geolocation ADS
Mipas Level 1B SPH
Calibrated Spectra MDS
Structure ADS
Summary Quality ADS
Offset Calibration ADS
Scan Information ADS
Geolocation ADS (LADS)
Gain Calibration ADS #2
Gain Calibration ADS #1
Level 0 SPH
DSD#1 for MDS containing VMR retrieval microwindows data
DSD for MDS containing p T retrieval microwindows data
VMR #1 retrieval microwindows ADS
P T retrieval microwindows ADS
1 MDSR per MDS
VMR profiles MDS (same format as for MIP_IG2_AX)
Temperature profiles MDS (same format as for MIP_IG2_AX)
Pressure profile MDS (same format as for MIP_IG2_AX)
P T continuum profiles MDS (same format as for MIP_IG2_AX)
GADS General (same format as for MIP_IG2_AX)
Level 0 MDSR
Values of unknown parameters MDS
Computed spectra MDS
Jacobian matrices MDS
General data
Data depending on occupation matrix location ADS
Microwindow grouping data ADS
LUTs for p T retrieval microwindows MDS
GADS General
P T retrieval microwindows ADS
ILS Calibration GADS
Auxilliary Products
MIP_MW1_AX: Level 1B Microwindow dictionary
MIP_IG2_AX: Initial Guess Profile data
MIP_FM2_AX: Forward Calculation Results
MIP_CS2_AX: Cross Sections Lookup Table
MIP_CS1_AX: MIPAS ILS and Spectral calibration
MIP_CO1_AX: MIPAS offset validation
MIP_CL1_AX: Line of sight calibration
MIP_CG1_AX: MIPAS Gain calibration
MIP_SP2_AX: Spectroscopic data
MIP_PS2_AX: Level 2 Processing Parameters
MIP_PS1_AX: Level 1B Processing Parameters
MIP_PI2_AX: A Priori Pointing Information
MIP_OM2_AX: Microwindow Occupation Matrix
MIP_MW2_AX: Level 2 Microwindows data
MIP_CA1_AX: Instrument characterization data
Level 0 Products
MIP_RW__0P: MIPAS Raw Data and SPE Self Test Mode
MIP_NL__0P: MIPAS Nominal Level 0
MIP_LS__0P: MIPAS Line of Sight (LOS) Level 0
Level 1 Products
MIP_NL__1P: MIPAS Geolocated and Calibrated Spectra
Level 2 Products
MIP_NLE_2P: MIPAS Extracted Temperature , Pressure and Atmospheric Constituents Profiles
MIP_NL__2P: MIPAS Temperature , Pressure and Atmospheric Constituents Profiles
Glossary
References
Glossaries of technical terms
Level 2 processing
Pointing
Miscellaneous hardware and optical terms
Spectrometry and radiometry
Data Processing
Alphabetical index of technical terms
Frequently Asked Questions
The MIPAS Instrument
Inflight performance verification
Instrument characteristics and performances
Preflight characteristics and expected performances
Subsystem description
Payload description and position on the platform
MIPAS Products and Algorithms
Data handling cookbook
Characterisation and calibration
Calibration
Latency, throughput and data volume
Auxiliary products
Level 2
Instrument specific topics
Algorithms and products
Level 2 products and algorithms
Products
The retrieval modules
Computation of cross-sections
Level 1b products and algorithms
Products
Calculate ILS Retrieval function
Level 1a intermediary products and algorithms
Product evolution history
Definition and convention
MIPAS Products User Guide
Image gallery
Further reading
How to use MIPAS data?
Summary of applications and products
Peculiarities of MIPAS
Geophysical coverage
Principles of measurement
Scientific background
MIPAS Product Handbook
Services
Site Map
Frequently asked questions
Glossary
Credits
Terms of use
Contact us
Search


 
 
 


2.4.4.1.3.3.1.3 Computation of cross-sections

The computation of cross-sections in equation is a very time consuming part of the forward model, due to the large number of spectral lines to be considered, the high spectral resolution required and the number of p, T combinations for which they have to be computed.
In the code the cross-sections can either be calculated line by line (LBL), using a pre-selected spectroscopic database and fast Voigt profile computation, or be read from look-up tables (LUTs).
The use of LUTs is advantageous only if they can be stored in random access memory (RAM), and the feasibility of this depends on the amount of memory required. Cross-section LUTs have to be tabulated for all the considered microwindows, for each absorber g, and for an appropriate range of p,T values. For a single microwindow (typical width 0.5 cm-1), the number of data points required is of the order of:

1000 spectral pts. x 100 pressures x 10 temperature pts. = 106 grid pointseq 2.24
A pictorial representation of a typical table is shown in figure2.25 . For a retrieval, up to 100 such tables have to be stored (tens of microwindows with a few absorbers per microwindow), requiring too much memory for most practical applications.
In order to overcome this difficulty, the solution suggested by Strow in Strow L. L., H. E. Motteler Ref. [1.62 ] was adopted. It consists of compressing this information using `Singular Value Decomposition'. Any matrix K (m x n) can be decomposed as the product of three other matrices:
image eq 2.25
where U (m x n) and V (n x n) are orthonormal matrices, and image is a diagonal matrix containing n singular values. Assuming that most of the information is contained in the j (<< n) largest singular values, the n dimension of the decomposition matrices can be truncated to give:
image eq 2.26
where the reduced matrices U' (m x j) and W' (j x n) are much smaller matrices than the original matrix K.
In this application, the matrix K contains the logarithm of the cross-section and is decomposed and stored as matrices U'(image ,j) and W'(j,x), where x represents any pair of (p,T) conditions. Typically the number j of singular values is less than 10, giving compression factors of the order of 10 - 100.
The number of singular values, as well as the number of pressure and temperature increments, are optimized so that the maximum difference between the resulting LUT and LBL convolved limb radiance is below the microwindow radiance noise criteria (equal to NESR/10). In figure2.26 the differences between LUT and LBL limb radiances before and after AILS convolution are reported for a representative microwindow.
The combined use of irregular grids and LUTs means that the absorption cross-sections need only be reconstructed at a subset of fine grid points. Therefore the m dimension is effectively reduced by an order of magnitude, which usually makes U' the smaller of the two components of the LUT.
image
Figure 2.25 Plot of the logarithm of the CO<sub>2</sub> cross-section (k) tabulated for the spectral interval 693.45-693.725 cm<sup>-1. Each of the 12 major cycles in p, T axis corresponds to a different temperature, and within each are 6 different -ln) values varying from Lorentz to Doppler broadening.

image
Figure 2.26 44 km tangent height radiance from a microwindow (frequency range 693.45 - 693.725 cm-1) selected for p,T retrieval. The upper plot shows the high-resolution radiance obtained with LBL calculation of cross-section (solid line) and with the use of LUTs (dotted line), as well as the convolved radiance (dashed line for LBL calculation and (+) for use of LUTs). The lower plot shows the difference between the two methods, before AILS convolution (solid line, left axis) and after AILS convolution (dashed line, right axis).

2.4.4.1.3.3.1.4 Computation of the radiative transfer integral

The radiative transfer integral (equation) is computed as a summation over discrete layers as follows:

image eq 2.27
where τ l,t is the single layer optical depth:
image eq 2.28
l = 1 and l = N (N is equal to twice the number of layers) are respectively the index of the farthest and the nearest integration step along the line of sight with respect to the observer, t is the index of the limb view, Bl,t is the Planck function computed for the equivalent temperature of the main gas of the retrieval in layer l and limb view t.image and imageare respectively the absorption cross section and the column of the gas g.
The second term of equation eq. 2.28 is the contribution to the single layer optical depth of all emission sources characterized by a constant amplitude in a microwindow, the so-called 'atmospheric continuum': imageis an altitude and microwindow dependent absorption cross-section, that is fitted by the retrieval program,imageis the air column.
Since the atmosphere is assumed horizontally homogeneous and with a spherical symmetry, the computation of equation eq. 2.27 is further accelerated taking into account that the two contributions of the same layer from opposite sides of the tangent point are characterized by different transmissions, but by the same emission.
Therefore equation eq. 2.27 becomes:
image eq 2.29
This expression is computed at the irregular frequency grid and at the grid of the simulated tangent altitudes, i.e. the grid containing the measured limb views 5.3.5. and the additional ones needed for the FOV convolution (see section 2.4.4.1.3.3.1.6. ).
The zero-level calibration correction (caused by self-emission of the instrument, scattering of light into the instrument or third order non-linearity of the detectors) has to be finally applied to the spectrum. This is performed adding to the atmospheric spectrum equation eq. 2.29 a microwindow dependent (but tangent altitude independent) offset which is then fitted by the retrieval program.

2.4.4.1.3.3.1.5 AILS convolution

The Instrument Line Shape (ILS) function is, by definition, the response of the spectrometer to a monochromatic radiance. In the case of a perfect Fourier transform spectrometer, observing a stable source of evenly distributed radiance, the ILS is equal to the convolution of the sinc eq. 5.3 function, associated with the finite spectral resolution of the instrument, with a term due to the finite angular aperture. If the angular aperture is circular, this term is equal to a rectangular function shifted in wavenumber and with a width that varies linearly with the wavelength Bell R. J. Ref. [1.2 ] . In a real instrument alignment errors and irregular angular aperture lead to a more elaborate ILS. Furthermore, the uneven distribution of the source radiance could introduce the major complication of a frequency and tangent altitude dependent ILS distortion. The atmospheric radiance at the limb is indeed characterized by an exponential energy distribution through the input diaphragm. Fortunately, it has been demonstrated Delbouille L. and G. Roland Ref. [1.24 ] that even the strongest exponential energy distribution across the field of view does not significantly affect the ILS. The main effect of the non-uniform energy distribution across the input aperture is instead that a significant difference can exist between the effective tangent altitude and the geometrical one, and this effect is taken into account by FOV convolution (section 2.4.4.1.3.3.1.6. ).
The ILS function, which is an input of the forward model, is therefore a function independent of the tangent altitude and will take into account alignment and aperture effects, but not the instrument responsivity and phase error corrections, which are corrected for in the Level 1B 2.4.3.2. processing.
Measured MIPAS spectra are apodized before entering the retrieval process. This choice is dictated by the use of selected spectral intervals ('microwindows') for the retrieval of the VMR profiles. The advantages of using 'microwindows' have been already discussed in section 2.4.4.1.3.1.1. . The measured spectrum is equal to the convolution of the atmospheric spectrum with the ILS, and since the ILS is characterized by ' sidelobes' decreasing linearly in amplitude, the atmospheric signal at a given frequency affects the measured signal in a broad spectral interval. Therefore, the simulation of a narrow microwindow of the measured spectrum requires the calculation of a broad spectral interval. Apodization reduces the amplitude of the side lobes of the ILS and accordingly the size of the spectral interval in which the atmospheric spectrum must be calculated. It causes a loss of spectral resolution, but if a reversible apodization function is used, no information is lost. When the spectrum is convolved with the apodization function, correlation between different spectral points is introduced which appear as off-diagonal terms in the VCM of the observations.
The 'Norton-Beer strong' Norton R. H., R.Beer Ref. [1.54 ] function is used as apodizing function. The effects of the ILS and the apodization are simultaneously considered by convolving the atmospheric radiance with the apodized instrument line shape (AILS), obtained convolving the ILS with the apodization function.
The atmospheric radiance computed at the irregular grid points should first be interpolated on the regular fine grid (spacing 0.0005 cm-1) and then convolved with the AILS function in order to obtain the spectrum on the coarse grid (spacing 0.025 cm-1). Actually, these two operations are simultaneously performed skipping the step of interpolating the spectrum on the regular fine grid, with a saving in both computation time and memory requirements.

2.4.4.1.3.3.1.6 Instrument field of view convolution

The main effect of the exponential variation of the atmospheric radiance as a function of the limb angle through the input aperture is that a non-negligible difference exists between the signal observed along the central line of the FOV and the integrated signal.
The spread of the FOV in the altitude domain is measured experimentally and tabulated in a FOV pattern distribution FOV(z) whose shape is constructed with a linear interpolation from a tabulated function. The FOV pattern is assumed to be constant as a function of the scan angle.
The effect of field of view is taken into account performing, for each spectral frequency, the convolution between the tangent altitude dependent spectrum and the FOV pattern. This convolution requires the forward model calculation for a number of lines of sight that span the vertical range of the FOV around the tangent altitude. In order to reduce the number of computations, in the Level 2 algorithm, the variation of the spectrum as a function of tangent altitude is determined by interpolating a polynomial through the spectra calculated at contiguous tangent altitudes in the range of the FOV pattern.
Where the radiance profile varies rapidly with tangent altitude, additional spectra may be simulated at intermediate tangent levels to maintain the numerical accuracy of the convolution. In figure2.27 the values of the spectrum at a significant frequency calculated with an analytical convolution at altitudes between 9.5 and 12.5 km are plotted as a function of the corresponding values of the spectrum calculated with a reference numerical convolution. The results of the analytical convolution for both quartic and parabolic interpolation are shown for a critical case (in the atmospheric model used for these simulations, the tropopause is located at 11 km). The deviation of the curve from a straight line indicates the presence of a potential error in the computation of the analytical derivative of the spectrum with respect to the tangent pressure (section 2.4.4.1.3.3.2. ).
The simultaneous computation of the whole sequence of limb scanning spectra, which is required by the global fit approach, allows a simple and efficient computation of the FOV convolution and avoids the reiterated computation of spectra with adjacent tangent altitudes.

image
Figure 2.27 FOV convolved limb radiance values, at a significant frequency, calculated analytically at altitudes between 9.5 and 12.5 km versus corresponding reference values obtained by means of numerical convolution. The analytical convolution performed using a quadratic interpolation (made drawing a parabola through 3 spectra with tangent altitudes in the range of FOV pattern) is compared with the one obtained drawing a polynomial of the fourth degree (quartic interpolation) through 5 spectra with tangent altitudes in the FOV pattern range. The reference FOV convolved spectral values are computed simulating spectra at 100 m distant tangent altitudes. The deviation of the curves from a straight line indicates the presence of an error in the interpolated spectrum and hence of a potential error in the computation of the analytical derivatives.

2.4.4.1.3.3.2 Jacobian calculation

Another important part of the retrieval code is the fast determination of the derivatives of the radiance with respect to the retrieval parameters. In the following we will first present the five different types of derivatives which have to be computed and then explain the procedure implemented for their calculation. As already stated in section 2.4.4.1.3.1. , the retrieval parameters are: (1) volume mixing ratios of atmospheric trace gases at tangent pressures, (2) atmospheric continuum values at tangent pressures, (3) the tangent pressures themselves, (4) temperature at tangent pressures, and (5) the zero-level calibration correction. As described in section 2.4.4.1.3.4.4. , a retrieval grid is chosen which coincides with the tangent points. As a consequence, the number of parameters in (1), (3) and (4) is equal to the number of tangent points. By contrast, the atmospheric continuum (2) and the zero-level calibration correction (5) are assumed to be microwindow dependent, and the atmospheric continuum also tangent altitude dependent. Thus, the number of atmospheric continuum parameters is equal to the number of microwindows times the number of tangent altitudes at which the individual microwindows are used, while the number of zero-level calibration correction parameters is equal to the number of microwindows.
The numerical computation of the derivatives requires, for each retrieved parameter, an extra forward model calculation with an increment applied to that parameter. Whenever possible, derivatives are computed analytically in the sense that analytical formulas of the derivatives are implemented in the program (this is the case of the retrieval parameters (1), (2), (3) and (5)). Where the calculation of sufficiently precise analytical derivatives requires computations as time consuming as the calculation of spectra (parameters (4)), an optimized numerical procedure is implemented.
Derivatives with respect to parameters (1) and (2) are handled in a similar way. The aim is the calculation of the derivatives of the spectrum L provided by equation eq. 2.27 with respect to the atmospheric retrieval parameters xn on each tangent level n. The expression for these derivatives reads:

image eq 2.30
.
If we neglect the 2nd order dependence of equivalent temperatures on the parameters xn , the first term of (4.12) eq. 2.30 is equal to 0. The second term requires the calculation of the derivative of the radiance with respect to the optical depth and the derivative of the optical depth with respect to the retrieval parameter. The derivative of the radiance with respect to the optical depth is equal to:
image eq 2.31
In this expression, the first term is the derivative of the emission of layer j attenuated by all layers between j and the observer, while the second term is the derivative of the attenuation for the radiation of each layer up to layer j-1 and represents the emission spectrum measured by the observer due to the first (j-1) layers.
The derivative of the optical depth with respect to each parameter of the atmospheric continuum reads simply:
image eq 2.32
whereimageis the continuum cross section of the j-th layer and image the continuum parameter, i.e the continuum cross section value at the n-th tangent level relative to a given microwindow. Due to the limited influence of the single change of a continuum parameter on the whole continuum profile, the only terms imagewhich are different from 0 are the ones corresponding to layers between tangent levels (n+1) and (n-1).
The derivative of the optical depth with respect to the VMR parameters is equal to:
image eq 2.33
Neglecting the dependence of equivalent temperature and pressure on the VMR parameters it follows that image. During the calculation of the Curtis-Godson equivalent quantities, only the derivatives of the partial columns of each layer with respect to the VMR at the tangent level have to be determined.
The analytical derivative of the spectrum with respect to the tangent pressure (3) can be very complicated, since changes in the column, in the line-shapes and in the temperature have to be considered when tangent pressure is perturbed. The problem was overcome by exploiting the fact that the effect of FOV convolution is to shift the 'effective' tangent pressure of the spectrum. Therefore, the derivative of the spectrum with respect to tangent pressure is calculated performing an analytical derivative of the expression that provides the convolution between the spectrum interpolated as a function of tangent pressure and the FOV pattern. The accuracy of the derivative calculated in this way is strictly connected with the accuracy of the interpolated spectrum in the range where the FOV pattern is defined (figure2.27 ).
The temperature derivatives are determined in a 'fast numerical' way. In contrast with slow numerical derivatives, in the 'fast numerical' calculation the derivatives are computed in parallel with the spectra, avoiding unnecessary repeated calculations. The implemented fast numerical derivatives with respect to tangent temperature make use of the limited influence of the change of one temperature parameter on the overall temperature profile: only the Curtis-Godson equivalent temperatures of the neighboring layers above and below the tangent altitude (relating to the considered parameter) are affected. Hence, in addition to the calculation of cross-sections and radiative transfer corresponding to the unperturbed original temperature profile, needed for the forward calculation, only cross-sections relating to the layers affected by the current temperature parameter change and radiative transfer as modified by the changed cross-sections are computed.
The derivatives with respect to the zero-level calibration correction (5) are equal to 1, since the zero-level calibration correction is simply an additive term of the simulated spectrum.
With the adopted optimizations the additional computing effort required for the Jacobian calculation is only twice the effort needed for one forward model run. This is a very interesting result, considering that the full numerical calculation of the derivatives would require as many forward model re-runs as many are the fitting parameters (≈ 100).

2.4.4.1.3.3.3 Convergence criteria

The convergence criteria adopted in our code are a compromise between the required accuracy of the parameters and the computing speed of the algorithm. The following three convergence conditions are used to achieve this:

  • Condition on linearity: at the current iteration 'iter' the relative difference between the actual image and the expected value of chi-square computed in the linear approximation (image) must be less than a fixed threshold t 1:

  • image eq 2.34

    where image is computed using the expression:
    image eq 2.35

     
  • Condition on attained accuracy: the relative correction that has to be applied to the parameters for the subsequent iteration is below a fixed threshold t2 i.e.:

  • image eq 2.36
    .
    Different thresholds are used for the different types of parameters depending on their required accuracy. Furthermore, whenever an absolute accuracy requirement is present for a parameter, the absolute variation of the parameter is checked instead of the relative variation considered in equation eq. 2.36 . The non-target parameters of the retrieval, such as continuum and instrumental offset parameters are not included in this check. Parameters equal to zero are excluded from this check as well.
  • Condition on computing time: the maximum number of iterations must be less than a given threshold.
The convergence is reached if either condition equation eq. 2.34 or condition equation eq. 2.36 is satisfied. If only the condition on computing time is satisfied, the retrieval is considered unsuccessful.

2.4.4.1.3.4 Choices and assumptions in the forward and retrieval algorithms

In this section we report de description of specific issues regarding both the strategy adopted in the version algorithm and the simplifications / shourtcomings implemented in the forward model

2.4.4.1.3.4.1 Use of complementary information in the inversion model

MIPAS limb scanning retrievals have redundancy of measurements, such that stable vertical profiles of atmospheric state parameters can be retrieved without constraining the retrieval with a-priori knowledge. On the other hand, when some complementary information on the unknown parameters is available, the quality of retrieved parameters can be improved by including this information in the retrieval process.
The external information can be either of 'general' type (as in the case of climatological data or model forecasts) which, as such, may apply to several MIPAS observations made at different locations and at different time, or of 'specific' type, which relates to individual MIPAS scans.
The exploitation of 'general' information is not desirable because the same external information can be used for the retrievals of several sequences so that the results of these retrievals are affected by a common bias and can no longer be considered independent measurements. The correlation between subsequent profiles would add extra complexity in the use of the data in geographical maps and averages.
On the other hand, complementary information relating to the individual MIPAS sequences (e.g. line of sight data, or data relating to the same air mass actually sounded by the considered MIPAS scan) can be profitably included in the retrieval, as stated in section 2.4.4.1.3.2.1. , using equation instead of equation. In fact, engineering data defining the instrument Line Of Sight (LOS) are updated at each scan and therefore constitute an effective and 'specific' source of information which can be routinely used in p,T retrievals without introducing a bias in the retrieved profiles.
The LOS engineering information consists of both a vector z containing the tangent altitudes of the current scan and of a VCM V z related to the vector zA relationship between the engineering tangent heights and the unknowns of the p,T retrieval is provided by hydrostatic equilibrium constraint. This law, generally fulfilled in normal atmospheric conditions, especially in the stratosphere, provides a relationship between pressure, temperature and altitude. Assuming known pressure and temperature distributions and a reference altitude image, the altitude image relating to pressure image is calculated as:

image eq 2.37
where M is the average molecular weight of the atmosphere, R the universal gas constant, gj the acceleration of gravity relating to the j-th atmospheric layer (its dependence on altitude and latitude is derived from the WGS84 model Department of Defense Ref. [1.25 ] ) and image is the average temperature of the j-th atmospheric layer.
The engineering measurements of tangent altitudes are linked with the unknowns of the inversion problem (p, T ) by equation eq. 2.37 . In particular, equation eq. 2.37 can be locally linearized providing:
image eq 2.38
Matrix K z is the Jacobian connecting the engineering tangent altitudes with the unknowns p,T and is obtained deriving equation with respect to p and T. In this case the solution of the retrieval problem is found by simultaneously inverting equation and equation eq. 2.38 . The solution formula is in this case provided by equation and is equal to:
image eq 2.39
In our approach climatological data only contribute to the definition of the first guess profile, and profiles which are used within the forward calculation but not retrieved (i.e. contaminants). The first guess is obtained by combining climatological data and the previously retrieved profile with optimal estimation equation. In this case, the optimal estimation helps in reducing the number of retrieval iterations, but does not directly contribute to the retrieval error.

2.4.4.1.3.4.2 Profile regularization

In some cases the retrieved profiles vary as a function of altitude with an oscillation that is greater than it is physically reasonable to expect. This oscillation is intrinsic to the retrieval problem, because the solution is represented in a base of functions different from the base of the observations identified by the Jacobian: if the base of the solution contains some components that are nearly orthogonal to the base of the measurements, these components are sensitive to small variations of noise with a consequent instability of the solution.
The techniques intended to reduce these instabilities are called 'regularisation' techniques. For instance, Tikhonov-Phillips [26] Ref. [1.63 ] regularisation consists in the minimization of the function:

image image,eq 2.40
where x is the unknown, L is an appropriate operator which determines the type of constraint, m is the regularisation parameter which determines the relative weight of the two conditions and x 0 is the a-priori estimate of the solution, which is usually set equal to zero.
The solution minimizing the function equation eq. 2.40 , in a Newtonian iteration, is given by:

image.eq 2.41

As indicated by equation eq. 2.40 , the Tikhonov-Phillips regularization minimizes a modifiedimage and, depending on the choice of the a-priori estimate x 0, a bias may be introduced in the retrieved profile. The bias can be avoided by choosing the a-priori estimate equal to the profile estimate of the current iteration: image. In this case the solution equation eq. 2.41 is equal to:

image eq 2.42
Levenberg-Marquardt solution is a particular case of equation eq. 2.42 , obtained with L=I.
Like in the case of the Levenberg-Marquardt method, also in the case of equation eq. 2.42 the regularization operates only on the correction imagethat is determined in the current iteration and stepwise the same exact solution is eventually reached. For this reason the choice of image is considered in the literature Rodgers C. D. Ref. [1.59 ] not to be a real regularization.
We find however that the choice of image serves our objective of not changing the function to be minimized (in order to avoid the associated bias) while successfully damping in the retrieval the undesirable components. The operator L provides a filter to the oscillatory components of the correction y. These components are added to the retrieved profile to the extent that they are well measured. In this way the convergence path, followed by the Newtonian iterations, is modified even if still aiming at the same convergence point. When the convergence criteria are satisfied a different solution is found in which the oscillatory components, if poorly determined by the measurements, have not yet been amplified as much as otherwise possible. This is obtained with a possible increase of the number of iterations but with an equally good value of the imagefunction. The apparent contradiction of an unbiased solution different from the 'exact' one and satisfying to the same convergence criteria can be explained considering that the ' damping' introduced by the operator L acts only on the poorly determined components, which by definition cause a small difference in the observations and therefore do not change significantly the image function.
An optional regularisation of the type equation eq. 2.42 is implemented in MIPAS Level 2 processor with L T L equal to a strip-diagonal matrix in which elements different from 0 are only the diagonal elements and the first off-diagonal ones. Due to the constant (user-defined) value of µ, this regularisation proves to be more effective than the Levenberg-Marquardt method in damping oscillations in both iteration and altitude domain. The value of µ is determined on the basis of operational needs.

2.4.4.1.3.4.3 Levels versus layers

The retrieval allows only the determination of a discrete representation of the vertical profile. Several options can be considered for the discretisation of the unknown profile, the most common being a representation at discrete levels with linearly interpolated values at intermediate altitudes Carlotti M. and B. Carli Ref. [1.14 ] and a representation at discrete layers with constant value within the thickness of the layer.
Since the spectrum contains information mainly on the average VMR in each layer and not on the specific values of VMR at adjacent levels and since a non-unique relation exists between VMR at adjacent levels and the average mixing ratio in the layer between the levels, more stable results can be obtained with the layer representation rather than with the level representation Clarmann T. v., H. Fischer, and H. Oelhaf Ref. [1.22 ] .
However, the level representation is more easily used in plots and is considered to result into a more user friendly product. Considering the small mathematical difference that in any case exists between the two representations, the representation at levels has been chosen.

2.4.4.1.3.4.4 Retrieval vertical grid

The vertical resolution and the accuracy with which the retrieved profile is determined are generally anti-correlated (see Ref. [1.14 ] ) and are strongly dependent on the grid where the retrieved points are represented ('retrieval grid'). In the case of onion peeling method, the unknown values can only be retrieved at the measurement grid, i.e. the grid of the measured tangent altitudes. With global fit, the 'retrieval grid' can be different from the 'measurement grid'. Since in Level 3 data processing global maps on pressure surfaces are produced, the possibility offered by the global fit of using fixed pressure levels which will in general be different from the tangent altitude levels could directly meet user needs.
The performances of the retrieval code when the two different retrieval grids are used have been compared for different measurement and retrieval scenarios with the method described in Carlotti M. and B. Carli Ref. [1.14 ] . This analytical procedure allows the estimation of the random error that is associated with each value of the retrieved profile when a given vertical resolution is adopted, or, conversely, of the vertical resolution that can be reached when the random errors must be contained within a given limit.
When the retrieval is performed on the measurement grid and then the profile is interpolated to a fixed grid, the vertical resolution of the resulting profile is degraded and its accuracy improved Carlotti M. and B. Carli Ref. [1.14 ] , Carli B., M. Ridolfi and CO Ref. [1.11 ] . It has been found that there is little difference in vertical resolution and accuracy between retrieving on the measured grid and retrieving on a fixed grid shifted with respect to the measured one.
On the other hand, if the retrieval is performed on the nominal 3 km grid and the measurements have been performed on a 'stretched' grid (e.g. 3.5 km spacing), undesirable oscillations are present in the retrieved profile. These can be avoided only if the retrieval grid is made coarser than any possible measurement grid or if some profile regularisation is imposed.
The final choice was to retrieve vertical profiles at an altitude grid defined by the tangent altitude levels, since this provides the best information that can be derived from the measurements and the most stable results.

2.4.4.1.3.4.5 Vertical resolution

The vertical resolution of the retrievals depends on both experimental choices (instantaneous FOV and scanning altitude step), and on the profile representation within the retrieval (spacing of the retrieved points). Improvements in the vertical resolution are always obtained at the expenses of either retrieval accuracy or, through the measurement time, horizontal resolution. A preliminary model of MIPAS FOV has been used, which assumes the FOV shape to be a trapezium with the greater base equal to 4 km and the smaller base equal to 3 km. Assuming this FOV shape, a 3 km spacing has been found to be a good resolution/accuracy compromise for both the measurements and the retrieval grid.

2.4.4.1.3.4.6 Assumptions

In order to limit the complexity of the code and meet the computing time requirements, some simplifications have been adopted in the forward model. In particular, some effects have up to now been neglected in both the spectroscopic and the atmospheric model.

Neglected effects in the spectroscopic model are:

  • line mixing Edwards D. P. and L. L. Strow Ref. [1.31 ] , Rosenkranz P. W Ref. [1.60 ] occurring when collisions between a radiating molecule and the broadening gas molecules cause the transfer of population between rotational-vibrational states. Line mixing affects especially the Q-branches where transitions between ro-vibrational energy levels closer than KBT (KB is the Boltzmann constant, T is the temperature) are packed together. The most apparent effect of line-mixing is a reduction of the cross-section in the wings of the branch. The impact of line-mixing effects, mainly significant for CO2 lines, is reduced by using an appropriate selection of microwindows.
  • pressure shift Rosenkranz P. W Ref. [1.60 ] , that is significant only at high pressures, is not foreseen to affect MIPAS spectra, because MIPAS penetrates to the tangent altitude of 8 km as a minimum.
Both these effects could be taken into account without an increase of the computing time if they are modeled by the program that generates the LUTs.

Concerning the atmospheric model, the following assumptions have been made:

  • Assumption on local thermodynamic equilibrium (LTE).
Level 2 algorithm assumes the atmosphere in local thermodynamic equilibrium: this means that the temperature of the Boltzmann distribution is equal to the kinetic temperature and that the source function in equation eq. 2.18 is equal to the Planck function at the local kinetic temperature. This LTE model is expected to be valid at the lower altitudes where kinetic collisions are frequent. In the stratosphere and mesosphere excitation mechanisms such as photochemical processes and solar pumping, combined with the lower collision relaxation rates make possible that many of the vibrational levels of atmospheric constituents responsible for infrared emissions have excitation temperatures which differ from the local kinetic temperature. It has been found Lopez-Puertas M. and CO Ref. [1.48 ] that many CO2 bands are strongly affected by non-LTE. However, since the handling of Non-LTE would severely increase the retrieval computing time, it was decided to select only microwindows whose emission is in thermodynamic equilibrium to avoid Non-LTE calculations in the forward model.
  • Assumption of horizontally homogeneous atmosphere
Limb sounding attains good sensitivity due to the long path lengths obtainable, but this necessarily requires measurements which `average' the atmosphere over long horizontal distances. With limb-scanning, there is the associated problem that the profile of acquired tangent points is sheared horizontally, partly by the variation in elevation angle and partly by the satellite motion. A third problem is the assumption that the retrieved value at one altitude can be used to model the contribution of the atmosphere at that level along the ray paths for lower tangent heights, whereas in reality these paths all intersect the altitude surface at different locations. Each of these effects has a horizontal length scale of the order of several hundred kilometers, and ignoring these effects is the equivalent of assuming that the atmosphere is horizontally homogeneous over this distance.
Studies Carli B., M. Ridolfi and CO Ref. [1.11 ] have shown that the retrieval accuracy is particularly sensitive to horizontal temperature gradients. For example, ignoring a temperature gradient of 3 K / 100 km (a typical maximum, e.g. associated with crossing the polar vortex) can lead to composition retrieval errors of tens of %, although these errors are localized and usually associated with regions in which the atmospheric composition is also changing rapidly.
Several approaches Carlotti M., B.M.Dinelli and CO Ref. [1.12 ] , Carlotti M., B.M.Dinelli, P.Raspollini, M.Ridolfi Ref. [1.71 ] have been considered in order to allow for horizontal inhomogeneity, but none of them appear to be suitable for NRT operational processing. We must, therefore, be aware of the assumption of horizontally homogeneous atmosphere when observing air masses with steep gradients.

2.4.4.1.3.5 Algorithm validation

To validate the approximations implemented in the forward model internal to the Level 2 processor (that is called 'Optimized Forward Model' = OFM), comparisons were made with a specially developed line-by-line code based on GENLN2 Edwards D. P Ref. [1.30 ] . This code was compared with several existing codes and was elected as our reference forward model (RFM) Edwards D. P Ref. [1.29 ] . The main results of the RFM / OFM intercomparisons are:

  • Ray-tracing: for N2O, 10 km tangent-height path (representing the most 'difficult' case involving both large VMR gradients and refraction effects) RFM-OFM calculations differ by less than 0.7% in the slant column calculations, less than 0.004% in the Curtis-Godson pressure calculation and less than 0.002 K in Curtis-Godson temperature calculation.
  • Cross-section calculations: RFM and OFM full spectral calculations agree to better than 1 % near major absorption features.
  • Limb spectral calculations: RFM-OFM limb radiance calculations agree to within NESR/4 (the values of the NESR in the different spectral ranges are reported on the figure below:
    image
    Figure 2.28

    The factor 4 is intended to account for the possible building up of systematic effects and for the achievement of a NESR better than the requirements.


The retrieval code has been validated by performing retrievals from spectra generated by its own forward model and by the RFM. Tests are in progress (January 2001) using with spectra obtained with the balloon instrument MIPAS-B2.
The results obtained so far indicate that both forward model error, i.e. error due to imperfect modeling of the atmosphere, and convergence error, i.e. error due to the fact that the inversion procedure does not find the real minimum of the χ2 function, are much smaller than the measurement error due to radiometric noise.

2.4.4.1.3.6 Performances

2.4.4.1.3.6.1 Accuracy performance of Level 2 retrieval algorithm

The main error sources that affect the accuracy of the retrieved profiles are:

  • noise error, due to the mapping of radiometric noise in the retrieved profiles;
  • temperature error, which maps into VMR retrieved profiles;
  • systematic error, due to incorrect input parameters.
The amplitude of noise error has been evaluated (through equation eq. 2.13 ) with test retrievals that use observations simulated starting from assumed atmospheric profiles (reference profiles) and perturbed with random noise of amplitude consistent with MIPAS noise specification.

The effect of temperature error on VMR retrievals is determined using tabulated propagation matrices which estimate the effect for different measuring conditions. Current results indicate that temperature error can be a significant component of the error budget and consideration is being given to methods to improve the accuracy of temperature retrieval. (details on p,T error propagation 2.4.4.3. )

Errors of the third type include systematic errors, such as spectroscopic errors or errors due to imperfect knowledge of the VMR profiles of non-target species. These errors are taken into account in the definition of the optimum size of each microwindow and for the selection of the optimal set of microwindows 2.4.4.4. that should be used for the retrieval. The quantifiers that are calculated for these operations can also be used for the determination of the total error budget.

The relevance of systematic errors in the total error budget depends on whether optimistic or conservative error estimates are used. The current estimate of the ultimate retrieval accuracy is summarized in the following plot that reports the total error as a function of the altitude for each of the retrieved constituents and for temperature.

image
Figure 2.29


 
 

2.4.4.1.3.6.2 Runtime performance

Note: here is a section illustrating only the runtime performance of the ORM. What about the L2 processor ????
The runtime performances of the ORM have been tested using different computers. Tests have been performed on simulated observations using two different sets of microwindows, a preliminary standard set and a set which optimizes the trade-off between accuracy and run-time performance. In these tests we used initial guess profiles of the retrieval that are sufficiently close to the reference profiles (the ones used to simulate the observations), so that convergence is reached in only one iteration. The results of these tests are shown in the table below.
Considering that the measurement time per scan is 75 seconds and that more than one computer can be used for the operational analysis data, we can conclude that the run-time requirements are fully satisfied also for retrievals that need more than one iteration.
 
 

Table 2.2 Table: Runtime (sec.) for p,T and 5 target species retrieval (1 iteration)
Computer description Standard set of MWs Optimized set of MWs
SUN SPARC station 20
120 MHz CPU, 128 Mb RAM
550 (*)
348 (*)
PENTIUM PC
200 MHz CPU, 256 Mb RAM
352
210
Ultra Sparc station 5
181
Not Available
IBM RS6000 Model 397
149
Not Available
Digital DEC-SERVER Mod. 4100
600 MHz CPU, 1 Gb RAM
74
51

(*) This run-time is strongly affected by the use of swap space
 
 
 
 
 


Keywords: ESA European Space Agency - Agence spatiale europeenne, observation de la terre, earth observation, satellite remote sensing, teledetection, geophysique, altimetrie, radar, chimique atmospherique, geophysics, altimetry, radar, atmospheric chemistry