Minimize Swarm Data Quality User Note: Current Dataset and History

1. "Latest_baselines" Dataset

The tables below present the "Latest_Baselines" Level 1b and Level 2 Datasets with their respective Product Baselines. Detailed explanations of the most relevant data quality features are given in the following sections.
For a further explanation of Dataset and Baseline concepts, please see the Definition of Dataset and Product Baseline.
For more information on the Swarm products, please see the Swarm Level 1b Product Definition.


Table 1: L1B Product Baselines presented in "Latest_baselines" Folder

 

 

Sat Alpha

Sat Bravo

Sat Charlie

Level 1b

Product Baseline

Validity

Product Baseline

Validity

Product Baseline

Validity

Orbit and Attitude products (ORBATT)

04

From 25/11/2013 to present

04

From 25/11/2013 to present

04

From 25/11/2013 to present

1 Hz Magnetic products (MAGx_LR_1B)

05

From 25/11/2013 to present

05

From 28/11/2013 to present

05

From 04/12/2013 to present

50 Hz Magnetic products and other auxiliary magnetic products
(MAGx_HR_1B, MAGx_CA_1B, VFMxAUX_1B, ASMxAUX_1B)

05

From 18/07/2015 to present

05

From 18/07/2015 to present

05

From 18/07/2015 to present

Plasma products (EFIx_PL_1B)

05

From 18/07/2015 to present

05

From 18/07/2015 to present

05

From 18/07/2015 to present

Accele products (SC_xDYN_1B)

04

From 28/11/2013 to present

04

From 28/11/2013 to present

04

From 28/11/2013 to present

Precise Swarm Orbit Counter (ORBCNT)

00

From 22/11/2013 to present

00

From 22/11/2013 to present

00

From 22/11/2013 to present


Table 2: L2 Cat-2 Product Baselines presented in "Latest_baselines" Folder
 

 

Sat Alpha

Sat Bravo

Sat Charlie

Level 2 - Cat2

Product Baseline

Validity

Product Baseline

Validity

Product Baseline

Validity

Field Aligned Currents - Single

03

From 26/11/2013 to present

03

From 28/11/2013 to present

03

From 04/12/2013 to present

Field Aligned Currents -
SW A and C combined

03

From 15/14/2014 to present

03

N/A

03

From 05/04/2014 to present

Ionospheric Bubble Index

03

From 09/12/2013 to present

03

From 09/12/2013 to present

03

From 01/12/2013 to present

Total Electron Content

03

From 25/11/2013 to present

03

From 25/11/2013 to present

03

From 25/11/2013 to present

Equatorial Electric Field

01

From 27/08/2015 to present

01

From 27/08/2015 to present

N/A

N/A


1.1  LATEST_BASELINES DATASET: QUALITY INFORMATION

 

1.1.1 Orbit and Attitude products    

 

On 17 September 2018, some modifications have been introduced in the way GPS and STR products are computed:

 

a) GPS

 

Change in ORBATT of carrier phase observation to get full-cycle ambiguities

Until now, the Swarm carrier phase observations were affected by half cycle ambiguities. The Swarm GPS receivers report in a dedicated flag whether the navigation data stream is inverted or not. The new RINEX converter uses the flag to generate carrier phase observations with full cycle ambiguities. It is noted that full cycle ambiguities allow using algorithms for integer ambiguity fixing, which enable more accurate precise orbit determination.
 
Align data in RINEX files to GPS time (GPS_RO clock offset and drift)

The Swarm GPS receiver clocks have a clock offset of roughly 0.2 µs, which corresponds to an offset of 60m in the pseudo-range observations. This offset is used to correct the pseudo-range observations and to align the clock offset with the receiver internal navigation solution. This bias has been hardcoded in the processor to avoid changes in interfaces. The value is documented in the header of the Rinex files.
The old RINEX files are aligned with the drifting IMT (Instrument Measurement Time), which causes large jumps at midnight, when the daily clock offset is adjusted by the processor. In order to avoid such large jumps, the new RINEX files are aligned with GPS time instead of IMT.
 
Reduction of receiver clock noise

In the previous baseline of L1B data, a navigation solution and clock offset are computed on-ground from the observations and ephemerides. The clock offset calculated in that way shows a noise level of about 30 ns, corresponding to noise level of 10 m for pseudo-range observations. The new processor uses instead the on-board navigation, which lead to a noise level for clock offsets of 3 ns, corresponding to a noise level of 1 m for pseudo-range observations.
In the previous version of the processor, GPS time was not stored with sufficient precision, resulting in noise in the code observations, which increased as GPS time increased. The new processor avoids that the fractional part of the GPS time is lost and the code observations therefore show constant low noise level over time.

 

b)  Star Tracker

 

STR Inter-Boresight Angles correction model

An anomalous variation (up to ~ 0.85 arc-sec/°C) in the Star Tracker (STR) Inter Boresight Angle (IBA) for Camera Head Unit (CHU) pairs has been observed. A STR IBA correction model that uses as input CHU and STR temperatures has been proposed by the instrument manufacturers. After an extensive validation process it has been decided to include this model in the L1B algorithm and operational processor. This model allows pre-flight and in-flight IBA measurements to match mostly within arc-seconds for Alpha and Charlie, and with slightly larger offsets for Bravo. An improvement (higher than 10 arc-seconds) of the STR attitudes accuracy is also observed.
 
An increase of the frequency of the STR L0 product from 1 Hz to 2 Hz

A high noise in the STR attitude accuracy is observed on the STR data at 1 Hz rate. Tests performed on-ground demonstrated that by doubling the STR data rate from 1Hz to 2Hz would improve significantly the pointing measurement is obtained. For this reason, the L1B processor has been adapted to handle STR data at 2 Hz. This means that very soon the time accuracy of the STR data will change from 1 to 2 Hz.


1.1.2 Magnetic products  

The improvements in the new Product Baseline of magnetic data have been made possible with the joint efforts of a dedicated group of experts, who analysed instrument and housekeeping data and elaborated a model for correcting data from external disturbance effects, predominantly Sun-driven and manifested as scalar residuals between the ASM and VFM data. After the application of this correction model, we have reduced significantly the differences between ASM and VFM data. The overall rms of such differences were around 1 nT before the correction, while after the correction the rms is roughly 300 pT, for all spacecraft. The expert group is working hard to further improve the disturbance models.
The key points are the following:

  1. Early in the Swarm mission it was recognised that the measurements from the two magnetometer instruments (ASM and VFM) on-board each spacecraft measured slightly different absolute magnetic field values. This effect has been analysed and interpreted as a magnetic disturbance of varying strength and direction. Such disturbances are highly correlated with the Sun incident angle with respect to the spacecraft and are manifested as scalar residuals between the ASM and VFM data. The experts endeavoured to model the disturbance parameterized in terms of a spherical harmonic model as a function of Sun incident angles, and this model has proven to provide excellent results.
  2. In the current processing strategy, the calibration parameters for the vector measurements estimated on a daily basis are replaced by the co-estimation of the adjustment of the VFM pre-flight calibration parameters and the magnetic vector disturbance model. This disturbance model (the so-called "dB_Sun)" is considered as a stray field vector at the VFM instrument location. A detailed description of this correction model can be found here. The applied dB_Sun for the correction is then stored in the following magnetic products: MAGx_HR_1B, MAGx_LR_1B and MAGx_CA_1B.
  3. In order to cope with the permanent failure of the ASM onboard Swarm Charlie, the magnetic scalar reference measurements for the disturbance model characterisation and VFM calibration described above have successfully been generated using the ASM measurements from Swarm Alpha. For more detailed information, please refer to the Scalar Mapping Technical Note.


    Separation of pre-flight from the in-flight VFM calibration parameters
    In the current version of the L1B operational processor a separation of the pre-flight and the in-flight VFM calibration parameters is introduced. More precisely, the pre-flight calibration parameters are applied to Level1binst.VFM (i.e., calibrated vector field components), while the inflight estimated calibration parameters are added to the diagonal scaling matrices in the computation of B'VFM (i.e., fully calibrated and corrected vector field components). In this way, we have the parameter B_pre (= Level1binst.VFM) parameter in MAGx_CA_1B products as a source of "original residuals".

    Change the data resampling used for the interpolation of the ASM measurements to UTC second
    Experts have found that the resampling of the ASM data from t_ASM to exact UTC is removing a part of the high frequency content of the ASM data. In order to avoid this, a change of the ASM data resampling (non-smoothing cubic B-splines) has been implemented in the Level 1B processing algorithms.
     
    Fixes in the computation of ASM stray fields
    Due to an old bug in the L1b data processor, the ASM stray fields, dF_AOCS and dF_other, stored in the Swarm MAGx_LR_1B and MAGx_CA_1B data with Product Baseline 03 and 04 from the beginning of the mission to 21/02/2015 have not been computed correctly. This issue has no impact on the full correction of the ASM data from the total of the stray fields (dF_Total) which is instead implemented and applied in the right way.
     
    Update of vector data calibration and disturbance characterization
    An update of the VFM pre-flight calibration parameters and of the dB_sun has been implemented in the Level 1B processing algorithms.


1.1.3 Plasma products  

 

Currently five main datasets exist for Swarm Plasma products:

  • A Preliminary Plasma Dataset: it is the result of the efforts of the Swarm Electric Field Instrument (EFI) Principal Investigators (PIs) and their teams, and is meant to partly replace the official dataset when it is not available.
  • The Plasma "EFIx_LP_1B" Dataset: it consists of the official Swarm plasma products as from Swarm Level 1b Product Definition, available in the "Latest Baselines" folder of the ESA FTP server.
  • An "Extended" Langmuir Probe dataset: this is an experimental dataset where separate measurements from the single probes are given, in order to investigate open issues on data and for data validation purposes
  • Electron density derived from Faceplate current: this is an experimental dataset that offers a different evaluation of the electron density, based on the measurements of the faceplate current at 16 Hz rate.
  • 2 Hz TII cross-track ion flow dataset: this is an experimental dataset, based on measurements of the TII cross-track data at 2 Hz.
  • 16 Hz TII cross-track ion flow dataset: this is an experimental dataset, based on measurements of the TII cross-track data at 16 Hz.

The use of such data by the scientific community is encouraged by the EFI PIs, provided interested scientists and users carefully read the  User Note concerning the state of the art of plasma data quality and the information about data formats and content. In case of doubts, the PIs will be pleased to answer your questions:

University of Calgary

Swedish Institute for Space Physics (IRF), in Uppsala:

Please always include ESA in CC in your communications: swarm@eo-sppa.org
The datasets are available on the ESA FTP server accessible by all users, under the path: ~/Advanced/Plasma_Data/


1.1.4  Accelerometer Products

 

The Level 1B accelerometer data (ACCx_PR_1B) are not distributed to the regular users as they are heavily perturbed by a variety of disturbances. The main problems are:

  • High number of steps, spikes, pulses observed in linear and angular acceleration;
  • Strong dependency on temperature variations.

Intensive investigations resulted in the conclusion that a fully automatic correction of the disturbances is not reliable, at least in the foreseeable future.
The ACCx_PR_1B products with baseline 04 contain the accelerometer data as well as data relevant for POD and spacecraft dynamics investigations, e.g. the mass of the spacecraft. Since the decision not to disseminate accelerometer data is preventing Swarm data users from accessing data relevant for POD and spacecraft dynamics investigations, it was decided to split the ACCx_PR_1B data products into two products:

  • Revised ACCx_PR_1B data products that contain only the accelerometer data
  • New SC_xDYN_1B data products that contain all data that were present in the old ACCx_PR_1B data products, but will not be contained in the revised ACCx_PR_1B data products

The new SC_xDYN_1B data products will also contain three new variables that specify which thrusters were activated for how long, including the resulting thrust force acting on the spacecraft. The information will be extracted from the Level 1a AOCS and BUS house-keeping data contained in the HK_xAOCS1A and HK_xBUS_1A data products, respectively, which are also not available to Swarm data users.
The revised ACCx_PR_1B data products will remain inaccessible to Swarm data users, whereas the new SC_xDYN_1B data products will be disseminated to all Swarm data users. For accessing accelerometer data, Swarm data users are advised to use the Level 2 ACCCCAL_2 data products.


1.1.5 Precise Swarm Orbit Counter

 

The AUXxORBCNT product provides precise Swarm orbit number information to the scientific user community. The start of an orbit is defined as the Equator crossing of the ascending node (i.e. the northbound Equator crossing). Detailed information for this product is available here


1.1.6 Level 2 Cat-2 products

 

Following the correction of the Magnetic Level 1B data (MAGx_LR_1B), a new Baseline (03) of FAC, IBI and TEC products has been introduced. For the product types: FACxTMS_2F, IBIxTMS_2F, and TECxTMS_2F, data have been therefore regenerated since, approximately, the beginning of the mission. For the product type: FAC_TMS_2F (FAC from combined S/C A and C measurements) the coverage does not start from beginning of mission, but from 15/04/2014, when the lower pair reached their definitive orbit configuration. Detailed quality information for the following products is available: FAC – single S/C solutionFAC - dual S/C solutionIBI and TEC.
EEF: Equatorial Electric field data were made available for the first time since 27/08/2015, only for S/C Alpha and Bravo. Unfortunately, the unavailability of the ASM instrument on S/C Charlie makes impossible the production of EEF data for such spacecraft. Detailed quality information on this products is available here.


2. Entire_mission_data

In the "Entire_mission_data" folder on the ESA FTP server users will find the full data coverage of the entire mission, regardless of any consideration of interoperability among the same product type, i.e. regardless of the baseline each product member of this folder belongs to.