General Structure of Product

The structure of the products produced for delivery to the PDGS follows the requirements of [AD-12] as represented in Figure 1 below.

Each product is composed of two physical files:

• XML Header file having extension .HDR

• Product file having extension .DBL

The XML Header file is an ASCII file containing data information that users can easily access for identifying the product without needs to look inside the product file.

It consists of:

• Fixed Header, a common header for all files in the Swarm Ground Segment

• Variable Header, including

• Main Product Header (MPH) containing general information.

• Specific Product Header (SPH) containing product specific information. The Specific Product Header will also contain references to auxiliary and/or input files relevant for the specific product.

The product file is the real product containing the processing results.

---

 

XML Header File

The XML Header file contains information identifying the product.

The Fixed Header (hereafter called Standard Swarm Header) is the common header for all files in the Swarm Ground Segment, which means it is applied to all files flowing amongst the sub-systems composing the PDGS.

The format of the Standard Swarm Header is under ESA responsibility and is specified in [AD-12] and [AD-13].

The Variable Header (hereafter called Product Header) is the header with format and content depending on the file type and kind of product.

The next sub-paragraphs specify the content of these headers.

Level 2 Products Fixed Header (Standard Swarm Header)

The Standard Swarm Header is completely ASCII and based on XML syntax and conventions proposed in [AD-12].

Field #	Description	Units	Format
1	File_Name	Tag
	Product file name (without extension) (see Section 4.4 in [AD-15])		55*uc
2	File_Description	Tag
	Description such as mid column in Table 5-1 - Table 5-3 in [AD-15] (For CAT-2 AUX products, allowed entries are described in [AD-16].)		x*uc
3	Notes	Tag
	This field will be empty.
4	Mission	Tag
	Will be "Swarm".		5*uc
5	File_Class	Tag
	Type of processing; character 9 - 18 in the file name (see Section 4.4 in [AD-15]), this can be: OPER = Routine operations RPRO = Reprocessing		4*uc
6	File_Type	Tag
	Product name, first column in Table 5-1- Table 5-3 in [AD-15]		10*uc
7	Validity_Period	Tag
7.1	Validity_Start	Tag
	Start time of validity period UTC=yyyy-mm-ddThh:mm:ss		23*uc
7.2	Validity_Stop	Tag
	Stop time of validity period UTC=yyyy-mm-ddThh:mm:ss		23*uc
8.	File_Version	Tag
	Version number of the product Start with 0001 and increases by one if following product versions are generated.		%04d
9.	Source	Tag
9.1	System	Tag
	Name of the Ground Segment component: L2PS: for CAT_1 products APDF: for CAT-2 products		4*uc
9.2	Creator	Tag
	Name of Level 2 processor , such as CI for comprehensive inversion, or IBI for bubble index, SwarmLevel2Create for CAT-1 AUX products, ADF Preprocessor for CAT-2 AUX products.		x*uc
9.3	Creator_Version	Tag
	Version of Level 2 processor VV.rr		5*uc
9.4	Creation_Date	Tag
	Date of file creation: UTC=yyyy-mm-ddThh:mm:ss		23*uc

Table 1: Level 2 Fixed Header

Level 2 Products Variable Header (Product Header)

The XML Variable Header (hereafter called Product Header) for the Level 2 products is composed by:

• an XML Main Product Header (XML MPH), and
• an XML Specific Product Header (XML SPH)

Each header is completely ASCII and based on XML syntax and conventions proposed in the [AD-12].

The following sections describe the MPH and SPH for Level 2, Cat-1 Auxiliary products, and Cat-2 Auxiliary products AUX_LIT_2F, AUX_COR_2F and AUX_IGR_2F. MPH and SPH for all other CAT-2 AUX products are described in [AD-16].

XML Main Product Header (XML MPH)

The Main Product Header (MPH) has the following format - very similar to the Level 0 and Level 1b Main Product Header ([AD-5], [AD-14]), see also [AD-13]:

Field #	Description	Units	Format
1	MPH	Tag
Following fields 1.1 - 1.9 are valid for all L2, Cat-1 Auxiliary products, and Cat-2 AUX_LIT_2F, AUX_COR_2F and AUX_IGR_2F 'Auxiliary products' (not for other Cat-2 Auxiliary products)
1.1	Product	Tag
	Product file name (without extension) (see Section 4.4 in [AD-15])		55*uc
1.2	Product_Format	Tag
	Format of the Product file: CDF, PDF or ASCII		5*uc
1.3	Proc_Stage_Code	Tag
	character 9 – 18 in the file name (see Section 4.4 in [AD-15]), this can be: OPER = Routine operations RPRO = Reprocessing		4*uc
1.3	Ref_Doc	Tag
	Reference Document describing the product, shall be SW-DS-DTU-GS-0001		17*uc
Data Processing Information
1.4	Proc_Center	Tag
	Processing Center ID code: BGS = British Geological Survey DEOS = DEOS DTU = DTU Space ETH = Eidgenössische Technische Hochschule GFZ = GeoForschungsZentrum Potsdam IPGP = Institut de Physique du Globe de Paris PDGS = Payload Data Ground Segment		4*uc
1.5	Proc_Time	Tag
	Processing Time, UTC (Product Generation Time) UTC=yyyy-mm-dd Thh:mm:ss.uuuuuu		30*uc
1.6	Software_Version	Tag
	Processor Name and software version number ProcessorName/VV.rr (Processor name as tag 9.2 of Fixed header)		x*uc
Product Confidence Data Information
1.7	Product_Err	Tag
	Product Error Flag. Set to 0 if product is fully ok, non-zero if any problems or errors are present		uc
Product Size Information
1.8	Tot_Size	Tag
	unit="bytes"	Attribute
	Total size of product	bytes	%+021d
1.9	CRC	Tag
	Cyclic Redundancy Code computed as overall value of all records of the Measurement Data Set. Will be set to "-0000000001" if not computed.		%+011d

Table 2: Level 2 Main Product Header (MPH)

XML Specific Product Header (XML SPH)

The formats of the Specific Product Headers (SPHs) are described. The SPHs are separately described for Level-2 Products (Cat-1 and Cat-2) and for CAT-1 AUX products. The SPH for CAT-1 AUX products applies also to the CAT-2 AUX products AUX_IGR_2F, AUX_COR_2F and AUX_LIT_2F. The SPH's for all other CAT-2 AUX products are described in [AD-16].

Following fields 1.1 – 1.6 are valid for L2 Cat-1 products
Field #	Description	Units	Format
1	SPH	Tag
1.1	SPH_Descriptor	Tag
	Product Name, first column of Tables 5.1 - 5.3 in [AD-15]		10*uc
1.2	Original_Filename	Tag
	Name of original files, e.g, "*.SP3" for GPS orbit files		x*uc
Information on Time of Data
1.3	Sensing_Time_Interval	Tag
1.3.1	Sensing_Start	Tag
	Start time in UTC of sensing data UTC=yyyy-mm-ddThh:mm:ss.uuuuuu		30*uc
1.3.2	Sensing_Stop	Tag
	Stop time in UTC of sensing data UTC=yyyy-mm-ddThh:mm:ss.uuuuuu		30*uc
Product Confidence Section - various information on the quality of the product, such as number of missing or erroneous ISPs (Instrument Source Packets) and number of rejected or suspicious samples.
1.4	Product_Confidence_Data	Tag
1.4.1	Quality_Indicator	Tag
	General product quality indicator, 000 if best quality is expected, 001 if the quality is acceptable.		%03d
List of Input files
1.5	List_of_Input_File_Names	Tag
	count="n"	Attribute
This part is repeated n times, one for each input file
1.5.i	File_Name		*uc
	Name of processor input file i
List of Output files
1.6	List_of_Output_File_Names	Tag
	count="n"	Attribute
This part is repeated n times, one for each output file
1.6.i	File_Name		*uc
	Name of processor output file i
Following fields 1.1 - 1.5 are valid for L2 Cat-2 products
Field #	Description	Units	Format
1	SPH	Tag
1.1	SPH_Descriptor	Tag
	Product Name, first column of Tables 5.1 - 5.3 in [AD-15]		10*uc
Information on Time of Data
1.2	Orbit_Information	Tag
1.2.1	Sensing_Start	Tag
	Start time in UTC of sensing data UTC=yyyy-mm-ddThh:mm:ss.uuuuuu		30*uc
1.2.2	Sensing_Stop	Tag
	Stop time in UTC of sensing data UTC=yyyy-mm-ddThh:mm:ss.uuuuuu		30*uc
Information on manoeuvre
1.3	Maneuver_Information	Tag
	count="n"	Attribute
1.3.i	Maneuver_Id i=1, ..., n	Tag
	The ith, distinct maneuver identification code[RD-3]		%03d
Product Confidence Section - various information on the quality of the product, such as number of missing or erroneous ISPs (Instrument Source Packets) and number of rejected or suspicious samples.
1.4	Product_Confidence_Data	Tag
1.4.1	Quality_Indicator	Tag
	General product quality indicator FAC_TMS_F and FACx_TMS_2F: 000 no problems at all occurred 001 quality_indicators is 001 in the L1b header of one of the MAGx_LR_1B 010 Flags(1) or Flags(2) is/are larger than 0 (data gaps occurred) 100 if one or all of Flags(3...8) are larger than 0 (problems with auxiliary data for magnetospheric field calculation occurred); And combinations of quality indicators IBIxTMS_2F: 000 no problems at all occurred 001 quality_indicators is 001 in the L1b header of one of the MAGx_LR_1B or EFIx_PL_1B 010 all of the Bubble_Index within the output are -1 (unanalyzable) TECxTMS_2F: 000 no problems at all occurred 001 L1b header of one of the MODx_SC_1B or GPSx_RO_1B quality_indicators is 001 010 daily average of Relative_STEC_RMS or daily average of DCB_Error is larger than 2 TECU EFFxTMS_2F: 000: no problems at all occurred 001: L1b header of one of the MAGx_LR_1B quality indicators is 001 010: Flags(4) is larger than 0 (data gap prevented processing)		%03d
List of input files
1.5	List_of_DSDs	Tag
	count="n"	Attribute
This part is repeated n times, one for each input file
1.5.i	DSD	Tag
	Data Set i descriptor, i = 1,2,…,n
1.5.i.1	Data_Set_Name	Tag
	Product name (see Table 5-2 Final Level 2 products in [AD-15])		10*uc
1.5.i.2	Data_Set_Type	Tag
	Type of Data Set: M - measurement R - reference		uc
1.5.i.3	File_Name	Tag
	Name of Input File (without extension)		55*uc
1.5.i.4	Data_Set_Offset	Tag
	unit="bytes" Offset (in bytes) of first byte of first DS record within Product File. Only used if Data_Set_Type = "M", otherwise set to zeros.	Attribute	%+021d
1.5.i.5	Data_Set_Size	Tag
	unit="bytes" Total number of bytes in DS Only used if Data_Set_Type = "M", otherwise set to zeros.	Attribute	%+021d
1.5.i.6	Num_of_Records	Tag
	Number of Data Set records. Only used if Data_Set_Type = "M", otherwise set to zeros.		%+011d
1.5.i.7	Record_Size	Tag
	unit="bytes" Size of Data Set records If variable set to -0000000001 Only used if Data_Set_Type = "M", otherwise set to zeros.	Attribute	%+011d
1.5.i.8	Byte_Order	Tag
	Byte ordering information. 3210 → Big-endian 0123 → Little-endian Only used if Data_Set_Type = "M", otherwise set to "0000".		4*uc

Table 3: Level 2 Specific Product Header (SPH) - For L2 Cat-1 and L2 Cat-2 products

 

Field #	Description	Units	Format
1	SPH	Tag
1.1	SPH_Descriptor	Tag
	Product Name, first column of Tables 5.1 – 5.3 in [AD-15]		10*uc
Information on Time of Data
1.2	Validity_Time_Interval	Tag
1.2.1	Validity_Start	Tag
	Start time in UTC of sensing data UTC=yyyy-mm-dd Thh:mm:ss.uuuuuu		33*uc
1.2.2	Validity_Stop	Tag
	Stop time in UTC of sensing data UTC=yyyy-mm-dd Thh:mm:ss.uuuuuu		33*uc
Product Confidence Section – various information on the quality of the product, such as number of missing or erroneous ISPs (Instrument Source Packets) and number of rejected or suspicious samples.
1.4	Product_Confidence_Data	Tag
1.4.1	Quality_Indicator	Tag
	Generel product quality indicator, 000 if best quality is expected. A different number than 000, if problems occurred during processing.		%03d
Originator information
1.5	Source	Tag
1.5.1	Original_Filename	Tag
	Name of the original file before reformatting Examples: AUX_KP__2_ : kpyyyy.wdc AUX_DST_2_ : dstyyyymm.ig2 AUX_F10_2_ : DAILYPLT.OBS AUX_IMF_2_ : omni2_h0_mrg1hr_yyyymmdd_v01.cdf AUX_OBS_2_ : *yyyy*.min (if only obsyyyymmddqmin.min files as input) *yyyy.wdc (if only obsyyyy.wdc as input) wdc_mixed (if obsyyyymmddqmin.min and obsyyyy.wdc as input) AUX_COR_2_ : freely chosen by the chain AUX_LIT_2_ : freely chosen by the chain AUX_IGR_2_ : igrf12coeffs.txt AUX_COR_2F : as for AUX_COR_2_ AUX_LIT_2F : as for AUX_LIT_2_ AUX_IGR_2F : as for AUX_IGR_2_ with yyyy=year, mm=month and dd=day		x*uc
1.5.2	Creator_Date	Tag
	Date of creation of the original file UTC=yyyy-mm-ddThh:mm:ss		23*uc
1.5.3	Creator	Tag
	Name of the external facility creating the original file, e.g., AUX_KP__2_ : Niemegk observatory AUX_DST_2_ : WDC Kyoto AUX_F10_2_ : NGDC NOAA AUX_IMF_2_ : NASA AUX_OBS_2_ : INTERMAGNET and WDC Edinburgh AUX_COR_2_ : GFZ AUX_LIT_2_ : IPGP AUX_IGR_2_ : IAGA AUX_COR_2F : as for AUX_COR_2_ AUX_LIT_2F : as for AUX_LIT_2_ AUX_IGR_2F : as for AUX_IGR_2_		x*uc

Table 4: Level 2 Specific Product Header (SPH) - For CAT-1 AUX products and for AUX_IGR_2F, AUX_COR_2F and AUX_LIT_2F

Input Files

Input files to the Level 2 Processor (Level 1b and auxiliary products) used in the generation of the products are specified in the specific product header (XML Specific Product Header (XML SPH)).

---

 

Product File Formats

The L2 and Auxiliary Product File consist of data sets with specific formats that have been agreed during the SCARF Technical Meeting 2 on February 22/23, 2010. Five different formats exist:

• ASCII Listing (SHC format)
• ASCII Listing (SP3 format)
• ASCII Listing (XML)
• ASCII Listing (others)
• CDF-format
• PDF-files

The following section describes the format categories in more detail. The format category for the individual products is provided in Table 5-1 – Table 5-3 in [AD-15].

ASCII Listing (SHC format)

All core and lithosphere magnetic field model products (final and intermediate) obey the SHC format. Missing data values are marked with NaN.

The Gauss coefficients g and h of spherical harmonic models are given as snapshots at different time instants. A static field (i.e. the high-degree lithospheric field) is given as one snapshot; the core field is provided as a series of snapshot models.

Each file contains one or more blocks. Each of these blocks describes a certain range of Gauss coefficients with the same description of time dependence (i.e. either static, linear time dependent, cubic splines, or order-6 splines, or others).

Each block contains a header part and a coefficient part of length K = N_max*(N_max+2) – (N_min-1)*(N_min+1) lines with the Gauss coefficients stored row-wise in natural order (g₁⁰, g₁¹, h₁¹, g₂⁰ g₂¹ h₂¹, …). The first column of this coefficient part contains spherical harmonic degree n, the second contains spherical harmonic order m. Non-negative values of the order refer to the Gauss coefficients g_n^m while negative orders indicate the Gauss coefficients h_n^m. Rows 3 to N_times+2 of this coefficient part contains the Gauss coefficients of the N_times snapshot models, provided at the time instants t (given in decimal years).

Such a series of snapshot models is sufficient to completely describe a model if its time dependence is constant, a Taylor expansion in time, or a piecewise linear varying spline (order 2 spline).

In the case of a time dependence described by splines of order larger than 2 (e.g. cubic splines, i.e. spline of order 4), additional information is needed to reconstruct the time dependence from the N_times snapshot models. Within SCARF, following format applies: providing spline_order snapshot models for each knot interval (i.e. an "oversampling" of the model), indicated by the value N_step (which is equal to spline_order-1 in this case). This is explained in Table 4-5 SHA format used in SCARF in [AD-15]. Every N_step value of time t (in decimal years) are the spline knots, and therefore N_step = spline_order - 1.. It is possible to read the whole block using space-delimited free format read instructions.