The recent 10 year anniversary of ESA’s Swarm mission provides a new decadal perspective of its magnetic field measurements, enabling scientists to make strides in their understanding of Earth’s system, from its outer atmosphere to its turbulent core.

One key to unlocking the full potential of this treasure trove of Swarm data is the ESA-funded VirES service (Virtual workspaces for Earth scientists).

The VirES toolset is a virtual research environment that offers more direct and in-depth exploitation of Swarm data, thanks to a data retrieval web interface, multi-dimensional geographical visualisation, interactive plotting and on-demand processing tools.

The three-satellite Swarm constellation has a complex product portfolio, with many derived products, some of which draw on data from other missions.

Interactive web-based data retrieval

Navigating, accessing, understanding, and effectively using all these data products is challenging. The VirES service tackles these challenges by providing a web-based development environment in the form of a JupyterLab workspace.

Thanks to a graphical web interface, Application Programming Interfaces (APIs) and Python-based tooling, the VirES is a versatile toolset for users of Swarm data. The service has an open access policy, making it easy for research scientists, engineers, and students to fully exploit Swarm data.

VirES is developed and operated by EOX IT Services, and has been in operation since 2016. The service is supported by the Swarm Data, Innovation, and Science Cluster (DISC), a consortium of 33 partner institutions and companies from 19 countries in Europe and North America.

Scientists need flexibility when manipulating data

While the graphical interface provides a comfortable experience to visualise the data and models, the underlying VirES server provides robust access mechanisms for on-demand processing, to help evaluate model outputs and auxiliary information.

This functionality is exposed over OGC (Open Geospatial Consortium) APIs, and the Heliophysics API, so that the service can be interacted with programmatically, and users can connect it with other systems.

Researchers often need as much flexibility as possible to work with data, preferring to connect with existing analysis tools and compose their own processing code. To meet this need, the VirES service is supplemented with a Jupyter environment (VRE: Virtual Research Environment) that allows interactive programming through the web browser. 

The VRE is bundled with domain-specific Python tooling from the scientific community and a helpful guidebook to help people get started. This is part of a wider effort to bring Python tooling to Swarm science.

Tool allows easy visualisation and processing

Nils Olsen, Professor at the Technical University of Denmark and one of the initiators behind VirES, says, “In the Swarm project we want not only to provide the measurements taken by the satellites, but also to allow users to easily visualise and process the various data products. With VirES/VRE we initiated such a service, thereby increasing access to the Swarm data for the community.”

Antonio de la Fuente, ESA's VirES Technical Officer, says, "ESA started this project back in 2015, as part of its technology research programme. Since then, it has been extensively expanded to become an operational service. The VirES/VRE platform is a crucial asset that supports other missions, such as Aeolus, and serves as a reference for future platforms, enabling open collaboration in data manipulation, agile algorithm development, and high-level product generation.”

ESA’s role in supporting the academic community

The International Association of Geomagnetism and Aeronomy (IAGA) runs biennial one-week schools for early career scientists using the VRE service to run interactive programming exercises.

Many universities have also been putting the service to good use in their education programmes. For example, the University of Cologne uses VirES for their Masters-level course “Geomagnetism and Space Weather”.

At the University of Bergen, as part of an introductory course in space physics, students are using the VirES to download Swarm magnetic field data and make complex geomagnetic calculations, without prior knowledge of Python.

Learn more about using Swarm data with a virtual research platform, during the Swarm 10 Year Anniversary Science Conference, on 8 to 12 April 2024, in Copenhagen, Denmark, and at a designated poster presentation on Swarm VirES at the upcoming European Geosciences Union general assembly (EGU) from 15 to 19 April in Vienna, Austria.