MAGxLR_1B (Magnetic field 1Hz)#

Abstract: Access to the low rate (1Hz) magnetic data (level 1b product), together with geomagnetic model evaluations (level 2 products).

%load_ext watermark
%watermark -i -v -p viresclient,pandas,xarray,matplotlib
Python implementation: CPython
Python version       : 3.11.6
IPython version      : 8.18.0

viresclient: 0.15.0
pandas     : 2.1.3
xarray     : 2023.12.0
matplotlib : 3.8.2
from viresclient import SwarmRequest
import datetime as dt
import matplotlib.pyplot as plt

request = SwarmRequest()

Product information#

This is one of the main products from Swarm - the 1Hz measurements of the magnetic field vector (B_NEC) and total intensity (F). These are derived from the Vector Field Magnetometer (VFM) and Absolute Scalar Magnetomer (ASM).

Documentation:

Measurements are available through VirES as part of collections with names containing MAGx_LR, for each Swarm spacecraft:

request.available_collections("MAG", details=False)
{'MAG': ['SW_OPER_MAGA_LR_1B',
  'SW_OPER_MAGB_LR_1B',
  'SW_OPER_MAGC_LR_1B',
  'SW_FAST_MAGA_LR_1B',
  'SW_FAST_MAGB_LR_1B',
  'SW_FAST_MAGC_LR_1B']}

The measurements can be used together with geomagnetic model evaluations as shall be shown below.

Check what “MAG” data variables are available#

request.available_measurements("MAG")
['F',
 'dF_Sun',
 'dF_AOCS',
 'dF_other',
 'F_error',
 'B_VFM',
 'B_NEC',
 'dB_Sun',
 'dB_AOCS',
 'dB_other',
 'B_error',
 'q_NEC_CRF',
 'Att_error',
 'Flags_F',
 'Flags_B',
 'Flags_q',
 'Flags_Platform',
 'ASM_Freq_Dev']

Check the names of available models#

request.available_models(details=False)
['IGRF',
 'LCS-1',
 'MF7',
 'CHAOS-Core',
 'CHAOS-Static',
 'CHAOS-MMA-Primary',
 'CHAOS-MMA-Secondary',
 'CHAOS-MIO',
 'MCO_SHA_2C',
 'MCO_SHA_2D',
 'MLI_SHA_2C',
 'MLI_SHA_2D',
 'MLI_SHA_2E',
 'MMA_SHA_2C-Primary',
 'MMA_SHA_2C-Secondary',
 'MMA_SHA_2F-Primary',
 'MMA_SHA_2F-Secondary',
 'MIO_SHA_2C-Primary',
 'MIO_SHA_2C-Secondary',
 'MIO_SHA_2D-Primary',
 'MIO_SHA_2D-Secondary',
 'AMPS',
 'MCO_SHA_2X',
 'CHAOS',
 'CHAOS-MMA',
 'MMA_SHA_2C',
 'MMA_SHA_2F',
 'MIO_SHA_2C',
 'MIO_SHA_2D',
 'SwarmCI']

Fetch some MAG data and models#

We can fetch the data and the model predictions (evaluated on demand) at the same time. We can also subsample the data - here we subsample it to 10-seconds by specifying the “PT10S” sampling_step.

request.set_collection("SW_OPER_MAGA_LR_1B")
request.set_products(
    measurements=["F", "B_NEC"],
    models=["CHAOS-Core", "MCO_SHA_2D"],
    sampling_step="PT10S"
)
data = request.get_between(
    # 2014-01-01 00:00:00
    start_time = dt.datetime(2014,1,1, 0),
    # 2014-01-01 01:00:00
    end_time = dt.datetime(2014,1,1, 1)
)

See a list of the source files#

data.sources
['SW_OPER_MAGA_LR_1B_20140101T000000_20140101T235959_0701_MDR_MAG_LR',
 'SW_OPER_MCO_SHA_2D_20131126T000000_20180101T000000_0401',
 'SW_OPER_MCO_SHA_2X_19970101T000000_20260206T235959_0805']

Load as a pandas dataframe#

Use expand=True to extract vectors (B_NEC…) as separate columns (…_N, …_E, …_C)

df = data.as_dataframe(expand=True)
df.head()
F F_MCO_SHA_2D Radius Longitude Spacecraft F_CHAOS-Core Latitude B_NEC_CHAOS-Core_N B_NEC_CHAOS-Core_E B_NEC_CHAOS-Core_C B_NEC_N B_NEC_E B_NEC_C B_NEC_MCO_SHA_2D_N B_NEC_MCO_SHA_2D_E B_NEC_MCO_SHA_2D_C
Timestamp
2014-01-01 00:00:00 22867.4800 22874.211386 6878309.23 -14.116674 A 22874.102418 -1.228938 20113.300310 -4127.295736 -10081.921542 20103.4954 -4125.6029 -10086.8732 20113.623790 -4127.463942 -10081.454557
2014-01-01 00:00:10 22814.4928 22820.941311 6878381.18 -14.131424 A 22820.824671 -1.862521 19824.823720 -4162.950034 -10508.865313 19815.0284 -4160.3663 -10514.3464 19825.161759 -4163.127529 -10508.410571
2014-01-01 00:00:20 22763.1833 22769.369047 6878452.06 -14.146155 A 22769.247190 -2.496090 19533.204562 -4197.343160 -10921.302465 19523.4054 -4194.6083 -10926.9383 19533.553823 -4197.529033 -10920.860399
2014-01-01 00:00:30 22713.2926 22719.238127 6878521.88 -14.160861 A 22719.113563 -3.129644 19238.986055 -4230.408364 -11319.151105 19229.1766 -4227.8647 -11324.7658 19239.343493 -4230.601798 -11318.721282
2014-01-01 00:00:40 22664.6401 22670.304568 6878590.62 -14.175534 A 22670.179741 -3.763184 18942.712049 -4262.080309 -11702.366477 18932.8056 -4260.1382 -11708.0771 18943.075066 -4262.280613 -11701.947711

… or as an xarray dataset:#

ds = data.as_xarray()
ds
<xarray.Dataset>
Dimensions:           (Timestamp: 360, NEC: 3)
Coordinates:
  * Timestamp         (Timestamp) datetime64[ns] 2014-01-01 ... 2014-01-01T00...
  * NEC               (NEC) <U1 'N' 'E' 'C'
Data variables:
    Spacecraft        (Timestamp) object 'A' 'A' 'A' 'A' 'A' ... 'A' 'A' 'A' 'A'
    F                 (Timestamp) float64 2.287e+04 2.281e+04 ... 4.021e+04
    F_MCO_SHA_2D      (Timestamp) float64 2.287e+04 2.282e+04 ... 4.021e+04
    Radius            (Timestamp) float64 6.878e+06 6.878e+06 ... 6.868e+06
    B_NEC_MCO_SHA_2D  (Timestamp, NEC) float64 2.011e+04 ... 3.557e+04
    Longitude         (Timestamp) float64 -14.12 -14.13 -14.15 ... 153.6 153.6
    B_NEC             (Timestamp, NEC) float64 2.01e+04 -4.126e+03 ... 3.558e+04
    F_CHAOS-Core      (Timestamp) float64 2.287e+04 2.282e+04 ... 4.02e+04
    Latitude          (Timestamp) float64 -1.229 -1.863 -2.496 ... 48.14 48.77
    B_NEC_CHAOS-Core  (Timestamp, NEC) float64 2.011e+04 ... 3.557e+04
Attributes:
    Sources:         ['SW_OPER_MAGA_LR_1B_20140101T000000_20140101T235959_070...
    MagneticModels:  ["CHAOS-Core = 'CHAOS-Core'(max_degree=20,min_degree=1)"...
    AppliedFilters:  []

Fetch the residuals directly#

Adding residuals=True to .set_products() will instead directly evaluate and return all data-model residuals

request = SwarmRequest()
request.set_collection("SW_OPER_MAGA_LR_1B")
request.set_products(
    measurements=["F", "B_NEC"],
    models=["CHAOS-Core", "MCO_SHA_2D"],
    residuals=True,
    sampling_step="PT10S"
)
data = request.get_between(
    start_time = dt.datetime(2014,1,1, 0),
    end_time = dt.datetime(2014,1,1, 1)
)
df = data.as_dataframe(expand=True)
df.head()
F_res_CHAOS-Core Radius Longitude F_res_MCO_SHA_2D Spacecraft Latitude B_NEC_res_MCO_SHA_2D_N B_NEC_res_MCO_SHA_2D_E B_NEC_res_MCO_SHA_2D_C B_NEC_res_CHAOS-Core_N B_NEC_res_CHAOS-Core_E B_NEC_res_CHAOS-Core_C
Timestamp
2014-01-01 00:00:00 -6.622418 6878309.23 -14.116674 -6.731386 A -1.228938 -10.128390 1.861042 -5.418643 -9.804910 1.692836 -4.951658
2014-01-01 00:00:10 -6.331871 6878381.18 -14.131424 -6.448511 A -1.862521 -10.133359 2.761229 -5.935829 -9.795320 2.583734 -5.481087
2014-01-01 00:00:20 -6.063890 6878452.06 -14.146155 -6.185747 A -2.496090 -10.148423 2.920733 -6.077901 -9.799162 2.734860 -5.635835
2014-01-01 00:00:30 -5.820963 6878521.88 -14.160861 -5.945527 A -3.129644 -10.166893 2.737098 -6.044518 -9.809455 2.543664 -5.614695
2014-01-01 00:00:40 -5.539641 6878590.62 -14.175534 -5.664468 A -3.763184 -10.269466 2.142413 -6.129389 -9.906449 1.942109 -5.710623

Plot the scalar residuals#

… using the pandas method:#

ax = df.plot(
    y=["F_res_CHAOS-Core", "F_res_MCO_SHA_2D"],
    figsize=(15,5),
    grid=True
)
ax.set_xlabel("Timestamp")
ax.set_ylabel("[nT]");
../_images/b23ed3ce222779a28d6c3a0f0f17d67e850c9b13e76968b55eb97c681cf4fb5f.png

… using matplotlib interface#

NB: we are doing plt.plot(x, y) with x as df.index (the time-based index of df), and y as df[".."]

plt.figure(figsize=(15,5))
plt.plot(
    df.index,
    df["F_res_CHAOS-Core"],
    label="F_res_CHAOS-Core"
)
plt.plot(
    df.index,
    df["F_res_MCO_SHA_2D"],
    label="F_res_MCO_SHA_2D"
)
plt.xlabel("Timestamp")
plt.ylabel("[nT]")
plt.grid()
plt.legend();
../_images/cc180e78b92089ff50a07020f7d1464ab81b427f86269648534daf90e786475b.png

… using matplotlib interface (Object Oriented style)#

This is the recommended route for making more complicated figures

fig, ax = plt.subplots(figsize=(15,5))
ax.plot(
    df.index,
    df["F_res_CHAOS-Core"],
    label="F_res_CHAOS-Core"
)
ax.plot(
    df.index,
    df["F_res_MCO_SHA_2D"],
    label="F_res_MCO_SHA_2D"
)
ax.set_xlabel("Timestamp")
ax.set_ylabel("[nT]")
ax.grid()
ax.legend();
../_images/cc180e78b92089ff50a07020f7d1464ab81b427f86269648534daf90e786475b.png

Plot the vector components#

fig, axes = plt.subplots(nrows=3, ncols=1, figsize=(15,10), sharex=True)
for component, ax in zip("NEC", axes):
    for model_name in ("CHAOS-Core", "MCO_SHA_2D"):
        ax.plot(
            df.index,
            df[f"B_NEC_res_{model_name}_{component}"],
            label=model_name
        )
    ax.set_ylabel(f"{component}\n[nT]")
    ax.legend()
axes[0].set_title("Residuals to models (NEC components)")
axes[2].set_xlabel("Timestamp");
../_images/cd745d52400cfdb2d12322ef8f1851eaf4dc88d3ef01bea42c778435a4f0b853.png

Similar plotting, using the data via xarray instead#

xarray provides a more sophisticated data structure that is more suitable for the complex vector data we are accessing, together with nice stuff like unit and other metadata support. Unfortunately due to the extra complexity, this can make it difficult to use right away.

ds = data.as_xarray()
ds
<xarray.Dataset>
Dimensions:               (Timestamp: 360, NEC: 3)
Coordinates:
  * Timestamp             (Timestamp) datetime64[ns] 2014-01-01 ... 2014-01-0...
  * NEC                   (NEC) <U1 'N' 'E' 'C'
Data variables:
    Spacecraft            (Timestamp) object 'A' 'A' 'A' 'A' ... 'A' 'A' 'A' 'A'
    B_NEC_res_MCO_SHA_2D  (Timestamp, NEC) float64 -10.13 1.861 ... 3.677 9.109
    F_res_CHAOS-Core      (Timestamp) float64 -6.622 -6.332 ... 4.011 3.99
    Radius                (Timestamp) float64 6.878e+06 6.878e+06 ... 6.868e+06
    Longitude             (Timestamp) float64 -14.12 -14.13 ... 153.6 153.6
    F_res_MCO_SHA_2D      (Timestamp) float64 -6.731 -6.449 ... 3.122 3.076
    B_NEC_res_CHAOS-Core  (Timestamp, NEC) float64 -9.805 1.693 ... 3.848 8.99
    Latitude              (Timestamp) float64 -1.229 -1.863 ... 48.14 48.77
Attributes:
    Sources:         ['SW_OPER_MAGA_LR_1B_20140101T000000_20140101T235959_070...
    MagneticModels:  ["CHAOS-Core = 'CHAOS-Core'(max_degree=20,min_degree=1)"...
    AppliedFilters:  []
fig, axes = plt.subplots(nrows=3, ncols=1, figsize=(15,10), sharex=True)
for i, ax in enumerate(axes):
    for model_name in ("CHAOS-Core", "MCO_SHA_2D"):
        ax.plot(
            ds["Timestamp"],
            ds[f"B_NEC_res_{model_name}"][:, i],
            label=model_name
        )
    ax.set_ylabel("NEC"[i] + " [nT]")
    ax.legend()
axes[0].set_title("Residuals to models (NEC components)")
axes[2].set_xlabel("Timestamp");
../_images/97850aa8111c71cb4e203bd2e337352c07d2c8c2c0813ea3f9b1b9b6faec57c9.png

Note that xarray also allows convenient direct plotting like:

ds["B_NEC_res_CHAOS-Core"].plot.line(x="Timestamp");
../_images/386a306e434cdb5003e7ec3f302d3617ea31c7241f203cc9a934d0b1972e8fac.png

Access multiple MAG datasets simultaneously#

It is possible to fetch data from multiple collections simultaneously. Here we fetch the measurements from Swarm Alpha and Bravo. In the returned data, you can differentiate between them using the “Spacecraft” column.

request = SwarmRequest()
request.set_collection("SW_OPER_MAGA_LR_1B", "SW_OPER_MAGC_LR_1B")
request.set_products(
    measurements=["F", "B_NEC"],
    models=["CHAOS-Core",],
    residuals=True,
    sampling_step="PT10S"
)
data = request.get_between(
    start_time = dt.datetime(2014,1,1, 0),
    end_time = dt.datetime(2014,1,1, 1)
)
df = data.as_dataframe(expand=True)
df.head()
F_res_CHAOS-Core Radius Longitude Spacecraft Latitude B_NEC_res_CHAOS-Core_N B_NEC_res_CHAOS-Core_E B_NEC_res_CHAOS-Core_C
Timestamp
2014-01-01 00:00:00 -6.622418 6878309.23 -14.116674 A -1.228938 -9.804910 1.692836 -4.951658
2014-01-01 00:00:10 -6.331871 6878381.18 -14.131424 A -1.862521 -9.795320 2.583734 -5.481087
2014-01-01 00:00:20 -6.063890 6878452.06 -14.146155 A -2.496090 -9.799162 2.734860 -5.635835
2014-01-01 00:00:30 -5.820963 6878521.88 -14.160861 A -3.129644 -9.809455 2.543664 -5.614695
2014-01-01 00:00:40 -5.539641 6878590.62 -14.175534 A -3.763184 -9.906449 1.942109 -5.710623
df[df["Spacecraft"] == "A"].head()
F_res_CHAOS-Core Radius Longitude Spacecraft Latitude B_NEC_res_CHAOS-Core_N B_NEC_res_CHAOS-Core_E B_NEC_res_CHAOS-Core_C
Timestamp
2014-01-01 00:00:00 -6.622418 6878309.23 -14.116674 A -1.228938 -9.804910 1.692836 -4.951658
2014-01-01 00:00:10 -6.331871 6878381.18 -14.131424 A -1.862521 -9.795320 2.583734 -5.481087
2014-01-01 00:00:20 -6.063890 6878452.06 -14.146155 A -2.496090 -9.799162 2.734860 -5.635835
2014-01-01 00:00:30 -5.820963 6878521.88 -14.160861 A -3.129644 -9.809455 2.543664 -5.614695
2014-01-01 00:00:40 -5.539641 6878590.62 -14.175534 A -3.763184 -9.906449 1.942109 -5.710623
df[df["Spacecraft"] == "C"].head()
F_res_CHAOS-Core Radius Longitude Spacecraft Latitude B_NEC_res_CHAOS-Core_N B_NEC_res_CHAOS-Core_E B_NEC_res_CHAOS-Core_C
Timestamp
2014-01-01 00:00:00 -10.413440 6877665.96 -14.420068 C 5.908082 -10.533725 2.390138 -1.289309
2014-01-01 00:00:10 -10.000957 6877747.64 -14.434576 C 5.274386 -10.283579 2.485700 -1.748991
2014-01-01 00:00:20 -9.667835 6877828.36 -14.449141 C 4.640702 -10.166119 2.401007 -2.174997
2014-01-01 00:00:30 -9.409666 6877908.12 -14.463755 C 4.007030 -10.259289 2.011146 -2.763503
2014-01-01 00:00:40 -9.142413 6877986.90 -14.478412 C 3.373371 -10.333386 1.515452 -3.171075

… or using xarray#

ds = data.as_xarray()
ds.where(ds["Spacecraft"] == "A", drop=True)
<xarray.Dataset>
Dimensions:               (Timestamp: 360, NEC: 3)
Coordinates:
  * Timestamp             (Timestamp) datetime64[ns] 2014-01-01 ... 2014-01-0...
  * NEC                   (NEC) <U1 'N' 'E' 'C'
Data variables:
    Spacecraft            (Timestamp) object 'A' 'A' 'A' 'A' ... 'A' 'A' 'A' 'A'
    F_res_CHAOS-Core      (Timestamp) float64 -6.622 -6.332 ... 4.011 3.99
    Radius                (Timestamp) float64 6.878e+06 6.878e+06 ... 6.868e+06
    Longitude             (Timestamp) float64 -14.12 -14.13 ... 153.6 153.6
    B_NEC_res_CHAOS-Core  (Timestamp, NEC) float64 -9.805 1.693 ... 3.848 8.99
    Latitude              (Timestamp) float64 -1.229 -1.863 ... 48.14 48.77
Attributes:
    Sources:         ['SW_OPER_MAGA_LR_1B_20140101T000000_20140101T235959_070...
    MagneticModels:  ["CHAOS-Core = 'CHAOS-Core'(max_degree=20,min_degree=1)"]
    AppliedFilters:  []