MAGxLR_1B (Magnetic field 1Hz)

Abstract: Access to the low rate (1Hz) magnetic data (level 1b product), together with geomagnetic model evaluations (level 2 products).

%load_ext watermark
%watermark -i -v -p viresclient,pandas,xarray,matplotlib
Python implementation: CPython
Python version       : 3.9.7
IPython version      : 8.0.1

viresclient: 0.10.1
pandas     : 1.4.1
xarray     : 0.21.1
matplotlib : 3.5.1
from viresclient import SwarmRequest
import datetime as dt
import matplotlib.pyplot as plt

request = SwarmRequest()

Product information

This is one of the main products from Swarm - the 1Hz measurements of the magnetic field vector (B_NEC) and total intensity (F). These are derived from the Vector Field Magnetometer (VFM) and Absolute Scalar Magnetomer (ASM).

Documentation:

Measurements are available through VirES as part of collections with names containing MAGx_LR, for each Swarm spacecraft:

request.available_collections("MAG", details=False)
{'MAG': ['SW_OPER_MAGA_LR_1B', 'SW_OPER_MAGB_LR_1B', 'SW_OPER_MAGC_LR_1B']}

The measurements can be used together with geomagnetic model evaluations as shall be shown below.

Check what “MAG” data variables are available

request.available_measurements("MAG")
['F',
 'dF_AOCS',
 'dF_other',
 'F_error',
 'B_VFM',
 'B_NEC',
 'dB_Sun',
 'dB_AOCS',
 'dB_other',
 'B_error',
 'q_NEC_CRF',
 'Att_error',
 'Flags_F',
 'Flags_B',
 'Flags_q',
 'Flags_Platform',
 'ASM_Freq_Dev']

Check the names of available models

request.available_models(details=False)
['IGRF',
 'LCS-1',
 'MF7',
 'CHAOS-Core',
 'CHAOS-Static',
 'CHAOS-MMA-Primary',
 'CHAOS-MMA-Secondary',
 'MCO_SHA_2C',
 'MCO_SHA_2D',
 'MLI_SHA_2C',
 'MLI_SHA_2D',
 'MLI_SHA_2E',
 'MMA_SHA_2C-Primary',
 'MMA_SHA_2C-Secondary',
 'MMA_SHA_2F-Primary',
 'MMA_SHA_2F-Secondary',
 'MIO_SHA_2C-Primary',
 'MIO_SHA_2C-Secondary',
 'MIO_SHA_2D-Primary',
 'MIO_SHA_2D-Secondary',
 'AMPS',
 'MCO_SHA_2X',
 'CHAOS',
 'CHAOS-MMA',
 'MMA_SHA_2C',
 'MMA_SHA_2F',
 'MIO_SHA_2C',
 'MIO_SHA_2D',
 'SwarmCI']

Fetch some MAG data and models

We can fetch the data and the model predictions (evaluated on demand) at the same time. We can also subsample the data - here we subsample it to 10-seconds by specifying the “PT10S” sampling_step.

request.set_collection("SW_OPER_MAGA_LR_1B")
request.set_products(
    measurements=["F", "B_NEC"],
    models=["CHAOS-Core", "MCO_SHA_2D"],
    sampling_step="PT10S"
)
data = request.get_between(
    # 2014-01-01 00:00:00
    start_time = dt.datetime(2014,1,1, 0),
    # 2014-01-01 01:00:00
    end_time = dt.datetime(2014,1,1, 1)
)

See a list of the source files

data.sources
['SW_OPER_MAGA_LR_1B_20140101T000000_20140101T235959_0505_MDR_MAG_LR',
 'SW_OPER_MCO_SHA_2D_20131126T000000_20180101T000000_0401',
 'SW_OPER_MCO_SHA_2X_19970101T000000_20220206T235959_0709']

Load as a pandas dataframe

Use expand=True to extract vectors (B_NEC…) as separate columns (…_N, …_E, …_C)

df = data.as_dataframe(expand=True)
df.head()
Longitude F F_MCO_SHA_2D Spacecraft Radius F_CHAOS-Core Latitude B_NEC_MCO_SHA_2D_N B_NEC_MCO_SHA_2D_E B_NEC_MCO_SHA_2D_C B_NEC_N B_NEC_E B_NEC_C B_NEC_CHAOS-Core_N B_NEC_CHAOS-Core_E B_NEC_CHAOS-Core_C
Timestamp
2014-01-01 00:00:00 -14.116674 22867.5503 22874.211509 A 6878309.22 22874.518390 -1.228938 20113.623921 -4127.463956 -10081.454567 20103.5246 -4126.2621 -10086.9888 20113.280053 -4126.995331 -10083.028645
2014-01-01 00:00:10 -14.131424 22814.5656 22820.941425 A 6878381.17 22821.246013 -1.862521 19825.161844 -4163.127549 -10508.410652 19815.0914 -4160.9933 -10514.4074 19824.774929 -4162.642623 -10509.994055
2014-01-01 00:00:20 -14.146155 22763.2585 22769.369161 A 6878452.05 22769.667074 -2.496090 19533.553905 -4197.529054 -10920.860481 19523.4946 -4195.1968 -10926.9664 19533.121514 -4197.028325 -10922.447337
2014-01-01 00:00:30 -14.160861 22713.3703 22719.238240 A 6878521.87 22719.525385 -3.129644 19239.343572 -4230.601819 -11318.721366 19229.2386 -4228.4747 -11324.8335 19238.864097 -4230.085746 -11320.305498
2014-01-01 00:00:40 -14.175534 22664.7202 22670.304681 A 6878590.61 22670.577358 -3.763184 18943.075144 -4262.280634 -11701.947796 18932.8807 -4260.8424 -11708.0897 18942.547863 -4261.749636 -11703.522899

… or as an xarray dataset:

ds = data.as_xarray()
ds
<xarray.Dataset>
Dimensions:           (Timestamp: 360, NEC: 3)
Coordinates:
  * Timestamp         (Timestamp) datetime64[ns] 2014-01-01 ... 2014-01-01T00...
  * NEC               (NEC) <U1 'N' 'E' 'C'
Data variables:
    Spacecraft        (Timestamp) object 'A' 'A' 'A' 'A' 'A' ... 'A' 'A' 'A' 'A'
    Longitude         (Timestamp) float64 -14.12 -14.13 -14.15 ... 153.6 153.6
    B_NEC_MCO_SHA_2D  (Timestamp, NEC) float64 2.011e+04 ... 3.557e+04
    F                 (Timestamp) float64 2.287e+04 2.281e+04 ... 4.021e+04
    B_NEC_CHAOS-Core  (Timestamp, NEC) float64 2.011e+04 ... 3.557e+04
    B_NEC             (Timestamp, NEC) float64 2.01e+04 -4.126e+03 ... 3.558e+04
    F_MCO_SHA_2D      (Timestamp) float64 2.287e+04 2.282e+04 ... 4.021e+04
    Radius            (Timestamp) float64 6.878e+06 6.878e+06 ... 6.868e+06
    F_CHAOS-Core      (Timestamp) float64 2.287e+04 2.282e+04 ... 4.02e+04
    Latitude          (Timestamp) float64 -1.229 -1.863 -2.496 ... 48.14 48.77
Attributes:
    Sources:         ['SW_OPER_MAGA_LR_1B_20140101T000000_20140101T235959_050...
    MagneticModels:  ["CHAOS-Core = 'CHAOS-Core'(max_degree=20,min_degree=1)"...
    RangeFilters:    []

Fetch the residuals directly

Adding residuals=True to .set_products() will instead directly evaluate and return all data-model residuals

request = SwarmRequest()
request.set_collection("SW_OPER_MAGA_LR_1B")
request.set_products(
    measurements=["F", "B_NEC"],
    models=["CHAOS-Core", "MCO_SHA_2D"],
    residuals=True,
    sampling_step="PT10S"
)
data = request.get_between(
    start_time = dt.datetime(2014,1,1, 0),
    end_time = dt.datetime(2014,1,1, 1)
)
df = data.as_dataframe(expand=True)
df.head()
Longitude F_res_CHAOS-Core F_res_MCO_SHA_2D Spacecraft Radius Latitude B_NEC_res_MCO_SHA_2D_N B_NEC_res_MCO_SHA_2D_E B_NEC_res_MCO_SHA_2D_C B_NEC_res_CHAOS-Core_N B_NEC_res_CHAOS-Core_E B_NEC_res_CHAOS-Core_C
Timestamp
2014-01-01 00:00:00 -14.116674 -6.968090 -6.661209 A 6878309.22 -1.228938 -10.099321 1.201856 -5.534233 -9.755453 0.733231 -3.960155
2014-01-01 00:00:10 -14.131424 -6.680413 -6.375825 A 6878381.17 -1.862521 -10.070444 2.134249 -5.996748 -9.683529 1.649323 -4.413345
2014-01-01 00:00:20 -14.146155 -6.408574 -6.110661 A 6878452.05 -2.496090 -10.059305 2.332254 -6.105919 -9.626914 1.831525 -4.519063
2014-01-01 00:00:30 -14.160861 -6.155085 -5.867940 A 6878521.87 -3.129644 -10.104972 2.127119 -6.112134 -9.625497 1.611046 -4.528002
2014-01-01 00:00:40 -14.175534 -5.857158 -5.584481 A 6878590.61 -3.763184 -10.194444 1.438234 -6.141904 -9.667163 0.907236 -4.566801

Plot the scalar residuals

… using the pandas method:

ax = df.plot(
    y=["F_res_CHAOS-Core", "F_res_MCO_SHA_2D"],
    figsize=(15,5),
    grid=True
)
ax.set_xlabel("Timestamp")
ax.set_ylabel("[nT]");
../_images/03a1_Demo-MAGx_LR_1B_21_0.png

… using matplotlib interface

NB: we are doing plt.plot(x, y) with x as df.index (the time-based index of df), and y as df[".."]

plt.figure(figsize=(15,5))
plt.plot(
    df.index,
    df["F_res_CHAOS-Core"],
    label="F_res_CHAOS-Core"
)
plt.plot(
    df.index,
    df["F_res_MCO_SHA_2D"],
    label="F_res_MCO_SHA_2D"
)
plt.xlabel("Timestamp")
plt.ylabel("[nT]")
plt.grid()
plt.legend();
../_images/03a1_Demo-MAGx_LR_1B_23_0.png

… using matplotlib interface (Object Oriented style)

This is the recommended route for making more complicated figures

fig, ax = plt.subplots(figsize=(15,5))
ax.plot(
    df.index,
    df["F_res_CHAOS-Core"],
    label="F_res_CHAOS-Core"
)
ax.plot(
    df.index,
    df["F_res_MCO_SHA_2D"],
    label="F_res_MCO_SHA_2D"
)
ax.set_xlabel("Timestamp")
ax.set_ylabel("[nT]")
ax.grid()
ax.legend();
../_images/03a1_Demo-MAGx_LR_1B_25_0.png

Plot the vector components

fig, axes = plt.subplots(nrows=3, ncols=1, figsize=(15,10), sharex=True)
for component, ax in zip("NEC", axes):
    for model_name in ("CHAOS-Core", "MCO_SHA_2D"):
        ax.plot(
            df.index,
            df[f"B_NEC_res_{model_name}_{component}"],
            label=model_name
        )
    ax.set_ylabel(f"{component}\n[nT]")
    ax.legend()
axes[0].set_title("Residuals to models (NEC components)")
axes[2].set_xlabel("Timestamp");
../_images/03a1_Demo-MAGx_LR_1B_27_0.png

Similar plotting, using the data via xarray instead

xarray provides a more sophisticated data structure that is more suitable for the complex vector data we are accessing, together with nice stuff like unit and other metadata support. Unfortunately due to the extra complexity, this can make it difficult to use right away.

ds = data.as_xarray()
ds
<xarray.Dataset>
Dimensions:               (Timestamp: 360, NEC: 3)
Coordinates:
  * Timestamp             (Timestamp) datetime64[ns] 2014-01-01 ... 2014-01-0...
  * NEC                   (NEC) <U1 'N' 'E' 'C'
Data variables:
    Spacecraft            (Timestamp) object 'A' 'A' 'A' 'A' ... 'A' 'A' 'A' 'A'
    Longitude             (Timestamp) float64 -14.12 -14.13 ... 153.6 153.6
    B_NEC_res_MCO_SHA_2D  (Timestamp, NEC) float64 -10.1 1.202 ... 2.782 8.984
    F_res_CHAOS-Core      (Timestamp) float64 -6.968 -6.68 ... 5.017 5.017
    B_NEC_res_CHAOS-Core  (Timestamp, NEC) float64 -9.755 0.7332 ... 2.904 9.926
    F_res_MCO_SHA_2D      (Timestamp) float64 -6.661 -6.376 ... 3.153 3.108
    Radius                (Timestamp) float64 6.878e+06 6.878e+06 ... 6.868e+06
    Latitude              (Timestamp) float64 -1.229 -1.863 ... 48.14 48.77
Attributes:
    Sources:         ['SW_OPER_MAGA_LR_1B_20140101T000000_20140101T235959_050...
    MagneticModels:  ["CHAOS-Core = 'CHAOS-Core'(max_degree=20,min_degree=1)"...
    RangeFilters:    []
fig, axes = plt.subplots(nrows=3, ncols=1, figsize=(15,10), sharex=True)
for i, ax in enumerate(axes):
    for model_name in ("CHAOS-Core", "MCO_SHA_2D"):
        ax.plot(
            ds["Timestamp"],
            ds[f"B_NEC_res_{model_name}"][:, i],
            label=model_name
        )
    ax.set_ylabel("NEC"[i] + " [nT]")
    ax.legend()
axes[0].set_title("Residuals to models (NEC components)")
axes[2].set_xlabel("Timestamp");
../_images/03a1_Demo-MAGx_LR_1B_30_0.png

Note that xarray also allows convenient direct plotting like:

ds["B_NEC_res_CHAOS-Core"].plot.line(x="Timestamp");
../_images/03a1_Demo-MAGx_LR_1B_32_0.png

Access multiple MAG datasets simultaneously

It is possible to fetch data from multiple collections simultaneously. Here we fetch the measurements from Swarm Alpha and Bravo. In the returned data, you can differentiate between them using the “Spacecraft” column.

request = SwarmRequest()
request.set_collection("SW_OPER_MAGA_LR_1B", "SW_OPER_MAGC_LR_1B")
request.set_products(
    measurements=["F", "B_NEC"],
    models=["CHAOS-Core",],
    residuals=True,
    sampling_step="PT10S"
)
data = request.get_between(
    start_time = dt.datetime(2014,1,1, 0),
    end_time = dt.datetime(2014,1,1, 1)
)
df = data.as_dataframe(expand=True)
df.head()
Longitude F_res_CHAOS-Core Spacecraft Radius Latitude B_NEC_res_CHAOS-Core_N B_NEC_res_CHAOS-Core_E B_NEC_res_CHAOS-Core_C
Timestamp
2014-01-01 00:00:00 -14.116674 -6.968090 A 6878309.22 -1.228938 -9.755453 0.733231 -3.960155
2014-01-01 00:00:10 -14.131424 -6.680413 A 6878381.17 -1.862521 -9.683529 1.649323 -4.413345
2014-01-01 00:00:20 -14.146155 -6.408574 A 6878452.05 -2.496090 -9.626914 1.831525 -4.519063
2014-01-01 00:00:30 -14.160861 -6.155085 A 6878521.87 -3.129644 -9.625497 1.611046 -4.528002
2014-01-01 00:00:40 -14.175534 -5.857158 A 6878590.61 -3.763184 -9.667163 0.907236 -4.566801
df[df["Spacecraft"] == "A"].head()
Longitude F_res_CHAOS-Core Spacecraft Radius Latitude B_NEC_res_CHAOS-Core_N B_NEC_res_CHAOS-Core_E B_NEC_res_CHAOS-Core_C
Timestamp
2014-01-01 00:00:00 -14.116674 -6.968090 A 6878309.22 -1.228938 -9.755453 0.733231 -3.960155
2014-01-01 00:00:10 -14.131424 -6.680413 A 6878381.17 -1.862521 -9.683529 1.649323 -4.413345
2014-01-01 00:00:20 -14.146155 -6.408574 A 6878452.05 -2.496090 -9.626914 1.831525 -4.519063
2014-01-01 00:00:30 -14.160861 -6.155085 A 6878521.87 -3.129644 -9.625497 1.611046 -4.528002
2014-01-01 00:00:40 -14.175534 -5.857158 A 6878590.61 -3.763184 -9.667163 0.907236 -4.566801
df[df["Spacecraft"] == "C"].head()
Longitude F_res_CHAOS-Core Spacecraft Radius Latitude B_NEC_res_CHAOS-Core_N B_NEC_res_CHAOS-Core_E B_NEC_res_CHAOS-Core_C
Timestamp
2014-01-01 00:00:00 -14.420068 -10.407173 C 6877665.99 5.908082 -10.454239 1.825332 -0.439921
2014-01-01 00:00:10 -14.434576 -10.032660 C 6877747.67 5.274386 -10.246696 1.962515 -1.050964
2014-01-01 00:00:20 -14.449141 -9.740029 C 6877828.39 4.640702 -10.156212 1.867993 -1.480355
2014-01-01 00:00:30 -14.463755 -9.523576 C 6877908.15 4.007030 -10.259488 1.473833 -2.004028
2014-01-01 00:00:40 -14.478412 -9.298362 C 6877986.93 3.373371 -10.349996 1.005463 -2.391989

… or using xarray

ds = data.as_xarray()
ds.where(ds["Spacecraft"] == "A", drop=True)
<xarray.Dataset>
Dimensions:               (Timestamp: 360, NEC: 3)
Coordinates:
  * Timestamp             (Timestamp) datetime64[ns] 2014-01-01 ... 2014-01-0...
  * NEC                   (NEC) <U1 'N' 'E' 'C'
Data variables:
    Spacecraft            (Timestamp) object 'A' 'A' 'A' 'A' ... 'A' 'A' 'A' 'A'
    Longitude             (Timestamp) float64 -14.12 -14.13 ... 153.6 153.6
    F_res_CHAOS-Core      (Timestamp) float64 -6.968 -6.68 ... 5.017 5.017
    B_NEC_res_CHAOS-Core  (Timestamp, NEC) float64 -9.755 0.7332 ... 2.904 9.926
    Radius                (Timestamp) float64 6.878e+06 6.878e+06 ... 6.868e+06
    Latitude              (Timestamp) float64 -1.229 -1.863 ... 48.14 48.77
Attributes:
    Sources:         ['SW_OPER_MAGA_LR_1B_20140101T000000_20140101T235959_050...
    MagneticModels:  ["CHAOS-Core = 'CHAOS-Core'(max_degree=20,min_degree=1)"]
    RangeFilters:    []