.ipynb .pdf

repository open issue

JupyterHub

Contents

IPDxIRR_2F (Ionospheric plasma densities)

Abstract: Access to the derived plasma characteristics at 1Hz (level 2 product).

%load_ext watermark
%watermark -i -v -p viresclient,pandas,xarray,matplotlib

Python implementation: CPython
Python version       : 3.9.7
IPython version      : 8.0.1

viresclient: 0.10.1
pandas     : 1.4.1
xarray     : 0.21.1
matplotlib : 3.5.1

from viresclient import SwarmRequest
import datetime as dt
import matplotlib.pyplot as plt
from matplotlib.dates import DateFormatter

request = SwarmRequest()

IPDxIRR_2F product information

Derived plasma characteristics at 1Hz, for each Swarm spacecraft.

Documentation:

• https://earth.esa.int/web/guest/missions/esa-eo-missions/swarm/data-handbook/level-2-product-definitions#IPDxIRR_2F

Check what “IPD” data variables are available

request.available_collections("IPD", details=False)

{'IPD': ['SW_OPER_IPDAIRR_2F', 'SW_OPER_IPDBIRR_2F', 'SW_OPER_IPDCIRR_2F']}

request.available_measurements("IPD")

['Ne',
 'Te',
 'Background_Ne',
 'Foreground_Ne',
 'PCP_flag',
 'Grad_Ne_at_100km',
 'Grad_Ne_at_50km',
 'Grad_Ne_at_20km',
 'Grad_Ne_at_PCP_edge',
 'ROD',
 'RODI10s',
 'RODI20s',
 'delta_Ne10s',
 'delta_Ne20s',
 'delta_Ne40s',
 'Num_GPS_satellites',
 'mVTEC',
 'mROT',
 'mROTI10s',
 'mROTI20s',
 'IBI_flag',
 'Ionosphere_region_flag',
 'IPIR_index',
 'Ne_quality_flag',
 'TEC_STD']

Fetch three hours of IPD data

request.set_collection("SW_OPER_IPDAIRR_2F")
request.set_products(measurements=request.available_measurements("IPD"))
data = request.get_between(
    dt.datetime(2014,12,21, 0),
    dt.datetime(2014,12,21, 3)
)

Load and plot using pandas/matplotlib

df = data.as_dataframe()
df.head()

	Ionosphere_region_flag	Te	Grad_Ne_at_20km	IBI_flag	mROTI20s	Longitude	delta_Ne20s	Ne	TEC_STD	Radius	...	delta_Ne40s	RODI20s	PCP_flag	Foreground_Ne	Grad_Ne_at_100km	Background_Ne	IPIR_index	mROT	RODI10s	ROD
Timestamp
2014-12-21 00:00:00.196999936	0	2212.28	-1.047788	-1	0.002676	-128.771412	10266.500	1255163.2	3.131452	6.840395e+06	...	3811.1	7764.002532	0	1305371.000	-0.084919	1343599.375	7	-0.011013	10238.517220	0.0
2014-12-21 00:00:01.196999936	0	2165.19	0.338403	-1	0.002732	-128.772618	2830.850	1250357.6	3.122495	6.840404e+06	...	16343.3	7181.496228	0	1292046.000	-0.144009	1343599.375	6	-0.011013	3263.138721	0.0
2014-12-21 00:00:02.196999936	0	1544.87	0.133643	-1	0.002750	-128.773822	0.000	1265851.3	3.113831	6.840413e+06	...	3246.4	7181.496228	0	1312436.750	-0.058276	1343599.375	6	-0.008833	3263.138721	0.0
2014-12-21 00:00:03.196999936	0	1228.50	1.443077	-1	0.003277	-128.775026	2194.925	1312436.8	3.104260	6.840422e+06	...	848.3	7390.308480	0	1312436.750	-0.144613	1343599.375	6	-0.008151	3263.138721	0.0
2014-12-21 00:00:04.196999936	0	2681.51	-1.948789	-1	0.003744	-128.776229	9491.525	1253999.0	3.097485	6.840430e+06	...	6448.3	7554.331699	0	1315059.375	-0.039358	1343599.375	6	-0.007051	3263.138721	0.0

5 rows × 29 columns

df.columns

Index(['Ionosphere_region_flag', 'Te', 'Grad_Ne_at_20km', 'IBI_flag',
       'mROTI20s', 'Longitude', 'delta_Ne20s', 'Ne', 'TEC_STD', 'Radius',
       'Grad_Ne_at_50km', 'mVTEC', 'mROTI10s', 'Num_GPS_satellites',
       'Spacecraft', 'Grad_Ne_at_PCP_edge', 'delta_Ne10s', 'Ne_quality_flag',
       'Latitude', 'delta_Ne40s', 'RODI20s', 'PCP_flag', 'Foreground_Ne',
       'Grad_Ne_at_100km', 'Background_Ne', 'IPIR_index', 'mROT', 'RODI10s',
       'ROD'],
      dtype='object')

fig, axes = plt.subplots(nrows=7, ncols=1, figsize=(20,18), sharex=True)
df.plot(ax=axes[0], y=['Background_Ne', 'Foreground_Ne', 'Ne'], alpha=0.8)
df.plot(ax=axes[1], y=['Grad_Ne_at_100km', 'Grad_Ne_at_50km', 'Grad_Ne_at_20km'])
df.plot(ax=axes[2], y=['RODI10s', 'RODI20s'])
df.plot(ax=axes[3], y=['ROD'])
df.plot(ax=axes[4], y=['mROT'])
df.plot(ax=axes[5], y=['delta_Ne10s', 'delta_Ne20s', 'delta_Ne40s'])
df.plot(ax=axes[6], y=['mROTI20s', 'mROTI10s'])
axes[0].set_ylabel("[cm$^{-3}$]")
axes[1].set_ylabel("[cm$^{-3}$m$^{-1}$]")
axes[2].set_ylabel("[cm$^{-3}$s$^{-1}$]")
axes[3].set_ylabel("[cm$^{-3}$m$^{-1}$]")
axes[4].set_ylabel("[TECU s$^{-1}$]")
axes[5].set_ylabel("[cm$^{-3}$m$^{-1}$]")
axes[6].set_ylabel("[TECU s$^{-1}$]")
axes[6].set_xlabel("Timestamp")

for ax in axes:
    # Reformat time axis
    # https://www.earthdatascience.org/courses/earth-analytics-python/use-time-series-data-in-python/customize-dates--matplotlib-plots-python/
    ax.xaxis.set_major_formatter(DateFormatter("%Y-%m-%d\n%H:%M:%S"))
    ax.legend(loc="upper right")
    ax.grid()
fig.subplots_adjust(hspace=0)

Load as xarray

ds = data.as_xarray()
ds

<xarray.Dataset>
Dimensions:                 (Timestamp: 10800)
Coordinates:
  * Timestamp               (Timestamp) datetime64[ns] 2014-12-21T00:00:00.19...
Data variables: (12/29)
    Spacecraft              (Timestamp) object 'A' 'A' 'A' 'A' ... 'A' 'A' 'A'
    Ionosphere_region_flag  (Timestamp) int32 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0
    Te                      (Timestamp) float64 2.212e+03 ... 1.723e+03
    Grad_Ne_at_20km         (Timestamp) float64 -1.048 0.3384 ... 1.002 0.9148
    IBI_flag                (Timestamp) int32 -1 -1 -1 -1 -1 ... -1 -1 -1 -1 -1
    mROTI20s                (Timestamp) float64 0.002676 0.002732 ... 0.0114
    ...                      ...
    Grad_Ne_at_100km        (Timestamp) float64 -0.08492 -0.144 ... 0.9621
    Background_Ne           (Timestamp) float64 1.344e+06 1.344e+06 ... 4.29e+05
    IPIR_index              (Timestamp) int32 7 6 6 6 6 6 6 6 ... 4 4 4 4 4 4 4
    mROT                    (Timestamp) float64 -0.01101 -0.01101 ... 0.09621
    RODI10s                 (Timestamp) float64 1.024e+04 3.263e+03 ... 503.7
    ROD                     (Timestamp) float64 0.0 0.0 ... 7.28e+03 7.28e+03
Attributes:
    Sources:         ['SW_OPER_IPDAIRR_2F_20141221T000000_20141221T235959_0301']
    MagneticModels:  []
    RangeFilters:    []

xarray.Dataset

• Dimensions:
  • Timestamp: 10800
• Coordinates: (1)
  • Timestamp
    (Timestamp)
    datetime64[ns]
    2014-12-21T00:00:00.196999936 .....

    description :

    CDF_EPOCH of the measurement.

    array(['2014-12-21T00:00:00.196999936', '2014-12-21T00:00:01.196999936',
           '2014-12-21T00:00:02.196999936', ..., '2014-12-21T02:59:57.196999936',
           '2014-12-21T02:59:58.196999936', '2014-12-21T02:59:59.196999936'],
          dtype='datetime64[ns]')

• Data variables: (29)
  • Spacecraft
    (Timestamp)
    object
    'A' 'A' 'A' 'A' ... 'A' 'A' 'A' 'A'

    units :

    -

    description :

    Spacecraft identifier (values: 'A', 'B', 'C' or '-' if not available).

    array(['A', 'A', 'A', ..., 'A', 'A', 'A'], dtype=object)

  • Ionosphere_region_flag
    (Timestamp)
    int32
    0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0

    units :

    -

    description :

    0: equator, 1: mid-latitudes; 2: auroral oval; 3: polar cap.

    array([0, 0, 0, ..., 0, 0, 0], dtype=int32)

  • Te
    (Timestamp)
    float64
    2.212e+03 2.165e+03 ... 1.723e+03

    units :

    K

    description :

    Electron temperature, directly copied from the Langmuir probe files.

    array([2212.28, 2165.19, 1544.87, ..., 1730.93, 1727.83, 1722.59])

  • Grad_Ne_at_20km
    (Timestamp)
    float64
    -1.048 0.3384 ... 1.002 0.9148

    units :

    cm^-3/m

    description :

    The electron density gradient in a running window calculated via linear regression over 5 data points for the 2 Hz electron density data.

    array([-1.047788  ,  0.33840267,  0.13364267, ...,  0.91515467,
            1.00201333,  0.91482933])

  • IBI_flag
    (Timestamp)
    int32
    -1 -1 -1 -1 -1 ... -1 -1 -1 -1 -1

    units :

    -

    description :

    Plasma Bubble Index, copied from the level-2 Ionospheric Bubble Index product, IBIxTMS_2F.

    array([-1, -1, -1, ..., -1, -1, -1], dtype=int32)

  • mROTI20s
    (Timestamp)
    float64
    0.002676 0.002732 ... 0.0114

    units :

    TECU/s

    description :

    Median of Rate Of change of TEC Index (ROTI) from all available GPS satellites above 30 degrees. The ROTI of each satellite is the standard deviation of ROT over 20 seconds.

    array([0.00267555, 0.00273237, 0.00274996, ..., 0.01015663, 0.01085673,
           0.01140045])

  • Longitude
    (Timestamp)
    float64
    -128.8 -128.8 ... -175.4 -175.4

    units :

    deg

    description :

    Position in ITRF - Longitude.

    array([-128.77141157, -128.77261789, -128.77382226, ..., -175.36866182,
           -175.36925577, -175.36985342])

  • delta_Ne20s
    (Timestamp)
    float64
    1.027e+04 2.831e+03 ... 1.702e+03

    units :

    cm^-3

    description :

    Derived by subtracting Ne by its median filtered value in 20 seconds. As a result, delta_Ne20s indicates the electron density fluctuations smaller than 150 km.

    array([10266.5 ,  2830.85,     0.  , ...,  1800.4 ,  1819.9 ,  1701.65])

  • Ne
    (Timestamp)
    float64
    1.255e+06 1.25e+06 ... 6.468e+05

    units :

    cm^-3

    description :

    Plasma density, directly copied from the Langmuir probe files.

    array([1255163.2, 1250357.6, 1265851.3, ...,  631863.3,  639288.8,
            646784.8])

  • TEC_STD
    (Timestamp)
    float64
    3.131 3.122 3.114 ... 2.866 2.891

    units :

    TECU

    description :

    STD of VTEC from all GPS satellites.

    array([3.13145204, 3.12249454, 3.11383092, ..., 2.84071331, 2.86645798,
           2.89068396])

  • Radius
    (Timestamp)
    float64
    6.84e+06 6.84e+06 ... 6.835e+06

    units :

    m

    description :

    Position in ITRF - Radius.

    array([6840394.98513586, 6840403.89273225, 6840412.75499263, ...,
           6835430.74735898, 6835442.25596207, 6835453.76413   ])

  • Grad_Ne_at_50km
    (Timestamp)
    float64
    -0.4039 0.1449 ... 0.9347 0.9775

    units :

    cm^-3/m

    description :

    The electron density gradient in a running window calculated via linear regression over 13 data points for the 2 Hz electron density data.

    array([-0.40394   ,  0.1448767 , -0.1237337 , ...,  0.89959912,
            0.9347433 ,  0.97747429])

  • mVTEC
    (Timestamp)
    float64
    51.79 51.77 51.75 ... 20.84 20.94

    units :

    TECU

    description :

    Median of VTEC from all available GPS satellites above 30 degrees.

    array([51.78693878, 51.76898678, 51.74690333, ..., 20.73871127,
           20.8393707 , 20.94157002])

  • mROTI10s
    (Timestamp)
    float64
    0.001472 0.001386 ... 0.005609

    units :

    TECU/s

    description :

    Median of Rate Of change of TEC Index (ROTI) from all available GPS satellites above 30 degrees. The ROTI of each satellite is the standard deviation of ROT over 10 seconds.

    array([0.0014718 , 0.00138551, 0.00131007, ..., 0.00471655, 0.00559606,
           0.00560861])

  • Num_GPS_satellites
    (Timestamp)
    int32
    4 4 4 4 4 4 4 4 ... 6 6 6 6 6 6 6 6

    units :

    -

    description :

    Total number of tracked GPS satellites above 20 degrees.

    array([4, 4, 4, ..., 6, 6, 6], dtype=int32)

  • Grad_Ne_at_PCP_edge
    (Timestamp)
    float64
    0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0

    units :

    cm^-3/m

    description :

    The linear electron density gradient calculated over the edges of a patch. This variable is non-zero only at the edges of polar cap patches.

    array([0., 0., 0., ..., 0., 0., 0.])

  • delta_Ne10s
    (Timestamp)
    float64
    67.88 1.296e+04 ... 1.702e+03

    units :

    cm^-3

    description :

    Derived by subtracting Ne by its median filtered value in 10 seconds. As a result, delta_Ne10s indicates the electron density fluctuations smaller than 75 km.

    array([   67.875, 12961.6  ,     0.   , ...,  1800.4  ,  1819.9  ,
            1701.65 ])

  • Ne_quality_flag
    (Timestamp)
    int32
    20000 20000 20000 ... 20000 20000

    units :

    -

    description :

    Quality flag for the Ne data and the derived data from Ne, e.g., background density, foreground density etc.

    array([20000, 20000, 20000, ..., 20000, 20000, 20000], dtype=int32)

  • Latitude
    (Timestamp)
    float64
    -4.694 -4.757 ... 24.74 24.68

    units :

    deg

    description :

    Position in ITRF - Latitude.

    array([-4.69353311, -4.75741622, -4.82129752, ..., 24.8065803 ,
           24.74263539, 24.67869052])

  • delta_Ne40s
    (Timestamp)
    float64
    3.811e+03 1.634e+04 ... 1.702e+03

    units :

    cm^-3

    description :

    Derived by subtracting Ne by its median filtered value in 40 seconds. As a result, delta_Ne40s indicates the electron density fluctuations smaller than 300 km.

    array([ 3811.1 , 16343.3 ,  3246.4 , ...,  1800.4 ,  1819.9 ,  1701.65])

  • RODI20s
    (Timestamp)
    float64
    7.764e+03 7.181e+03 ... 943.2 907.9

    units :

    cm^-3/s

    description :

    Rate Of change of Density Index (RODI) is the standard deviation of ROD over 20 seconds.

    array([7764.00253246, 7181.49622793, 7181.49622793, ...,  923.42482842,
            943.21431292,  907.90593797])

  • PCP_flag
    (Timestamp)
    int32
    0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0

    units :

    -

    description :

    The polar cap patch flag: 0 if the plasma density measurement occurred OUTSIDE a polar cap patch. 1 if the plasma density measurement occurred at one of the edges of a polar cap patch (no plasma velocity measurements are available). 2 if the plasma density measurement occurred at the LEADING edge of a polar cap patch. 3 if the plasma density measurement occurred at the TRAILING edge of a polar cap patch. 4 if the plasma density measurement occurred INSIDE a polar cap patch proper. When no ion drift data is available, the leading and trailing edges cannot be distinguished. In this case the polar cap patch flag is set to 4 throughout the patch proper and to 1 throughout both edges.

    array([0, 0, 0, ..., 0, 0, 0], dtype=int32)

  • Foreground_Ne
    (Timestamp)
    float64
    1.305e+06 1.292e+06 ... 6.488e+05

    units :

    cm^-3

    description :

    Foreground density, as calculated from ndens using a percentile filter of 7 datapoints (2 Hz raw data) and 50 as the percentile.

    array([1305371.   , 1292046.   , 1312436.75 , ...,  635649.   ,
            643381.5  ,  648826.625])

  • Grad_Ne_at_100km
    (Timestamp)
    float64
    -0.08492 -0.144 ... 0.9369 0.9621

    units :

    cm^-3/m

    description :

    The electron density gradient in a running window calculated via linear regression over 27 data points for the 2 Hz electron density data.

    array([-0.08491926, -0.14400869, -0.05827591, ...,  0.91132594,
            0.9368809 ,  0.96208895])

  • Background_Ne
    (Timestamp)
    float64
    1.344e+06 1.344e+06 ... 4.29e+05

    units :

    cm^-3

    description :

    Background density, as calculated from Ne using a percentile filter of 551 datapoints (2 Hz raw data) and 35 as the percentile.

    array([1343599.375  , 1343599.375  , 1343599.375  , ...,  422318.09375,
            425527.09375,  429034.6875 ])

  • IPIR_index
    (Timestamp)
    int32
    7 6 6 6 6 6 6 6 ... 4 4 4 4 4 4 4 4

    units :

    -

    description :

    The numeric index for plasma fluctuations and irregularities: 0-3 low, 4-5 medium, and > 6 high level of ionosphericplasma irregularities.

    array([7, 6, 6, ..., 4, 4, 4], dtype=int32)

  • mROT
    (Timestamp)
    float64
    -0.01101 -0.01101 ... 0.09621

    units :

    TECU/s

    description :

    Median of Rate Of change of TEC (ROT) from all available GPS satellites above 30 degrees.

    array([-0.01101324, -0.01101324, -0.00883297, ...,  0.09318886,
            0.09159881,  0.0962132 ])

  • RODI10s
    (Timestamp)
    float64
    1.024e+04 3.263e+03 ... 654.8 503.7

    units :

    cm^-3/s

    description :

    Rate Of change of Density Index (RODI) is the standard deviation of ROD over 10 seconds.

    array([10238.51721984,  3263.13872093,  3263.13872093, ...,
             744.15328606,   654.83197202,   503.72685779])

  • ROD
    (Timestamp)
    float64
    0.0 0.0 0.0 ... 7.28e+03 7.28e+03

    units :

    cm^-3/s

    description :

    Rate Of change of Density

    array([   0. ,    0. ,    0. , ..., 7201.6, 7279.6, 7279.6])

• Attributes: (3)

  Sources :

  ['SW_OPER_IPDAIRR_2F_20141221T000000_20141221T235959_0301']

  MagneticModels :

  []

  RangeFilters :

  []

Alternative plot setup

To plot the data from xarray, we need a different plotting setup. This does however give us more control over the plot. The units are extracted directly from the xarray object.

fig, axes = plt.subplots(nrows=7, ncols=1, figsize=(20,18), sharex=True)
def subplot(ax=None, y=None, **kwargs):
    """Plot combination of variables onto a given axis"""
    units = ds[y[0]].units
    for var in y:
        ax.plot(ds["Timestamp"], ds[var], label=var, **kwargs)
        if units != ds[var].units:
            raise ValueError(f"Units mismatch for {var}")
    ax.set_ylabel(f"[{units}]")
    # Reformat time axis
    # https://www.earthdatascience.org/courses/earth-analytics-python/use-time-series-data-in-python/customize-dates--matplotlib-plots-python/
    ax.xaxis.set_major_formatter(DateFormatter("%Y-%m-%d\n%H:%M:%S"))
    ax.legend(loc="upper right")
    ax.grid()
subplot(ax=axes[0], y=['Background_Ne', 'Foreground_Ne', 'Ne'])
subplot(ax=axes[1], y=['Grad_Ne_at_100km', 'Grad_Ne_at_50km', 'Grad_Ne_at_20km'])
subplot(ax=axes[2], y=['RODI10s', 'RODI20s'])
subplot(ax=axes[3], y=['ROD'])
subplot(ax=axes[4], y=['mROT'])
subplot(ax=axes[5], y=['delta_Ne10s', 'delta_Ne20s', 'delta_Ne40s'])
subplot(ax=axes[6], y=['mROTI20s', 'mROTI10s'])
fig.subplots_adjust(hspace=0)

previous

EFIx_LP_1B (Langmuir probe 2Hz)

next

TECxTMS_2F (Total electron content)