.ipynb .pdf

repository open issue

JupyterHub

Contents

IPDxIRR_2F (Ionospheric plasma densities)

Abstract: Access to the derived plasma characteristics at 1Hz (level 2 product).

%load_ext watermark
%watermark -i -v -p viresclient,pandas,xarray,matplotlib

Python implementation: CPython
Python version       : 3.9.7
IPython version      : 8.0.1

viresclient: 0.11.3
pandas     : 1.4.1
xarray     : 2023.8.0
matplotlib : 3.5.1

from viresclient import SwarmRequest
import datetime as dt
import matplotlib.pyplot as plt
from matplotlib.dates import DateFormatter

request = SwarmRequest()

IPDxIRR_2F product information

Derived plasma characteristics at 1Hz, for each Swarm spacecraft.

Documentation:

• https://earth.esa.int/web/guest/missions/esa-eo-missions/swarm/data-handbook/level-2-product-definitions#IPDxIRR_2F

Check what “IPD” data variables are available

request.available_collections("IPD", details=False)

{'IPD': ['SW_OPER_IPDAIRR_2F', 'SW_OPER_IPDBIRR_2F', 'SW_OPER_IPDCIRR_2F']}

request.available_measurements("IPD")

['Ne',
 'Te',
 'Background_Ne',
 'Foreground_Ne',
 'PCP_flag',
 'Grad_Ne_at_100km',
 'Grad_Ne_at_50km',
 'Grad_Ne_at_20km',
 'Grad_Ne_at_PCP_edge',
 'ROD',
 'RODI10s',
 'RODI20s',
 'delta_Ne10s',
 'delta_Ne20s',
 'delta_Ne40s',
 'Num_GPS_satellites',
 'mVTEC',
 'mROT',
 'mROTI10s',
 'mROTI20s',
 'IBI_flag',
 'Ionosphere_region_flag',
 'IPIR_index',
 'Ne_quality_flag',
 'TEC_STD']

Fetch three hours of IPD data

request.set_collection("SW_OPER_IPDAIRR_2F")
request.set_products(measurements=request.available_measurements("IPD"))
data = request.get_between(
    dt.datetime(2014,12,21, 0),
    dt.datetime(2014,12,21, 3)
)

Load and plot using pandas/matplotlib

df = data.as_dataframe()
df.head()

	Grad_Ne_at_PCP_edge	delta_Ne40s	Ne	TEC_STD	IPIR_index	Ne_quality_flag	PCP_flag	Grad_Ne_at_50km	Ionosphere_region_flag	RODI10s	...	Background_Ne	Longitude	IBI_flag	Te	Spacecraft	Num_GPS_satellites	delta_Ne10s	mROT	delta_Ne20s	Foreground_Ne
Timestamp
2014-12-21 00:00:00.196999936	0.0	3811.1	1255163.2	3.131451	7	20000	0	-0.403940	0	10238.517220	...	1343599.375	-128.771412	-1	2212.278353	A	4	67.875	-0.011013	10266.500	1305371.000
2014-12-21 00:00:01.196999936	0.0	16343.3	1250357.7	3.122494	6	20000	0	0.144877	0	3263.138721	...	1343599.375	-128.772618	-1	2165.194729	A	4	12961.600	-0.011013	2830.850	1292046.000
2014-12-21 00:00:02.196999936	0.0	3246.4	1265851.3	3.113830	6	20000	0	-0.123734	0	3263.138721	...	1343599.375	-128.773822	-1	1544.874194	A	4	0.000	-0.008833	0.000	1312436.750
2014-12-21 00:00:03.196999936	0.0	848.3	1312436.8	3.104259	6	20000	0	-0.131441	0	3263.138721	...	1343599.375	-128.775026	-1	1228.501871	A	4	12393.550	-0.008151	2194.925	1312436.750
2014-12-21 00:00:04.196999936	0.0	6448.3	1253999.0	3.097484	6	20000	0	-0.403369	0	3263.138721	...	1343599.375	-128.776229	-1	2681.512355	A	4	21700.700	-0.007051	9491.525	1315059.375

5 rows × 29 columns

df.columns

Index(['Grad_Ne_at_PCP_edge', 'delta_Ne40s', 'Ne', 'TEC_STD', 'IPIR_index',
       'Ne_quality_flag', 'PCP_flag', 'Grad_Ne_at_50km',
       'Ionosphere_region_flag', 'RODI10s', 'mROTI10s', 'ROD', 'mVTEC',
       'mROTI20s', 'Radius', 'RODI20s', 'Grad_Ne_at_20km', 'Latitude',
       'Grad_Ne_at_100km', 'Background_Ne', 'Longitude', 'IBI_flag', 'Te',
       'Spacecraft', 'Num_GPS_satellites', 'delta_Ne10s', 'mROT',
       'delta_Ne20s', 'Foreground_Ne'],
      dtype='object')

fig, axes = plt.subplots(nrows=7, ncols=1, figsize=(20,18), sharex=True)
df.plot(ax=axes[0], y=['Background_Ne', 'Foreground_Ne', 'Ne'], alpha=0.8)
df.plot(ax=axes[1], y=['Grad_Ne_at_100km', 'Grad_Ne_at_50km', 'Grad_Ne_at_20km'])
df.plot(ax=axes[2], y=['RODI10s', 'RODI20s'])
df.plot(ax=axes[3], y=['ROD'])
df.plot(ax=axes[4], y=['mROT'])
df.plot(ax=axes[5], y=['delta_Ne10s', 'delta_Ne20s', 'delta_Ne40s'])
df.plot(ax=axes[6], y=['mROTI20s', 'mROTI10s'])
axes[0].set_ylabel("[cm$^{-3}$]")
axes[1].set_ylabel("[cm$^{-3}$m$^{-1}$]")
axes[2].set_ylabel("[cm$^{-3}$s$^{-1}$]")
axes[3].set_ylabel("[cm$^{-3}$m$^{-1}$]")
axes[4].set_ylabel("[TECU s$^{-1}$]")
axes[5].set_ylabel("[cm$^{-3}$m$^{-1}$]")
axes[6].set_ylabel("[TECU s$^{-1}$]")
axes[6].set_xlabel("Timestamp")

for ax in axes:
    # Reformat time axis
    # https://www.earthdatascience.org/courses/earth-analytics-python/use-time-series-data-in-python/customize-dates--matplotlib-plots-python/
    ax.xaxis.set_major_formatter(DateFormatter("%Y-%m-%d\n%H:%M:%S"))
    ax.legend(loc="upper right")
    ax.grid()
fig.subplots_adjust(hspace=0)

Load as xarray

ds = data.as_xarray()
ds

<xarray.Dataset>
Dimensions:                 (Timestamp: 10800)
Coordinates:
  * Timestamp               (Timestamp) datetime64[ns] 2014-12-21T00:00:00.19...
Data variables: (12/29)
    Spacecraft              (Timestamp) object 'A' 'A' 'A' 'A' ... 'A' 'A' 'A'
    Grad_Ne_at_PCP_edge     (Timestamp) float64 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0
    delta_Ne40s             (Timestamp) float64 3.811e+03 ... 1.702e+03
    Ne                      (Timestamp) float64 1.255e+06 1.25e+06 ... 6.468e+05
    TEC_STD                 (Timestamp) float64 3.131 3.122 ... 2.866 2.891
    IPIR_index              (Timestamp) int32 7 6 6 6 6 6 6 6 ... 4 4 4 4 4 4 4
    ...                      ...
    Te                      (Timestamp) float64 2.212e+03 ... 1.723e+03
    Num_GPS_satellites      (Timestamp) int32 4 4 4 4 4 4 4 4 ... 6 6 6 6 6 6 6
    delta_Ne10s             (Timestamp) float64 67.88 1.296e+04 ... 1.702e+03
    mROT                    (Timestamp) float64 -0.01101 -0.01101 ... 0.09621
    delta_Ne20s             (Timestamp) float64 1.027e+04 ... 1.702e+03
    Foreground_Ne           (Timestamp) float64 1.305e+06 ... 6.488e+05
Attributes:
    Sources:         ['SW_OPER_IPDAIRR_2F_20141221T000000_20141221T235959_0302']
    MagneticModels:  []
    AppliedFilters:  []

xarray.Dataset

• Dimensions:
  • Timestamp: 10800
• Coordinates: (1)
  • Timestamp
    (Timestamp)
    datetime64[ns]
    2014-12-21T00:00:00.196999936 .....

    description :

    CDF_EPOCH of the measurement.

    array(['2014-12-21T00:00:00.196999936', '2014-12-21T00:00:01.196999936',
           '2014-12-21T00:00:02.196999936', ..., '2014-12-21T02:59:57.196999936',
           '2014-12-21T02:59:58.196999936', '2014-12-21T02:59:59.196999936'],
          dtype='datetime64[ns]')

• Data variables: (29)
  • Spacecraft
    (Timestamp)
    object
    'A' 'A' 'A' 'A' ... 'A' 'A' 'A' 'A'

    units :

    -

    description :

    Spacecraft identifier (values: 'A', 'B', 'C' or '-' if not available).

    array(['A', 'A', 'A', ..., 'A', 'A', 'A'], dtype=object)

  • Grad_Ne_at_PCP_edge
    (Timestamp)
    float64
    0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0

    units :

    cm^-3/m

    description :

    The linear electron density gradient calculated over the edges of a patch. This variable is non-zero only at the edges of polar cap patches.

    array([0., 0., 0., ..., 0., 0., 0.])

  • delta_Ne40s
    (Timestamp)
    float64
    3.811e+03 1.634e+04 ... 1.702e+03

    units :

    cm^-3

    description :

    Derived by subtracting Ne by its median filtered value in 40 seconds. As a result, delta_Ne40s indicates the electron density fluctuations smaller than 300 km.

    array([ 3811.1 , 16343.3 ,  3246.4 , ...,  1800.45,  1819.9 ,  1701.65])

  • Ne
    (Timestamp)
    float64
    1.255e+06 1.25e+06 ... 6.468e+05

    units :

    cm^-3

    description :

    Plasma density, directly copied from the Langmuir probe files.

    array([1255163.2, 1250357.7, 1265851.3, ...,  631863.3,  639288.8,
            646784.8])

  • TEC_STD
    (Timestamp)
    float64
    3.131 3.122 3.114 ... 2.866 2.891

    units :

    TECU

    description :

    STD of VTEC from all GPS satellites.

    array([3.13145112, 3.12249362, 3.11383   , ..., 2.8407127 , 2.86645736,
           2.89068334])

  • IPIR_index
    (Timestamp)
    int32
    7 6 6 6 6 6 6 6 ... 4 4 4 4 4 4 4 4

    units :

    -

    description :

    The numeric index for plasma fluctuations and irregularities: 0-3 low, 4-5 medium, and > 6 high level of ionosphericplasma irregularities.

    array([7, 6, 6, ..., 4, 4, 4], dtype=int32)

  • Ne_quality_flag
    (Timestamp)
    int32
    20000 20000 20000 ... 10000 10000

    units :

    -

    description :

    Quality flag for the Ne data and the derived data from Ne, e.g., background density, foreground density etc.

    array([20000, 20000, 20000, ..., 10000, 10000, 10000], dtype=int32)

  • PCP_flag
    (Timestamp)
    int32
    0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0

    units :

    -

    description :

    The polar cap patch flag: 0 if the plasma density measurement occurred OUTSIDE a polar cap patch. 1 if the plasma density measurement occurred at one of the edges of a polar cap patch (no plasma velocity measurements are available). 2 if the plasma density measurement occurred at the LEADING edge of a polar cap patch. 3 if the plasma density measurement occurred at the TRAILING edge of a polar cap patch. 4 if the plasma density measurement occurred INSIDE a polar cap patch proper. When no ion drift data is available, the leading and trailing edges cannot be distinguished. In this case the polar cap patch flag is set to 4 throughout the patch proper and to 1 throughout both edges.

    array([0, 0, 0, ..., 0, 0, 0], dtype=int32)

  • Grad_Ne_at_50km
    (Timestamp)
    float64
    -0.4039 0.1449 ... 0.9347 0.9775

    units :

    cm^-3/m

    description :

    The electron density gradient in a running window calculated via linear regression over 13 data points for the 2 Hz electron density data.

    array([-0.40394   ,  0.1448767 , -0.1237337 , ...,  0.89959927,
            0.93474374,  0.97747502])

  • Ionosphere_region_flag
    (Timestamp)
    int32
    0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0

    units :

    -

    description :

    0: equator, 1: mid-latitudes; 2: auroral oval; 3: polar cap.

    array([0, 0, 0, ..., 0, 0, 0], dtype=int32)

  • RODI10s
    (Timestamp)
    float64
    1.024e+04 3.263e+03 ... 654.8 503.7

    units :

    cm^-3/s

    description :

    Rate Of change of Density Index (RODI) is the standard deviation of ROD over 10 seconds.

    array([10238.51721984,  3263.13872093,  3263.13872093, ...,
             744.15587296,   654.82993341,   503.71589016])

  • mROTI10s
    (Timestamp)
    float64
    0.001472 0.001386 ... 0.005609

    units :

    TECU/s

    description :

    Median of Rate Of change of TEC Index (ROTI) from all available GPS satellites above 30 degrees. The ROTI of each satellite is the standard deviation of ROT over 10 seconds.

    array([0.0014718 , 0.00138551, 0.00131007, ..., 0.00471655, 0.00559606,
           0.00560861])

  • ROD
    (Timestamp)
    float64
    0.0 0.0 0.0 ... 7.28e+03 7.28e+03

    units :

    cm^-3/s

    description :

    Rate Of change of Density

    array([   0. ,    0. ,    0. , ..., 7201.8, 7279.6, 7279.6])

  • mVTEC
    (Timestamp)
    float64
    51.79 51.77 51.75 ... 20.84 20.94

    units :

    TECU

    description :

    Median of VTEC from all available GPS satellites above 30 degrees.

    array([51.78693356, 51.76898156, 51.74689813, ..., 20.73870507,
           20.8393645 , 20.94156382])

  • mROTI20s
    (Timestamp)
    float64
    0.002676 0.002732 ... 0.0114

    units :

    TECU/s

    description :

    Median of Rate Of change of TEC Index (ROTI) from all available GPS satellites above 30 degrees. The ROTI of each satellite is the standard deviation of ROT over 20 seconds.

    array([0.00267555, 0.00273238, 0.00274996, ..., 0.01015663, 0.01085673,
           0.01140045])

  • Radius
    (Timestamp)
    float64
    6.84e+06 6.84e+06 ... 6.835e+06

    units :

    m

    description :

    Position in ITRF - Radius.

    array([6840394.97965252, 6840403.88651707, 6840412.74948835, ...,
           6835430.75082371, 6835442.26034348, 6835453.76761761])

  • RODI20s
    (Timestamp)
    float64
    7.764e+03 7.181e+03 ... 943.2 907.9

    units :

    cm^-3/s

    description :

    Rate Of change of Density Index (RODI) is the standard deviation of ROD over 20 seconds.

    array([7764.00253246, 7181.49622793, 7181.49622793, ...,  923.42601747,
            943.21446108,  907.90485614])

  • Grad_Ne_at_20km
    (Timestamp)
    float64
    -1.048 0.3384 ... 1.002 0.9148

    units :

    cm^-3/m

    description :

    The electron density gradient in a running window calculated via linear regression over 5 data points for the 2 Hz electron density data.

    array([-1.047788  ,  0.33840267,  0.13364267, ...,  0.91515733,
            1.00201333,  0.91482933])

  • Latitude
    (Timestamp)
    float64
    -4.694 -4.757 ... 24.74 24.68

    units :

    deg

    description :

    Position in ITRF - Latitude.

    array([-4.69353277, -4.75741624, -4.82129755, ..., 24.80658022,
           24.7426353 , 24.67869043])

  • Grad_Ne_at_100km
    (Timestamp)
    float64
    -0.08492 -0.144 ... 0.9369 0.9621

    units :

    cm^-3/m

    description :

    The electron density gradient in a running window calculated via linear regression over 27 data points for the 2 Hz electron density data.

    array([-0.08491926, -0.14400869, -0.05827591, ...,  0.91132596,
            0.93688094,  0.96208904])

  • Background_Ne
    (Timestamp)
    float64
    1.344e+06 1.344e+06 ... 4.29e+05

    units :

    cm^-3

    description :

    Background density, as calculated from Ne using a percentile filter of 551 datapoints (2 Hz raw data) and 35 as the percentile.

    array([1343599.375  , 1343599.375  , 1343599.375  , ...,  422318.09375,
            425527.09375,  429034.6875 ])

  • Longitude
    (Timestamp)
    float64
    -128.8 -128.8 ... -175.4 -175.4

    units :

    deg

    description :

    Position in ITRF - Longitude.

    array([-128.77141179, -128.77261789, -128.77382227, ..., -175.36866165,
           -175.3692556 , -175.36985325])

  • IBI_flag
    (Timestamp)
    int32
    -1 -1 -1 -1 -1 ... -1 -1 -1 -1 -1

    units :

    -

    description :

    Plasma Bubble Index, copied from the level-2 Ionospheric Bubble Index product, IBIxTMS_2F.

    array([-1, -1, -1, ..., -1, -1, -1], dtype=int32)

  • Te
    (Timestamp)
    float64
    2.212e+03 2.165e+03 ... 1.723e+03

    units :

    K

    description :

    Electron temperature, directly copied from the Langmuir probe files.

    array([2212.27835299, 2165.19472943, 1544.8741945 , ..., 1730.92749345,
           1727.82763148, 1722.59162917])

  • Num_GPS_satellites
    (Timestamp)
    int32
    4 4 4 4 4 4 4 4 ... 6 6 6 6 6 6 6 6

    units :

    -

    description :

    Total number of tracked GPS satellites above 20 degrees.

    array([4, 4, 4, ..., 6, 6, 6], dtype=int32)

  • delta_Ne10s
    (Timestamp)
    float64
    67.88 1.296e+04 ... 1.702e+03

    units :

    cm^-3

    description :

    Derived by subtracting Ne by its median filtered value in 10 seconds. As a result, delta_Ne10s indicates the electron density fluctuations smaller than 75 km.

    array([   67.875, 12961.6  ,     0.   , ...,  1800.45 ,  1819.9  ,
            1701.65 ])

  • mROT
    (Timestamp)
    float64
    -0.01101 -0.01101 ... 0.09621

    units :

    TECU/s

    description :

    Median of Rate Of change of TEC (ROT) from all available GPS satellites above 30 degrees.

    array([-0.01101323, -0.01101323, -0.00883297, ...,  0.09318886,
            0.09159881,  0.0962132 ])

  • delta_Ne20s
    (Timestamp)
    float64
    1.027e+04 2.831e+03 ... 1.702e+03

    units :

    cm^-3

    description :

    Derived by subtracting Ne by its median filtered value in 20 seconds. As a result, delta_Ne20s indicates the electron density fluctuations smaller than 150 km.

    array([10266.5 ,  2830.85,     0.  , ...,  1800.45,  1819.9 ,  1701.65])

  • Foreground_Ne
    (Timestamp)
    float64
    1.305e+06 1.292e+06 ... 6.488e+05

    units :

    cm^-3

    description :

    Foreground density, as calculated from ndens using a percentile filter of 7 datapoints (2 Hz raw data) and 50 as the percentile.

    array([1305371.   , 1292046.   , 1312436.75 , ...,  635649.   ,
            643381.5  ,  648826.625])

• Indexes: (1)
  • Timestamp
    PandasIndex

    PandasIndex(DatetimeIndex(['2014-12-21 00:00:00.196999936',
                   '2014-12-21 00:00:01.196999936',
                   '2014-12-21 00:00:02.196999936',
                   '2014-12-21 00:00:03.196999936',
                   '2014-12-21 00:00:04.196999936',
                   '2014-12-21 00:00:05.197007872',
                   '2014-12-21 00:00:06.196999936',
                   '2014-12-21 00:00:07.197007872',
                   '2014-12-21 00:00:08.196999936',
                   '2014-12-21 00:00:09.197007872',
                   ...
                   '2014-12-21 02:59:50.196999936',
                   '2014-12-21 02:59:51.196999936',
                   '2014-12-21 02:59:52.197007872',
                   '2014-12-21 02:59:53.196999936',
                   '2014-12-21 02:59:54.197007872',
                   '2014-12-21 02:59:55.196999936',
                   '2014-12-21 02:59:56.197007872',
                   '2014-12-21 02:59:57.196999936',
                   '2014-12-21 02:59:58.196999936',
                   '2014-12-21 02:59:59.196999936'],
                  dtype='datetime64[ns]', name='Timestamp', length=10800, freq=None))

• Attributes: (3)

  Sources :

  ['SW_OPER_IPDAIRR_2F_20141221T000000_20141221T235959_0302']

  MagneticModels :

  []

  AppliedFilters :

  []

Alternative plot setup

To plot the data from xarray, we need a different plotting setup. This does however give us more control over the plot. The units are extracted directly from the xarray object.

fig, axes = plt.subplots(nrows=7, ncols=1, figsize=(20,18), sharex=True)
def subplot(ax=None, y=None, **kwargs):
    """Plot combination of variables onto a given axis"""
    units = ds[y[0]].units
    for var in y:
        ax.plot(ds["Timestamp"], ds[var], label=var, **kwargs)
        if units != ds[var].units:
            raise ValueError(f"Units mismatch for {var}")
    ax.set_ylabel(f"[{units}]")
    # Reformat time axis
    # https://www.earthdatascience.org/courses/earth-analytics-python/use-time-series-data-in-python/customize-dates--matplotlib-plots-python/
    ax.xaxis.set_major_formatter(DateFormatter("%Y-%m-%d\n%H:%M:%S"))
    ax.legend(loc="upper right")
    ax.grid()
subplot(ax=axes[0], y=['Background_Ne', 'Foreground_Ne', 'Ne'])
subplot(ax=axes[1], y=['Grad_Ne_at_100km', 'Grad_Ne_at_50km', 'Grad_Ne_at_20km'])
subplot(ax=axes[2], y=['RODI10s', 'RODI20s'])
subplot(ax=axes[3], y=['ROD'])
subplot(ax=axes[4], y=['mROT'])
subplot(ax=axes[5], y=['delta_Ne10s', 'delta_Ne20s', 'delta_Ne40s'])
subplot(ax=axes[6], y=['mROTI20s', 'mROTI10s'])
fig.subplots_adjust(hspace=0)

previous

EFIx_LP_1B (Langmuir probe 2Hz)

next

TECxTMS_2F (Total electron content)