.ipynb .pdf

repository open issue

JupyterHub

Contents

IPDxIRR_2F (Ionospheric plasma densities)

Abstract: Access to the derived plasma characteristics at 1Hz (level 2 product).

%load_ext watermark
%watermark -i -v -p viresclient,pandas,xarray,matplotlib

Python implementation: CPython
Python version       : 3.9.7
IPython version      : 8.0.1

viresclient: 0.11.0
pandas     : 1.4.1
xarray     : 0.21.1
matplotlib : 3.5.1

from viresclient import SwarmRequest
import datetime as dt
import matplotlib.pyplot as plt
from matplotlib.dates import DateFormatter

request = SwarmRequest()

IPDxIRR_2F product information

Derived plasma characteristics at 1Hz, for each Swarm spacecraft.

Documentation:

• https://earth.esa.int/web/guest/missions/esa-eo-missions/swarm/data-handbook/level-2-product-definitions#IPDxIRR_2F

Check what “IPD” data variables are available

request.available_collections("IPD", details=False)

{'IPD': ['SW_OPER_IPDAIRR_2F', 'SW_OPER_IPDBIRR_2F', 'SW_OPER_IPDCIRR_2F']}

request.available_measurements("IPD")

['Ne',
 'Te',
 'Background_Ne',
 'Foreground_Ne',
 'PCP_flag',
 'Grad_Ne_at_100km',
 'Grad_Ne_at_50km',
 'Grad_Ne_at_20km',
 'Grad_Ne_at_PCP_edge',
 'ROD',
 'RODI10s',
 'RODI20s',
 'delta_Ne10s',
 'delta_Ne20s',
 'delta_Ne40s',
 'Num_GPS_satellites',
 'mVTEC',
 'mROT',
 'mROTI10s',
 'mROTI20s',
 'IBI_flag',
 'Ionosphere_region_flag',
 'IPIR_index',
 'Ne_quality_flag',
 'TEC_STD']

Fetch three hours of IPD data

request.set_collection("SW_OPER_IPDAIRR_2F")
request.set_products(measurements=request.available_measurements("IPD"))
data = request.get_between(
    dt.datetime(2014,12,21, 0),
    dt.datetime(2014,12,21, 3)
)

Load and plot using pandas/matplotlib

df = data.as_dataframe()
df.head()

	TEC_STD	IPIR_index	PCP_flag	ROD	delta_Ne40s	Grad_Ne_at_20km	Grad_Ne_at_50km	mVTEC	Ne	Grad_Ne_at_PCP_edge	...	Latitude	Spacecraft	mROTI10s	RODI10s	Radius	Grad_Ne_at_100km	Te	Ne_quality_flag	mROTI20s	Num_GPS_satellites
Timestamp
2014-12-21 00:00:00.196999936	3.131452	7	0	0.0	3811.1	-1.047788	-0.403940	51.786939	1255163.2	0.0	...	-4.693533	A	0.001472	10238.517220	6.840395e+06	-0.084919	2212.28	20000	0.002676	4
2014-12-21 00:00:01.196999936	3.122495	6	0	0.0	16343.3	0.338403	0.144877	51.768987	1250357.6	0.0	...	-4.757416	A	0.001386	3263.138721	6.840404e+06	-0.144009	2165.19	20000	0.002732	4
2014-12-21 00:00:02.196999936	3.113831	6	0	0.0	3246.4	0.133643	-0.123734	51.746903	1265851.3	0.0	...	-4.821298	A	0.001310	3263.138721	6.840413e+06	-0.058276	1544.87	20000	0.002750	4
2014-12-21 00:00:03.196999936	3.104260	6	0	0.0	848.3	1.443077	-0.131441	51.728764	1312436.8	0.0	...	-4.885179	A	0.001930	3263.138721	6.840422e+06	-0.144613	1228.50	20000	0.003277	4
2014-12-21 00:00:04.196999936	3.097485	6	0	0.0	6448.3	-1.948789	-0.403369	51.711319	1253999.0	0.0	...	-4.949060	A	0.002434	3263.138721	6.840430e+06	-0.039358	2681.51	20000	0.003744	4

5 rows × 29 columns

df.columns

Index(['TEC_STD', 'IPIR_index', 'PCP_flag', 'ROD', 'delta_Ne40s',
       'Grad_Ne_at_20km', 'Grad_Ne_at_50km', 'mVTEC', 'Ne',
       'Grad_Ne_at_PCP_edge', 'delta_Ne20s', 'Foreground_Ne', 'Background_Ne',
       'Ionosphere_region_flag', 'delta_Ne10s', 'mROT', 'RODI20s', 'Longitude',
       'IBI_flag', 'Latitude', 'Spacecraft', 'mROTI10s', 'RODI10s', 'Radius',
       'Grad_Ne_at_100km', 'Te', 'Ne_quality_flag', 'mROTI20s',
       'Num_GPS_satellites'],
      dtype='object')

fig, axes = plt.subplots(nrows=7, ncols=1, figsize=(20,18), sharex=True)
df.plot(ax=axes[0], y=['Background_Ne', 'Foreground_Ne', 'Ne'], alpha=0.8)
df.plot(ax=axes[1], y=['Grad_Ne_at_100km', 'Grad_Ne_at_50km', 'Grad_Ne_at_20km'])
df.plot(ax=axes[2], y=['RODI10s', 'RODI20s'])
df.plot(ax=axes[3], y=['ROD'])
df.plot(ax=axes[4], y=['mROT'])
df.plot(ax=axes[5], y=['delta_Ne10s', 'delta_Ne20s', 'delta_Ne40s'])
df.plot(ax=axes[6], y=['mROTI20s', 'mROTI10s'])
axes[0].set_ylabel("[cm$^{-3}$]")
axes[1].set_ylabel("[cm$^{-3}$m$^{-1}$]")
axes[2].set_ylabel("[cm$^{-3}$s$^{-1}$]")
axes[3].set_ylabel("[cm$^{-3}$m$^{-1}$]")
axes[4].set_ylabel("[TECU s$^{-1}$]")
axes[5].set_ylabel("[cm$^{-3}$m$^{-1}$]")
axes[6].set_ylabel("[TECU s$^{-1}$]")
axes[6].set_xlabel("Timestamp")

for ax in axes:
    # Reformat time axis
    # https://www.earthdatascience.org/courses/earth-analytics-python/use-time-series-data-in-python/customize-dates--matplotlib-plots-python/
    ax.xaxis.set_major_formatter(DateFormatter("%Y-%m-%d\n%H:%M:%S"))
    ax.legend(loc="upper right")
    ax.grid()
fig.subplots_adjust(hspace=0)

Load as xarray

ds = data.as_xarray()
ds

<xarray.Dataset>
Dimensions:                 (Timestamp: 10800)
Coordinates:
  * Timestamp               (Timestamp) datetime64[ns] 2014-12-21T00:00:00.19...
Data variables: (12/29)
    Spacecraft              (Timestamp) object 'A' 'A' 'A' 'A' ... 'A' 'A' 'A'
    TEC_STD                 (Timestamp) float64 3.131 3.122 ... 2.866 2.891
    IPIR_index              (Timestamp) int32 7 6 6 6 6 6 6 6 ... 4 4 4 4 4 4 4
    PCP_flag                (Timestamp) int32 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0
    ROD                     (Timestamp) float64 0.0 0.0 ... 7.28e+03 7.28e+03
    delta_Ne40s             (Timestamp) float64 3.811e+03 ... 1.702e+03
    ...                      ...
    Radius                  (Timestamp) float64 6.84e+06 6.84e+06 ... 6.835e+06
    Grad_Ne_at_100km        (Timestamp) float64 -0.08492 -0.144 ... 0.9621
    Te                      (Timestamp) float64 2.212e+03 ... 1.723e+03
    Ne_quality_flag         (Timestamp) int32 20000 20000 20000 ... 20000 20000
    mROTI20s                (Timestamp) float64 0.002676 0.002732 ... 0.0114
    Num_GPS_satellites      (Timestamp) int32 4 4 4 4 4 4 4 4 ... 6 6 6 6 6 6 6
Attributes:
    Sources:         ['SW_OPER_IPDAIRR_2F_20141221T000000_20141221T235959_0301']
    MagneticModels:  []
    AppliedFilters:  []

xarray.Dataset

• Dimensions:
  • Timestamp: 10800
• Coordinates: (1)
  • Timestamp
    (Timestamp)
    datetime64[ns]
    2014-12-21T00:00:00.196999936 .....

    description :

    CDF_EPOCH of the measurement.

    array(['2014-12-21T00:00:00.196999936', '2014-12-21T00:00:01.196999936',
           '2014-12-21T00:00:02.196999936', ..., '2014-12-21T02:59:57.196999936',
           '2014-12-21T02:59:58.196999936', '2014-12-21T02:59:59.196999936'],
          dtype='datetime64[ns]')

• Data variables: (29)
  • Spacecraft
    (Timestamp)
    object
    'A' 'A' 'A' 'A' ... 'A' 'A' 'A' 'A'

    units :

    -

    description :

    Spacecraft identifier (values: 'A', 'B', 'C' or '-' if not available).

    array(['A', 'A', 'A', ..., 'A', 'A', 'A'], dtype=object)

  • TEC_STD
    (Timestamp)
    float64
    3.131 3.122 3.114 ... 2.866 2.891

    units :

    TECU

    description :

    STD of VTEC from all GPS satellites.

    array([3.13145204, 3.12249454, 3.11383092, ..., 2.84071331, 2.86645798,
           2.89068396])

  • IPIR_index
    (Timestamp)
    int32
    7 6 6 6 6 6 6 6 ... 4 4 4 4 4 4 4 4

    units :

    -

    description :

    The numeric index for plasma fluctuations and irregularities: 0-3 low, 4-5 medium, and > 6 high level of ionosphericplasma irregularities.

    array([7, 6, 6, ..., 4, 4, 4], dtype=int32)

  • PCP_flag
    (Timestamp)
    int32
    0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0

    units :

    -

    description :

    The polar cap patch flag: 0 if the plasma density measurement occurred OUTSIDE a polar cap patch. 1 if the plasma density measurement occurred at one of the edges of a polar cap patch (no plasma velocity measurements are available). 2 if the plasma density measurement occurred at the LEADING edge of a polar cap patch. 3 if the plasma density measurement occurred at the TRAILING edge of a polar cap patch. 4 if the plasma density measurement occurred INSIDE a polar cap patch proper. When no ion drift data is available, the leading and trailing edges cannot be distinguished. In this case the polar cap patch flag is set to 4 throughout the patch proper and to 1 throughout both edges.

    array([0, 0, 0, ..., 0, 0, 0], dtype=int32)

  • ROD
    (Timestamp)
    float64
    0.0 0.0 0.0 ... 7.28e+03 7.28e+03

    units :

    cm^-3/s

    description :

    Rate Of change of Density

    array([   0. ,    0. ,    0. , ..., 7201.6, 7279.6, 7279.6])

  • delta_Ne40s
    (Timestamp)
    float64
    3.811e+03 1.634e+04 ... 1.702e+03

    units :

    cm^-3

    description :

    Derived by subtracting Ne by its median filtered value in 40 seconds. As a result, delta_Ne40s indicates the electron density fluctuations smaller than 300 km.

    array([ 3811.1 , 16343.3 ,  3246.4 , ...,  1800.4 ,  1819.9 ,  1701.65])

  • Grad_Ne_at_20km
    (Timestamp)
    float64
    -1.048 0.3384 ... 1.002 0.9148

    units :

    cm^-3/m

    description :

    The electron density gradient in a running window calculated via linear regression over 5 data points for the 2 Hz electron density data.

    array([-1.047788  ,  0.33840267,  0.13364267, ...,  0.91515467,
            1.00201333,  0.91482933])

  • Grad_Ne_at_50km
    (Timestamp)
    float64
    -0.4039 0.1449 ... 0.9347 0.9775

    units :

    cm^-3/m

    description :

    The electron density gradient in a running window calculated via linear regression over 13 data points for the 2 Hz electron density data.

    array([-0.40394   ,  0.1448767 , -0.1237337 , ...,  0.89959912,
            0.9347433 ,  0.97747429])

  • mVTEC
    (Timestamp)
    float64
    51.79 51.77 51.75 ... 20.84 20.94

    units :

    TECU

    description :

    Median of VTEC from all available GPS satellites above 30 degrees.

    array([51.78693878, 51.76898678, 51.74690333, ..., 20.73871127,
           20.8393707 , 20.94157002])

  • Ne
    (Timestamp)
    float64
    1.255e+06 1.25e+06 ... 6.468e+05

    units :

    cm^-3

    description :

    Plasma density, directly copied from the Langmuir probe files.

    array([1255163.2, 1250357.6, 1265851.3, ...,  631863.3,  639288.8,
            646784.8])

  • Grad_Ne_at_PCP_edge
    (Timestamp)
    float64
    0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0

    units :

    cm^-3/m

    description :

    The linear electron density gradient calculated over the edges of a patch. This variable is non-zero only at the edges of polar cap patches.

    array([0., 0., 0., ..., 0., 0., 0.])

  • delta_Ne20s
    (Timestamp)
    float64
    1.027e+04 2.831e+03 ... 1.702e+03

    units :

    cm^-3

    description :

    Derived by subtracting Ne by its median filtered value in 20 seconds. As a result, delta_Ne20s indicates the electron density fluctuations smaller than 150 km.

    array([10266.5 ,  2830.85,     0.  , ...,  1800.4 ,  1819.9 ,  1701.65])

  • Foreground_Ne
    (Timestamp)
    float64
    1.305e+06 1.292e+06 ... 6.488e+05

    units :

    cm^-3

    description :

    Foreground density, as calculated from ndens using a percentile filter of 7 datapoints (2 Hz raw data) and 50 as the percentile.

    array([1305371.   , 1292046.   , 1312436.75 , ...,  635649.   ,
            643381.5  ,  648826.625])

  • Background_Ne
    (Timestamp)
    float64
    1.344e+06 1.344e+06 ... 4.29e+05

    units :

    cm^-3

    description :

    Background density, as calculated from Ne using a percentile filter of 551 datapoints (2 Hz raw data) and 35 as the percentile.

    array([1343599.375  , 1343599.375  , 1343599.375  , ...,  422318.09375,
            425527.09375,  429034.6875 ])

  • Ionosphere_region_flag
    (Timestamp)
    int32
    0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0

    units :

    -

    description :

    0: equator, 1: mid-latitudes; 2: auroral oval; 3: polar cap.

    array([0, 0, 0, ..., 0, 0, 0], dtype=int32)

  • delta_Ne10s
    (Timestamp)
    float64
    67.88 1.296e+04 ... 1.702e+03

    units :

    cm^-3

    description :

    Derived by subtracting Ne by its median filtered value in 10 seconds. As a result, delta_Ne10s indicates the electron density fluctuations smaller than 75 km.

    array([   67.875, 12961.6  ,     0.   , ...,  1800.4  ,  1819.9  ,
            1701.65 ])

  • mROT
    (Timestamp)
    float64
    -0.01101 -0.01101 ... 0.09621

    units :

    TECU/s

    description :

    Median of Rate Of change of TEC (ROT) from all available GPS satellites above 30 degrees.

    array([-0.01101324, -0.01101324, -0.00883297, ...,  0.09318886,
            0.09159881,  0.0962132 ])

  • RODI20s
    (Timestamp)
    float64
    7.764e+03 7.181e+03 ... 943.2 907.9

    units :

    cm^-3/s

    description :

    Rate Of change of Density Index (RODI) is the standard deviation of ROD over 20 seconds.

    array([7764.00253246, 7181.49622793, 7181.49622793, ...,  923.42482842,
            943.21431292,  907.90593797])

  • Longitude
    (Timestamp)
    float64
    -128.8 -128.8 ... -175.4 -175.4

    units :

    deg

    description :

    Position in ITRF - Longitude.

    array([-128.77141157, -128.77261789, -128.77382226, ..., -175.36866182,
           -175.36925577, -175.36985342])

  • IBI_flag
    (Timestamp)
    int32
    -1 -1 -1 -1 -1 ... -1 -1 -1 -1 -1

    units :

    -

    description :

    Plasma Bubble Index, copied from the level-2 Ionospheric Bubble Index product, IBIxTMS_2F.

    array([-1, -1, -1, ..., -1, -1, -1], dtype=int32)

  • Latitude
    (Timestamp)
    float64
    -4.694 -4.757 ... 24.74 24.68

    units :

    deg

    description :

    Position in ITRF - Latitude.

    array([-4.69353311, -4.75741622, -4.82129752, ..., 24.8065803 ,
           24.74263539, 24.67869052])

  • mROTI10s
    (Timestamp)
    float64
    0.001472 0.001386 ... 0.005609

    units :

    TECU/s

    description :

    Median of Rate Of change of TEC Index (ROTI) from all available GPS satellites above 30 degrees. The ROTI of each satellite is the standard deviation of ROT over 10 seconds.

    array([0.0014718 , 0.00138551, 0.00131007, ..., 0.00471655, 0.00559606,
           0.00560861])

  • RODI10s
    (Timestamp)
    float64
    1.024e+04 3.263e+03 ... 654.8 503.7

    units :

    cm^-3/s

    description :

    Rate Of change of Density Index (RODI) is the standard deviation of ROD over 10 seconds.

    array([10238.51721984,  3263.13872093,  3263.13872093, ...,
             744.15328606,   654.83197202,   503.72685779])

  • Radius
    (Timestamp)
    float64
    6.84e+06 6.84e+06 ... 6.835e+06

    units :

    m

    description :

    Position in ITRF - Radius.

    array([6840394.98513586, 6840403.89273225, 6840412.75499263, ...,
           6835430.74735898, 6835442.25596207, 6835453.76413   ])

  • Grad_Ne_at_100km
    (Timestamp)
    float64
    -0.08492 -0.144 ... 0.9369 0.9621

    units :

    cm^-3/m

    description :

    The electron density gradient in a running window calculated via linear regression over 27 data points for the 2 Hz electron density data.

    array([-0.08491926, -0.14400869, -0.05827591, ...,  0.91132594,
            0.9368809 ,  0.96208895])

  • Te
    (Timestamp)
    float64
    2.212e+03 2.165e+03 ... 1.723e+03

    units :

    K

    description :

    Electron temperature, directly copied from the Langmuir probe files.

    array([2212.28, 2165.19, 1544.87, ..., 1730.93, 1727.83, 1722.59])

  • Ne_quality_flag
    (Timestamp)
    int32
    20000 20000 20000 ... 20000 20000

    units :

    -

    description :

    Quality flag for the Ne data and the derived data from Ne, e.g., background density, foreground density etc.

    array([20000, 20000, 20000, ..., 20000, 20000, 20000], dtype=int32)

  • mROTI20s
    (Timestamp)
    float64
    0.002676 0.002732 ... 0.0114

    units :

    TECU/s

    description :

    Median of Rate Of change of TEC Index (ROTI) from all available GPS satellites above 30 degrees. The ROTI of each satellite is the standard deviation of ROT over 20 seconds.

    array([0.00267555, 0.00273237, 0.00274996, ..., 0.01015663, 0.01085673,
           0.01140045])

  • Num_GPS_satellites
    (Timestamp)
    int32
    4 4 4 4 4 4 4 4 ... 6 6 6 6 6 6 6 6

    units :

    -

    description :

    Total number of tracked GPS satellites above 20 degrees.

    array([4, 4, 4, ..., 6, 6, 6], dtype=int32)

• Attributes: (3)

  Sources :

  ['SW_OPER_IPDAIRR_2F_20141221T000000_20141221T235959_0301']

  MagneticModels :

  []

  AppliedFilters :

  []

Alternative plot setup

To plot the data from xarray, we need a different plotting setup. This does however give us more control over the plot. The units are extracted directly from the xarray object.

fig, axes = plt.subplots(nrows=7, ncols=1, figsize=(20,18), sharex=True)
def subplot(ax=None, y=None, **kwargs):
    """Plot combination of variables onto a given axis"""
    units = ds[y[0]].units
    for var in y:
        ax.plot(ds["Timestamp"], ds[var], label=var, **kwargs)
        if units != ds[var].units:
            raise ValueError(f"Units mismatch for {var}")
    ax.set_ylabel(f"[{units}]")
    # Reformat time axis
    # https://www.earthdatascience.org/courses/earth-analytics-python/use-time-series-data-in-python/customize-dates--matplotlib-plots-python/
    ax.xaxis.set_major_formatter(DateFormatter("%Y-%m-%d\n%H:%M:%S"))
    ax.legend(loc="upper right")
    ax.grid()
subplot(ax=axes[0], y=['Background_Ne', 'Foreground_Ne', 'Ne'])
subplot(ax=axes[1], y=['Grad_Ne_at_100km', 'Grad_Ne_at_50km', 'Grad_Ne_at_20km'])
subplot(ax=axes[2], y=['RODI10s', 'RODI20s'])
subplot(ax=axes[3], y=['ROD'])
subplot(ax=axes[4], y=['mROT'])
subplot(ax=axes[5], y=['delta_Ne10s', 'delta_Ne20s', 'delta_Ne40s'])
subplot(ax=axes[6], y=['mROTI20s', 'mROTI10s'])
fig.subplots_adjust(hspace=0)

previous

EFIx_LP_1B (Langmuir probe 2Hz)

next

TECxTMS_2F (Total electron content)