An evaluation of International Reference Ionosphere electron density in the polar cap and cusp using EISCAT Svalbard radar measurements

Incoherent scatter radar measurements are an important source for studies of ionospheric plasma parameters. In this paper the EISCAT Svalbard radar (ESR) long-term database is used to evaluate the International Reference Ionosphere (IRI) model. The ESR started operations in 1996, and the accumulated database up to 2012 thus covers 16 years, giving an overview of the ionosphere in the polar cap and cusp during more than one solar cycle. Data from ESR can be used to obtain information about primary plasma parameters: electron density, electron and ion temperature, and line-ofsight plasma velocity from an altitude of about 50 and up to 1600 km. Monthly averages of electron density and temperature and ion temperature and composition are also provided by the IRI model from an altitude of 50 to 2000 km. We have compared electron density data obtained from the ESR with the predicted electron density from the IRI-2016 model. Our results show that the IRI model in general fits the ESR data well around the F2 peak height. However, the model seems to underestimate the electron density at lower altitudes, particularly during winter months. During solar minimum the model is also less accurate at higher altitudes. The purpose of this study is to validate the IRI model at polar latitudes.


Introduction
Electron density in the polar cap F-region ionosphere is produced by solar extreme ultraviolet radiation, transport of plasma density structures from lower latitudes and particle precipitation and is reduced by recombination and transport to lower latitudes.The solar wind has a significant influence on the dynamics of the high-latitude ionosphere as it controls the electrodynamics and therefore transport by E × B plasma drift and by particle precipitation.Thus the highlatitude ionosphere is a highly variable region where structures form, recombine and are transported in and out in response to transient changes in the solar wind and/or the interplanetary magnetic field.
Plasma density structures in the high-latitude F region are transported anti-sunward across the polar cap and sunward in the auroral zone due to electric convection (e.g., Cowley and Lockwood, 1992).This transport can increase the electron density in the nightside significantly when structures produced by solar extreme ultraviolet radiation in the sunlit ionosphere follow the convection lines into the polar cap.As an example, plasma originating at midlatitudes can be transported to high latitudes and into the polar cap in a form of a tongue of ionization which greatly enhances the plasma density in the polar cap, cusp and auroral zone (e.g., Foster et al., 2005).Tongues of ionization can also be segmented into 100-1000 km sized islands of enhanced electron density, called polar cap patches (e.g., Lockwood and Carlson, 1992;Zhang et al., 2013).The patches are transported over the polar cap following the convection pattern.
In addition to transport of solar-produced plasma from lower latitudes, soft particle precipitation is an important source of F-region plasma density irregularities at high latitudes (e.g., Kelley et al., 1982).Soft particle precipitation in the cusp region where particles precipitate directly from the magnetosheath can produce polar cap patches that can be convected over the polar cap (e.g., Walker et al., 1999;Oksavik et al., 2006;Goodwin et al., 2015).As a result of plasma transport and particle precipitation, the high-latitude F region ionosphere is nonuniform and highly dynamic.This is a challenge for models aiming to accurately represent the high-latitude ionosphere.
The International Reference Ionosphere (IRI) model is widely used for ionospheric and magnetospheric research, also at high latitudes.IRI is an empirical model which provides monthly averages of ionospheric parameters from an altitude of 50 to 2000 km (Bilitza, 1990(Bilitza, , 2001)).Among the data sources used to build the IRI model are incoherent scatter radars (ISRs), the ISIS and Alouette topside sounders, rocket and satellite observations, as well as the worldwide network of ionosondes.As ionosondes are an essential data source for the IRI model, the IRI model is known to be less accurate at high and low latitudes, where the ionosonde coverage is lower compared to midlatitudes (e.g., Bilitza and Reinisch, 2008).
In the present study we use ISR data from the EISCAT Svalbard radar (ESR), covering the polar cap and cusp, to evaluate the IRI model-predicted electron density in the F region.As the radar started its operations in the 1990s, the accumulated database now contains data from more than one solar cycle.Long time series of ionospheric data are essential when the aim is to study the performance of the IRI model during different diurnal, seasonal and solar activity conditions.Using the ESR data allows us to evaluate the IRI model in the region where the model is known to be less accurate under various conditions.
Previous studies have also used observational data to evaluate the IRI model (e.g., Themens et al., 2014;Wichaipanich et al., 2013;Kim et al., 2011;Chuo and Lee, 2008;Lei et al., 2006;Zhang and Holt, 2004), but few of these have evaluated the model at latitudes as high as the auroral zone or the polar cap, or with long enough time series to evaluate the model at different parts of the solar cycle.Brum et al. (2011) and Lei et al. (2006) used data from ISRs in their studies.Brum et al. (2011) used Arecibo ISR data from experiments performed between 1985 and 2009, covering three different solar cycles, to evaluate the IRI-predicted F2 layer critical frequency (foF2) and height (hmF2) at midlatitudes.They found an overestimation of foF2 at day and an underestimation at night.For hmF2 they found an underestimation, especially during high solar activity.However, after applying a correction for solar activity, they found that the IRI model reproduced the seasonal variation well.Lei et al. (2006) compared ISR data from Millstone Hill, at midlatitudes, and ESR, in the polar cap, with the IRI-2001 model.They used the ISR data to evaluate the IRI electron density and plasma temperature profiles.As expected due to the poor data coverage at high latitudes, they found that the model performed best at midlatitudes.Lei et al. (2006) used 1-month-long data from October 2002 from Millstone Hill and the ESR and therefore did not study any solar cycle or seasonal variations.
Several studies have compared ionosonde data with the IRI model.At midlatitudes, Kim et al. (2011) compared 10 years of NmF2 and hmF2 data from a digisonde located at the Korean Peninsula with the IRI-2007 model.They looked at different diurnal, seasonal, solar activity and geomagnetic conditions and found that although there was good agreement between the observed and IRI-predicted NmF2, there were significant differences between the model and observed hmF2 during midnight, especially during high solar activity.At auroral latitudes, Oyeyemi et al. (2010) compared hmF2 observed at three different ionosonde stations with the hmF2 values predicted by IRI-2001 for three separate years at different parts of the solar cycle.They found the best agreement when the solar activity was high.
In a study by Themens et al. (2014) data from four ionosonde stations located within the polar cap were used to evaluate the hmF2, peak density (NmF2), M(3000)F2 and the bottomside thickness parameter B0 predicted by the IRI-2007 model during the latest extended solar minimum from 2008 to 2010.In addition, data from the Resolute Advanced Modular ISR were used to evaluate the IRI-predicted topside thickness.They evaluated the IRI peak height and density, and topside and bottomside thickness and found differences which they attributed to errors in the modeling of the IRI M(3000)F2 factor and poor representation of diurnal and seasonal variability.
Although Lei et al. (2006), Oyeyemi et al. (2010) and Themens et al. ( 2014) have compared the IRI model with highlatitude data, such comparisons have mainly been made at midlatitudes.It is therefore highly relevant to evaluate the IRI model for the high-latitude region, and in this study we compare the IRI model with the ESR data from the polar cap and cusp.

Data and methodology
The EISCAT Svalbard radar, located at 78.15 • N, 16.02 • E (geographic coordinates) and 75.43 • N, 110.68 • E (geomagnetic coordinates), is one of three incoherent scatter radar systems operated by the EISCAT Scientific Association.In addition to the ESR, an ultra-high-frequency system and a very-high-frequency system are located near Tromsø, Norway, with additional receiver systems in Kiruna, Sweden, and Sodankylä, Finland.The radars are usually operated in a campaign mode, and the data are therefore not continuous.Typically, ESR operates around 1000-2000 h a year, but as part of an IPY-ICESTAR project the ESR was operated nearly continuously from March 2007 to February 2008.Data from ESR can be used to obtain information about primary plasma parameters: electron density, electron and ion temperature, and line-of-sight plasma velocity (many other parameters can be derived from them) from an altitude of about 50 and up to 1600 km.The complete data set therefore provides a comprehensive overview of the ionosphere in the polar cap and cusp during a range of different diurnal, seasonal, geomagnetic and solar activity conditions.
To get access to the complete ESR data set, we have used the Madrigal database, which is an archive of data from a range of different upper atmosphere instruments.Nearly all the experiments since the EISCAT radars were put into operation are available through Madrigal.All ESR data used in this study have been retrieved from the EISCAT Madrigal database.
The Madrigal data have been collected from different experiment modes with different time and altitude resolution and altitude span.As the ESR measurements are not continuous and have different time and altitude resolution, the data have been integrated in 3 h daily intervals for 3-month seasonal periods in each 20 km altitude bin as shown in Fig. 1.The seasonal binning is based on the equinoxes and solstices.This means that spring includes February, March and April, summer includes May, June and July, autumn includes August, September and October and winter is November, December and January (consecutive months).Erroneous data that sometimes appear in Madrigal were filtered out of the integration.The filtering excluded records with electron densities lower than 10 8 m −3 and records with electron densities higher than 10 12 m −3 .An exception was made for the upper electron density limit during solar maximum years 1999, 2000, 2001 and 2002.Due to the higher solar activity the filtering during these years only excluded records with electron densities higher than 10 13 m −3 .
The ESR results were compared with the IRI-2016 model.ESR has two parabolic antennas: a 42 m diameter antenna fixed in the field-aligned direction and a fully steerable antenna with a diameter of 32 m.In order to make the comparison between the IRI model with the ESR data as accurate as possible , ESR data were only used if the elevation angle was larger than 75 • .The IRI model is updated as new data become available, and in this study the latest IRI-2016 model is used.Standard options were used for the IRI model.An IRI profile was generated for each unique time where an ESR profile were used.The IRI profile covered altitudes between 200 and 500 km with a step size of 20 km.Each IRI profile therefore had one value in each 20 km altitude bin.Since seasonal averages were used for the ESR data, and not monthly averages as produced by IRI, the IRI-produced electron densities were also binned and averaged according to season and 3 h daily intervals.
To further investigate the observed difference in the electron densities from ESR measurements and IRI-produced electron densities, the total electron content (TEC) and hmF2 parameters were also estimated.The TEC calculation was done by integrating the electron density over each altitude bin.ESR TEC was only calculated for seasons and daily time intervals when there were data in each altitude bin to ensure that ESR TEC and IRI TEC could be compared.
ESR hmF2 was estimated for each profile in a way similar to Vickers et al. (2014).Cubic spline interpolation was used to set a fixed distance of 10 km between each point in the profile.The maximum electron density was then found in each interpolated profile, and a second-degree polynomial was fitted to five points in the peak area centered around the maximum.The maximum of the fitted polynomial was then used as an estimate of the hmF2.To ensure a sufficient number of interpolation points around the maximum, we searched for hmF2 in the altitude range 180 to 500 km.Profiles from which it was difficult to extract any clear maximum were excluded.This filtering excluded ESR profiles in which the maximum was found in the lowest or highest range gate and profiles in which the electron density doubled between two adjacent points (possible outliers).All the estimated hmF2s were then categorized by season and 3 h time intervals, and the average hmF2 in each bin was found.tire database without binning, but after the electron densities outside the range 10 8 to 10 12 m −3 have been excluded.If the IRI model had been a perfect fit to the ESR data, all the data would have been located in the bar where the ratio is 1, as indicated by the red line.From Fig. 2, it is apparent that the model both overestimates and underestimates the electron density as measured by the radar; however, the model is significantly biased towards an underestimation.

Comparison of the ratio ESR / IRI
Figure 3 shows the ratio between the ESR-measured electron density and electron density produced by IRI in each bin.As for Fig. 1, the eight top panels each represent a 3 h integration interval.The bottom panel displays the sunspot number, which shows the solar cycle variation.The four columns in each year represent the seasons in the order of spring, summer, autumn and winter.
During solar maximum (1999)(2000)(2001)(2002), the red color in Fig. 3 shows that the IRI model clearly underestimates the electron density for altitudes below the F2 peak.The underestimation is visible in all seasons and at any time of the day, but largest at nighttime and during winter.For example, during winter in 2001 and 2002, the ratio of ESR to IRI electron density is 2.5 or above for all altitudes between 200 and 500 km in the time interval 18:00-21:00 UT.During summer, in the same time interval, the agreement is much better between the IRI model and the ESR data.There is still some underestimation, but this is mainly concentrated to below 260 km altitude, and the ratio of ESR to IRI electron density is 2 or less.Also at higher altitudes, above the F2 peak, the IRI model underestimates the electron density.However, the ratio is less for this high-altitude underestimation and it usually covers a smaller altitude range than the underestimation below the F2 peak.
As the solar activity declines towards solar minimum, the situation changes.Compared to the solar maximum, there is better agreement between the IRI model and the ESR data at the altitudes below and above the F2 peak area during the years 2003-2004.However, the dark blue color in Fig. 3 shows that for these years the IRI model overestimates the electron density in the area around the peak density.
During the extended solar minimum (2006-2010) the situation becomes more similar to the solar maximum years.The IRI model fits the ESR data best in the area around the peak height but underestimates the electron density below ∼ 260 km and above ∼ 440 km altitude.At altitudes above ∼ 440 the underestimation is slightly stronger during solar minimum than during solar maximum.On the other hand, below the peak height the altitude range where the model underestimates the electron density is smaller than during solar maximum, particularly at nighttime.
High electron density gives a larger signal-to-noise ratio in the radar measurements.We therefore expect more outliers when the electron density is low.To ensure that the larger number of outliers does not affect the results, the electron density distribution at different parts of the solar cycle was examined.Based on this examination, the lower limit on  electron density was set to 10 8 m −3 and the upper limit to 10 13 m −3 during solar maximum and 10 12 m −3 during other parts of the solar cycle.Also, at higher altitudes the electron density is lower, increasing the risk of erroneous data entering the analysis due to the lower signal-to-noise ratio.Therefore, it was decided to only use EISCAT data from below 500 km where the signal-to-noise ratio is in general sufficient.An exception is winter data during solar minimum, where data above 400 km can be unreliable.These data have therefore been removed from Figs. 2-5.

Comparison with previous versions of the IRI model
A new topside model, the NeQuick model (e.g., Radicella, 2009), has been used as the standard option by the IRI model since IRI-2007.Coïsson et al. (2006) compared the NeQuick and the IRI topside model with topside profiles from the ISIS-2 satellite and found that the NeQuick topside model provides a better representation of the topside ionosphere than the IRI topside model.IRI-2012 also introduced a new model for bottomside thickness, ABT-2009(Altadill et al., 2009), which has since been used as the standard option (Bilitza et al., 2014).As the IRI model offers several options to choose from, including those used as standard options in the previous versions of the IRI model, it is of interest to investigate whether another choice of options would give a better agreement with the ESR electron density.Therefore, we have chosen to also compare earlier versions of the IRI model with the ESR electron density to check whether a different set of standard options would give a better representation of the electron density in the high-latitude ionosphere.
Figure 4 shows the ratio ESR electron density over IRI, for the IRI-2001, IRI-2012and IRI-2016during 2001 (solar maximum).Above ∼ 440 km the IRI-2001 model represents the polar cap ionosphere during spring and summer slightly better than the IRI-2012 and IRI-2016 model.The median ratio over the spring and summer plots for altitudes 440-500 km is ∼ 1.35 and ∼ 1.37 for the IRI-2012 and IRI-2016 models, respectively, but only ∼ 0.97 for the IRI-2001 model.A different situation is seen during winter where IRI-2012 and IRI-2016 clearly perform better than the IRI-2001 model.In general, the performance of the IRI-2012 and IRI-2016 is similar, but some differences can be observed.For example, both models underestimate the electron density below 300 km, but the underestimation is stronger for the new IRI-2016 model, particularly around the equinoxes where the median ratios are ∼ 1.81 and ∼ 2.29 for the IRI-2012 andIRI-2016 model, respectively.Figure 5 is similar to Fig. 4 but compares the performance during a year in the extended solar minimum (2008).Here, all three versions of the IRI model show similar behavior.However, in contrast to the situation during solar maximum, the IRI-2016 model is slightly better at reproducing the ionosphere below 300 km during the extended solar minimum than IRI-2012.Below 300 km the median ratio over all seasons is ∼ 1.39 for IRI-2012 and ∼ 1.31 for IRI-2016.Also the mean ratio over the plot for bins below 300 km confirms that the underestimation is stronger for the IRI-2012 model than for IRI-2016.The mean ratio is ∼ 3.12 for IRI-2012 and ∼ 1.87 for IRI-2016.

Comparison of the total electron content (TEC)
From the binned ESR data the TEC between 200 and 500 km was also calculated and a comparison between ESR TEC and IRI TEC is shown in Fig. 6, where TEC is given in TEC units (1 TECU = 10 16 electrons m −2 ). Figure 6 shows that the TEC is best reproduced during the solar minimum and that the IRI model underestimates the TEC during the solar maximum.The IRI model reproduces TEC well during the extended solar minimum but underestimates the TEC during solar maximum.Although the general tendency is that IRI-TEC underestimates the ESR-TEC, there are also examples of IRI overestimating the electron density.For example in spring 1999, one can observe an overestimation of TEC in Fig. 6, consistent with the overestimation of the electron density at altitudes around the peak height and above, as seen in Fig. 3.The underestimation observed during the solar maximum 2001-2002 is consistent with the results Themens and Jayachandran (2016) found using Canadian GPS-TEC data during the most recent solar maximum.They attributed this underestimation to the topside thickness parameterization in the IRI model.

Comparison of the height of the maximum density in the F2 layer
Also the heights of the F2 layer peak (hmF2) as given from the IRI model were compared with hmF2 calculated from the ESR data.Both the estimated ESR hmF2s and the hmF2s from the IRI model were binned according to season and time of day, and the median in each bin was found.For the IRI model, we obtained the hmF2 from all the times an ESR profile was available.The results are seen in Fig. 7, where the blue line represents the estimated ESR hmF2s and the red line the IRI hmF2s.As shown, the general tendency seems to be that the agreement is best during solar maximum, while the IRI model overestimates the hmF2 during solar minimum.
The IRI model seems to reproduce the hmF2 measured by the ESR best in the time intervals 03:00-12:00 UT. Figure 7 shows that the IRI hmF2 has greater seasonal variation than the ESR hmF2, especially at nighttime.

Conclusions
Electron density from the IRI-2016 model has been compared with data from the EISCAT Svalbard radar covering more than one solar cycle.The results of this comparison could be useful for users and developers of the IRI model, since it is possible from our study to identify the time periods and altitude ranges where the model might need improvement.Also, an inclusion of the entire ESR data set might contribute to improving the performance of the IRI model at high latitudes.The most important results are summarized as follows: -The IRI model is found to be biased towards an underestimation of the electron density in the polar cap and cusp.This underestimation is most severe at nighttime and during solar maximum.A large discrepancy between the IRI TEC and ESR TEC during the solar maximum (1999)(2000)(2001)(2002) is consistent with the findings of Themens et al. (2014) from the latest solar maximum.
- -An overestimation of the hmF2 occurs both during solar minimum and solar maximum, but it seems to be slightly stronger during nighttime than during daytime.
At nighttime the IRI model hmF2 has clear seasonal variations, while the hmF2 observed with the ESR radar has very little seasonal variation.
-Finally, the comparison shows that the IRI model performs best at altitudes close to the hmF2.At these altitudes (around 350 km during solar maximum and 260 km solar minimum), the IRI model reproduces the electron density more accurately than at higher or lower altitudes.

Figure 1 .
Figure 1.Altitude profiles of electron density (log10 N e , m −3 ) measured by ESR and integrated over 3 h in a day for 3 months(spring, summer, autumn, winter).Panels correspond to the 3 h integration intervals.

Figure 2 Figure 2 .
Figure2shows a histogram of how all the ESR data compare with the IRI model.This is the distribution of the en-

Figure 3 .
Figure 3. Altitude profiles of the ratio between ESR and IRI electron densities.The eight top panels correspond to the different 3 h integration intervals.The last panel shows the 3-month averaged sunspot number.The four columns for each year represent the seasons in the order of spring, summer, autumn and winter.

Figure 4 .
Figure 4. Altitude profiles of the ratio between ESR and IRI electron densities for three different versions of the IRI model (IRI-2001, IRI-2012 and IRI-2016) in 2001.The data have been binned according to season, as indicated, and as daily 3 h averages.The rows show how the IRI-2001, IRI-2012 and IRI-2016 models reproduce the polar ionosphere during 2001 respectively.

Figure 5 .
Figure 5. Same as Fig. 4, but for 2008, in the extended solar minimum, instead of 2001.

Figure 6 .
Figure 6.Total electron content from ESR (blue) and IRI model (red) given in TEC units (TECU).Each panel corresponds to a 3 h integration bin, and each year contains four seasonal bins (spring, summer, autumn and winter).

Figure 7 .
Figure 7.The height of the F2 peak from ESR (blue) and IRI model (red).Each panel corresponds to a 3 h integration bin, and each year contains four seasonal bins (spring, summer, autumn and winter).Error bars correspond to 1 standard deviation.
Also previous versions of the IRI model have been compared with the ESR electron density.Comparisons with the IRI-2001 and the IRI-2012 model show that there are no major differences in the performance of the IRI-2016 model and of previous versions.There are, however, some small differences.Most noticeably, the IRI-2012 and IRI-2016 model reproduce the electron density during winter 2001 significantly better than the IRI-2001 model.During solar minimum, the IRI-2016 model seems to be slightly improved for altitudes below the hmF2 compared with the IRI-2012 model.