Sentinel data variety and volume

The CEDA Archive now holds over 10 Petabytes of satellite data from the Sentinels - this increases in volume at over 7 Terabytes per day. We provide a mirror archive for Sentinel products, duplicating the ESA archives so that data products required by the Earth observation community are easily accessible. We update 13 Sentinel products daily with another 8 being updated monthly - many of these products are essential for researchers studying global issues such as sea surface temperature, air quality, and land-use change. 

Due to the ever-increasing volume of data, the team makes use of the differing types of archive storage to ensure data is shared in the most cost-effective and user-friendly way. Fast disk holds the latest data so users can access this instantly, older data is stored on tape as this has less user demand and is a cheaper storage solution. The Near Line Archive (NLA) allows users to pull back data from tape to access the data interactively (more information on the NLA in the 2017-18 annual report).

Providing useful data to the community  

Many researchers were interested in investigating how COVID-19 lockdowns were changing countries emissions. Sentinel 5P products, which are used for air quality monitoring, were perfect for this. CEDA staff responded to this community need by updating and backfilling many of the relevant products so that comparisons could be made. 

CEDA has aimed to archive the latest version of Sea and Land Surface Temperature Radiometer (SLSTR) data produced by Sentinel 3 satellites. Data released by ESA this year have been downloaded from ESA and NASA to populate the CEDA Archive. The latest version has provided users with access to the earlier products which hadn’t previously been released as well as older products with the updated processing version.

Images: Satellite images from Sentinel 5p that shows the global levels of Nitrogen Dioxide (NO2) in February 2019 (pre covid pandemic) versus February 2020 (during covid pandemic). Credit: processing by CEDA staff, ESA Copernicus data 

Working with external data providers  

CEDA also archives processed data products such as the Department for Environment, Food and Rural Affairs (Defra) and Joint Nature Conservation Committee (JNCC) Analysis Ready Data (ARD) for both Sentinel 1 and 2 over the UK. These new datasets take the initial processing task away from users - allowing researchers to immediately start tackling issues such as monitoring wildfires or detecting habitat change, rather than spending time preprocessing the data. These products have grown in size to almost 100 TB and were the largest volume downloaded via the web from the CEDA Archive for any Sentinel products during the last year. These data are produced by the Defra Earth Observation Data Service (EODS) on Microsoft Azure and JNCC Simple ARD Service hosted on JASMIN. 

During the next year, a new satellite (Sentinel 6) will be launched with sea surface level data expected to be added to the CEDA Archive. The team will build on past experience to ensure these important data are archived in the most efficient way, allowing users access to the most relevant and up to date data.  

Find out about other work we do in the CEDA Annual Report (20-21).