Data News

One of the TDRS satellites which GPM uses for downloading data is currently in safehold.  This means that all downloads scheduled for that satellite wiil obviously not be able to be met. This will increase the latency of all GPM NRT products.  

One missed pass has already occurred and led to increased latency. Several others will occur during the night and into tomorrow.

Keywords:

PPS has completed the production of the GIS-friendly TIF/TFW products created from the IMERG Early dataset.  These products include running accumulations at 3hr and 1day.

The Early TIF/TFW GIS-friendly products have been created back to the beginning of the mission: March 2014. They can be found on jsimpson server in the directories:

/NRTPUB/imerg/gis/early/YYYY/MM

Documentation for the TIF/TFW products is contained in the document README.GIS.pdf in /NRTPUB/imerg/gis. 
 

Keywords:

Date of Event:  15 June 2017
Start Time of Event:  13:15 UTC
Schedule stop time:   14:15 UTC

GSFC networking personnel will be replacing network equipment that will directly impact GPM MOC, both its connection to PPS AND, more significantly, its connection to White Sands (WSC) which is the download location for GPM data.

Since this replacement will provided needed increase in network throughput capacity, the update is required.

During the outage, PPS SDPS (tnitta) will receive NO GPM DATA.  

While only an hour is currently planned for the outage as with any network modifications, changes might take longer or be completed earlier. 

During the outage no GPM realtime data will be available.

Keywords:

In an effort to keep users informed, below is the current status of the IMERG V04 products.

Summary

1. As of 02:20 UTC on 9 May 2017, IMERG Early and Late are running in V04 with backward compatibility adjustments to V05 inputs.
2. From that point to about 11:30 UTC on 12 May 2017, there are long, thin streaks of artifacts in IMERG Early and Late due to GPROF-ATMS.
3. From that point to about 15:00 UTC on 30 May 2017, there are unrealistically large areas of light rain in moist tropical ocean areas in IMERG Early and Late due to GPROF-ATMS and GPROF-MHS. Thereafter, the content of microwave data is somewhat reduced.

These artifacts will remain in the V04 Early and Late products, and be resolved in the expected upgrade to V05 later this summer.

Details

  1. At 02:20 UTC on 9 May 2017, Version 04 processing was shut down for GPM core products and subsequently started as Version 05.  At this point IMERG Early and Late continued to run in V04, with backward compatibility adjustments to V05 inputs.  This might introduce some additional uncertainty in the results.  IMERG Final will be computed with a few-month lag up through April 2017, but then must be halted until V05 is instituted because we wish to keep this research data record as consistent as possible.
  2. At about 11:30 UTC on 12 May 2017, the Version 05 GPROF-ATMS Level 2 precipitation estimates were turned off because the scan positions were slghtly misnavigated, which caused frequent unreasonable values in the last 6 footprints.  A corrected version was restarted about 23:00 UTC on 15 May.  The IMERG Early and Late products from the start of Version 05 processing for core products (9 May) up to this correction include the erroneous data (long, thin streaks of high values).
  3. At about 15:00 UTC on 30 May 2017, the Version 05 GPROF-ATMS and GPROF-MHS Level 2 precipitation estimates were given quality flags of 3 in the outer swath regions, suggesting that users (in particular IMERG) should not use these footprints due to unrealistic areas of widespread light rain in moist tropical oceanic cases. [Diagnosis of the problem is on-going by the GPROF team.  It is not obvious that these footprints yield unrealistic values outside of moist tropical ocean areas, but at present not enough is known to do a more sophisticated screening.]  As a result, the swaths are narrowed by footprints 1-18 and 79-96 for ATMS and 1-15 and 76-90 for MHS.  This should increase the frequency of morphed and IR-based input data.  The IMERG Early and Late results from the start of Version 05 processing for core products (9 May) up to this correction have erroneous data in these footprints, at least in moist tropical oceanic conditions. 

Please contact us for any questions.

Keywords:

PPS is releasing GPM V05 DPR L1-L3 and Combined L2 and L3 V05 data products  today, Wednesday May 17, 2017. This V05 data will be available beginning with May 01, 2017 data and forward. The DPR Level 3 monthly products will be released when validated.

Reprocessed V05 DPR and Combined data for March 2014 through April 2017 will take several weeks to complete. This data will appear in PPS's arthurhou archive as we reprocess (6 to 8 weeks - estimated completion time).

Registered users can obtain data from:
ftp://arthurhou.pps.eosdis.nasa.gov/
and can order this through STORM:
https://storm.pps.eosdis.nasa.gov/

GPM V05 release notes and other important documents for version 5 products can be found at:
https://pps.gsfc.nasa.gov/GPMprelimdocs.html
and:
https://pps.gsfc.nasa.gov/atbd.html
PPS Public Website Home:
https://pps.gsfc.nasa.gov/

PPS Toolkit TKIO version 3.80.26 should be used to read the GPM V05 products. This latest version of TKIO is also backwards compatible with GPM Version 04 product formats. TKIO 3.80.26 and associated documentation is available from: ftp://gpmweb2.pps.eosdis.nasa.gov/pub/PPStoolkit/GPM/tkio-3.80.26/

Please contact us for any questions.

Keywords:

Effective approximately 04:00 UTC, 9 May 2017 the near-realtime swath products (GMI, partner constellation, radar and combined) will be upgraded from V04 to V05. 

During the conversion the latency of near-realtime will be affected as all the new code and support files need to be modified to carry out this upgrade. We hope this will take no more than 2 hours but can not guarantee it might not be longer.

We will start with the upgrade of radar products, followed as quickly as possible by GMI and then the partner constellation products. We are starting with the radar as this requires close coordination with JAXA and the arrival of the first L1B Ku and Ka V05A products from JAXA will actually initiate the conversion activities.

The format for most products has remained substantially the same. HOWEVER, the GPROF precipitation Level 2 products for GMI and the partner constellation radiometers has substantially changed. Existing programs reading the GPROF products will likely no longer work. File specifications can be retrieved from pps.gsfc.nasa.gov. 

IMERG will remain at V04 but will be changed from V04A to V04B. This is to indicate that the input products have changed. During the upgrade IMERG will not be available until completion of the upgrade and certification by the IMERG algorithm team that the product is correct. This certification process may take some time and until the product is certified, it will NOT be available.

There are no format changes in IMERG. However, there might be a slight differences in the precipitation values. 

The PPS I/O toolkit used from V05 will be 3.80.26.
 
V05 release notes (caveats for use) are available in /NRTPUB/V5ReleaseInfo directory on jsimpson.pps.eosdis.nasa.gov.

Please contact us for questions.

Keywords:

The GPM project has reprocessed all the original NRT IMERG data both EARLY and LATE to product version V04A.  In addition, this reprocessing brought the IMERG  dataset back to the beginning of the GPM mission in March 2014.  The original NRT IMERG started its processing in March of 2015.

Currently only the HDF5 products have been reprocessed to V04A.  The TIFF/TFW versions will also be reprocessed but this will take a while longer.

The V04 products are located in the directories early/YYYYMM and late/YYYYMM respectively. A subdirectory V03 has been created within those directories and the V03 data was moved into the subdirectory.

Keywords:

The Precipitation Processing System has begun the transition to GPM Version 5 products.

The satellite observations made in April 2017 will be the last observations to be included in Version 4 of the GPM data products, with one exception. Starting with observations made in May 2017, PPS will generate Version 5 GPM data products, again with one exception.  The exception in both cases is that  the PPS Realtime system will continue to generate the Version 4 Early and Late IMERG data products.  Version 4 data products have been removed from the STORM data ordering system (https://storm.pps.eosdis.nasa.gov/storm/ to make room for Version 5 data products).  Version 4 data products,  however, will continue to be available for download from the GPM's "arthurhou" FTP archive or from the "jsimpson" FTP archive in the case of Early and Late IMERG.

GPM V04 standing orders and stand-alone orders will be processed and distributed through April 30th, 2017. Starting with data date May 1, 2017 all products will be produced as V05. With the start of V05 processing the V04 data will only be accessible from the arthurhou FTP archives (ftp://arthurhou.pps.eosdis.nasa.gov/gpmallversions/V04/). As V05 products become available later this month, they will appear in STORM and on "arthurhou".

The PPS Toolkit for GPM V05 products is tkio-3.80.26 and is also backwards compatible with V04 products.

On Tuesday May 02, 2017, PPS will complete GPM and Partner V04 products. All GPM and Partner V04 products will be processed through data date April 30, 2017 with the exception of the following:

  • IMERG Final Run V04, which is still in production, will end with data date Feb 28, 2017.
  • IMERG Early and Late Run will continue to be produced as V04, using V05 GPM inputs. 
  • -L2-L3 GPROF Climatology products will end with data date Jan 30, 2017.


PPS will reprocess the entire mission as V05 starting with March 2014 data. All users should transition to the improved V05 products to ensure that they are using the latest algorithms.

All V05 data products will be held until they have been reviewed and verified by the Science Team. GPM V05 Products will be announced and released to the public upon their approval.

Users with standing orders will experience an interruption in data delivery as we make our transition to GPM and partner V05 production.

GPM V05 release notes and other documentation will be available through our ftp archives and the PPS public website prior to the public release. We will include this information in a future announcement.

Please contact us if you have any concerns, comments or questions.

Thank you for your patience during this transition.
 

Keywords:

During the initial setup up for V04 IMERG, an error was made in the configuration for running the EARLY version of IMERG. This affected the V04A early data from Feb 5 - March 12, 2017. 

This error caused the early and the late products to be identical. Given that the LATE product is the more accurate product this might not be an issue for many users. However, users who need a consistent approach in the early data will need to download IMERG early Feb 5-March 12, 2017 again.

On  April 12, 2017 at about 16:55 UTC Feb 5-Feb 28, 2017 files have been rerun and are available from their usual location on jsimpson.pps.eosdis.nasa.gov. Both the standard HDF5 IMERG products and their derived GIS TIFF/TFW have been rerun and are available for this period.

The March 1-12, 2017 will take another week or so to re-run.

Please contact us for any questions.

Keywords:

PPS is announcing the release of GPM IMERG V04 Final products today, Wednesday March 22, 2017.  Version 04 IMERG data is available on PPS's arthurhou ftp archive for the dates of March 2014 - July 2015. Processing will continue over the next several weeks until the data has been made available up to the present.

This data can be found at:
ftp://arthurhou.pps.eosdis.nasa.gov/gpmallversions/V04 

Individual data orders, subsets or subscription orders can also be set up through PPS's STORM:
https://storm.pps.eosdis.nasa.gov/storm/

Release notes and ATBD documentation for IMERG V04 can be found at:
ftp://arthurhou.pps.eosdis.nasa.gov/gpmdata/documents/

Version 04 supersedes all prior IMERG versions, and users are urged to utilize the new V04 datasets as soon as practical.

More Documentation and information can be found here:
https://pps.gsfc.nasa.gov/GPMprelimdocs.html
https://pps.gsfc.nasa.gov/atbd.html

Main Web Page:
https://pps.gsfc.nasa.gov/

Thank you for your kind attention. 

Please inform the PPS Helpdesk if you have any questions, concerns or comments:
helpdesk@mail.pps.eosdis.nasa.gov

Keywords:

Pages