VIIRS/AHI Flood Product
Processing regions are defined for VIIRS and AHI data to generate flood detection products and time-series composites. AHI processing is broken down into five regions, with specific channels required for software execution. Runtime, data volume, and processing specifics are detailed for each region, providing insights into efficient data processing strategies.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
VIIRS/AHI Flood Product Coverage over Australia
AHI Processing region I have AHI processing is broken down into 5 region Region 1 Northeast Asia Region 2 Southeast Asia Region 3 Maritime Continent Region 4 - Australia Region 5 New Zealand Processing Regions 1 & 2 would not be necessary; Regions 3 and/or 5 could also be skipped Regions can be redefined as any lat/lon box 4 channels are required: 3, 4, 5, 13 Software H8_AHI_Flood_Detection_Version01 generate a flood detection product for each AHI image time AHI_Composition_Process generate a time-series composite of the individual images. (I run this after each image has been processed; alternately, it could be run hourly or daily depending on user needs) Executables file size ~25 MB, asset data ~130 GB (this is includes some files that are only applicable to the Western hemisphere) Data volume Approximately 16 GB per day for full-disk; could be ~1/5 the data volume if only running 1/5 the coverage area The satellite imagery data volume is on the order of 125 GB per day for the full disk. The imagery does not need to be kept after the product is run, and the volume could be reduced by not using all of the full disk sections. Runtime On the order of 20 minutes to process all 5 regions in a full disk Because imagery is available every 10 minutes and it takes 20 minutes to process, it is important to be able process multiple cases simultaneously. Presumably if the domain were to be reduced to ~1/5 of the full disk, the runtime would also be reduced
VIIRS Processing region Granuals can be run to create a product with a domain based on the image location, or region of interest can be predefined and the product will populate only the locations with valid data over the region of interest. For direct broadcast processing over CONUS, I process imagery into 7 fixed regions For global processing, each granule is processed with each product having a unique domain Presumably the Australia could be either 1 large region or a few smaller regions GITCO, GMTOC, SVI01, SVI02, SVI03, SVI04, and SVI05 data files are required (cloud mask product is optional, having it as a dependency can increase product latency) Software VIIRS_Swath_Projection project the data VIIRS_Flood_Detection_Version01 detect flooding Executables file size ~25 MB, asset data ~30 GB (this is includes some files that are only applicable to the Western hemisphere) Data volume On the order of 10 GB per day for CONUS Including the satellite imagery data volume and intermediary files generated, on the order of 200 GB per day is used for CONUS. The imagery and intermediary files does not need to be kept after the product is run, and the volume could be reduced. Runtime On the order of 40 minutes to process a direct broadcast overpass region Multiple regions are processed simultaneously With a smaller domain, runtime would reduced
VIIRS daily composite & VIIRS/AHI joint product Run Mosaick_Subset_Flood_Process for, NPP, J01, & AHI The hemisphere is divided into tiles and a file is created for each tile It takes ~15 minutes to run a full hemisphere of VIIRS; a smaller domain would be faster AHI runtime is only a few minutes for full hemisphere Run ABI_Composition_Process for each AHI region It only takes a few minutes to run each of the 5 AHI regions Run VIIRS_Composite This combines the NPP and J01 results (from Modaick_Subset_Flood_Process) into a single file per region (136 global regions) It takes ~15 minutes to run the entire hemisphere Run Merge_ABI_VIIRS_Flood This combines the AHI & VIIRS results for each of the 136 global regions It takes ~15 minutes to run the entire hemisphere Data volume is ~75 GB/day (much less if not running full disk AHI and global VIIRS)
Outputs The flood products can be generated in HDF of NetCDF formats Scripts exist to convert output into GeoTIFF and Shapefile formats Newer scripts relay on Python & GDAL software Older software uses IDL & McIDAS It should be possible to run without McIDAS & IDL, but processing at CIMSS still uses some of the scripts with software dependencies. It is possible to set up a dedicated RealEarth server to display product A virtual machine can be set up either on the same hardware that generates the products, or a dedicated machine
Hardware At CIMSS, to support multiple simultaneous processes, we run on a system with 40 CPUs, 125GB of memory, and 20 TB of hard drive space The computing resources support global processing, to run a smaller domain fewer resources would be adequate.
Processing Running the flood software is fairly straightforward Creating scripts for routine processing is less straightforward The scripts developed at CIMSS may not be readily portable to other locations Example code can be shared, but scripts will need to be customized Paths to data directories The frequency code is run Apply logic to only process imagery with daylight and skip dark/nighttime imagery Imagery can be processed as soon as it arrives, daily composites could run & updated with each new image time. It might make sense to run some products only once daily Adjustments to some configuration files and scripts will need to be made to run different (smaller) regions of interest (scripts at CIMSS are set up to run globally, hemispherically, or over CONUS)
