Fast distributed and parallel pre-processing on massive satellite data using grid computing

Abstract

Distributed/parallel-processing system like sun grid engine (SGE) that utilizes multiple nodes/cores is proposed for the faster processing of large sized satellite image data. After verification, distributed process environment for pre-processing performance can be improved by up to 560.65% from single processing system. Through this, analysis performance in various fields can be improved, and moreover, near-real time service can be achieved in near future.

Keywords

satellite data / image processing / computation intensive computing

References

[1] MustaphaM A, Sei-IchiS, LihanT. Satellite-measured seasonal variations in primary production in the scallop-farming region of the Okhotsk Sea [J]. ICES Journal of Marine Science, 2009, 66(7): 1557-1569

[2] SuwannathatsaS, WongwisesP. Chlorophyll distribution by oceanic model and satellite data in the bay of Bengal and Andaman sea [J]. Oceanological and Hydrobiological Studies, 2013, 42(2): 132-138

[3] KeithA, EuroconsultS B. Satellite-based earth observation: Market prospects to 2018 [R]. Euroconsult, 2009

[4] KozhevnikovaA V, TarasenkovM V, BelovV V. Parallel computations for solving problems of the reconstruction of the reflection coefficient of the Earth’s surface by satellite data [J]. Atmospheric and Oceanic Optics, 2013, 26(4): 326-328

[5] http://seadas.gsfc.nasa.gov/doc/toplevel/ancinfo.html

[6] KwakJ, YoonJ, JungY, HahmJ, ParkDongin. Large-scale data analysis based on hadoop for astroinformatics [J]. Journal of KIISE, 2011, 17(11): 587-591

[7] BorthakurD. The hadoop distributed file system: Architecture and design [R]. Apache Software Foundation, 2007

[8] ShethA. A new landscape for distributed and parallel data management [J]. Distributed and Parallel Databases, 2012, 30(2): 101-103

[9] HongS, ShinY, ChangJaewoo. Optimization and performance analysis of cloud computing platform for distributed processing of big data [J]. Journal of Korea Spatial Information Society, 2011, 19(4): 55-71

[10] http://gridscheduler.sourceforge.net/howto/GridEngineHowto.html

[11] NakamuraA, ParkJ G, MatsushitaK, MackinK J, NunohiroE. Development and evaluation of satellite image data analysis infrastructure [J]. Artificial Life and Robotics, 2012, 16(4): 511-513

[12] El-Kenawy El-SayedM T, El-Desoky Ali Ibraheem, Al-RahamawyM F. Distributing graphic rendering using grid computing with load balancing [J]. International Journal of Computer Applications, 2012, 47(9): 1-6