Forestry big data platform by Knowledge Graph

Abstract

Using the advantages of web crawlers in data collection and distributed storage technologies, we accessed to a wealth of forestry-related data. Combined with the mature big data technology at its present stage, Hadoop’s distributed system was selected to solve the storage problem of massive forestry big data and the memory-based Spark computing framework to realize real-time and fast processing of data. The forestry data contains a wealth of information, and mining this information is of great significance for guiding the development of forestry. We conducts co-word and cluster analyses on the keywords of forestry data, extracts the rules hidden in the data, analyzes the research hotspots more accurately, grasps the evolution trend of subject topics, and plays an important role in promoting the research and development of subject areas. The co-word analysis and clustering algorithm have important practical significance for the topic structure, research hotspot or development trend in the field of forestry research. Distributed storage framework and parallel computing have greatly improved the performance of data mining algorithms. Therefore, the forestry big data mining system by big data technology has important practical significance for promoting the development of intelligent forestry.

Keywords

Intelligent forestry / Co-word analysis / Knowledge Graph / Big data

References

[1] Boettcher SEP, Wolfe JM. Searching for the right word: hybrid visual and memory search for words. Atten Percept Psychophys, 2015, 77(4): 1-11.

[2] Bovenzi M. Metrics of whole-body vibration and exposure–response relationship for low back pain in professional drivers: a prospective cohort study. Int Arch Occup Environ Health, 2009, 82(7): 893-917.

[3] Callon M, Courtial JP, Laville F. Co-word analysis as a tool for describing the network of interactions between basic and technological research: the case of polymer chemistry. Scientometrics, 1991, 22(1): 155-205.

[4] Dong XL, Gabrilovich E, Heitz G, Horn W, Lao N, Murphy K (2014) Knowledge vault: a web-scale approach to probabilistic knowledge fusion, pp 601–610. https://doi.org/10.1145/2623330.2623623

[5] Dong XL, Gabrilovich E, Heitz G, Horn W, Murphy K, Sun S. From data fusion to knowledge fusion. Proc VLDB Endow, 2015, 7(10): 881-892.

[6] Fisher DH. Knowledge acquisition via incremental conceptual clustering. Mach Learn, 1987, 2(2): 139-172.

[7] Frank S, Fürst C, Pietzsch F. Cross-sectoral resource management: how forest management alternatives affect the provision of biomass and other ecosystem services. Forests, 2015, 6(3): 533-560.

[8] Gong XY, Bohu Li, Chai XD, Gu M. Survey on big data platform technology. J SystSimul, 2014, 26(3): 489-496. (in Chinese with English abstract)

[9] Groc CD (2011) Babouk: focused web crawling for corpus compilation and automatic terminology extraction. In: IEEE/WIC/ACM international conference on web intelligence and intelligent agent technology, vol 1, pp 497–498

[10] Gu Y, Ji L, Jiang Z, He J (2012) Endless and scalable knowledge table extraction from semi-structured websites. In: IEEE, international conference on data mining workshops. IEEE Computer Society, pp 835–842

[11] Hou J, Zhang PD, Zhang BR, Yuan XZ. The development status of Chinese forestry biomass resources and the suggestion. Renew Energy Resources, 2009, 27: 113-117.

[12] Jiang L, Li B, Song M (2011) The optimization of HDFS based on small files. In: IEEE international conference on broadband network and multimedia technology, pp 912–915

[13] Jie W, Yang S, Wang Y, Cheng H (2016) The crawling and analysis of agricultural products big data based on Jsoup. In: International conference on fuzzy systems & knowledge discovery. https://doi.org/10.1109/FSKD.2015.7382112

[14] Li Z (2012) Parsing the internal structure of words: a new paradigm for Chinese word segmentation. Meeting of the Association for Computational Linguistics: Human Language Technologies

[15] Li L, Hao T, Chi T. Evaluation on China’s forestry resources efficiency based on big data. J Clean Prod, 2016, 142: 513-523.

[16] Liu GY, Hu JM, Wang HL. A co-word analysis of digital library field in China. Scientometrics, 2012, 91(1): 203-217.

[17] Mohai P. The forest service since the national forest management act. Policy Stud J, 2010, 23(2): 247-252.

[18] Paulheim H. Knowledge Graph refinement: a survey of approaches and evaluation methods. Semantic Web, 2017, 8(3): 489-508.

[19] Pujara J, Miao H, Getoor L, Cohen W. Knowledge graph identification. The semantic web—ISWC 2013, 2013, Berlin: Springer.

[20] Ruslandi AR, Sist P, Peña-Claros M, Thomas R, Putz FE. Beyond equitable data sharing to improve tropical forest management. Int For Rev, 2014, 16(4): 497-503.

[21] Shvachko KV. HDFS scalability: the limits to growth. Login, 2010, 35(2): 6-16.

[22] Stevanovic D, An A, Vlajic N. Feature evaluation for web crawler detection with data mining techniques. Expert Syst Appl, 2012, 39(10): 8707-8717.

[23] Sun W (2012) A stacked sub-word model for joint Chinese word segmentation and part-of-speech tagging. Meeting of the Association for Computational Linguistics: Human Language Technologies. https://doi.org/10.2514/6.2006-1683

[24] Tang L, Shao G, Dai L. Roles of digital technology in China's sustainable forestry development. Int J Sustain Dev World Ecol, 2009, 16(2): 94-101.

[25] Wu B, Zhang M, Zeng H, Zhang X, Yan N, Meng J (2016) Agricultural monitoring and early warning in the era of big data. J Remote Sens 20(5)

[26] Zhang M, Deng Z, Che W, Liu T. Combining statistical model and dictionary for domain adaption of Chinese word segmentation. J Chin Inf Process, 2012, 26(2): 8-12.

[27] Zhao Y, Li G, Li F, Li D (2011) The establishment and application of spatial decisions support systems of digital forestry. International Conference on Geoinformatics. https://doi.org/10.1109/CTS.2014.6867550

[28] Zheng LM. Design of crawler based on html parser information extraction. Microcomput Inf, 2009, 25(15): 123-124.