A Seismic Data Processing System based on Fast Distributed File System

Jun Li; Changsen Pan; Menghan Lu

doi:10.24297/ijct.v14i5.3986

Authors

Jun Li Department of Automation,CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System, University of Science and Technology of China, Hefei
Changsen Pan Department of Automation,CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System, University of Science and Technology of China, Hefei
Menghan Lu Department of Automation,CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System, University of Science and Technology of China, Hefei

DOI:

https://doi.org/10.24297/ijct.v14i5.3986

Keywords:

Big Data, Seismic Exploration, SEGY File Format, Fast Distributed File System

Abstract

Big data has attracted an increasingly number of attentions with the advent of the cloud era, and in the field of seismic exploration, the amount of data created by seismic exploration has also experienced an incredible growth in order to satisfy the social needs. In this case, it is necessary to build a highly-effective system of data storage and process. In our paper, we aim at the properties of the seismic data and the requirement to the performance of IO, and establish a distributed file system with the goal of processing seismic data based on the Fast Distributed File System (Fast DFS), then test our system through a series of operations such as file write and read, and the results show that our file system is very proper and effective when processing seismic data.

Downloads

Download data is not yet available.

References

[1] Patel A B, Birla M, Nair U. Addressing big data problem using Hadoop and Map Reduce [C]//Engineering (NUiCONE), 2012 Nirma University International Conference on. IEEE, 2012: 1-5.
[2] Rajaraman A, Ullman J D. Mining of massive datasets [M]. Cambridge University Press, 2012.
[3] https://nutch.apache.org/
[4] Sagiroglu S, Sinanc D. Big data: A review[C]//Collaboration Technologies and Systems (CTS), 2013 International Conference on. IEEE, 2013: 42-47.
[5] Ghemawat S, Gobioff H, Leung S T. The Google file system [C]//ACM SIGOPS Operating Systems Review. ACM, 2003, 37(5): 29-43.
[6] Dean J, Ghemawat S. MapReduce: simplified data processing on large clusters [J]. Communications of the ACM, 2008, 51(1): 107-113.
[7] Claerbout J F. Fundamentals of geophysical data processing [J]. 1985.
[8] Howard J H, Kazar M L, Menees S G, et al. Scale and performance in a distributed file system [J]. ACM Transactions on Computer Systems (TOCS), 1988, 6(1): 51-81.
[9] Zhou H W. Practical Seismic Data Analysis[M]. Cambridge University Press, 2014.
[10] Shvachko K, Kuang H, Radia S, et al. The hadoop distributed file system [C]//Mass Storage Systems and Technologies (MSST), 2010 IEEE 26th Symposium on. IEEE, 2010: 1-10.
[11] http://code.google.com/p/fastdfs/
[12] Jia B, Wlodarczyk T W, Rong C. Performance considerations of data acquisition in hadoop system [C]//Cloud Computing Technology and Science (CloudCom), 2010 IEEE Second International Conference on. IEEE, 2010: 545-549.
[13] http://bbs.chinaunix.net/forum-240-1.html
[14] Borthakur D, Gray J, Sarma J S, et al. Apache Hadoop goes realtime at Facebook [C]//Proceedings of the 2011 ACM SIGMOD International Conference on Management of data. ACM, 2011: 1071-1080.