doi: 10.4304/jsw.9.10.2598-2606
Distributed Storage and Processing Method for Big Data Sensing Information of Machine Operation Condition
Abstract—The traditional relational database cannot satisfy the requirements of the high speed and real-time storage and processing for the distributed Big Data sensing information in the Wide Area Network environment. In this context, the No-SQL database HBase is used to store the big data sensing information of machine operation condition collected by Fiber Bragg Grating sensor network. The distributed storage environment and the optimal database table scheme is built. Moreover, the HBase Rowkey is designed in detail to sharpen the retrieval speed and avoid the server hot point accumulation. Meanwhile, the real-time outlier detection method with the working situational constraint is proposed to monitor the machine working condition. It is implemented by the multi-dimensional histogram statistics method in the Map Reduce distributed environment. Hence, the traditional threshold monitoring method is improved and the false alarm problem is eliminated. Through balancing the performance between HBase and the relational database MySQL, the real-time storage rate of the proposed method can satisfy at least 20 machines running concurrency with 4000 Hz Fiber Bragg Grating sampling frequency by HBase. Also, the effectiveness of real-time outlier detection method is proved by the practical operation data processing.
Index Terms—Machine operation condition, Fiber Bragg Grating sensing, Big Data, distributed storage and processing
Cite: Fan Zhang, Zude Zhou, Wenjun Xu, "Distributed Storage and Processing Method for Big Data Sensing Information of Machine Operation Condition," Journal of Software vol. 9, no. 10, pp. 2598-2606, 2014.
General Information
ISSN: 1796-217X (Online)
Abbreviated Title: J. Softw.
Frequency: Quarterly
APC: 500USD
DOI: 10.17706/JSW
Editor-in-Chief: Prof. Antanas Verikas
Executive Editor: Ms. Cecilia Xie
Abstracting/ Indexing: DBLP, EBSCO,
CNKI, Google Scholar, ProQuest,
INSPEC(IET), ULRICH's Periodicals
Directory, WorldCat, etcE-mail: jsweditorialoffice@gmail.com
-
Oct 22, 2024 News!
Vol 19, No 3 has been published with online version [Click]
-
Jan 04, 2024 News!
JSW will adopt Article-by-Article Work Flow
-
Apr 01, 2024 News!
Vol 14, No 4- Vol 14, No 12 has been indexed by IET-(Inspec) [Click]
-
Apr 01, 2024 News!
Papers published in JSW Vol 18, No 1- Vol 18, No 6 have been indexed by DBLP [Click]
-
Jun 12, 2024 News!
Vol 19, No 2 has been published with online version [Click]