Volume 9 Number 10 (Oct. 2014)
Home > Archive > 2014 > Volume 9 Number 10 (Oct. 2014) >
JSW 2014 Vol.9(10): 2598-2606 ISSN: 1796-217X
doi: 10.4304/jsw.9.10.2598-2606

Distributed Storage and Processing Method for Big Data Sensing Information of Machine Operation Condition

Fan Zhang, Zude Zhou, Wenjun Xu

School of Information Engineering, Wuhan University of Technology, Wuhan 430070, China; Key Lab. of Fiber Optic Sensing Technology and Information Processing, Ministry of Education, Wuhan 430070, China

Abstract—The traditional relational database cannot satisfy the requirements of the high speed and real-time storage and processing for the distributed Big Data sensing information in the Wide Area Network environment. In this context, the No-SQL database HBase is used to store the big data sensing information of machine operation condition collected by Fiber Bragg Grating sensor network. The distributed storage environment and the optimal database table scheme is built. Moreover, the HBase Rowkey is designed in detail to sharpen the retrieval speed and avoid the server hot point accumulation. Meanwhile, the real-time outlier detection method with the working situational constraint is proposed to monitor the machine working condition. It is implemented by the multi-dimensional histogram statistics method in the Map Reduce distributed environment. Hence, the traditional threshold monitoring method is improved and the false alarm problem is eliminated. Through balancing the performance between HBase and the relational database MySQL, the real-time storage rate of the proposed method can satisfy at least 20 machines running concurrency with 4000 Hz Fiber Bragg Grating sampling frequency by HBase. Also, the effectiveness of real-time outlier detection method is proved by the practical operation data processing.

Index Terms—Machine operation condition, Fiber Bragg Grating sensing, Big Data, distributed storage and processing

[PDF]

Cite: Fan Zhang, Zude Zhou, Wenjun Xu, "Distributed Storage and Processing Method for Big Data Sensing Information of Machine Operation Condition," Journal of Software vol. 9, no. 10, pp. 2598-2606, 2014.

General Information

  • ISSN: 1796-217X (Online)

  • Abbreviated Title: J. Softw.

  • Frequency:  Quarterly

  • APC: 500USD

  • DOI: 10.17706/JSW

  • Editor-in-Chief: Prof. Antanas Verikas

  • Executive Editor: Ms. Cecilia Xie

  • Abstracting/ Indexing: DBLP, EBSCO,
           CNKIGoogle Scholar, ProQuest,
           INSPEC(IET), ULRICH's Periodicals
           Directory, WorldCat, etc

  • E-mail: jsweditorialoffice@gmail.com

  • Oct 22, 2024 News!

    Vol 19, No 3 has been published with online version   [Click]

  • Jan 04, 2024 News!

    JSW will adopt Article-by-Article Work Flow

  • Apr 01, 2024 News!

    Vol 14, No 4- Vol 14, No 12 has been indexed by IET-(Inspec)     [Click]

  • Apr 01, 2024 News!

    Papers published in JSW Vol 18, No 1- Vol 18, No 6 have been indexed by DBLP   [Click]

  • Jun 12, 2024 News!

    Vol 19, No 2 has been published with online version   [Click]