文档介绍：分布式系统中局部处理机的设计与实现
摘要：针对企业生产过程中存在大量原始数据需要实时处理的问题，设计并实现了一个基于自定义架构的局部处理机。在设计之初以Hadoop的并行架构为参考，对MapReduce的工作原理和缓存方式进行了分析，在此基础上根据实际生产环境设计了一个“多类线程协同处理”的程序架构，并辅以两类自定义的数据缓存方式，保证了分布式系统中的局部处理机在接收、计算、上传各环节的并发性和正确性。该系统投入实际生产并连续使用一年有余，实现了将企业多个车间生成的原始数据进行实时处理的预期目标，具有很好的稳定性、有效性和可扩展性。实际应用结果表明，自定义的程序架构和有效的缓存方式能实现大量数据的同步处理及分析。
关键词：Hadoop；MapReduce；数据缓存；并发；局部处理机；分布式系统
中图分类号： 文献标志码：A
Abstract：Concerning the problem that there is a lot of data which need to be realtime processed during the production process， the local processor， based on multithread and coprocessing architecture and two data buffer mechanisms was accomplished. As a reference
， multifunctional thread in Hadoop’s parallel architecture has an impressed impact on the design of the local processor， especially MapReduce principle. Based on the userdefined architecture， the local processor ensures data concurrency and correctness during receiving， computing and uploading. The system has been put into production for over one year. It can meet the enterprise requirements and has good stability， realtime， effectiveness and scalablility. The application result shows that the local processor can achieve synchronized analysis and processing of mass data.
Key words： Hadoop； MapReduce； data buffer； concurrency； local processor； distributed system
0 引言
在实际应用中，很多企业基于LabVIEW（Laboratory Virtual Instrument