Author(s)

Chenxiang Zhang

Corresponding Author

Chenxiang Zhang

Abstract

How to extract useful information from massive data quickly becomes the most difficult problem that application software developers encounter in curriculum development. Based on the analysis of the key technical foundation and other existing distributed storage and calculation researches on Hadoop cluster technology combination, as well as their business needs and the actual hardware and software capabilities, the paper proposes a large-scale Hadoop data processing based on model and data structure design program in several processes of organizing and using the programming methods, introduces the development of the model, model of log data preprocessing and its application to large website.

Keywords

Massive Data, Data Processing, Hadoop