Abstract: This paper proposed a distributed search algorithm for the poor based on Hadoop, which combined the data feature extraction and text clustering technology to aggregate the similar text, established the corresponding text feature vector space according to the query accuracy requirements, and filtered the data with poor correlation so that it did not participate in the search, so as to improve the execution efficiency of the system and reduce the internal execution speed. The experiment results showed that the algorithm can be combined with parameter settings to personalize the use of different application scenarios and achieve the best personalized effect, which had great application value.

Key words: Hadoop, big data search, data mining, characteristic item, sorting result sets