Processing financial data in big data distributed environment using sentiment analysis

Abstract

This chapter covers design, implementation and evaluation of a system, which predicts future stock prices basing on analyses of data from social media services. The authors took advantage of available large datasets from Twitter micro blogging platform and stock markets. Data was collected during three months and processed for further analysis. Machine learning was employed to conduct sentiment classification of data coming from social networks in order to estimate future stock prices. Calculations were performed in distributed environment according to Map Reduce programming model. Evaluation and discussion of preliminary results of predictions for different time intervals and input datasets proved efficiency of chosen approach.[...]