互联网搜索数据与社会经济行为的相关性已被多篇文献所证实,然而对于这项研究的基础工作——数据预处理,目前尚缺乏系统的方法。本文提出一套完整的搜索数据预处理流程,包括搜索关键词的选择、时差关系判定、关键词指数合成等步骤,并对各关键步骤给出了处理方法及标准。通过该方法可以得到稳定且高拟合度的先行关键词指数。本文以股票市场中上证指数为研究对象,实证检验得出,合成后的先行关键词指数与上证指数的拟合优度高达0.979。Granger检验证实了对上证指数具有显著的预测能力,回归结果显示关键词指数每变动1个百分点,后一期的上证指数将同方向变动0.518个百分点。
The correlations between Internet search data and socio-economic behavior have been confirmed by many researches, but the basis of this study-data preprocessing, is short of general methodology now. In this paper, we present a systematic method for Internet search data preprocessing, which includes the critical steps: keywords selection, time difference measurement, and leading index composition. Using this method, we can get the leading keywords index with stable leading relation and high degree of fit. Specifically, the correlation coefficient between our leading keywords index and Shanghai Composite Index reaches 0. 979, Granger test confirms that leading keywords index has significant predictive ability for Shanghai Composite Index, and the regression results show that each percentage point change of keywords index, Shanghai Composite Index moves 0. 518 percentage points in the same direction in next period.