随着微博、照片分享等社会化媒体的快速发展,每天产生了大量的短文本内容如评论、微博等,对其进行深入挖掘有重大的应用价值和学术意义.该文选取微博作为例子,详细阐述我们提出的方法.微博信息流因其简短和实时的特性而具有非常大的价值,已经成为市场营销,股票预测、舆情监控等应用的重要信息源.尽管如此,微博内容特征极其稀疏、上下文语境提取困难,使得微博信息的挖掘面临着很大挑战.因此,我们提出一种基于Wikipedia的微博语义概念扩展方法,通过自动识别那些与微博信息语义相关的Wikipedia概念来丰富它的内容特征,从而有效提高微博信息数据挖掘和分析的效果.该文工作首先通过可链接性剪枝、概念关联和消歧,发现微博信息中重要的n-gram所对应的Wikipedia概念;其次,采用基于概念文档关联矩阵的NMF分解(非负矩阵分解)方法获取Wikipedia概念之间的语义近邻,为微博信息扩展相关的语义概念.基于TREC 2011的微博数据集和Wikipedia 2011数据集进行实验,与已有两个相关研究工作比较,该文提出的方法取得了较好的效果.
The emergence of social media services is seeing a large amount of short text such as tweets and reviews are generated every day. Mining those data attracts more interests from both industry and academia. And such data has already become an important source of information for marketing, stock prediction, etc. However, mining short text is non-trival since of extremely sparse text and lack of context. Thus we propose to enrich short text content by automatically identifying concepts in open knowledge bases such as Wikipedia, which are semantically related to them. In our work, firstly, through linkable pruning, concept linking and disambiguation, important n-grams in tweet and their related Wikipedia concepts are linked. Secondly, NMF (non-negative matrix factorization) is used to factorize concept-document matrix to get concepts' semantic neighbors. And related concepts are then expended for tweets. Experiments on the collection of tweets from TREC 2011 and Wikipedia 2011 show that our approach gets effective results.