中文摘要：

H.264/AVC视频编码标准采用了基于率失真最优化（RDO ）技术的多种编码模式选择的预测编码方法，取得了很高的率失真效率，同时也引入了非常高的计算复杂度．为了降低RDO全搜索方法的预测复杂度，本文提出了一种分层判决结构的快速帧间模式选择方法．根据H ．264/AVC帧间预测编码的结构特点，将宏块的各种模式分为4个层次．研究运动估计预测原理，利用预测编码的时间复杂度与空间复杂度之间的耦合关系，得到每个层次的自适应判决阈值，使得快速模式选择方法应用于每一个层次，在保证率失真性能的条件下，及时退出多余的全搜索率失真代价函数计算．实验结果表明，该算法平均减少了45.79％的编码时间，同时，保持了视觉质量和输出码率不变．对比结果也显示，与Zeng方法相比，得到了0.05dB的PSNR增益；相比Choi方法，本文方法节省了约20％的计算时间；本文方法优于这两种快速模式选择方法．

英文摘要：

The rate distortion optimization （RDO） which uses mode-selection based prediction coding can achieve the high-est coding efficiency in H .264/AVC video coding standard .However ,exhaustive checking of all the prediction modes increases computational complexity significantly .In this paper ,we propose a hierarchical decision structure for fast inter-frame mode selection to reduce the complexity of the exhaustive RDO mode decision process .Different modes of macroblock are first classified into 4 lay-ers in terms of the characteristics of the structure of inter-frame prediction coding in H .264/AVC .Using the coupled relationship between the spatial and temporal complexity of the motion-estimation prediction coding ,the adaptive threshold for each layer can be calculated .The obtained threshold can then be applied to each layer to terminate the mode selection process while maintaining al-most the same R-D performance in H.264 .Experimental results show a 45 .79% reduction on computational time and the sane RD performance as the baseline coder by using the proposed fast selection method .The results also demonstrate that the proposed algo-rithm is superior to two recent fast mode decision algorithms ,with the peak signal-to-noise ratio （PSNR） being improved by 0 .05dB on average when compared to Zeng′s algorithm and the entire encoding time being reduced by about 20% on average when com-pared to Choi′s mode decision algorithm .