The relative pose between inertial and visual sensors equipped in autonomous robots is calibrated in two steps. In the first step, the sensing system is moved along a line, the orientations in the relative pose are computed from at least five corresponding points in the two images captured before and after the movement. In the second step, the translation parameters in the relative pose are obtained with at least two corresponding points in the two images captured before and after one step motion. Experiments are conducted to verify the effectiveness of the proposed method.