This paper proposes a new contrastive self-supervised learning paradigm for bearing remaining useful life (RUL) prediction based on CNN-LSTM models. It addresses the dilemma of scarce labels and data imbalance in Prognostics and Health Management (PHM) by designing a specific pretext task to mine the potential degradation-related information in unlabelled data. In this paper, we propose a method to build contrastive sample pairs using sequence order information. Then, a Siamese CNN encoder guided by the customized contrastive loss is designed to maximize the differences between encoding features of the contrastive sample pairs. After that, the CNN's parameters are partly frozen, and its encoded features are used as the input of the subsequent LSTM layer to predict the RUL. Finally, on the labeled dataset, LSTM is fine-tuned to optimize the ability of CNN-LSTM for RUL prediction. The proposed method is validated on “PRONOSTIA Bearing Dataset”. The obtained results and the analysis of the hidden layer output highlight the performance of the proposed approach, which outperforms the supervised learning paradigm in terms of maintaining the ability to capture sequential discriminatory information for better RUL prediction, especially in the case of a reduced amount of labeled data.