The use of recurrent neural networks in several applications has allowed to capture impressive results, especially in various applications such as video prediction and it has become a promising direction of scientific research. In this paper, we introduce a novel algorithm for video prediction called "Robust Spatiotemporal Convolutional Long Short-Term Memory (Robust-ST-ConvLSTM) Algorithm" that outperforms the state-of-the-art approaches. Robust-ST-ConvLSTM is a memory flow algorithm based on higher order ConvLSTM. This memory flow algorithm is holding the spatiotemporal information to optimize and control the prediction abilities of the ConvLSTM cell. Our approach is developed in the specific context of predicting future frames based on historical observations, and we experimentally validate the ability of the proposed algorithm on two spatiotemporal datasets, including a moving variant of MNIST dataset of handwritten digits, and KTH which is a human motion dataset.