In complex real-world decision problems, ensuring safety and addressing uncertainties are crucial aspects. In this work, we present an uncertainty-aware Reinforcement Learning agent designed for risk-sensitive applications in continuous action spaces. Our method quantifies and leverages both epistemic and aleatoric uncertainties to enhance agent's learning and to incorporate risk assessment into decision-making processes. We conduct numerical experiments to evaluate our work on a modified version of Lunar Lander with variable and risky landing conditions. We show that our method outperforms both Deep Deterministic Policy Gradient (DDPG) and TD3 algorithms by reducing collisions and having significant faster training. In addition, it enables the trained agent to learn a risk-sensitive policy that balances performance and risk based on a specific level of sensitivity to risk required for the task.