Real-world decision problems, such as Domestic Hot Water (DHW) production, require the consideration of multiple, possibly conflicting objectives. This work suggests an adaptation of Deep Q-Networks (DQN) to solve multi-objective sequential decision problems using scalarization functions. The adaptation was applied to train multiple agents to control DHW systems in order to find possible trade-offs between comfort and energy cost reduction. Results have shown the possibility of finding multiple policies to meet preferences of different users. Trained agents were tested to ensure hot water production with variable energy prices (peak and off-peak tariffs) for several consumption patterns and they can reduce energy cost from 10.24 % without real impact on users’ comfort and up to 18 % with slight impact on comfort.