To keep up with current technological developments, the engineering of multi-agent systems (MAS) has to provide solutions to: (i) support large scale systems, (ii) cope with open systems, and (iii) support humans in the loop. In this paper, we claim that the World Wide Web provides a suitable middleware for engineering MAS that address these challenges in an integrated manner. Even though approaches to engineer Web-based MAS have already been explored in the MAS community, existing proposals do not achieve a complete integration with the Web architecture. We approach this problem from a new angle: we design the agent environment as a hypermedia application. We apply REST, the architectural style of the Web, to introduce a resource-oriented abstraction layer for agent environments that decouples the application environment from its deployment context. Higher-level environment abstractions can then be implemented on top of this lower-level abstraction layer. To demonstrate our approach, we implemented a multi-agent application for the Internet of Things in which software agents can seamlessly navigate, use and cooperate in an environment deployed over multiple Web services (e.g., Facebook, Twitter) and constrained devices.