In this paper, we claim that the World Wide Web provides a suitable middleware for Internet-scale, open, and human-centric multi-agent systems. The novelty of our approach is to design the agent environment as a hypermedia application. We apply REST, the architectural style of the Web, to introduce a resource-oriented abstraction layer that decouples the application environment from its deployment context. Higher-level environment abstractions can then be implemented on top of this lower-level abstraction layer. To demonstrate our approach, we implemented a multi-agent application for the Internet of Things in which software agents can seamlessly navigate, use, and cooperate in an environment deployed over multiple Web services (e.g., Facebook, Twitter) and constrained devices.