Ten years ago, researchers in multi-agent systems became more and more aware that agent systems consist of more than only agents. The series of workshops on Environments for Multi-Agent Systems (E4MAS 2004-2006) emerged from this awareness. One of the primary outcomes of this endeavor was a principled understanding that the agent environment should be considered as a primary design abstraction, equally important as the agents. A special issue in JAAMAS 2007 contributed a set of influential papers that define the role of agent environments, describe their engineering, and outline challenges in the field that have been the drivers for numerous follow up research efforts. The goal of this paper is to wrap up what has been achieved in the past 10 years and identify challenges for future research on agent environments. Instead of taking a broad perspective, we focus on three particularly relevant topics of modern software intensive systems: large scale, openness, and humans in the loop. For each topic, we reflect on the challenges outlined 10 years ago, present an example application that highlights the current trends, and from that outline challenges for the future. We conclude with a roadmap on how the different challenges could be tackled. Keywords: agent environment, multi-agent systems, middleware, large-scale systems, open systems, human in the loop.