In the last few years, the growing complexity of current applications has led to the design of self-organizing systems presenting self-* properties. Those systems are composed of several autonomous interactive entities. They behave autonomously and present interesting characteristics allowing them to handle dynamics coming from exogenous and endogenous changes. As those systems are different from classical ones, new or updated characterization and evaluation criteria are required for analyzing the contribution of self-* properties and system performance. This work is a first step for evaluating self-* systems. Its main purpose is to propose a simulation based evaluation framework which aims at guiding the evaluation of this kind of systems from the design phase all the way to the execution results: evaluation of the system at run-time, intrinsic and development methodologies characterization.