``Meta-Simulation Design and Analysis for Large Scale
Networks''
David Bauer, Jr.
Ph.D. Thesis, Department of Computer Science, Rensselaer
Polytechnic Institute, December 2005.
ABSTRACT
Performance analysis techniques are fundamental to the process of
network protocol design and operations. A variety of techniques have
been used by researchers in different contexts: analytic models (eg:
TCP models, web models, self-similar models, topology models),
simulation platforms (eg: ns-2, SSFnet, GloMoSim, Genesis),
prototyping platforms (eg: MIT Click Router toolkit~\cite{click},
XORP~\cite{xorp}), tools for systematic design-of-experiments and
exploring parameter state spaces (eg: Recursive Random Search,
STRESS~\cite{stress}), experimental emulation platforms (eg:
Emulab), real-world overlay deployment platforms (eg: Planetlab),
and real-world measurement and data-sets (eg: CAIDA~\cite{caida},
Rocketfuel~\cite{SMW}).
The high-level motivation behind the use of these tools is simple:
to gain varying degrees of qualitative and quantitative
understanding of the behavior of the system-under-test. This
high-level purpose translates into a number of specific lower-level
objectives, such as: validation of protocol design and performance
for a wide range of parameter values (parameter sensitivity),
understanding of protocol stability and dynamics, and studying
feature interactions between protocols. Broadly, we may summarize
the objective as a quest for general invariant relationships between
network parameters and protocol dynamics.
To address these needs, we developed an experiment design platform
that will allow us to empirically model and heuristically search for
optimizing protocol response. In general the protocol response is a
function of a large vector of parameters, i.e., is a response
surface in a large-dimensional parameter space (perhaps tens of
thousands or more dimensions). We build off recent work at
Rensselaer on an efficient search algorithm (called Recursive Random
Search) for large-dimensional parameter optimization, and empirical
modeling of protocol performance characteristics especially in
``interesting'' regions of the parameter state space. The result of
this work includes a unified search, empirical modeling and
optimization framework with demonstrated ability to pose meaningful
large-scale network design questions and provide ``good'' models
rapidly.