= Federation in Support of Multi-Party Experiments = Multi-party experiments are a research activity undertaken by multiple experimenters or groups of experimenters that requires differing views of the environment. For example, a multi-party security experiment may pit an attack designer against a defense designer, each with limited visibility and control of the system. Such environments may also support collaboration. Conceptually, multiple users come to DETER with experiment fragments and constraints on how those fragments are combined and DETER creates the composite environment that meets those constraints. [[Image(model.png)]] DETER's support for this is evolving. Our initial implementation uses the simple composition system to combine partial topologies automatically, subject to constraints, and the federation to construct the composite topologies in DETER as federated experiments with individual service configurations. This page documents the proof of concept implementation. More work is ongoing in making this system more broadly usable and expressive. == Combining partial topologies == One aspect of combining multiple environments is building a single overall topology, even if no individual player will be able to see all of it. We use a simple label-matching constraint resolver to combine topology fragments. Each node in a fragment may be annotated with constraints that describe how it can be attached to nodes in other fragments. Each node has one or more attachment points, each of which as one or more labels attached to it, called the attachment point's type. Additionally each attachment point may only be attached to points that have labels specified in its "Link to" list. The labels on each attachment point are just labels that the designer chooses. They shape the composition based on the matching constraints on other attachnemt points. In the example below, the two left most nodes have a single attachment point of type "Single". They can only be attached to points of type "Node". The next node has connections of type "Node" that can connect to either "Node" or "Single" connections. They combine to form a 3-node fragment with one attachment point type "Node" that can connect to either "Node" or "Single" connections. [[Image(composition.png)]] These fragments can either be described in [FeddAbout#ExperimentDescriptions the extended ns-2] language or in simple xml containing a [http://fedd.deterlab.net/wiki/FeddPluginArchitecture#TopologyDescriptionLanguage topdl] description. The three-connection node is encoded in ns-2 as: {{{ # simple DETER topology # COMPOSITION: bot0:optional:Node:Single,Node # COMPOSITION: bot0:optional:Node:Single,Node # COMPOSITION: bot0:optional:Node:Single,Node set ns [new Simulator] source tb_compat.tcl set bot0 [$ns node] tb-set-node-testbed $bot0 "deter" $ns rtproto Static $ns run }}} The comments containing the {{{COMPOSITION}}} keyword contain the constraints. The first colon-separated field gives the node name to which the constraint applies, the second whether the field is optional or required, the third the type of connector, and the last a comma-separated list of acceptable types to connect. Optional constraints need not be satisfied, though our satisfier tries to satsify as many as possible. This fragment has one attachment point for each commented attachment point description. The same component in XML is: {{{ bot0 False Node Single Node bot0 False Node Single Node bot0 False Node Single Node 1.0 bot0 type pc testbed deter failureaction fatal }}} In ns-2 the constraints are encoded as comments as an expedient. DETER and fedd are moving away from the programmatic topology expression, and the relative difficulties of encoding such constranints in ns-2/tcl vs. XML is an example of why. The overhead of adding new ns-2/tcl is much higher than parsing a few new XML elements. == Composing Fragments == The simple composer is included in the fedd git repository as [source:fedd/compose.py]. It takes one or more fragment files as described above and produces an new fragment file in either xml or ns-2, suitable for passing to [FeddCommands#fedd_create.py fedd_create.py] or back to the composer. The compose command takes the following arguments: '''--help''':: Display help text '''--url''':: The url of a running fedd that is willing to translate ns-2 to topdl/xml. {{{https://users.isi.deterlab.net:23235}}} will work. Not needed if all inputs are topdl/XML. '''--multifile=''' ''file,number'':: Include the given ''file'' in the composition ''number'' times. '''--output=''' ''file'':: Save the composed fragment in ''file''. The suffix of the file determines the output format, {{{tcl}}} or {{{ns}}} are saved as ns-2, and {{{topdl}}} or {{{xml}}} are topdl/XML. If this option is not given, the fragment goes to standard out as XML, though the '''---format''' option can set the output explicitly. '''--format=''' ''format'': Override the format of the output. Choices are: {{{tcl}}} or {{{ns}}} to output ns-2, and {{{topdl}}} or {{{xml}}} to output topdl/XML. '''--add_testbeds''':: Add a testbed attribute to each component. Each view of the multi-party experiment is instantiated in a separate testbed. If this option is given, the same testbed attribute is assigned to each node in each fragment. All the nodes in the first fragment are assigned to {{{testbed000}}} all in the second to {{{testbed001}}}, etc. '''--output_testbeds''':: Include testbed assignments in the output fragment. '''--lax''':: Do not return an error if all required constraints cannot be met. '''--same_node''':: Allow loops to the same node in the output topology. (Default is off) '''--same_topology''':: Allow nodes in the same fragment to satisfy each others constraints. (Default is off) '''--same_pair''':: Allow multiple links between the same two nodes in different topologies to be created. (Default is off) '''--seed=''' ''number'':: Seed the random number generator. '''--certfile=''' ''file'':: Load a fedid from ''file'' for ns-2 to topdl translations. Ususally not needed. '''--config=''' ''config'':: Take the options from ''file'' in the format below. After the options, any parameter is taken to be a filename to include in the composition. === Configuration File === The composer has many options, and invocations that use only options can be long. To simplify matters, a simple key/value configuration file can be used. Each line in the configuration file corresponds to a command line parameter. Options that take a value are separated from that option by an equals sign (=), and the '''--multifile''' option can be abbreviated as '''file'''. The following command line {{{ compose.py --output_testbeds --url=http://users.isi.deterlab.net:23235 --multifile=usa_trio.tcl,3 --multifile=etrio.tcl,2 --multifile=bot.tcl,7 --multifile=atrio.tcl,1 --format=ns us_hex.tcl attack.tcl defend.tcl irc.tcl --output=topo.tcl }}} can be replaced with {{{ compose.py --config=./compose.config }}} if {{{compose.config}}} contains the following: {{{ url=http://users.isi.deterlab.net:23235 output_testbeds file=usa_trio.tcl,3 file=etrio.tcl,2 file=bot.tcl,7 file=atrio.tcl,1 us_hex.tcl attack.tcl defend.tcl irc.tcl output=topo.tcl }}} Note that input files included once can be directly named in the configuration file. If you would like to replicate this example, the files are attached to this page. == Instantiating the experiment == Once the experiment description has been composed, it can be created using [FeddCommands#fedd_create.py fedd_create.py]. Each view in the experiment must be assigned to a separate testbed. When operating on a single testbed, each sub-experiment is specified using the ''testbed''/''subname'' syntax. For example, an experiment representing an attacker and one representing a defender may be assigned to '''deter/attack''' and '''deter/defend''' testbeds respectively. Both sub-experiments will be instantiated separately on deter. Similarly, each sub-experiment can have its [FeddAbout#ExperimentServices experiment services] specified separately. Using '''project_export''' to export the filesystems and accounts of a given project to a sub experiment and '''hide_hosts''' to make any unnecessary hosts invisible outside the region, individual views can be created.