Federation in Support of Multi-Party Experiments

Multi-party experiments are a research activity undertaken by multiple experimenters or groups of experimenters that requires differing views of the environment. For example, a multi-party security experiment may pit an attack designer against a defense designer, each with limited visibility and control of the system. Such environments may also support collaboration.

Conceptually, multiple users come to DETER with experiment fragments and constraints on how those fragments are combined and DETER creates the composite environment that meets those constraints.

DETER's support for this is evolving. Our initial implementation uses the simple composition system to combine partial topologies automatically, subject to constraints, and the federation to construct the composite topologies in DETER as federated experiments with individual service configurations.

This page documents the proof of concept implementation. More work is ongoing in making this system more broadly usable and expressive.

Combining partial topologies

One aspect of combining multiple environments is building a single overall topology, even if no individual player will be able to see all of it. We use a simple label-matching constraint resolver to combine topology fragments. Each node in a fragment may be annotated with constraints that describe how it can be attached to nodes in other fragments. Each node has one or more attachment points, each of which as one or more labels attached to it, called the attachment point's type. Additionally each attachment point may only be attached to points that have labels specified in its "Link to" list.

The labels on each attachment point are just labels that the designer chooses. They shape the composition based on the matching constraints on other attachnemt points.

In the example below, the two left most nodes have a single attachment point of type "Single". They can only be attached to points of type "Node". The next node has connections of type "Node" that can connect to either "Node" or "Single" connections. They combine to form a 3-node fragment with one attachment point type "Node" that can connect to either "Node" or "Single" connections.

These fragments can either be described in the extended ns-2 language or in simple xml containing a ​topdl description. The three-connection node is encoded in ns-2 as:

# simple DETER topology
# COMPOSITION: bot0:optional:Node:Single,Node
# COMPOSITION: bot0:optional:Node:Single,Node
# COMPOSITION: bot0:optional:Node:Single,Node

set ns [new Simulator]
source tb_compat.tcl


set bot0 [$ns node]
tb-set-node-testbed $bot0 "deter"

$ns rtproto Static
$ns run 

The comments containing the COMPOSITION keyword contain the constraints. The first colon-separated field gives the node name to which the constraint applies, the second whether the field is optional or required, the third the type of connector, and the last a comma-separated list of acceptable types to connect. Optional constraints need not be satisfied, though our satisfier tries to satsify as many as possible.

This fragment has one attachment point for each commented attachment point description.

The same component in XML is:

<component>
 <constraints>
  <constraint>
   <name>bot0</name>
   <required>False</required>
   <provides>Node</provides>
   <accepts>Single</accepts>
   <accepts>Node</accepts>
  </constraint>
  <constraint>
   <name>bot0</name>
   <required>False</required>
   <provides>Node</provides>
   <accepts>Single</accepts>
   <accepts>Node</accepts>
  </constraint>
  <constraint>
   <name>bot0</name>
   <required>False</required>
   <provides>Node</provides>
   <accepts>Single</accepts>
   <accepts>Node</accepts>
  </constraint>
 </constraints>
 <experiment>
  <version>1.0</version>
  <elements>
   <computer>
    <name>bot0</name>
    <attribute>
     <attribute>type</attribute>
     <value>pc</value>
    </attribute>
    <attribute>
     <attribute>testbed</attribute>
     <value>deter</value>
    </attribute>
    <attribute>
     <attribute>failureaction</attribute>
     <value>fatal</value>
    </attribute>
   </computer>
  </elements>
 </experiment>
</component>

In ns-2 the constraints are encoded as comments as an expedient. DETER and fedd are moving away from the programmatic topology expression, and the relative difficulties of encoding such constranints in ns-2/tcl vs. XML is an example of why. The overhead of adding new ns-2/tcl is much higher than parsing a few new XML elements.

Composing Fragments

The simple composer is included in the fedd git repository as fedd/compose.py​. It takes one or more fragment files as described above and produces an new fragment file in either xml or ns-2, suitable for passing to fedd_create.py or back to the composer. The compose command takes the following arguments:

--help

Display help text

--url

The url of a running fedd that is willing to translate ns-2 to topdl/xml. This requires a fedd with access to an otcl interpreter with ns-2 extensions that will output topdl. Most users will not want to configure that fedd, but https://users.isi.deterlab.net:23235 will work and is publically accessible. Not needed if all inputs are topdl/XML.

--multifile= file,number

Include the given file in the composition number times.

--output= file

Save the composed fragment in file. The suffix of the file determines the output format, tcl or ns are saved as ns-2, and topdl or xml are topdl/XML. If this option is not given, the fragment goes to standard out as XML, though the ---format option can set the output explicitly. --format= format: Override the format of the output. Choices are: tcl or ns to output ns-2, and topdl or xml to output topdl/XML.

--add_testbeds

Add a testbed attribute to each component. Each view of the multi-party experiment is instantiated in a separate testbed. If this option is given, the same testbed attribute is assigned to each node in each fragment. All the nodes in the first fragment are assigned to testbed000 all in the second to testbed001, etc.

--output_testbeds

Include testbed assignments in the output fragment.

--lax

Do not return an error if all required constraints cannot be met.

--same_node

Allow loops to the same node in the output topology. (Default is off)

--same_topology

Allow nodes in the same fragment to satisfy each others constraints. (Default is off)

--same_pair

Allow multiple links between the same two nodes in different topologies to be created. (Default is off)

--seed= number

Seed the random number generator.

--certfile= file

Load a fedid from file for ns-2 to topdl translations. Ususally not needed.

--config= config

Take the options from file in the format below.

After the options, any parameter is taken to be a filename to include in the composition.

Configuration File

The composer has many options, and invocations that use only options can be long. To simplify matters, a simple key/value configuration file can be used. Each line in the configuration file corresponds to a command line parameter. Options that take a value are separated from that option by an equals sign (=), and the --multifile option can be abbreviated as file. The following command line

compose.py --output_testbeds --url=http://users.isi.deterlab.net:23235 --multifile=usa_trio.tcl,3 --multifile=etrio.tcl,2 --multifile=bot.tcl,7 --multifile=atrio.tcl,1 --format=ns us_hex.tcl attack.tcl defend.tcl irc.tcl --output=topo.tcl

can be replaced with

compose.py --config=./compose.config

if compose.config contains the following:

url=http://users.isi.deterlab.net:23235
output_testbeds
file=usa_trio.tcl,3
file=etrio.tcl,2
file=bot.tcl,7
file=atrio.tcl,1
us_hex.tcl
attack.tcl
defend.tcl
irc.tcl
output=topo.tcl

Note that input files included once can be directly named in the configuration file. If you would like to replicate this example, the files are attached to this page.

Instantiating the experiment

Once the experiment description has been composed, it can be created using fedd_create.py. Each view in the experiment must be assigned to a separate testbed. When operating on a single testbed, each sub-experiment is specified using the testbed/subname syntax. For example, an experiment representing an attacker and one representing a defender may be assigned to deter/attack and deter/defend testbeds respectively. Both sub-experiments will be instantiated separately on deter.

Similarly, each sub-experiment can have its experiment services specified separately. Using project_export to export the filesystems and accounts of a given project to a sub experiment and hide_hosts to make any unnecessary hosts invisible outside the region, individual views can be created.