Changes between Version 16 and Version 17 of FeddUsersGuide


Ignore:
Timestamp:
Jun 9, 2014 2:51:25 PM (10 years ago)
Author:
faber
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • FeddUsersGuide

    v16 v17  
    318318
    319319The two formats are generally interchangeable using federation tools, though some external tools will prefer one or the other.
     320
     321= Using The Desktop Plugin In ExoGENI =
     322
     323The DETER lightweight federation system is a simple way to join your ExoGENI experiment with a DETER experiment.  The ExoGENI experiment can dynamically join an experiment in DETER when the DETER experiment is created using the federation system.  When the DETER experiment is terminated, the ExoGENI environment remains intact and can be reconnected to a similar experiment or to a different DETER experiment.  The technology in general is also called [FeddDesktop DETER desktop federation].
     324
     325The federation system accomplishes this by using a node that the experimenter adds to the ExoGENI topology that talks to DETER through its ExoGENI management interface.  Traffic to or from DETER will currently pass through the management interfacs as well, though we expect to expand to using other ExoGENI facilities in the near future.
     326
     327The federation node runs an image DETER provides and needs some simple configuration from the ExoGENI experimenter to coordinate with DETER.  In the simplest case, that configuration consists of:
     328
     329 * The IP address on which DETER can reach the management interface
     330 * The DETER(s) user who is allowed in create experiments that talk to this ExoGENI slice
     331
     332The rest of this document describes how to get and use the lightweight federation technology in ExoGENI.
     333
     334== Getting The Lightweight Federation Image and Getting DETER Federation Rights ==
     335
     336The lightweight federation image can currently be found at http://www.isi.edu/~faber/tmp/fedd.xml .  Its SHA1 hash is {{{05cf5d86906c11cdb35ece535d2539fe38481d17}}}.
     337
     338Adding these lines to your {{{.flukes.properties}}} file on your desktop will make the image accessible from flukes menus.  (Change the number in {{{image1}}} to be an unused number if you already have custom images defined.
     339
     340{{{
     341image1.name=Fedd-enabled Ubuntu 12.04
     342image1.url=http://www.isi.edu/~faber/tmp/fedd.xml
     343image1.hash=05cf5d86906c11cdb35ece535d2539fe38481d17
     344}}}
     345
     346In addition, users will need to get ther DETER accounts authorized to create federated experiments.  DETER administration controls this facility because it allows users to reach outside DETER, which we normally do not allow.  To get an account authorized for federation, contact [mailto:faber@isi.edu].
     347
     348== Connecting a Single ExoGENI node to a DETER Experiment ==
     349
     350This section is a tutorial description of setting up a federated experiment to an ExoGENI node.
     351
     352=== Configuring ExoGENI ===
     353
     354The simplest form of lightweight federation is to conect a single ExoGNEI node to a deter experiment.  We do this by creating a single-node ExoGENI slide that runs teh federation software and connecting a DETER experiment to it when that experiment is created.
     355
     356To create the single-node ExoGENI experiment, create one a single node slide using the federation image above.  When the slice exists, log in to the node and configure the federation software.  To do that the experimenter needs to know the two facts above.
     357
     358The image provides a script to set up a single node for federation.  The script and the federation configuration both are stored in the {{{/usr/local/etc/fedd}}} directory.  To configure the federation system, log into the node, change directory to {{{/usr/local/etc/fedd}}} and run the {{{init_fedd}}} script there.  It takes 2 parameters, the management IP address and the experimenter to authorize.  The following sequence will configure fedd to allow user "faber" to contact DETER using IP address 192.1.242.14:
     359
     360{{{
     361~# cd /usr/local/etc/fedd
     362/usr/local/etc/fedd# ./init_fedd 192.1.242.14 faber
     363}}}
     364
     365When that script completes, start the federation system on the ExoGENI node.  When debugging we recommend leaving a window open and running the daemon as:
     366
     367{{{
     368# fedd.py --config /usr/local/etc/fedd/desktop.conf --debug
     369}}}
     370
     371You can also run it in the background, logging to {{{/var/log/fedd.log}}} by:
     372
     373{{{
     374# touch /var/log/fedd.log
     375# fedd.py --config /usr/local/etc/fedd/desktop.conf --logfile /var/log/fedd.log &
     376}}}
     377
     378{{{fedd.py}}} expects the logfile to exist, hence the {{{touch}}} command.
     379
     380There are some detailed debugging messages that libraries and other dependent software produce that are visible with {{{--debug}}} that are lost during normal logging.
     381
     382=== Creating the experiment on DETER ===
     383
     384To connect the ExoGENI node to a DETER experiment, specify an federated experiment with the node that is to be on ExoGENI as being on testbed "desktop".  Here is an example DETER experiment description of that format:
     385
     386{{{
     387# simple DETER topology federated to a desktop
     388#
     389
     390set ns [new Simulator]
     391source tb_compat.tcl
     392
     393set a [$ns node]
     394set b [$ns node]
     395set c [$ns node]
     396set d [$ns node]
     397set e [$ns node]
     398set f [$ns node]
     399tb-set-node-testbed $a "deter"
     400tb-set-node-testbed $c "deter"
     401tb-set-node-testbed $d "deter"
     402tb-set-node-testbed $e "deter"
     403tb-set-node-testbed $f "deter"
     404
     405tb-set-node-testbed $b "desktop"
     406
     407set link0 [ $ns duplex-link $a $b 1Gb 0ms DropTail]
     408set link1 [ $ns duplex-link $a $c 1Gb 0ms DropTail]
     409set link2 [ $ns duplex-link $a $d 1Gb 0ms DropTail]
     410set link3 [ $ns duplex-link $e $c 1Gb 0ms DropTail]
     411set link4 [ $ns duplex-link $f $c 1Gb 0ms DropTail]
     412
     413$ns rtproto Static
     414$ns run
     415}}}
     416
     417You can [attachment:desk.tcl download a copy of that file].
     418
     419That file specified a topology that looks like this.  Computers are boxes and network connections are lines.  The larger blue outlines show which testbed each computer is in.
     420
     421[[Image(desk.png)]]
     422
     423To instantiate that topology, run the command:
     424
     425{{{
     426fedd_create.py --file desk.tcl --experiment_name $EXPNAME --map desktop:https://$MGMT_IP:23231
     427}}}
     428
     429where {{{$EXPNAME}}} is replaced with a short mnemonic name for the combined experiment and {{{$MGMT_IP}}} is the external management IP of the node running the federation software.
     430
     431When that returns, it will return something like:
     432
     433{{{
     434localname: faber-smart5
     435fedid: 2b7b6852a2db53d3e77431937e1da01d8fbf335d
     436status: starting
     437}}}
     438
     439DETER is coordinating between its local federation controllers and the one running on the ExoGENI node, allocating resources, and stitching them.  You can check the status using the {{{fedd_multistatus.py}}} command:
     440
     441{{{
     442users:~$ fedd_multistatus.py
     443faber-smart5:2b7b6852a2db53d3e77431937e1da01d8fbf335d:active
     444faber-smart1:9a57f78345e04136e8696a3439b476f3bc2f444d:failed
     445faber-smart2:7afc1dbbc29b6c7f46128d6dc0502136a95cb6cf:starting
     446}}}
     447
     448That output shows three federated experiments.  The experiments are:
     449
     450 * faber-smart5 which can be accessed and used for experimentation
     451 * faber-smart1 which failed and cannot be accessed
     452 * faber-smart2 which is in the process of being created.  It will eventually transition to active or failed.
     453
     454An experimenter can poll using {{{fedd_multistatus.py}}} or watch the debugging log using {{{fedd_spewlog.py}}}.  The second looks like (where $EXPNAME is faber-smart5):
     455
     456{{{
     457users:~$ fedd_spewlog.py --experiment_name $EXPNAME
     45803 Mar 14 16:46:50 fedd.experiment_control.faber-smart5 Calling StartSegment at https://192.1.242.14:23231
     45903 Mar 14 16:46:50 fedd.experiment_control.faber-smart5 Calling StartSegment at https://users.isi.deterlab.net:23231
     460Allocatation complete03 Mar 14 16:47:50 fedd.experiment_control.faber-smart5 Waiting for sub threads (it has been 1 mins)
     46103 Mar 14 16:48:50 fedd.experiment_control.faber-smart5 Waiting for sub threads (it has been 2 mins)
     46203 Mar 14 16:46:52 fedd.access.faber-smart5 State is none
     46303 Mar 14 16:46:52 fedd.access.faber-smart5 [make_null_experiment]: Creating experiment
     46403 Mar 14 16:47:14 fedd.access.faber-smart5 [make_null_experiment]: Create succeeded
     46503 Mar 14 16:47:14 fedd.access.faber-smart5 [start_segment]: creating script file
     46603 Mar 14 16:47:14 fedd.access.faber-smart5 [scp_file]: /usr/bin/scp -o IdentitiesOnly yes -o StrictHostKeyChecking no -o ForwardX11 no -i /usr/local/etc/fedd/deter/fedd_rsa /tmp/tmpFlZEWb faber@users.isi.deterlab.net:tmpFlZEWb
     46703 Mar 14 16:47:14 fedd.access.faber-smart5 [ssh_cmd]: /usr/bin/ssh -n -o 'IdentitiesOnly yes' -o 'StrictHostKeyChecking no' -o 'ForwardX11 no' -i /usr/local/etc/fedd/deter/fedd_rsa faber@users.isi.deterlab.net sh -x tmpFlZEWb
     46803 Mar 14 16:47:14 fedd.access.faber-smart5 [scp_file]: /usr/bin/scp -o IdentitiesOnly yes -o StrictHostKeyChecking no -o ForwardX11 no -i /usr/local/etc/fedd/deter/fedd_rsa /tmp/access-8Yev3U/fedgw_rsa.pub faber@users.isi.deterlab.net:/proj/DeterTest/exp/faber-smart5/tmp/fedgw_rsa.pub
     46903 Mar 14 16:47:15 fedd.access.faber-smart5 [scp_file]: /usr/bin/scp -o IdentitiesOnly yes -o StrictHostKeyChecking no -o ForwardX11 no -i /usr/local/etc/fedd/deter/fedd_rsa /tmp/access-8Yev3U/fedgw_rsa faber@users.isi.deterlab.net:/proj/DeterTest/exp/faber-smart5/tmp/fedgw_rsa
     47003 Mar 14 16:47:15 fedd.access.faber-smart5 [scp_file]: /usr/bin/scp -o IdentitiesOnly yes -o StrictHostKeyChecking no -o ForwardX11 no -i /usr/local/etc/fedd/deter/fedd_rsa /tmp/access-8Yev3U/hosts faber@users.isi.deterlab.net:/proj/DeterTest/exp/faber-smart5/tmp/hosts
     47103 Mar 14 16:47:15 fedd.access.faber-smart5 [scp_file]: /usr/bin/scp -o IdentitiesOnly yes -o StrictHostKeyChecking no -o ForwardX11 no -i /usr/local/etc/fedd/deter/fedd_rsa /tmp/access-8Yev3U/ca.pem faber@users.isi.deterlab.net:/proj/DeterTest/exp/faber-smart5/tmp/ca.pem
     47203 Mar 14 16:47:16 fedd.access.faber-smart5 [scp_file]: /usr/bin/scp -o IdentitiesOnly yes -o StrictHostKeyChecking no -o ForwardX11 no -i /usr/local/etc/fedd/deter/fedd_rsa /tmp/access-8Yev3U/node.pem faber@users.isi.deterlab.net:/proj/DeterTest/exp/faber-smart5/tmp/node.pem
     47303 Mar 14 16:47:16 fedd.access.faber-smart5 [scp_file]: /usr/bin/scp -o IdentitiesOnly yes -o StrictHostKeyChecking no -o ForwardX11 no -i /usr/local/etc/fedd/deter/fedd_rsa /tmp/access-8Yev3U/deterdesktoptunnel0.gw.conf faber@users.isi.deterlab.net:/proj/DeterTest/exp/faber-smart5/tmp/deterdesktoptunnel0.gw.conf
     47403 Mar 14 16:47:16 fedd.access.faber-smart5 [scp_file]: /usr/bin/scp -o IdentitiesOnly yes -o StrictHostKeyChecking no -o ForwardX11 no -i /usr/local/etc/fedd/deter/fedd_rsa /tmp/access-8Yev3U/client.conf faber@users.isi.deterlab.net:/proj/DeterTest/exp/faber-smart5/tmp/client.conf
     47503 Mar 14 16:47:17 fedd.access.faber-smart5 [scp_file]: /usr/bin/scp -o IdentitiesOnly yes -o StrictHostKeyChecking no -o ForwardX11 no -i /usr/local/etc/fedd/deter/fedd_rsa /tmp/access-8Yev3U/experiment.tcl faber@users.isi.deterlab.net:/proj/DeterTest/exp/faber-smart5/tmp/experiment.tcl
     47603 Mar 14 16:47:17 fedd.access.faber-smart5 [scp_file]: /usr/bin/scp -o IdentitiesOnly yes -o StrictHostKeyChecking no -o ForwardX11 no -i /usr/local/etc/fedd/deter/fedd_rsa /tmp/access-8Yev3U/software/fedkit.tgz faber@users.isi.deterlab.net:/proj/DeterTest/software//faber-smart5/fedkit.tgz
     47703 Mar 14 16:47:18 fedd.access.faber-smart5 [scp_file]: /usr/bin/scp -o IdentitiesOnly yes -o StrictHostKeyChecking no -o ForwardX11 no -i /usr/local/etc/fedd/deter/fedd_rsa /tmp/access-8Yev3U/software/seer-fbsd71-current.tgz faber@users.isi.deterlab.net:/proj/DeterTest/software//faber-smart5/seer-fbsd71-current.tgz
     47803 Mar 14 16:47:18 fedd.access.faber-smart5 [modify_exp]: Modifying faber-smart5
     47903 Mar 14 16:47:35 fedd.access.faber-smart5 [modify_exp]: Modify succeeded
     48003 Mar 14 16:47:35 fedd.access.faber-smart5 [swap_exp]: Swapping faber-smart5 in
     48103 Mar 14 16:49:46 fedd.access.faber-smart5 [swap_exp]: Swap succeeded
     48203 Mar 14 16:49:46 fedd.access.faber-smart5 [get_mapping] Generating mapping
     48303 Mar 14 16:49:47 fedd.access.faber-smart5 Node mapping complete
     48403 Mar 14 16:49:47 fedd.access.faber-smart5 Link mapping complete03 Mar 14 16:49:48 fedd.experiment_control.faber-smart5 [start_segment]: Experiment faber-smart5 active
     485active
     486}}}
     487
     488The {{{--experiment_name}}} parameter is the same as on the {{{fedd_create.py}}} command, or selected from {{{fedd_multistatus.py}}} output.
     489
     490The {{{fedd_spewlog.py}}} command prints changes to the log as they appear and terminates when the experiment transitions to either active or failed state.
     491
     492=== Interacting with the Experiment ===
     493
     494Once an experiment is active, it completes stitching itself together and a user can log in to the various nodes using the native testbed mechanisms.  That stitching may take a minute or two after the federation system declares the experiment active.  Additionally it may take the dynamic routing some time to converge, depending on the complexity of the topology.
     495
     496Inside the DETER experiment, one can log into the nodes and interact with them by node name as usual.  Details are [https://trac.deterlab.net/wiki/Tutorial/UsingNodes here].  With node b on ExoGENI and a on DETER, as shown above, this sequence shows the transparent connection.  A user logs into node a in the local experiment (experiment faber-smart5 in in project detertest) and pings node b from node a.
     497
     498{{{
     499users.isi.deterlab.net:~$ ssh a.faber-test.detertest
     500a:~$ ping b
     501PING b-link0 (10.0.3.2) 56(84) bytes of data.
     50264 bytes from b-link0 (10.0.3.2): icmp_req=1 ttl=64 time=152 ms
     50364 bytes from b-link0 (10.0.3.2): icmp_req=2 ttl=64 time=76.3 ms
     504}}}
     505
     506While for all intents and purposes, b (in ExoGENI) is part of the experiment, the long ping times make it easy to identify:
     507
     508{{{
     509a:~$ ping e
     510PING e-link3 (10.0.1.1) 56(84) bytes of data.
     51164 bytes from e-link3 (10.0.1.1): icmp_req=1 ttl=63 time=0.797 ms
     51264 bytes from e-link3 (10.0.1.1): icmp_req=2 ttl=63 time=0.437 ms
     513}}}
     514
     515Similarly, one can log into the ExoGENI node and see the DETER nodes by the same names:
     516
     517{{{
     518root@server-18393:~# ping a
     519PING a-link2 (10.0.0.2) 56(84) bytes of data.
     52064 bytes from a-link2 (10.0.0.2): icmp_req=1 ttl=64 time=76.7 ms
     52164 bytes from a-link2 (10.0.0.2): icmp_req=2 ttl=64 time=76.5 ms
     522}}}
     523
     524It may be surprising, but the ExoGENI node can route to nodes throughout our multi-hop DETER topology:
     525
     526{{{
     527root@server-18393:~# ping c
     528PING c-link3 (10.0.1.2) 56(84) bytes of data.
     52964 bytes from c-link3 (10.0.1.2): icmp_req=1 ttl=63 time=153 ms
     53064 bytes from c-link3 (10.0.1.2): icmp_req=2 ttl=63 time=76.7 ms
     531}}}
     532
     533The federated experiment in DETER runs [http://en.wikipedia.org/wiki/Ospf ospf] on each node, and fedd.py starts an ospf daemon on the ExoGENI node as well.  The ExoGENI node simply discovers the routing table from that connection.  We will show how to expolit this connection to interconnect more interesting topologoes.
     534
     535=== Tearing The Experiment Down ===
     536
     537To tear down the experiment, use the command:
     538
     539{{{
     540users:~$ fedd_terminate.py --experiment_name $EXPNAME
     541}}}
     542
     543This releases the DETER resources and disconnects the ExoGENI node.  The ExoGENI slice remains intact and can be connected to another DETER experiment.
     544
     545If an experimenter tears down the ExoGENI slice before the {{{fedd_terminate.py}}} command is issued, or there is some other problem, the {{{--force}}} flag can be given to make {{{fedd_terminate.py}}} purge all state that the federation system can reach.
     546
     547== A More Complex ExoGENI Topology ==
     548
     549To connect a more complex ExoGENI topology, we look at routing from a two-node ExoGENI world into the same DETER topology.  The ExoGENI topology looks like this:
     550
     551[[Image(ExoGENI.png)]]
     552
     553Node0 is running the fedd and will be accessible as before at hostname "b".  In addition we will make Node1 accessible throughout the DETER topology by its IP address.
     554
     555When we create the ExoGENI topology we use the AutoIP feature of [https://geni-orca.renci.org/trac/wiki/flukes Flukes] to assign addresses.  By default Flukes puts the addresses on the 172.16.0.0/30 subnet.  We move those addresses to the 10.16.0.0/24 subnet by editing the nodes' properties.  DETER uses the 10.0.0.0/8 network block for experiment interfaces.  It uses the 172.16.0.0/30 net for addressing [http://containers.deterlab.net containers] management interfaces.
     556
     557When the slice is ready, log in to Node0 and run the {{{init_fedd}}} utility as before.  In addition, add the following lines to {{{/usr/local/etc/fedd/desktop.config}}}:
     558
     559{{{
     560# Export Interfaces (interfaces to run OSPF on/export to DETER).
     561# Comma-separated list of interface names
     562export_interfaces: eth1
     563
     564# Export Networks (networks to export to OSPF - these usually correspond
     565# to export_interfaces).  Comma separated
     566export_networks: 10.16.0.0/24
     567}}}
     568
     569With those settings, {{{fedd.py}}} will export any routes discovered on {{{eth1}}} and network 10.16.0.0/24 to the ospfd running in DETER.
     570
     571The simplest way to export a complex ExoGENI topology is to just run ospfd inside the ExoGENI topology.  With these settings, the combined experiment will be able to route throughout the experiment as soon as routing converges.  For this simple topology to work, we log in to Noed1 in exoGENI and add a route to all the DETER nodes through Node0:
     572
     573{{{
     574# ip route add 10.0.0.0/16 via 10.16.0.1
     575}}}
     576
     577Start up fedd.py on Node0 as before and run the same {{{fedd_create.py}}} command on DETER.
     578
     579In addition to being able to contact Node0 directly, nodes in the DETER experiment now see a route to the 10.16.0.0/24 network:
     580
     581{{{
     582a:~$ ip route
     583default via 192.168.1.254 dev eth4
     58410.0.0.0/24 dev eth2  proto kernel  scope link  src 10.0.0.2
     58510.0.1.0/24 via 10.0.4.2 dev eth1  proto zebra  metric 20
     58610.0.2.0/24 via 10.0.4.2 dev eth1  proto zebra  metric 20
     58710.0.3.0/24 dev eth0  proto kernel  scope link  src 10.0.3.1
     58810.0.4.0/24 dev eth1  proto kernel  scope link  src 10.0.4.1
     58910.16.0.0/24 via 10.0.3.2 dev eth0  proto zebra  metric 20
     590192.168.0.0/22 dev eth4  proto kernel  scope link  src 192.168.1.188
     591192.168.252.0/22 via 192.168.1.254 dev eth4  proto zebra
     592}}}
     593
     594That route reaches Node1:
     595
     596{{{
     597a:~$ ping 10.16.0.2
     598PING 10.16.0.2 (10.16.0.2) 56(84) bytes of data.
     59964 bytes from 10.16.0.2: icmp_req=1 ttl=63 time=139 ms
     60064 bytes from 10.16.0.2: icmp_req=2 ttl=63 time=69.9 ms
     60164 bytes from 10.16.0.2: icmp_req=3 ttl=63 time=69.4 ms
     602}}}
     603
     604While some attention needs to be paid to routing if routing protocols are not run in ExoGENI, this facility allows a wide range of connectivity between the testbeds.