Changes between Initial Version and Version 1 of FeddAbout

Mar 14, 2014 10:35:42 AM (6 years ago)

Initial import


  • FeddAbout

    v1 v1  
     1= The DETER Federation Architecture (DFA) =
     3This is a basic overview of the DETER federation system.  More details
     4on this implementation and configuration are in later documents.  You
     5can also [wiki:FeddPubs read these academic papers] that capture the evolution of the
     6design to this point.
     8DFA is a system that allows a researcher to construct experiments that
     9span testbeds by dynamically acquiring resources from other testbeds and
     10configuring them into a single experiment.  As closely as possible that
     11experiment will mimic a single DETER/Emulab experiment.
     13Though the experiment appears to be a cohesive whole, the testbeds that
     14loan the resources retain control of those resources.  Because testbeds
     15retain this control, each testbed may issue credential necessary for
     16manipulating the federated resources.  For example, a testbed that has
     17loaned nodes to an experiment may require the experimenter to present a
     18credential issued by that testbed (e.g., an SSH key or SSL certificate)
     19to reboot those nodes.  The system acquires those credentials on behalf
     20of experimenters and distributes them on behalf of testbeds.
     22Testbed administrators may use the system to establish regular policies
     23between testbeds to share resources across many users of a testbed.
     24Similarly, a single user with accounts on multiple testbeds can use the
     25same interfaces to coordinate experiments that share his testbed
     26resource, assuming sharing those resources does not violate the policy
     27of any of the constituent testbeds.
     29= Overview =
     33Fedd plays the role of the Federator in the diagram above.  It is divided into two parts:
     35 '''Experiment controller'''::
     36  Presents the abstraction of a federated experiment to the users.  Allows allocation, configuration, and release of resources to the federated experiment.
     37 '''Access Controller'''::
     38  Negotaties access and allocation of resources from federant testbeds.
     40The access controller is depicted by the regular plug-in shapes in the federator.  These are standard plug-in interfaces that allow standard access to a variety of testbed types.  Currently emulab style testbeds and DRAGON style transit networks are supported.  The regular system interface to fedd allows new systems to be added easily.
     42Depending on the nature of the access controller it may be implemented on the local testbed, on the same host as the experiment controller (even in the same process) or partially in both places.  This depends on the access mechanisms of the underlying testbed and the policies of the administrators.  We have implemented plugins for Emulab-style testbeds that run completely remotely and access the testbed through ssh and plug-ins that run on the Emulab testbed directly and communicate outside the testbed entirely through fedd interfaces.  Access controllers that run directly on the remote testbed have more discretion in executing privledged local commands than remote access controllers.
     44The [FeddPluginArchitecture access controller/experiment controller interface] is documented.
     46= Fedd: The DETER Federation Daemon =
     48Fedd is the name for the codebase that supports both the access controller and experiment controller function.  Generally any instance of fedd will be acting as either an experiment controller or an access controller.  We continue to move toward reflecting the distinction in function in the codebase, but this is not complete.
     50Generally fedd is configured by the testbed's administration to allow
     51controlled sharing of the resources, though the same software and
     52interfaces can be used by single users to coordinate federated
     55Creating a federated experiment consists of breaking an annotated
     56experiment into sub-experiments, gaining access to testbeds that can
     57host those experiments, and coordinating the creation and connection of
     58those sub-experiments.  Fedd insulates the user from the various
     59complexities of creating sub-experiments and dealing with cleanup and
     60partial failures.  Once an experiment is created, fedd returns
     61credentials necessary to administer the experiment.
     63On termination, fedd cleans up the various sub-experiments and
     64deallocates any dynamic resources or credentials.
     66= Access Control =
     68This section describes the basic model of access control that fedd
     69implements.  We start by explaining how users are represented between
     70fedds and then describe the process of acquiring and releasing access to
     71a testbed's resources.
     73This section documents the current access control system of fedd.  It is
     74undergoing development.
     76== Global Identifiers: Fedids ==
     78In order to avoid collisions between local user names the DFA defines a
     79global set of names called DETER federation IDs, or more concisely
     80fedids.  A fedid has two properties:
     82 * An entity claiming to be named by a fedid can prove it interactively
     83 * Two entities referring to the same fedid can be sure that they are referring to the same entity
     85An example implementation of a fedid is an RSA public key.  An entity
     86claiming to be identified by a given public key can prove it by
     87responding correctly to a challenge encrypted by that key.  For a large
     88enough key size, collisions are rare enough that the second property
     89holds probabilistically.
     91We adopt public keys as the basis for DFA fedids, but rather than tie
     92ourselves to one public key format, we use subject key identifiers as
     93derived using method (1) in RFC3280.  That is, we use the 160 bit SHA-1
     94hash of the public key.  While this allows a slightly higher chance of
     95collisions than using the key itself, the advantage of a single
     96representation in databases and configuration files outweighs the
     97increase in risk for this application.
     99== Global Identifiers: Three-level Names ==
     101In Emulab, projects are created by users within projects and those
     102attributes determine what resources can be accessed.  We generalized
     103this idea into a testbed, project, user triple that is used for access
     104control decisions.  A requester identified as ("DETER", "proj1",
     105"faber") is a user from the DETER testbed, proj1 project, user faber.
     106Testbeds contain projects and users, projects contain users, and users
     107do not contain anything.
     109Parts of the triple can be left blank: ( , ,"faber") is a user named faber
     110without a testbed or project affiliation.
     112Though the examples used above are simple strings, the outermost (i.e., leftmost) non-blank field
     113of a name must be a fedid to be valid, and are treated as the entity
     114asserting the name.  Contained fields can either be fedids or local
     115names.  This allows a testbed to continue to manage its namespace
     118If DETER has fedid:1234 then the name (fedid:1234, "proj1", "faber")
     119refers to the DETER user faber in proj1, if and only if the requester
     120can prove they are identified by fedid:1234.  Note that by making
     121decisions on the local names a testbed receiving a request is trusting
     122the requesting testbed about the membership of those projects and names
     123of users.
     125Testbeds make decisions about access based on these three level names.
     126For example, any user in the "emulab-ops" project of a trusted testbed may
     127be granted access to federated resources.  It may also be the case that
     128any user from a trusted testbed is granted some access, but that users
     129from the emulab-ops project of that testbed are granted access to more
     130kinds of resources.
     132These three level names are used only for testbed access control.  All
     133other entities in the system are identified by a single fedid.
     135== Granting Access ==
     137An access controller is responsible for mapping from the global identifier space into the local access control of the testbed
     138This may mean mapping a request into a local Emulab project and user, a local DRAGON certificate, or a local [ ProtoGENI] certificate.  As other plugins are added other mappings will appear
     140=== Emulab Projects ===
     142Once an access controller managing an Emulab has decided to grant a researcher access to resources, it
     143implements that decision by granting the researcher access to an Emulab
     144project with relevant permissions on the local testbed.  The terminology
     145is somewhat unfortunate in that the access controller is configured to grant access
     146based on the global three-level name that includes project and user
     147components and implements that decision by granting access to a local
     148Emulab project and Emulab user.
     150The Emulab project to which the fedd grants access may exist and contain
     151static users and resource rights, may exist but be dynamically
     152configured by fedd with additional resource rights and access keys, or
     153may be created completely by fedd.  Completely static projects are
     154primarily used when a user wants to tie together his or her accounts on
     155multiple testbeds that do not bar that behavior, but do not run fedd.
     157Whether to dynamically modify or dynamically create files depends
     158significantly on testbed administration policy and how widespread and
     159often federation is conducted.  In Emulabs projects are intended as
     160long-term entities, and creating and destroying them on a per-experiment
     161basis may not appeal to some users.  However, static projects require
     162some administrator investment per-project.
     164=== Certificate Systems: DRAGON and ProtoGENI ===
     166Access controllers on these systems map the request into a flat space of X.509 certificates and keys that provide the identity for these identity based systems.  Unlike the self-signed certificates representing [FeddAbout#GlobalIdentifiers:Fedids fedids], these are full X.509 certificates used to establish a chain of trust.  Different global users are mapped into certificates and keyes that match the access level intended.
     168= Experiments =
     170A federated experiment combines resources from multiple independently administered testbeds into a cohesive environment for experimentation.  Earlier versions of fedd enforced a !Master/Slave relationship with one emulab project exporting its environment to others.  While this model remains viable, current fedd implementations allow compositions of the services and views of the facilities as well as their resources.
     172Currently experiments can be described in the same ns2 syntax that DETER
     173and Emulab use, except that additional annotations for the testbed on
     174which to place each node have been added.  (The extension is
     175set-node-testbed, and described elsewhere.)  Most users will specify experiments in this format, initially. The testbed name here is a
     176local string meaningful to the experimenter.  Each fedd knows some set
     177of such strings, and experimenters can also inform fedd of the
     178appropriate mapping between local testbed name and fedd address to
     181Fedd also understands experiments expressed in the [FeddPluginArchitecture#TopologyDescriptionLanguage topdl] language that it uses to communicate with facilities.
     183An experimenter creates an experiment by presenting it to a fedd that
     184knows him or her - usually the fedd on their local testbed - along with
     185the master testbed name, local Emulab project to export and local
     186mappings.  That fedd will map the local user into the global (testbed,
     187project, user) namespace and acquire and configure resources to create
     188the experiment.  A fedd may try different mappings of the experimenter
     189into the global namespace in order to acquire all the resources.  For
     190example a local experimenter may be a member of several local Emulab
     191projects that map into different global testbed-scoped projects.
     193Once created the user can gather information on the resources allocated to the experiment and where to access them.  (Experiments are identified by
     194fedids).  The local name is mnemonic and can be used locally to access
     195information about the experiment and to terminate it.  The experimenter
     196can suggest a name when requesting experiment creation.
     198In order to create a fedid, a key pair is created.  The public and
     199private keys for the fedid representing the experiment are returned in
     200an encrypted channel from fedd to the requesting experimenter.  The
     201experimenter can use this key pair as a capability to grant access to the
     202experiment to other users.
     204While an experiment exists, experimenters can access the nodes directly
     205using the credentials passed back and can acquire information about the
     206experiment's topology from fedd.  Only the experimenter who created it
     207or one given the capability to manipulate it can get this information.
     209Finally, fedd will terminate an experiment on behalf of the experimenter
     210who created it, or one who has been granted control of it.  Fedd is
     211responsible for deallocating the running experiments and telling the
     212testbeds that this experimenter is done with their access.
     214The `fedd` installation includes a [ command line program] that allows users to create and delete federated experiments.  We are currently developing other interfaces, including a [ SEER ] interface.
     216=== Experiment Descriptions ===
     218An experiment description is currently given in a slightly extended version of the [ ns2 dialect that is used for an Emulab experiment description.] Specifically, the `tb-set-node-testbed` command has been added.  Its syntax is:
     221tb-set-node-testbed node_reference testbed_name
     224The `node_reference` is the tcl variable containing the node and the `testbed_string` is a name meaningful to the `fedd` creating the experiment.  The experiment creation interface has a mechanism that a requester can use to supply translations from testbed names to the URIs needed to contact that testbed's `fedd` for access.
     226Multiple segments of a federerated experiment can be embedded on the same testbed specifying the testbed_name as ''name''/''subname''.  In that syntax, ''name'' is the identifier of the testbed on which to embed the node and ''subname'' is a unique identifier within the federated experiment.  All the nodes assigned to '''deter/attack''' will be in the same sub-experiment and that will be a different sub-experiment from nodes in '''deter/defend'''.
     228In addition, `fedd` understands the `tb-set-default-failure-action` command.  This sets the default failure action for nodes in the experiment to one of '''fatal''', '''nonfatal''', or '''ignore'''.  Those values have the same meaning as in standard Emulabs. Nodes that do not have their failure mode reset by the `tb-set-node-failure-action` command will user the explicit default.  If no default is set, '''fatal''' is used.
     230=== Federated Experiment Creation ===
     232TO give an intuition for how `fedd` works, we present a sample experiment description and show how it would be turned into a federated experiment.  The experiment description is:
     235# simple DETER topology for playing with SEER
     237set ns [new Simulator]
     238source tb_compat.tcl
     240set a [$ns node]
     241set b [$ns node]
     242set c [$ns node]
     243set d [$ns node]
     244set e [$ns node]
     246set deter_os "FC6-STD"
     247set deter_hw "pc"
     248set ucb_os "FC6-SMB"
     249set ucb_hw "bvx2200"
     251tb-set-node-os $a $deter_os
     252tb-set-node-testbed $a "deter"
     253tb-set-hardware $a $deter_hw
     256tb-set-node-os $b $deter_os
     257tb-set-node-testbed $b "deter"
     258tb-set-hardware $b $deter_hw
     260tb-set-node-os $c $ucb_os
     261tb-set-node-testbed $c "ucb"
     262tb-set-hardware $c $ucb_hw
     264tb-set-node-os $d $ucb_os
     265tb-set-node-testbed $d "ucb"
     266tb-set-hardware $d $ucb_hw
     268tb-set-node-os $e $ucb_os
     269tb-set-node-testbed $e "ucb"
     270tb-set-hardware $e $ucb_hw
     273set link0 [ $ns duplex-link $a $b 100Mb 0ms DropTail]
     274set link1 [ $ns duplex-link $c $b 100Mb 0ms DropTail]
     275set lan0 [ $ns make-lan "$c $d $e" 100Mb 0ms ]
     277$ns rtproto Static
     278$ns run
     281That experiment file generates the following topology:
     285Each of the boxes is a node and the circle is a shared network.  Considering the values of the `set-node-testbed` calls, the experiment will be split up like this:
     289The assignment of nodes to testbeds implicitly creates links (or networks) that must be connected across the wide area, and one of the key jobs of `fedd` is to create these connections.  Fedd splits the experiment description into two descriptions (one for each testbed), adds extra nodes and startcmds to create the shared experiment, and swaps them in.  The sub-experiments look something like these:
     292set ns [new Simulator]
     293source tb_compat.tcl
     295set a [$ns node]
     296tb-set-hardware $a pc
     297tb-set-node-os $a FC6-STD
     298tb-set-node-tarfiles $a /usr /proj/emulab-ops//tarfiles/bwfed/fedkit.tgz
     299# tb-set-node-testbed $a "deter"
     300tb-set-node-startcmd $a "sudo -H /usr/local/federation/bin/make_hosts /proj/emulab-ops/exp/bwfed/tmp//hosts >& /tmp/federate \$USER"
     301tb-set-node-failure-action $a "fatal"
     302set b [$ns node]
     303tb-set-hardware $b pc
     304tb-set-node-os $b FC6-STD
     305tb-set-node-tarfiles $b /usr /proj/emulab-ops//tarfiles/bwfed/fedkit.tgz
     306# tb-set-node-testbed $b "deter"
     307tb-set-node-startcmd $b "sudo -H /usr/local/federation/bin/make_hosts /proj/emulab-ops/exp/bwfed/tmp//hosts >& /tmp/federate \$USER"
     308set control [$ns node]
     309tb-set-hardware $control pc
     310tb-set-node-os $control FC6-STD
     311tb-set-node-tarfiles $control /usr /proj/emulab-ops//tarfiles/bwfed/fedkit.tgz
     312# tb-set-node-testbed $control "deter"
     313tb-set-node-startcmd $control "sudo -H /usr/local/federation/bin/make_hosts /proj/emulab-ops/exp/bwfed/tmp//hosts >& /tmp/federate \$USER "
     314tb-set-node-failure-action $control "fatal"
     315# Link establishment and parameter setting
     316set link0 [$ns duplex-link $a $b 100000kb 0.0ms DropTail]
     318tb-set-ip-link $a $link0
     319tb-set-ip-link $b $link0
     320# federation gateway
     321set ucbtunnel0 [$ns node ]
     322tb-set-hardware $ucbtunnel0 pc3000_tunnel
     323tb-set-node-os $ucbtunnel0 FBSD7-TVF
     324tb-set-node-startcmd $ucbtunnel0 "sudo -H /usr/local/federation/bin/ -f /proj/emulab-ops/exp/bwfed/tmp/`hostname`.gw.conf >& /tmp/bridge.log"
     325tb-set-node-tarfiles $ucbtunnel0 /usr/ /proj/emulab-ops//tarfiles/bwfed/fedkit.tgz
     326# Link establishment and parameter setting
     327set link1 [$ns duplex-link $ucbtunnel0 $b 100000kb 0.0ms DropTail]
     329tb-set-ip-link $ucbtunnel0 $link1
     330tb-set-ip-link $b $link1
     332$ns rtproto Session
     333$ns run
     337set ns [new Simulator]
     338source tb_compat.tcl
     340set c [$ns node]
     341tb-set-hardware $c pc
     342tb-set-node-os $c FC6-SMB
     343tb-set-node-tarfiles $c /usr /proj/Deter//tarfiles/bwfed/fedkit.tgz
     344# tb-set-node-testbed $c "ucb"
     345tb-set-node-startcmd $c "sudo -H /bin/sh /usr/local/federation/bin/ >& /tmp/startup \$USER "
     346tb-set-node-failure-action $c "fatal"
     347set d [$ns node]
     348tb-set-hardware $d pc
     349tb-set-node-os $d FC6-SMB
     350tb-set-node-tarfiles $d /usr /proj/Deter//tarfiles/bwfed/fedkit.tgz
     351# tb-set-node-testbed $d "ucb"
     352tb-set-node-startcmd $d "sudo -H /bin/sh /usr/local/federation/bin/ >& /tmp/startup \$USER "
     353tb-set-node-failure-action $d "fatal"
     354set e [$ns node]
     355tb-set-hardware $e pc
     356tb-set-node-os $e FC6-SMB
     357tb-set-node-tarfiles $e /usr /proj/Deter//tarfiles/bwfed/fedkit.tgz
     358# tb-set-node-testbed $e "ucb"
     359tb-set-node-startcmd $e "sudo -H /bin/sh /usr/local/federation/bin/ >& /tmp/startup \$USER"
     360tb-set-node-failure-action $e "fatal"
     361set f [$ns node]
     362tb-set-hardware $f pc
     363tb-set-node-os $f FC6-SMB
     364tb-set-node-tarfiles $f /usr /proj/Deter//tarfiles/bwfed/fedkit.tgz
     365# tb-set-node-testbed $f "ucb"
     366tb-set-node-startcmd $f "sudo -H /bin/sh /usr/local/federation/bin/ >& /tmp/startup \$USER"
     367tb-set-node-failure-action $f "fatal"
     368# Create LAN
     369set lan0 [$ns make-lan "$c $d $e " 100000kb 0.0ms ]
     371# Set LAN/Node parameters
     372tb-set-ip-lan $c $lan0
     373tb-set-lan-simplex-params $lan0 $c 0.0ms 100000kb 0 0.0ms 100000kb 0
     376# Set LAN/Node parameters
     377tb-set-ip-lan $d $lan0
     378tb-set-lan-simplex-params $lan0 $d 0.0ms 100000kb 0 0.0ms 100000kb 0
     381# Set LAN/Node parameters
     382tb-set-ip-lan $e $lan0
     383tb-set-lan-simplex-params $lan0 $e 0.0ms 100000kb 0 0.0ms 100000kb 0
     386# federation gateway
     387set detertunnel0 [$ns node ]
     388tb-set-hardware $detertunnel0 pc3000_tunnel
     389tb-set-node-os $detertunnel0 FBSD7-TVF
     390tb-set-node-startcmd $detertunnel0 "sudo -H /usr/local/federation/bin/ -f /proj/Deter/exp/bwfed/tmp/`hostname`.gw.conf >& /tmp/bridge.log"
     391tb-set-node-tarfiles $detertunnel0 /usr/ /proj/Deter//tarfiles/bwfed/fedkit.tgz
     392# Link establishment and parameter setting
     393set link1 [$ns duplex-link $c $detertunnel0 100000kb 0.0ms DropTail]
     395tb-set-ip-link $c $link1
     396tb-set-ip-link $detertunnel0 $link1
     398$ns rtproto Session
     399$ns run
     402The resulting topology across two testbeds looks like this:
     406The red nodes are the connector nodes inserted by `fedd`.  They both transfer experimental traffic verbatim and tunnel services, like a shared filesystem.  Exactly what gets tunneled and how the connections are made is a function of the federation kit, the topic of the next section.
     408= Experiment Services =
     410An important part of the cohesive experiment framework created by the DFA is the web of services interconnecting the sub experiments.  A service, in this context, is fairly liberally defined; it is any one of several pre-defined ways of sharing information statically or dynamically between sub-experiments.  Static information includes the accounts to create on various machines or the hostnames to export to sub-experiments.  Dynamic information includes mounting filesystems or connecting user experiment tools, like [ SEER].
     412Each service has an ''exporter'' the testbed that provides the service, and zero or more ''importers'', testbeds that use the information.  In addition, services can have attribute/value pairs as parameters.
     414The specific services currently supported by fedd are:
     416 '''hide_hosts'''::
     417 By default, all hosts in the topology are visible across the experiment.  This allows some hosts to be completely local.  The hosts chosen are given by the hosts attribute.
     418 '''local_seer_control'''::
     419 Adds a local seer control node to sub-experiments in the exporting testbed.
     420 '''project_export'''::
     421 Exports the user configuration and filesystem named by the project attribute to the importing testbeds
     422 '''seer_master'''::
     423  Adds a second seer controller to the exporting testbed that is capable of aggregating the inputs from several local seer masters.  To create a seer instance that sees several sub experiments, create a local_seer_control in each and export a seer_master to all.
     424 '''SMB'''::
     425  Export a set of SMB/CIFS filesystems.
     426 '''userconfig'''::
     427  Export the account information from a given project or group.  Currently accepts the project attribute for that purpose.
     429Service advertisement, provision, and specification is an ongoing area of research in federation, and we expect these specifications to become more detailed and useful as that research progresses.
     431= The Federation Kit =
     433The federation kit is the software used by `fedd` to connect testbeds and tunnel services from the master to the other testbeds. Currently we have a single fedkit (available from the [FeddDownload download section]) that provides SSH tunnels to connect experimental testbeds at the ethernet layer, and that tunnels the emulab event system and provides shared file service via [ SMB].  That fedkit runs on any FreeBSD or Linux image that provides the SMB file system.
     435By splitting this function out, we intend to allow different installations of `fedd` to provide different interconnection and service tunneling function.  Currently the [FeddDownload DETER fedkit] is the only federation kit in use, and `fedd` defaults its startcmd options for use with it.
     437The federation kit has 2 roles
     439 * Configuring experiment nodes to use [FeddAbout#ExperimentServices services], such as shared file systems.
     440 * Configuring portal nodes to connect experiments
     442== Fedkit on Experiment Nodes ==
     444On experiment nodes, the fedkit starts dynamic routing, and optionally configures user accounts and samba filesystems if they are in use.  The system expects the following software to be available:
     446 * quagga routing system (an old gated installation will also work)
     447 * samba-client
     448 * smbfs
     450Those are the linux package names; equivalent FreeBSD packages will also work.  For software to be available is for it to be either installed or accessible using {{{yum}}} or {{{apt-get}}}.  DETER nodes have a local repository for that purpose.
     452The fedkit is installed in {{{/usr/local/federation}}} and when run places a log in {{{/tmp/federate}}}.
     454Services are initialized based on the contents of {{{/usr/local/federation/etc/client.conf}}}.  Possible values include:
     456 '''!ControlGateway'''::
     457  The DNS name (or IP address) of the node that will forward services
     458 '''Hide''':
     459  Do nat add this node to the node's view of the experiment.  Used for [FeddMulti multi-party experiments].
     460 '''!PortalAlias'''::
     461  A name that will be mapped to the same IP address as the control gateway.  SEER in particular expects nodes with certain functions to have certain names.
     462 '''!ProjectUser'''::
     463  The local user under which to mount shared project directories
     464 '''!ProjectName'''::
     465  Project name to derive shared project directories from
     466 '''Service'''::
     467  a string naming the [FeddAbout#ExperimentServices services] to initialize.
     468 '''SMBShare'''::
     469  The name of the share to mount
     473== Fedkit on Portal Nodes ==
     475On portal nodes the fedkit uses ssh to interconnect the segments and bridges traffic at layer 2.  If the portal node is a Linux image it needs to have the {{{bridge-tools}}} package available.  Like the fedkit on experiment nodes, it will attempt to load that software from repositories if it is not present.
     477The fedkit configures the portal based on the contents of a configuration file containing the following parameters:
     479 '''active'''::
     480  a boolean.  If true this portal will initiate ssh connections to its peer.
     481 '''nat_partner'''::
     482  a boolean.  If true the fedkit's peer is behind a network address translator.  Not used yet.
     483 '''tunnelip'''::
     484  a boolean.  If true use the DETER system for binding external addresses.
     485 '''peer'''::
     486  a string.  A list of DNS names or IP addresses.  Usually this is one value, the DNS name of the peer, but passive ends of NATted portals may use a list of addresses to establish routing.
     487 '''ssh_pubkey'''::
     488  a string.  A file in which the access controller has placed the ssh key shared by this portal and its peer.  These are nonce keys discarded after the experiment ends.
     489 '''ssh_privkey'''::
     490  a string.  A file in which the access controller has placed the ssh key shared by this portal and its peer.  These are nonce keys discarded after the experiment ends.
     492The passive portal node establishes routing connectivity to the active end, reconfigures the local sshd to allow link layer forwarding and to allow the active end to remotely configure it, and waits.  The active end connects through ssh, establishes a link layer forwarding tunnel and bridges that to the experimental interface.  It also forwards ports to connect experiment services.
     494= Interfaces to fedd =
     496The fedd interfaces are completely specified in WDSL and SOAP bindings.
     497For simplicity and backward compatibility, fedd can export the same
     498interfaces over XMLRPC, where the message formats are trivial
     501All fedd accesses are through HTTP over SSL.  Fedd can support
     502traditional SSL CA chaining for access control, but the intention is to
     503use self-signed certificates representing fedids.  The fedid semantics
     504are always enforced.
     506There are also several internal interfaces also specified in WSDL/SOAP
     507and accessible through XMLRPC used to simplify the distributed
     508implementation of fedd.  You can read about those in the [FeddDevelop development section].