Fedd Commands

These are the various user level programs that the fedd package installs, including fedd itself.

Fedd

This is the daemon itself, which is either an experiment controller or access controller (or both) depending on its configuration. After the fedd description, we cover the various support scripts included in the distribution, before addressing the command line interface.

Fedd.py

The main federation daemon; when this documentation refers to fedd is is the fedd.py executable that is running. It can run on any host as any user, provided permissions and keys are correct. Most of fedd's behavior is controlled by its configuration files and databases, there are a few command line arguments.

--config=filename

Use filename as the configuration file rather than the default /usr/local/etc/fedd.conf

--debug

Produce additional error output

--logfile=filename

Redirect log output to filename rather than the default standard output

Confirm_sshkey.py

The confirm_sshkey.py command is used by fedd to confirm that a local account can be accessed using an SSH public key. It does this by accessing the Emulab database, and must run on boss. It is included in the distribution for administrators who want to be able to confirm keys without patching addpubkey.

The command takes the same arguments as the extended version of addpubkey, which is to say the same arguments as the unextended addpubkey plus -C. Other than installing it and adjusting the confirmkey parameter in the [allocation] section of the configuration file users should rarely need this command. By default the fedd package confirm_sshkey.py in /usr/local/bin.

Exp_access_db.py

The exp_access_db.py command is a helper to generate experiment control component access databases. It can be run on boss to print data in that database's format that can be redirected into a file for use as a database. With no options it converts the entire users table of the database into an experiment control access database with an entry for each local project of which the user is a member. The fedid's are generated from the user's SSL certificates as they appear in the Emulab database.

That behavior can be constrained using the following options.

--user=username

Extract the information for username. Once --user is given once, only the users selected will be output. The option may be specified more than once.

--project=projectname

Extract information only for the given project. The option may be specified multiple times.

--no-project-access

Do not generate a line for any of the projects a user is in. Only generate lines of the form

fedid:xxx -> user

--no-user-access

Do not generate a line without an enclosing project. Only generate lines of the form

fedid:xxx -> (project,user)

Multiple user constraints are or-ed with each other as are multiple project constraints. Then the resulting or clauses are and-ed together. For example, the invocation

$ exp_access_db.py --user=faber --user=lahey --project=Deter --project=emulab-ops

produces output like

# faber
fedid:12ecc7415746281efa0ed58e180c51a5cba13a57->faber
fedid:12ecc7415746281efa0ed58e180c51a5cba13a57->(Deter,faber)
fedid:12ecc7415746281efa0ed58e180c51a5cba13a57->(emulab-ops,faber)
# lahey
fedid:d7da471ff2ba5b205d31ae379b1cf634c8dce1c6->lahey
fedid:d7da471ff2ba5b205d31ae379b1cf634c8dce1c6->(emulab-ops,lahey)

If the users are in other projects, those lines are suppressed; if the projects contain other users, those lines are suppressed. The comment lines before each cluster of output help administrators understand which data corresponds to a given user.

As with other commands in the fedd package, it is installed in /usr/local/bin by default.

Fedid.py

The fedid.py command prints the [Feddabout#Globalidentifiers:Fedids fedid] of the public key of an X.509 certificate to the standard output. This is handy in getting fedids for the various databases that fedd uses. The command is invoked with 0 or more --attribute=attr parameters and 0 or more filenames. The command prints the fedids and the attributes in the format of a global access database and can be used directly to construct those files. For example, this invocation:

$ fedid.py --attribute=split --attribute=allocate fedd.pem deterboss.pem

produces

fedid:ce90957dd5b7d20f9c3890c4599313b7f1cf31ea split,allocate
fedid:9b8e7f2b114956ad516befc93fa7e7eae3751bf9 split,allocate

User_to_project.py

The user_to_project.py command exists to provide command line functionality that Emulab keeps in a web page. It attaches a user to a project. It is called by a fedd running on boss when a project is dynamically created. Its default installation destination and the default value for the [experiment_control] parameter user_to_project both default to /usr/local/bin/user_to_project.py. It is unlikely a user will ever need to run this command.

Command Line

These scripts make up the command line interface to the federation system. They remain the most versatile and powerful way to create and manage federated experiments. Most of these commands communicate with the experiment controller through either a SOAP or XMLRPC interface to carry out their functions.

All are installed in /usr/local/bin by default.

All the command line tools accept the following arguments:

--cert=filename

Certificate from which to derive the user's fedid. By default the contents of .ssl/emulab.pem in the user's home directory is used.

--debug

Produce additional debugging output.

--serializeOnly

Do not contact the fedd, but just print the SOAP or XMLRPC message to the standard output.

--trusted=filename

Use the certificates in filename as certificate authorities to confirm the server's identity. Optional.

--url=fedd_url

Fedd to contact.

--transport=[xmlrpc|soap]

Use the given encoding of the service request.

--trace

Print the SOAP exchanges to stderr. Currently only the SOAP transport is supported.

fedd_create.py

This command creates a federated experiment from an experiment description. It does the equivalent of a call to fedd_new.py to make an allocation and then attaches resources and services as described in the description file. On completion it returns the name of the experiment, and if requested using the --experiment_cert parameter, a certificate that grants access to the experiment.

When fedd_create.py returns, the experiment has not yet fully been created, it is simply in progress. Users can poll the experiment controller by issuing fedd_multistatus.py? commands or wait for the experiment to be instantiated using fedd_spewlog.py.

fedd_create.py takes the following arguments:

--experiment_cert=filename

Store the certificate used to access the resulting experiment in filename. This certificate can be used as an input parameter to the terminate and informational commands below, but is an output parameter here.

--experiment_name=exp_name

Suggest exp_name to the fedd as a local identifier for the federated experiment. The actual local name chosen will be returned from the command.

--file=filename

File containing the experiment description.

--project=export_project

The local project to export from the master testbed.

--master=master_testbed

The master testbed. This should correspond to one of the annotations used for testbed names in the experiment. The fedd will use its experiment name mapping DB to resolve that name to a testbed.

--service=spec

Add the given service to the experiment being created. This parameter may be specified multiple times. Spec has the format:

name:exporter:importer1,importer2:attribute1=value1;attribute2=value2

Names are specified in the discussion of services in the overview.

fedd_ftopo.py

Returns a mapping between the virtual element names in the experiment description and a physical name that can be used to access the resource. Right now all the plug-ins that return such a mapping return a DNS name as the physical name.

fedd_ftopo.py takes the following parameters in addition to the standard ones:

--experiment_cert=filename

The filename must contain a certificate valid for the experiment's fedid. If so it is used to authenticate access to the experiment. A password may be required if one has been added to the private key.

--experiment_name=name

The local name of the experiment to query. Only the experiment creator can use this for access.

The output has the format:

element_name:physical_name:testbed

fedd_image.py

fedd_image.py creates visualizations of federated experiments at a box and lines level, showing the interconnections between elements. The visualizations can be generated from an active federated experiment, a topdl description, or an ns2 description. fedd-image.py calls out to graphviz's neato program to draw the graphs, so it or a compatible program must be available.

Internally, the various representations are converted to topdl and from there into the neato specification. An active experiment's topdl description must be pulled frmo teh experiment controller it was created on and a local ns2 description must be translated by a fedd into topdl. Topdl descriptions can be drawn without contacting a fedd.

fedd_image.py takes the following arguments in addition to the standard ones:

--experiment_cert=filename

The filename must contain a certificate valid for the experiment's fedid. If so it is used to authenticate access to the experiment. A password may be required if one has been added to the private key.

--experiment_name=name

The local name of the experiment to query. Only the experiment creator can use this for access.

--file=filename

File containing the experiment description.

--format=fmt

The image format to use. Valid choices are jpg, png, svg, and dot. dot is graphviz input.

--labels

Include node names and IP address information in the image. This is the default.

--no-labels

Omit node names and IP address information in the image.

--output=file

A file to store the image in. If --format is given, otherwise the last three characters of the filename are taken as a --format specification.

--pixels=pix

Output an image that is pix pixels, square.

--program=path

A neato compatible program to draw the graph. This may be helpful if graphviz is installed in a non-standard place or with a non-standard name.

fedd_info.py

This program provides various information about running (or failed) federated experiments. Many of the types of data it can return are accessible in more user-friendly ways by other commands. It is largely used for debugging.

The data types accessible are:

id

The local name and fedid of this experiment. Just the fedid is returned, not a certificate representing it, or its key.

embedding

A raw format of the information returned by [FeddCommands#fedd_ftopo.py]. Unless you're debugging fedd_ftopo.py we suggest that program.

experimentdescription

A topdl description of the global virtual topology of this experiment. Portal nodes and other elements that are transparent to the experiment (and potentially unknown to the experiment controller) are omitted.

log

The experiment controller log from the experiment's creation or failed creation attempt.

status

The experiment status. See below for details.

vtopo

vis

Emulab compatible descriptions of the virtual topology and visualization of the experiment. These are used to debug these compatibility interfaces.

An experiment's status can be one of the following values:

active

The experiment has resources allocated to it and has been successfully configured.

empty

The experiment exists, but no attempt has been made to attach resources to it.

failed

The experiment could not be started. This state allows users to retrieve logs and federant information from failed experiments. Once information is retrieved, this experiment should be terminated.

starting

The experiment is having resources allocated and elements configured.

terminating

The experiment is being terminated.

fedd_info.py takes the standard arguments and:

--experiment_cert=filename

The filename must contain a certificate valid for the experiment's fedid. If so it is used to authenticate access to the experiment. A password may be required if one has been added to the private key.

--experiment_name=name

The local name of the experiment to query. Only the experiment creator can use this for access.

--data=sel

Print the data given by sel, which can be any of the values above. The --data argument may be specified multiple times.

fedd_multiinfo.py

fedd_multiinfo.py is a batch version of fedd_info.py that returns the selected data for all experiments that the caller can access. It takes the standard arguments and a ---data=sel argument that has the same meaning as for fedd_info.py.

fedd_multistatus.py

fedd_multistatus.py provides a summary of status information for all experiments that the caller can access. It takes the standard arguments. Output is one line per experiment of the form:

local_name:fedid:status

fedd_new.py

This command creates an experiment but does not attach any resources to it. This is currently not very useful, but will become useful in the future. It takes the standard arguments and:

--experiment_cert=filename

Store the certificate used to access the resulting experiment in filename. This certificate can be used as an input parameter to the terminate and informational commands below, but is an output parameter here.

--experiment_name=exp_name

Suggest exp_name to the fedd as a local identifier for the federated experiment. The actual local name chosen will be returned from the command.

Fedd_client.py

The fedd_client.py command was the original command line interface to most of fedd's interfaces. It has been superceded by the individual scripts below. It allows users to create terminate and interrogate experiments as well as to make access requests and request experiment splits. The access and splitting commands are primarily for debugging. Users who wish to make use of that function can consult the command's internal help message, by issuing one of

$ fedd_client.py split --help 
$ fedd_client.py access --help

The general syntax for invoking the command is:

$ fedd_client.py operation parameters

where operation is one of create, terminate, info, vtopo, vis, multiinfo, multistatus, spewlog, image, ns_image, split, or access and the parameters are described below.

All commands take the following parameters:

--cert=filename

Certificate from which to derive the user's fedid. By default the contents of .ssl/emulab.pem in the user's home directory is used.

--debug

Produce additional debugging output.

--serializeOnly

Do not contact the fedd, but just print the SOAP or XMLRPC message to the standard output.

--trusted=filename

Use the certificates in filename as certificate authorities to confirm the server's identity. Optional.

--url=fedd_url

Fedd to contact.

--transport=[xmlrpc|soap]

Use the given encoding of the service request.

--trace

Print the SOAP exchanges to stderr. Currently only the SOAP transport is supported.

The create command takes the following additional parameters:

--experiment_cert=filename

Store the certificate used to access the resulting experiment in filename. This certificate can be used as an input parameter to the terminate and informational commands below, but is an output parameter here.

--experiment_name=exp_name

Suggest exp_name to the fedd as a local identifier for the federated experiment. The actual local name chosen will be returned from the command.

--file=filename

File containing the experiment description.

--project=export_project

The local project to export from the master testbed.

--master=master_testbed

The master testbed. This should correspond to one of the annotations used for testbed names in the experiment. The fedd will use its experiment name mapping DB to resolve that name to a testbed.

--ssh_key=file

Use the contents of file as the SSH key for service access in the experiment. By default the contents of .ssh/id_rsa.pub is used.

An example of experiment creation is:

$ fedd_client.py create --file=test_fedd.tcl --master=deter --project=emulab-ops --url=https://users.isi.deterlab.net:23235 --transport=xmlrpc

The results are the local name of the experiment and its fedid printed to standard output. If --experiment_file is given an X.509 certificate and private key are saved to the given file that can be used as a capability to access the experiment state. When the create command terminates the experiment is being created. Use the info or spewlog command to see when the experiment has been fully created.

The commands to access data about an experiment or to terminate it all take only two non-generic options:

--experiment_file=filename

The filename must contain a certificate valid for the experiment's fedid. If so it is used to authenticate access to the experiment. A password may be required if one has been added to the private key.

--experiment_name=name

The local name of the experiment to terminate. Only the experiment creator can use this for access.

The informational commands are vtopo, vis, and info. Vtopo and vis are equivalent to the analogous Emulab XMLRPC commands, and fedd_client.py prints thier results in raw XML. Info provides general access to information about an experiment's state. It takes an additonal parameter to control the data output:

--data =data_element

The data to output. May be specified more than once.

The valid choices for --data are:

federant

All information about the various federants. The XML format is available in the data description

id

The local name and fedid of the experiment

status

The current status of the experiment. One of active, starting, or failed

log

The current experiment allocation log (may be incomplete for experiments still starting)

access

The X.509 credentials used to access the experiment. Note that this is only available to those who already have access to the experiment, but that can be passed along to others to grant them access.

vis

visualization XML as above. (In fact vis is an alias for this.)

vtopo

topology information as above. (vtopo is an alias for this.)

The image, ns_image and status commands are more useful to users. The first two create simple image files from either an existing federated experiment or from the experiment description. image takes the same parameters as vtopo, vis, and info as well as:

--output=file

A file to store the image in. The output format is determined by the file suffix.

--format=fmt

The image format to use. Valid choices are jpg, png, svg, and dot. dot is graphviz input.

--labels

Include node names and IP address information in the image. This is the default.

--no-labels

Omit node names and IP address information in the image.

--pixels=pix

Output an image that is pix pixels, square.

ns_image takes the same additional parameters as image, but is takes create parameters instead. It generates the image from the description rather than the existing experiment. It does not create a federated experiment.

The status command is a modified info command that prints a single word that describes the status of an experiment:

active

The experiment exists

starting

The experiment is being created.

terminating

The experiment is beinfg terminated

failed

The experiment could not be started. This state allows users to retrieve logs and federant information from failed experiments. Once information is retrieved, this experiment should be terminated.

The multiinfo command takes the --data options that info takes, and returns the specified information for all experiments that the user has access to, separated by three dashes for each experiment.

The multistatus command returns the following information for each experiment that the user can access. Each experiment's information is on a single line, separated by colons. The information is the experiment's human-readable name, the experiment's fedid as a hex string, the experiment's status, the master testbed for the experiment, and the project name on that master testbed.

The spewlog command waits for a given experiment to go into either active or failed state and incrementally outputs the allocation log as it is generated at the fedd. In addition to the ususal information parameters it takes a --logfile=file parameter that specifies where to put the log file, and a --update=secs parameter that specifies how often to poll for log updates (default 10 seconds). When spewlog completes, it outputs the status of the experiment into the log file and terminates.

The terminate command stops a federated experiment and removes the access granted from the federants. It takes a --force parameter that terminates experiments that are starting, terminating or in an unusual state. It also reports a log of the steps to deallocate the experiments if --print_log is given or writes that log to a file if --logfile=file is given.

The split and access commands are really only for developers. They are not described here, but

$ fedd_client.py access --help
$ fedd_client.py split --help

provides information on their use.