Federating With DETER

This discusses how to connect a ​DETER or ​Emulab-based testbed to ​DETERLab using fedd, ​ABAC and their tools.

Introduction

This page takes a user through the process of adding federation components to an existing DETER and assigning user credentials that will allow users to access that DETER through the fedd interfaces, and then to use those same interfaces to access federated resources. We will also talk about how to allow federated users to use the local resources.

This document will discuss the relevant fedd components, what they do, and what they need in order to do those jobs. We also discuss the tools an experimenter uses to create and monitor federated experiments. Then we briefly discuss how to get the ABAC credentials necessary to access the federated environment.

Fedd really is two entities, an Access Controler that mediates access to federated resources and provides standard interfaces for allocating them, and an Experiment Controller that acts as a credential store and coordinates requests across multiple access controllers. The discussion of the Deter Federation Architecture has more to say about these entities, but this is enough to get started. DETERLab runs an experiment controller that can be reached at ​https://users.isi.deterlab.net:23235 and new federation users can use that experiment controller, but when setting up and administering a federated testbed it can be very helpful to install and configure an experiment controller locally.

Here is a block diagram of a user making a request for a federated experiment. We will show how to configure the credentials that users and the experiment controller pass to access controllers, how to configure the policies at the access controlers to allow access and map federated users into local users. We will also configure the experiment controller to find the access controlers and the access controllers to allocate resources (start DETER experiments) on behalf of local users.

Both the policy and credentials are ​ABAC credentials, but we provide tools to configure these for the common cases. Specifically we explain how to get a new testbed endorsed as a federation partner and how to allow federated partners access to local resources. Tools for finer-grained access control are in development, though ambitious users can use the raw ABAC tools to specify policy.

Road Map

The rest of this document explains how to do the following:

• Install the federation software
• Configure an experiment controller that translates local DETER projects and user identities into ABAC
• Configure an access controller to map those ABAC credentials back into local users and to allocate resources on their behalf
• How to get credentials endorsed from DETERLab to gain access to federated resources and use them.

Installation

There are detailed instructions for installation of fedd and the required software. As we prepare for the fedd 4.0 release, it is best to install from git.

Before downloading and installing the code, there are network connectivity requirements to consider. The experiment controller must be reachable by user tools and must be able to reach and be reached by access controllers. These are SSL protected TCP connections, though the port is configurable. Additionally, the some experimental nodes in the testbeds must be able to reach each other.

Configuring The Experiment Controller

The experiment controller needs a directory in which to store credentials and configuration information. This includes SSL credentials, so it should be protected. You can create a user to run the experiment controller if necessary.

This directory will contain the following configuration files

• fedd.conf - the main configuration file
• fedd.pem - a self-signed X.509 certificate identifying the controller
• exp_access_db - a human-readable database containing the mapping from experimenter identity to ABAC credentials (this is converted into ABAC)
• exp_map_db - a mapping of symbolic names to access controller URLs
• fedkit.tgz - the federation software run withing the experiment. Running make in the fedkit directory creates it.

In addition the following empty directories must be created for the controller to store experiment data in

• abac - the compiled ABAC credentials
• certs - experiment credentials
• repo - software that is distributed to access controlers for inclusion in federated experiments
• userconf - user configuration information for experiments (exported accounts...)

We recommend /usr/local/etc/fedd/experiment as the experiment controller's home directory.

fedd.conf

Here is an annotated fedd.conf file that describes the various parameters necessary to run an experiment controller:

[DEFAULT]

# This section is primarily to set internal variables for customization.  A 
# variable is inserted by %(variable)s .  NB: that s after the parens is
# literal.

# The Experiment Controllers home directory.  Used in susbtitutions.
base: /usr/local/etc/fedd
# Space in /var for the Experiment controller. Used in susbtitutions.
var: /var/db/fedd
# The DNS name the Experiment controller runs on. Used in susbtitutions.
hostname:users.isi.deterlab.net

[globals]

# Real EC configuration

# The certificate file that identifies this controller.
cert_file: %(base)s/fedd.pem

# The ports and protocols this node is supporting.  This says listen on poer 
# 23235 for SOAP connections and 23232 for XMLRPC connections.
services: 23235,23232:xmlrpc


[experiment_control]

# How to convert tcl (ns2) experiments into topdl.  Generally the safest way to
# do this is to contact DETER's EC using the ns2topdl_uri setting below.  If
# the testbed is a DETER testbed, you may be able to do the conversion locally
# using the tcl_splitter parameter below.  Pick one.

#tcl_splitter: /usr/testbed/opsdir/lib/ns2ir/parse.tcl
ns2topdl_uri: http://users.isi.deterlab.net:23235

# Staging area for software being sent to federated experiments.  This line 
# stores it under /var/db/fedd (from the var variable above.
repodir: %(var)s/repo

# Contact URL for retrieving software.
repo_url: https://%(hostname)s:23235

# Contact URL for retrieving experiment configuration data
store_url: https://%(hostname)s:23235

# File in which the dynamic experiment configuration data is stored.  You do 
# not need to create this file, nor manage it.
synch_store: %(base)s/store.dat

# ABAC configuration.  No other value for auth_type is currently valid.
auth_type = abac

# Location of the ABAC state
auth_dir = %(base)s/abac

# Time to cache status information, in seconds
info_cache = 300

# Logging level, in traditional syslog values.
log_level: debug

# Debugging parameters.  If leave_tempfiles is true, temporaries are not 
# deleted.  If create_debug is true, the local testbed is not accessed (no
# resources are allocated.)
#leave_tmpfiles: true
#create_debug: false

# Internal experiment controller state.  Completely managed internally
experiment_state: %(base)s/fedd.state

# List of testbed names to access URLs
mapdb: %(base)s/exp_map_db

# Software to install on each experiment node to form federated experiments.
# The first parameter is the directory in which to install it and the second
# the tar file to unpack.  These are the correct values for the fedkit
# distributed with fedd.
fedkit: /usr %(base)s/fedkit.tgz

The fedd.conf above is attached​ to this page for download and editing.

We have instructions for creating a certificate suitable for use as a cert_file.

A fedkit.tgz can be created from the fedkit directory of the git repository. Change to that directory and make.

The exp_map_db file is also attached​ and its contents are:

deter:https://users.isi.deterlab.net:23231
deter-test:https://users.isi.deterlab.net:23001
deter_internal:https://users.isi.deterlab.net:23233

These are the addresses of known access controllers. The tools can also request different mappings.

The most complex part of the configuration is setting up the ABAC policy.

Setting up the Experiment Controller's ABAC

For new federators we provide simple tools to create a policy based on the DETER projects and user accounts present in the loacl testbed. The ​exp_access_db.py command will read the database on the boss node and output a human-readable configuration file that tells what attributes the experiment controller will assert on behalf of the user.

In particular, the example at the link above produces a file that looks like this:

# faber
fedid:12ecc7415746281efa0ed58e180c51a5cba13a57->faber
fedid:12ecc7415746281efa0ed58e180c51a5cba13a57->(Deter,faber)
fedid:12ecc7415746281efa0ed58e180c51a5cba13a57->(emulab-ops,faber)
# lahey
fedid:d7da471ff2ba5b205d31ae379b1cf634c8dce1c6->lahey
fedid:d7da471ff2ba5b205d31ae379b1cf634c8dce1c6->(emulab-ops,lahey)

That output asserts that the holder of the X.509 identity with the hash 12ecc7415746281efa0ed58e180c51a5cba13a5 will be identified as the user faber and as a member of projects emulab-ops and Deter. DETER issues all users an X.509 certificate and that certificate is extracted by the script.

To create a file that asserts all users and projects in your testbed, run exp_access_db.py on your boss node and save the output in a file called exp_access_db.

To convert those assertions to ABAC in a format that access controllers will understand:

• copy exp_access_db to your experiment controller's home (/usr/local/etc/fedd/experiment)
• run fedd_to_abac.py to create the ABAC
  • fedd_to_abac.py --cert fedd.pem --dir /usr/local/etc/fedd/experiment/abac --make_dir exp_access_db should do it

At this point you should be able to run your experiment controller:

fedd.py --config=/usr/local/etc/fedd/experiment/fedd.conf --debug

You can specify a log file with the --log option.

Configuring the Access Controller

The access controller needs a similar set of files configured, but there are a couple key differences. The configuration file looks like:

[DEFAULT]
# The home directory of the access controller
base: /usr/local/etc/fedd/access

[globals]
# Access controller identity
cert_file: %(base)s/fedd.pem

# Services as for the experiment controller
services: 23231

[access]

# syslog level
log_level: debug

# File that holds internal state between access controller boots
access_state: %(base)s/deter_access.state

# The file that maps from ABAC credentials to local user and project name.
# Generated by tools, but human readable.
accessdb: %(base)s/deter_abac_map

# Directory where certificates identifying particular allocations are created
certdir: %(base)s/certs

# The ssh identity used to move software into experiments.  Federated users
# must allow this key access to their DETER accounts.
ssh_privkey_file: %(base)s/fedd_rsa
ssh_pubkey_file: %(base)s/fedd_rsa.pub

# Same as the experiment controller values
# create_debug: true
#leave_tmpfiles: true

# This is the correct value for an Emulab or DETER testbed
type: local_emulab

# ABAC configuration and directories
auth_type: abac
auth_dir: %(base)s/abac

# URI at which to access the Emulab XMLRPC interface.  Generally only the boss
# node's DNS name needs to be
boss: https://boss.isi.deterlab.net:3069/usr/testbed
ops: users.isi.deterlab.net

# Image to be loaded on portal nodes.  Some version of FreeBSD is best.
portal_image: FBSD7-STD

# Configurable federation setup commands.  For the standard fedkit on a DETER,
# these are correct.
portal_startcommand: sudo -H /usr/bin/perl -I/usr/local/federation/lib /usr/local/federation/bin/combo.pl --use_file >& /tmp/bridge.log
node_startcommand: sudo -H /usr/bin/perl -I/usr/local/federation/lib /usr/local/federation/bin/federate.pl >& /tmp/federate


# DNS domain of the testbed
domain: .isi.deterlab.net

# Analogous to the experiment controller
federation_software: /usr %(base)s/fedkit.tgz