Topdl Library

This page describes the python library for manipulating topologies that follow the topdl model and are generally output in the topdl language. It is broken up into a reference section describing the various python classes and procedures and an examples section.

Reference

Installing and Accessing

The topdl library is available as oart of the DETER module of the DETER federation system. Any system that has fedd installed will have topdl installed as well. In particular, users.isi.deterlab.net has the library installed.

To install the library separately, follow the instructions for getting fedd from git, make the distribution, and install the deter-data-version.tar.gz tar file using the same procedure for installing the fedd tar file. Installing all of fedd will work as well, of course.

Once the library is installed, python programs can access it by:

#!/usr/bin/env python

from deter import topdl

The Classes

The class hierarchy mirrors the entities of the topdl model. Where the description of the model glosses over the details of the classes that parameterize the top-level abstractions, this reference will discuss them all. Each of these classes is a python class, and can be created and manipulated independently, but in a proper topology they are grouped together.

A Substrate object has members that are Capacity and Latency objects. This listing shows that nesting:

• Substrate
  • Capacity (0 or 1)
  • Latency (0 or 1)
  • Attribute (0 or more)

The other common element, a Computer object, has the following nested objects:

• Computer
  • CPU (0 or more)
    • Attribute (0 or more)
  • OperatingSystem (0 or more)
    • Attribute
  • Software (0 or more)
    • Attribute (0 or more)
  • Storage
    • Attribute
  • Interface (0 or more)
    • Capacity (0 or 1)
    • Latency (0 or 1)
    • Attribute (0 or more)
  • Service (0 or more)
    • ServiceParam (0 or more)

Most classes can have attributes attached to them that contain extention attributes that different applications make use of. The presence or absence of an optional nested object does not mean the presence or absence of the thing it describes, but that the description is unbound. As we have seen a Computer without a nested CPU object does not describe a computer without a CPU, but a computer with no constraints on the CPU.

Adaptive Constructors

The constructors of the various topdl classes will accept contained classes in several formats. They will always accept an object of the relevant class. They will also take a dict that maps constructor parameters to values for the given class. Examples make this clearer than description.

The following calls to create a CPU object are equivalent:

cpu = topdl.CPU(type='Intel', attribute=[topdl.Attribute(attribute='clock_rate', value='1Ghz')])

and

cpu = topdl.CPU(type='Intel', attribute=[{'attribute': 'clock_rate', 'value': '1Ghz'}])

and

cpu = topdl.CPU(**{'type': 'Intel', 'attribute' : [ {'attribute': 'clock_rate', 'value': '1Ghz'}]})

That is, inner class parameters can be specified using dicts directly, and a top level dict can user the standard operator.

All topdl class constructors take the member names as parameters, and should be called using named parameters, as above. The only exceptions to this is the Substrate object's interfaces member and the Interface object's subs member. An Interface's element member can be specified, but generally is not.

Topdl Class Features

All topdl classes implement the following methods:

set_attribute(self, key, value)

Attach an user-defined attribute with the given key and value to this object; value and key are strings. The method includes a key/value pair (encoded as an attribute object) into the attribute member of the object. If an attribute with the given key exists, it is overwritten. This method accesses the attribute member of the given object, not any named members. Objects that do not have an attribute member silently discard the operation.

get_attribute(self, key)

Return the attribute of this object with the given key. The attribute - a string - is returned. If no such attribute exists, or the object has no attribute member, return None.

remove_attribute(self, key)

If an attribute with the given key exists, remove it.

clone(self)

Returns a deep copy of this object.

to_dict(self)

Return the object as a python dict. No copies are made, so make modifications with caution.

to_xml(self)

Return the XML encoding (from the topdl language) of this object (and nested objects)

Topology Class

This class holds an enitre topology. The utility functions that read and write files operate on it, and it is generally the unit of I/O for the library.

It has the following members:

substrates

A list of Substrate objects that contain the substrates of the topology. The list may be empty indicating no communcations are possible.

elements

A list of entities make up the network. These can be of any Computer objects, Testbed objects, Segment objects, or Other objects. The list may be empty indicating no elements are in the topology. attribute::: A list of Attribute objects that are attached to the machine. The list may be empty.

version

The version of topdl that this object encodes.

As usual, the constructor takes the member names as parameter names.

One useful specialized member is:

incorporate_elements(self)

That member sanity checks the substrates to make sure they are uniquely named, assigns names to any interfaces without them, and connects the element members of the Interface objects. If there are inconsistencies, it raises a topdl.ConsistencyError with more detail in the message.

incorporate_elements() is called by the Topology constructor as well and that constructor may throw the same exceptions.

Computer Class

A Computer object is a programmable computer in a topology. A webserver is well represented by a computer, as is a desktop. It has the following members:

name

The unique name of the computer in the topology, a string. Other names may be assigned in the localname attribute.

cpu

A list of CPU objects that describe the CPUs of the machine. The list may be empty, meaning no CPU requirements or information are present.

software

A list of Software objects that describe the software to install. The list may be empty, meaning no software is to be installed.

storage

A list of Storage objects that describe the storage present the machine. Storage objects can describe memory or persistent storage requirements. The list may be empty, meaning no storage requirements or information are present.

interface

A list of Interface objects that describe connections of the Computer to communications substrates. The list may be empty, meaning that the computer is not connected to any substrates.

attribute

A list of Attribute objects that are attached to the machine. The list may be empty.

localname

A list of strings giving alternative names for the object. When a machine is allocated to a testbed the DNS names assigned to the machine are given as localnames. The list may be empty, meaning no such alternatives exist.

status

A string indicating the current status of the Computer (if any). It may also be None, indicating no status is reported. Valid values are

• "empty" - no allocation of resources has been attempted - generally not applied to Computers
• "active" - Computer is functioning as part of an experiment
• "inactive" - Computer is allocated, but not functioning
• "starting" - Computer is transitioning from "inactive" to "active"
• "terminating" - Computer is transitioning from "active" to "inactive"
• "failed" - an allocation of resources has failed - generally not applied to Computers

service

A list of Service objects that describe the services this machine uses or supplies. These are generally used by the federation system, and their definition is somewhat in flux. The list may be empty.

operation

A list of strings that describe the valid operations on the machine. These are generally used by the federation system, and their definition is somewhat in flux.The list may be empty.

The constructor takes the member names as parameters. Only name is required.

A Computer should only be an element of one Topology. Use clone() to avoid this.

Substrate Class

The Substrate object represents a communication substrate. The elements that have interfaces connected to a substrate can communicate with one another. A Substrate may include default parameters for the latency and capacity of the communication channel.

The members are:

name

A string, the unique name of the substrate. {{{capacity}}:: A Capacity object that gives the default capacity of the substrate. It may be None, incdicating that there is no default. {{{latency}}:: A Latency object that gives the default latency of the substrate. It may be None, incdicating that there is no default.

attribute

A list of Attribute objects that are attached to the substrate. The list may be empty.

localname

A list of strings giving alternative names for the object. The list may be empty, meaning no such alternatives exist.

status

A string indicating the current status of the Substrate (if any). It may also be None, indicating no status is reported. Valid values are

• "empty" - no allocation of resources has been attempted - generally not applied to Substrates
• "active" - Substrate is functioning as part of an experiment
• "inactive" - Substrate is allocated, but not functioning
• "starting" - Substrate is transitioning from "inactive" to "active"
• "terminating" - Substrate is transitioning from "active" to "inactive"
• "failed" - an allocation of resources has failed - generally not applied to Substrates

service

A list of Service objects that describe the services this substrate uses or supplies. These are generally used by the federation system, and their definition is somewhat in flux. The list may be empty.

operation

A list of strings that describe the valid operations on the substrate. These are generally used by the federation system, and their definition is somewhat in flux.The list may be empty.

interfaces

A list of Interface objects connected to this substrate. The list may be empty, meaning that the substrate is not connected to any elements. This field is maintained by the Topology object holding this Substrate object. It is not a parameter to the constructor.

Other than interfaces, the constructor takes the member names as parameters. Only the name is required.

Because the enclosing Topology object maintains the interfaces member, it is dangerous to have the same Substrate object in more than one Topology. Use the clone() method to avoid this.

Interface Class

This class defines the attachment of an entity, e.g., a Computer, to a Substrate. As with Substrates and entities, each Interface should be attached to only one entity and be unique across a Topology. An Interface has the following members:

substrate

A list of strings, each naming a substrate to which this interface is connected. The list must have at least one entry.

capacity

A Capacity object indicating the communications capacity of the interface. It may be None to use the substrate default.

latency

A Latency object indicating the communications delay of the interface. It may be None to use the substrate default.

attribute

A list of Attribute objects that are attached to the interface. The list may be empty.

element

The entity to which this interface is attached. It may be a Computer object, Testbed object, Segment object, or Other object. This is generally managed by the Topology holding the entity and substrate with which the Interface is attached. It is wise to treat the field as read-only and valid only after a Topology has called incorporate_elements().

subs

A list of Substrates to which the interface is attached. It cannot be set through the constructor. This is generally managed by the Topology holding the entity and substrate with which the Interface is attached. It is wise to treat the field as read-only and valid only after a Topology has called incorporate_elements().

The constructor takes all the member names except subs as parameters. Most programs will not need to use element as a parameter to the constructor, either.

Attribute Class

An attribute is a user-defined annotation of one of the other classes: a key/value pair. An attribute has the following members:

attribute

A string, the key used to find the attribute.

value

A string, the value

While attributes can be manipulated directly liek any of the other topdl classes, it is most common to use the attribute manipulation functions of the annotated class to access them.

Should a program need to call the constructor, it takes the member names as named parameters.

CPU Class

This represents a CPU on a computer. It has the following members:

type

A string indicating the type of CPU. These are not yet standardized, but common formats like "i686" or "amd64" are used.

attribute

A list of Attribute objects that are attached to the CPU. The list may be empty.

The constructor takes the member names as named parameters.

OperatingSystem Class

This class captures the operating system requirements of a computer. It has the following members:

name

A string naming the operating system required. This is usually the operating system itself, e.g., Linux, FreeBSD, Windows XP. This may be None, indicating no preference or information.

version

A string identifying the version of the OS. For integrates systems like FreeBSD this is the system version. For distribution-based systems like Linux, this is the kernel version. See below for distribution versioning. This may be None, indicating no version information or preference.

distribution

A string that for operating systems that characterize OS distribution independently of kernel or base system, names the distribution in use, e.g., Ubuntu. This may be None, indicating no information or preference.

distributionversion

A string that for operating systems that characterize OS distribution independently of kernel or base system, names the version of the distribution in use, e.g., 12.04 for Ubuntu. This may be None, indicating no information or preference.

attribute

A list of Attribute objects that are attached to the operating system. The list may be empty.

The constructor takes the member names as parameters. Note that all the members may be None, but an empty !OperatingSystem object is not very useful. The flexibility is there to allow specifying an operating system version without a distribution or a distribution without a base version.

Storage Class

The storage requirements or information for a system. This class characterizes memory or permanent storage. Its members are:

amount

A floating point value indicating the number of megabytes of storage being considered. Values less than 0.000001 are treated as 0. {{{persistence}}:: A boolean value indicating whether this is persistent storage, such as a disk or SSD, or volitile such as memory.

attribute

A list of Attribute objects that are attached to the machine. The list may be empty.

Currently there is no field for media type. Use attributes.

The constructor takes the member names as named parameters.

Software Class

This indicates what software must be installed on a system. Its members are:

location

A string containing a URI from which to install the software, e.g., a file: URI pointing to a tarfile.

install

A string containing location in the local file system to base the software install. This may be None, indicating that the software distribution format includes this information, e.g., an rpm or debian software file.

The constructor takes the member names as named parameters.

Service Class

A service class encapsulates the ideas of a DETER federation service. Such services can be available from or used by a variety of topology elements. The Service class has the following members:

name

The name of the service. A few are defined.

importer

A list of entity names that are using or requesting the service. This list may be empty.

param

A list of ServiceParam objects that represent typed parameters to the service. The list may be empty.

description

A string describing the service. This may be None.

status

A string describing the current status of the service (if any). It may also be None, indicating no status is reported. Valid values are

• "empty" - no allocation of resources has been attempted - generally not applied to Services
• "active" - Service is functioning as part of an experiment
• "inactive" - Service is allocated, but not functioning
• "starting" - Service is transitioning from "inactive" to "active"
• "terminating" - Service is transitioning from "active" to "inactive"
• "failed" - an allocation of resources has failed - generally not applied to Services

The constructor will take the member names as named parameters.

ServiceParam Class

A description of the type of parameter to a Service will accept. It has the following members:

name

Name of the parameter, a string.

type

A string containing the type of the parameter. Valid types are:

• string
• int
• float

The constructor will take the member names as named parameters.

Examples