= Topdl Library =

This page describes the python library for manipulating topologies that follow the [TopDl topdl] model and are generally output in the topdl language.  It is broken up into a reference section describing the various python classes and procedures and an examples section.

= Reference =

== Installing and Accessing ==

The topdl library is available as oart of the DETER module of the DETER federation system.  Any system that has fedd installed will have topdl installed as well.  In particular, {{{users.isi.deterlab.net}}} has the library installed.

To install the library separately, follow the [FeddDownload#GitAccess instructions for getting fedd from git], make the distribution, and install the {{{deter-data-}}}''version''{{{.tar.gz}}} tar file using the [FeddDownload#DownloadingandInstallingthetarfile same procedure for installing the fedd tar file].  Installing all of fedd will work as well, of course.

Once the library is installed, python programs can access it by:

{{{
#!/usr/bin/env python

from deter import topdl
}}}

== The Classes ==

The class hierarchy mirrors the entities of the [TopDl topdl model].  Where the description of the model glosses over the details of the classes that parameterize the top-level abstractions, this reference will discuss them all.  Each of these classes is a python class, and can be created and manipulated independently, but in a proper topology they are grouped together.

A Substrate object has members that are Capacity and Latency objects.  This listing shows that nesting:

 * Substrate
   * Capacity (0 or 1)
   * Latency (0 or 1)
   * Attribute (0 or more)

The other common element, a Computer object, has the following nested objects:

 * [TopdlLibrary#ComputerClass Computer]
  * [TopdlLibrary#CPUClass CPU] (0 or more)
    * [TopdlLibrary#AttributeClass Attribute] (0 or more)
  * [TopdlLibrary#OperatingSystemClass !OperatingSystem] (0 or more)
    * [TopdlLibrary#AttributeClass Attribute] (0 or more)
  * [TopdlLibrary#SoftwareClass Software] (0 or more)
    * [TopdlLibrary#AttributeClass Attribute] (0 or more)
  * [TopdlLibrary#StorageClass Storage] (0 or more)
    * [TopdlLibrary#AttributeClass Attribute] (0 or more)
  * [TopdlLibrary#InterfaceClass Interface] (0 or more)
    * [TopdlLibrary#CapacutyClass Capacity] (0 or 1)
    * [TopdlLibrary#LatencyClass Latency] (0 or 1)
    * [TopdlLibrary#AttributeClass Attribute] (0 or more)
  * [TopdlLibrary#ServiceClass Service] (0 or more)
    * [TopdlLibrary#ServiceParamClass !ServiceParam] (0 or more)

Most classes can have attributes attached to them that contain extention attributes that different applications make use of.  The presence or absence of an optional nested object does not mean the presence or absence of the thing it describes, but that the description is unbound.  As [TopDl#Representation we have seen] a Computer without a nested CPU object does not describe a computer without a CPU, but a computer with no constraints on the CPU.
  
=== Adaptive Constructors ===

The constructors of the various topdl classes will accept contained classes in several formats.  They will always accept an object of the relevant class.  They will also take a dict that maps constructor parameters to values for the given class.  Examples make this clearer than description.

The following calls to create a CPU object are equivalent:

{{{
cpu = topdl.CPU(type='Intel', attribute=[topdl.Attribute(attribute='clock_rate', value='1Ghz')])
}}}

and

{{{
cpu = topdl.CPU(type='Intel', attribute=[{'attribute': 'clock_rate', 'value': '1Ghz'}])
}}}


and 

{{{
cpu = topdl.CPU(**{'type': 'Intel', 'attribute' : [ {'attribute': 'clock_rate', 'value': '1Ghz'}]})
}}}

That is, inner class parameters can be specified using dicts directly, and a top level dict can user the standard ** operator.

All topdl class constructors take the member names as parameters, and should be called using named parameters, as above.  The only exceptions to this is the [TopdlLibrary#SubstrateClass Substrate object]'s {{{interfaces}}} member and the [TopdlLibrary#InterfaceClass Interface object]'s {{{subs}}} member.  An Interface's {{{element}}} member can be specified, but generally is not.

=== Topdl Class Features ===

All topdl classes implement the following methods:

 {{{set_attribute(self, key, value)}}}::
   Attach an user-defined attribute with the given key and value to this object; value and key are strings.  The method includes a key/value pair (encoded as an [TopdlLibrary#AttributeClass attribute object]) into the {{{attribute}}} member of the object.  If an attribute with the given key exists, it is overwritten. 
   This method accesses the attribute member of the given object, not any named members. Objects that do not have an {{{attribute}}} member silently discard the operation.
 {{{get_attribute(self, key)}}}::
   Return the attribute of this object with the given key.  The attribute - a string - is returned.  If no such attribute exists, or the object has no {{{attribute}}} member, return None.
 {{{remove_attribute(self, key)}}}::
   If an attribute with the given key exists, remove it.
 {{{clone(self)}}}::
   Returns a deep copy of this object.
 {{{to_dict(self)}}}::
   Return the object as a python dict.  No copies are made, so make modifications with caution.
 {{{to_xml(self)}}}::
   Return the XML encoding (from the topdl language) of this object (and nested objects)

=== Topology Class ===

This class holds an enitre topology.  The utility functions that read and write files operate on it, and it is generally the unit of I/O for the library.

It has the following members:

 {{{substrates}}}::
   A list of [TopdlLibrary#SubstrateClass Substrate objects] that contain the substrates of the topology.  The list may be empty indicating no communcations are possible.
 {{{elements}}}::
   A list of entities make up the network. These can be of any [TopdlLibrary#ComputerClass Computer objects], [TopdlLibrary#TestbedClass Testbed objects], [TopdlLibrary#SegmentClass Segment objects], or [TopdlLibrary#OtherClass Other objects]. The list may be empty indicating no elements are in the topology.
 {{{attribute}}}:::
   A list of [TopdlLibrary#AttributeClass Attribute objects] that are attached to the machine.  The list may be empty.
 {{{version}}}::
   The version of topdl that this object encodes.  

As usual, the constructor takes the member names as parameter names.

One useful specialized member is:

{{{
incorporate_elements(self)
}}}

That member sanity checks the substrates to make sure they are uniquely named, assigns names to any interfaces without them, and connects the {{{element}}} members of the [TopdlLibrary#InterfaceClass Interface objects].  If there are inconsistencies, it raises a {{{topdl.ConsistencyError}}} with more detail in the message.

{{{incorporate_elements()}}} is called by the {{{Topology}}} constructor as well and that constructor may throw the same exceptions.

=== Computer Class ===

A Computer object is a programmable computer in a topology.  A webserver is well represented by a computer, as is a desktop.  
It has the following members:

 {{{name}}}::
   The unique name of the computer in the topology, a string.  Other names may be assigned in the {{{localname}}} attribute.
 {{{cpu}}}::
   A list of [TopdlLibrary#CPUClass CPU objects] that describe the CPUs of the machine.  The list may be empty, meaning no CPU requirements or information are present.
 {{{software}}}::
   A list of [TopdlLibrary#SoftwareClass Software objects] that describe the software to install.  The list may be empty, meaning no software is to be installed.
 {{{storage}}}::
    A list of [TopdlLibrary#StorageClass Storage objects] that describe the storage present the machine.  Storage objects can describe memory or persistent storage requirements.  The list may be empty, meaning no storage requirements or information are present.
 {{{interface}}}::
   A list of [TopdlLibrary#InterfaceClass Interface objects] that describe connections of the Computer to communications substrates.  The list may be empty, meaning that the computer is not connected to any substrates.
 {{{attribute}}}::
   A list of [TopdlLibrary#AttributeClass Attribute objects] that are attached to the machine.  The list may be empty.
 {{{localname}}}::
   A list of strings giving alternative names for the object.  When a machine is allocated to a testbed the DNS names assigned to the machine are given as {{{localname}}}s.  The list may be empty, meaning no such alternatives exist.
 {{{status}}}::
   A string indicating the current status of the Computer (if any).  It may also be None, indicating no status is reported.  Valid values are
     * "empty" - no allocation of resources has been attempted - generally not applied to Computers
     * "active" - Computer is functioning as part of an experiment
     * "inactive" - Computer is allocated, but not functioning
     * "starting" - Computer is transitioning from "inactive" to "active"
     * "terminating" - Computer is transitioning from "active" to "inactive"
     * "failed" - an allocation of resources has failed - generally not applied to Computers
 {{{service}}}::
   A list of [TopdlLibrary#ServiceClass Service objects] that describe the services this machine uses or supplies.  These are generally used by the federation system, and their definition is somewhat in flux. The list may be empty.
 {{{operation}}}::
   A list of strings that describe the valid operations on the machine.  These are generally used by the federation system, and their definition is somewhat in flux.The list may be empty.

The constructor takes the member names as parameters.  Only {{{name}}} is required.

A {{{Computer}}} should only be an element of one {{{Topology}}}.  Use {{{clone()}}} to avoid this.

=== Substrate Class ===

The {{{Substrate}}} object represents a communication substrate.  The elements that have interfaces connected to a substrate can communicate with one another.  A {{{Substrate}}} may include default parameters for the latency and capacity of the communication channel.

The members are:

 {{{name}}}::
   A string, the unique name of the substrate.
 {{{capacity}}}::
   A [TopdlLibrary#CapacityClass Capacity object] that gives the default capacity of the substrate.  It may be None, incdicating that there is no default.
 {{{latency}}}::
   A [TopdlLibrary#LatencyClass Latency object] that gives the default latency of the substrate.  It may be None, incdicating that there is no default.
 {{{attribute}}}::
   A list of [TopdlLibrary#AttributeClass Attribute objects] that are attached to the substrate.  The list may be empty.
 {{{localname}}}::
   A list of strings giving alternative names for the object.  The list may be empty, meaning no such alternatives exist.
 {{{status}}}::
   A string indicating the current status of the Substrate (if any).  It may also be None, indicating no status is reported. Valid values are
     * "empty" - no allocation of resources has been attempted - generally not applied to Substrates
     * "active" - Substrate is functioning as part of an experiment
     * "inactive" - Substrate is allocated, but not functioning
     * "starting" - Substrate is transitioning from "inactive" to "active"
     * "terminating" - Substrate is transitioning from "active" to "inactive"
     * "failed" - an allocation of resources has failed - generally not applied to Substrates
 {{{service}}}::
   A list of [TopdlLibrary#ServiceClass Service objects] that describe the services this substrate uses or supplies.  These are generally used by the federation system, and their definition is somewhat in flux. The list may be empty.
 {{{operation}}}::
   A list of strings that describe the valid operations on the substrate.  These are generally used by the federation system, and their definition is somewhat in flux.The list may be empty. 
 {{{interfaces}}}::
   A list of [TopdlLibrary#InterfaceClass Interface objects] connected to this substrate.  The list may be empty, meaning that the substrate is not connected to any elements. This field is maintained by the [TopdlLibrary#TopologyClass Topology object] holding this Substrate object.  '''It is not a parameter to the constructor.'''

Other than {{{interfaces}}}, the constructor takes the member names as parameters.  Only the {{{name}}} is required.

Because the enclosing {{{Topology}}} object maintains the {{{interfaces}}} member, it is dangerous to have the same {{{Substrate}}} object in more than one {{{Topology}}}.  Use the {{{clone()}}} method to avoid this.

=== Interface Class ===

This class defines the attachment of an entity, e.g., a [TopdlLibrary#ComputerClass Computer], to a [TopdlLibrary#SubstrateClass Substrate].  As with Substrates and entities, each Interface should be attached to only one entity and be unique across a [TopdlLibrary#Topology Topology].  An Interface has the following members:

 {{{substrate}}}::
  A list of strings, each naming a substrate to which this interface is connected.  The list must have at least one entry.
 {{{capacity}}}::
  A [TopdlLibrary#CapacityObject Capacity object] indicating the communications capacity of the interface.  It may be None to use the substrate default.
 {{{latency}}}::
  A [TopdlLibrary#LatencyObject Latency object] indicating the communications delay of the interface.  It may be None to use the substrate default.
 {{{attribute}}}::
   A list of [TopdlLibrary#AttributeClass Attribute objects] that are attached to the interface.  The list may be empty.
 {{{element}}}::
   The entity to which this interface is attached.  It may be a [TopdlLibrary#ComputerClass Computer object], [TopdlLibrary#TestbedClass Testbed object], [TopdlLibrary#SegmentClass Segment object], or [TopdlLibrary#OtherClass Other object].  This is generally managed by the {{{Topology}}} holding the entity and substrate with which the {{{Interface}}} is attached.  It is wise to treat the field as read-only and valid only after a {{{Topology}}} has called {{{incorporate_elements()}}}.
 {{{subs}}}::
   A list of [TopdlLibrary#SubstrateClass Substrates] to which the interface is attached.  It '''cannot ''' be set through the constructor. This is generally managed by the {{{Topology}}} holding the entity and substrate with which the {{{Interface}}} is attached.  It is wise to treat the field as read-only and valid only after a {{{Topology}}} has called {{{incorporate_elements()}}}.

The constructor takes all the member names '''except subs''' as parameters.  Most programs will not need to use {{{element}}} as a parameter to the constructor, either.

=== Attribute Class ===

An attribute is a user-defined annotation of one of the other classes: a key/value pair.  An attribute has the following members:

 {{{attribute}}}::
  A string, the key used to find the attribute.
 {{{value}}}::
  A string, the value

While attributes can be manipulated directly liek any of the other topdl classes, it is most common to use the [TopdlLibrary#TopdlClassFeatures attribute manipulation functions] of the annotated class to access them.

Should a program need to call the constructor, it takes the member names as named parameters.

=== CPU Class ===

This represents a CPU on a computer.  It has the following members:

 {{{type}}}::
  A string indicating the type of CPU.  These are not yet standardized, but common formats like "i686" or "amd64" are used.
 {{{attribute}}}::
   A list of [TopdlLibrary#AttributeClass Attribute objects] that are attached to the CPU.  The list may be empty.

The constructor takes the member names as named parameters.

=== !OperatingSystem Class ===

This class captures the operating system requirements of a computer.  It has the following members:

 {{{name}}}::
  A string naming the operating system required.  This is usually the operating system itself, e.g., Linux, FreeBSD, Windows XP.  This may be None, indicating no preference or information.
 {{{version}}}::
  A string identifying the version of the OS.  For integrates systems like FreeBSD this is the system version.  For distribution-based systems like Linux, this is the kernel version.  See below for distribution versioning.  This may be None, indicating no version information or preference.
 {{{distribution}}}::
  A string that for operating systems that characterize OS distribution independently of kernel or base system, names the distribution in use, e.g., Ubuntu. This may be None, indicating no information or preference.
 {{{distributionversion}}}::
  A string that for operating systems that characterize OS distribution independently of kernel or base system, names the version of the distribution in use, e.g., 12.04 for Ubuntu. This may be None, indicating no information or preference.
 {{{attribute}}}::
   A list of [TopdlLibrary#AttributeClass Attribute objects] that are attached to the operating system.  The list may be empty.


The constructor takes the member names as parameters.  Note that all the members may be None, but an empty {{{!OperatingSystem}}} object is not very useful.  The flexibility is there to allow specifying an operating system version without a distribution or a distribution without a base version.

=== Storage Class ===

The storage requirements or information for a system.  This class characterizes memory or permanent storage.  Its members are:

 {{{amount}}}::
  A floating point value indicating the number of megabytes of storage being considered.  Values less than 0.000001 are treated as 0.
 {{{persistence}}::
  A boolean value indicating whether this is persistent storage, such as a disk or SSD, or volitile such as memory.  
 {{{attribute}}}::
   A list of [TopdlLibrary#AttributeClass Attribute objects] that are attached to the machine.  The list may be empty.

Currently there is no field for media type.  Use attributes.

The constructor takes the member names as named parameters.

=== Software Class ===

This indicates what software must be installed on a system.  Its members are:

 {{{location}}}::
  A string containing a URI from which to install the software, e.g., a file: URI pointing to a tarfile.
 {{{install}}}::
   A string containing location in the local file system to base the software install.  This may be None, indicating that the software distribution format includes this information, e.g., an rpm or debian software file.

The constructor takes the member names as named parameters.

=== Service Class ===

A service class encapsulates the ideas of a [FeddAbout#ExperimentServices DETER federation service].  Such services can be available from or used by a variety of topology elements.  The Service class has the following members:

 {{{name}}}::
  The name of the service.  A few [FeddAbout#ExperimentServices are defined].
 {{{importer}}}::
  A list of entity names that are using or requesting the service.  This list may be empty.
 {{{param}}}::
  A list of [TopdlLibrary#ServiceParamClass ServiceParam objects] that represent typed parameters to the service.  The list may be empty.
 {{{description}}}::
  A string describing the service.  This may be None.
 {{{status}}}::
  A string describing the current status of the service (if any).  It may also be None, indicating no status is reported. Valid values are
     * "empty" - no allocation of resources has been attempted - generally not applied to Services
     * "active" - Service is functioning as part of an experiment
     * "inactive" - Service is allocated, but not functioning
     * "starting" - Service is transitioning from "inactive" to "active"
     * "terminating" - Service is transitioning from "active" to "inactive"
     * "failed" - an allocation of resources has failed - generally not applied to Services

The constructor will take the member names as named parameters.

=== !ServiceParam Class ===

A description of the type of parameter to a [TopdlLibrary#ServiceClass Service] will accept.  It has the following members:

 {{{name}}}::
   Name of the parameter, a string.
 {{{type}}}::
   A string containing the type of the parameter.  Valid types are:
    * string
    * int
    * float

The constructor will take the member names as named parameters.

=== Testbed Class ===

This class represents a testbed that has allocated resources or from which an allocation is requested.  While it is primarily used internally by the federation system, some other applications include a {{{Testbed}}} as a place to attach attributes that pertain to the whole topology.

It includes the following members:

 {{{uri}}}::
  A string containing the URI on which the testbed can be contacted.  Usually a fedd URI.
 {{{type}}::
  A string indicating the underlying type of the testbed.  Values include "deter," "emulab," and "protoGENI".
 {{{interface}}}::
   A list of [TopdlLibrary#InterfaceClass Interface objects] that describe connections of the Testbed to communications substrates.  The list may be empty, meaning that the testbed is not connected to any substrates.
 {{{attribute}}}::
   A list of [TopdlLibrary#AttributeClass Attribute objects] that are attached to the Testbed.  The list may be empty.
 {{{localname}}}::
   A list of strings giving alternative names for the testbed. The list may be empty, meaning no such alternatives exist.
 {{{status}}}::
   A string indicating the current status of the Testbed (if any).  It may also be None, indicating no status is reported.  Valid values are
     * "empty" - no allocation of resources has been attempted - generally not applied to Testbeds
     * "active" - Testbed is functioning as part of an experiment
     * "inactive" - Testbed is allocated, but not functioning
     * "starting" - Testbed is transitioning from "inactive" to "active"
     * "terminating" - Testbed is transitioning from "active" to "inactive"
     * "failed" - an allocation of resources has failed - generally not applied to Testbed
 {{{service}}}::
   A list of [TopdlLibrary#ServiceClass Service objects] that describe the services this Testbed uses or supplies.  These are generally used by the federation system, and their definition is somewhat in flux. The list may be empty.
 {{{operation}}}::
   A list of strings that describe the valid operations on the Testbed.  These are generally used by the federation system, and their definition is somewhat in flux.The list may be empty.

The constructor takes the member names as named parameters.

=== Segment Class ===

This class represents a sub-experiment of a federated experiment about which the details are unknown except for its interfaces.  It is primarily used internally by teh federation system.

It has the following members:

 {{{id}}}::
  An [TopdlLibrary#IDClass ID object] that identifies this segment.
 {{{type}}}::
  A string indicating the underlying type of the testbed.  Values include "deter," "emulab," and "protoGENI".
 {{{interface}}}::
   A list of [TopdlLibrary#InterfaceClass Interface objects] that describe connections of the segment to communications substrates.  The list may be empty, meaning that the segment is not connected to any substrates.
 {{{attribute}}}::
   A list of [TopdlLibrary#AttributeClass Attribute objects] that are attached to the segment.  The list may be empty.

=== ID Class ===

This class represents a polymorphic identifier as used by the federation system.  At most one of its members is set.

Those members are:

 {{{fedid}}}::
   A fedid object, representing the hash of a principal key.  This represents a [FeddAbout#GlobalIdentifiers:Fedids fedid].
 {{{uuid}}}::
   A string holding a UUID URI.
 {{{uri}}}::
   A string holding a generic URI
 {{{localname}}}::
   A string holding a locally scoped name
 {{{kerberosUsername}}::
   A kerberos user name.

The constructor accepts the member names as named parameters.

=== Other Class ===

This class represents a topology element that is not captured above.  It is a set of interfaces and a place to hang attributes, for use encoding kinds of elements unknown when topdl was specified.

The members are:

 {{{interface}}}::
   A list of [TopdlLibrary#InterfaceClass Interface objects] that describe connections of the element to communications substrates.  The list may be empty, meaning that the element is not connected to any substrates.
 {{{attribute}}}::
   A list of [TopdlLibrary#AttributeClass Attribute objects] that are attached to the element.  The list may be empty.

The constructor accepts the member names as named parameters.

= Examples =