Converting Descriptions in SOAP to Python data structures

The SOAP/xsd descriptions of fedd interfaces are portable across many programming languages and architectures, each with their own take on how to represent them internally. Because so much fedd code is written in python using the XSI package to encode and decode SOAP, we have placed some nodes on how to do the conversions.

The general layous of the files is that the message types are all in fedd_types.xsd​ while fedd.wsdl​ contains the SOAP boilerplate to turn it all into request/response messages. fedd_internal.wsdl​ is some internal fedd interfaces and topdl.xsd​ is the XSD defintions for topdl, fedd's topology description language.

Though the full details are described by the W3C in ​two ​documents, for fedd's purposes, the usage is simple enough. Messages are built up from simple XSD types composed into complex types. An element has a name and is part of a complex type and may either have a simple or complex type. Simple types in use by fedd include:

• ​xsd:int: an integer value
• ​xsd:string: a string value
• ​xsd:boolean: a boolean value
• ​xsd:base64Binary: an opaque binary value
• ​xsd:dateTime: an encoded date and time
• ​xsd:float: a floating point value
• ​xsd:double: a double precision floating point value

Each of these is encoded into python in the obvious ways. Doubles, floats and integers are python numbers; strings and base64Binaries are python strings; booleans are python variables set to True or False; and dateTimes are integers.

As simple element is encoded as a dictionary mapping between name and value. All the elements in the same sequence (an xsd grouping) are in the same dictionary. So, for example, the vtopolanType and vtoponodeType complex types (from wsdl/fedd_types.xsd​) are described in xsd as:

  <xsd:complexType name="visnodeType">
    <xsd:annotation>
      <xsd:documentation>
        Node in the visualization of a federated experiment (Emulab
        legacy).  Fields include the local hostname of the node, x,y
        coordinates in a 2-dimensional representation, and whether the
        node in the visualization is a host or a LAN.
      </xsd:documentation>
    </xsd:annotation>
    <xsd:sequence>
      <xsd:element name="name" type="xsd:string"/>
      <xsd:element name="x" type="xsd:int"/>
      <xsd:element name="y" type="xsd:int"/>
      <xsd:element name="type" type="xsd:string"/>
    </xsd:sequence>
  </xsd:complexType>

  <xsd:complexType name="vtopolanType">
    <xsd:annotation>
      <xsd:documentation>
        LAN in the virtual topology of a federated experiment (Emulab legacy).
        The fields are the name of the LAN/link (vname) the node that
        this description applies to (vnode), the IP of the connection,
        and performance information.
      </xsd:documentation>
    </xsd:annotation>
    <xsd:sequence>
      <xsd:element name="vname" type="xsd:string"/>
      <xsd:element name="vnode" type="xsd:string"/>
      <xsd:element name="ip" type="xsd:string"/>
      <xsd:element name="bandwidth" type="xsd:int"/>
      <xsd:element name="delay" type="xsd:float"/>
      <xsd:element name="member" type="xsd:string"/>
    </xsd:sequence>
  </xsd:complexType>

translate into python dicts like:

vtoponodeType = {
  'vnode': 'name',
  'ips': '10.1.1.1,10.1.2.1',
}

vtopolanType = {
  'vname': 'string',
  'vnode': 'string2',
  'ip': '10.1.1.1',
  'bandwidth': 100,
  'delay': 0.35,
  'member': 'member_string',
}

A complex type that includes another element that is a sequence simply includes a dict keyed by the element name.

Some elements are repeated multiple times in a type, or are optional, as noted by the maxOccurs and minOccurs attributes. Elements that have maxOccurs="1" and minOccurs="0" defined in the XSD are optional fields and may either occur in the sequence/dict or not. Elements with maxOccurs="unbounded" and minOccurs="0" defined in the XSD are repeated elements. If none appear, the key is missing from the dict, if one or more are present the value mapped to the element name in the dict is a list of hashes, one for each copy of the repeatable element present.

For example, the vtopoType from fedd_types.xsd​ includes a pair of repeated elements. It is defined in XSD as:

  <xsd:complexType name="vtopoType">
    <xsd:annotation>
      <xsd:documentation>
        The virtual topology of a federated experiment (Emulab legacy).
      </xsd:documentation>
    </xsd:annotation>
    <xsd:sequence>
      <xsd:element name="node" type="tns:vtoponodeType" minOccurs="0" 
        maxOccurs="unbounded" />
      <xsd:element name="lan" type="tns:vtopolanType" minOccurs="0"
        maxOccurs="unbounded"/>
    </xsd:sequence>
  </xsd:complexType>

An example of that datatype that only includes 2 nodes encoded as a dict is:

vtopoType = {
   "node": [ { 'name' : 'name1', 'ips': '10.1.1.1', }, { 'name': 'name2', 'ips': '10.1.2.1', }, ]
}

Some types are defined not as a sequence, but as a choice. For example, an IDType is defined (in fedd_types.xsd​ as the rest of these examples are) as:

  <xsd:complexType name="IDType">
    <xsd:annotation>
      <xsd:documentation>
        An ID is an identifier for a principal, service, or object.  This type
        is currently polymorphic o allow different implementations of type,
        though running code primarily uses localnames and fedids.
      </xsd:documentation>
    </xsd:annotation>
    <xsd:choice>
      <xsd:element name="uuid" type="xsd:base64Binary"/>
      <xsd:element name="fedid" type="xsd:base64Binary"/>
      <xsd:element name="uri" type="xsd:string"/>
      <xsd:element name="localname" type="xsd:string"/>
      <xsd:element name="kerberosUsername" type="xsd:string"/>
    </xsd:choice>
  </xsd:complexType>

When represented as a dict, that will dict will have only one key, chosen from the set { 'uuid', 'fedid', 'uri', 'localname', 'kerberosUsername' }.

Others include restricted values, for example the statusType is defined in fedd_types.xsd​:

  <xsd:simpleType name="statusType">
    <xsd:annotation>
      <xsd:documentation>
        The current state of the experiment.
      </xsd:documentation>
    </xsd:annotation>
    <xsd:restriction base="xsd:string">
      <xsd:enumeration value="empty"/>
      <xsd:enumeration value="active"/>
      <xsd:enumeration value="starting"/>
      <xsd:enumeration value="terminating"/>
      <xsd:enumeration value="failed"/>
    </xsd:restriction>
  </xsd:simpleType>

When instantiated as a dict, the values are constrained to be from the set { 'empty', 'active', 'starting', 'terminating', 'failed' }, and if another value is encoded, it results in an error.

With those rules is should be straightforward to convert from the XSD descriptions to the python datatypes.