Warewolf Easy Service Bus (ESB) Datalist

---

Datalist is responsible for data manipulation and storage within Warewolf. The Datalist itself is made up of four primary components: Translators, Compiler, Evaluation and Storage. Each component is described below.

Storage:

When the Warewolf server starts up, memory is reserved for use by the datalist. Using this reservation, data is stored as a BinaryDataListRow which is a byte array representation of the data. The BinaryDataListRow is based off protocol buffer concept to convert object to and from the byte array. At present the datalist only stores string data, i.e. all data in the system at a storage level is considered to be string. Although all data at the storage layer is string or character based the storage does understand NULL and during retrieval of data for a variable if the value is NULL an error occurs and that point in the execution of the activity or service. BinaryDataListEntry is the essentially an access point for the BinaryDataListRow as well as have extended information about the entry. A BinaryDataList is comprised of BinaryDataListEntries.

Translators:

Datalist translators are used to transform the data stored within the datalist into a certain format or from a certain format into the datalist, e.g. XML to datalist, data table to datalist or datalist to JSON. Translators are also used to transform data that comes into the Warewolf server and data that is emitted from the Warewolf server. Makes use of the storage layer to retrieve or store the values from the translation depending on which way the translation is happening. See StudioDataListXMLTranslator for implementation example.

Evaluation:

The evaluation component of the datalist is responsible to determining if a piece of data needs to be evaluated i.e. it is a variable and the data needs to be retrieved. Evaluation caters for recursive evaluation i.e. [[hello[[world]]]] would be recursively evaluated as [[hellobob]] where [[world]] had the value of “bob” and then [[hellobob]] as “my name is bob” where [[hellobob]] had the value “my name is bob” stored in it.

Compiler:

The compiler is where everything comes together, by using the evaluation and translator’s data is mapped and moved around in Warewolf. The ServerDataListCompiler is in charge for merging and shaping data based on the actions that are being performed. It is the access point to perform all data manipulations that are required by Warewolf. It is heavily used by sub-execution (execution a service that maps one variable to another) to ensure that the right data is inputted and outputted. It is also where data for debug is created.

Mapping:

Mapping in Warewolf refers to how variables related to each between services and from which variables data will retrieved and into which variables this data will be stored. Therefore variables have a ColumnIODirection which can be set as Input, Output or Both which allows the limiting of which variables can accept data and which variables emit data between services. The ColumnIODirection of a variable is set through the Studio variable list.