Menu ▾ ▴

KnowledgeContainers

James Edmondson

C++ Guide Series
Architecture | Knowledge Base | Networking | Containers | Threads | Optimizations | KaRL | Encryption | Checkpointing

MADARA Containers

The MADARA KaRL scripting environment is fast and efficient, but it can also be difficult to debug and work with for programmers more familiar with object-oriented programming in their host languages. To address the needs of object-oriented developers, MADARA also provides a set of containers which present facades into the Knowledge Base and mimic the look and feel of STL containers. These object-oriented containers are meant to be user-friendly and intuitive, but some guidance may be useful for users who do not know where they are and how they may be initialized.

1. Types of Containers

In this section, we breakdown the types of containers. For a complete list of containers, see the files located inside of the include/madara/knowledge_engine/containers directory of your source distribution, or see the Library documentation for the Containers namespace.

1.1 Instance Variables

Instance variables are facades into single variable locations within the Knowledge Base and are the most efficient of the MADARA containers. Values are set with a set function, available through the container, and retrieved using the asterisk operator (*) or with a to_integer (), to_double (), or to_string () function. MADARA currently supports the following instance variables:

1.1.1 Integer

Location: madara/knowledge/containers/Integer.h

Documentation: Integer

Integer is a container that provides interfaces for setting and getting an integer value from the Knowledge Base.

// setup includes and namespace alias for convenience
#include "madara/knowledge/containers/Integer.h"
namespace engine = madara::knowledge;
namespace containers = engine::containers;

// create a knowledge base and make a container that points inside it
engine::KnowledgeBase knowledge;
containers::Integer my_id (".id", knowledge);

// set id within the knowledge base to 1
my_id = 1;

...

// read the value later
std::cout << "My id is " << *my_id << std::endl;

1.1.2 Double

Location: madara/knowledge/containers/Double.h

Documentation: Double

Double is a container that provides interfaces for setting and getting a double value from the Knowledge Base.

// setup includes and namespace alias for convenience
#include "madara/knowledge/containers/Double.h"
namespace engine = madara::knowledge;
namespace containers = engine::containers;

// create a knowledge base and make a container that points inside it
engine::KnowledgeBase knowledge;
containers::Double funds (".funds", knowledge);

// set funds to 300.50
funds = 300.50;

...

// read the value later
std::cout << "My funds available are " << *funds << std::endl;

1.1.3 String

Location: madara/knowledge_engine/containers/String.h

Documentation: String

String is a container that provides interfaces for setting and getting a string value from the Knowledge Base.

// setup includes and namespace alias for convenience
#include "madara/knowledge/containers/Double.h"
namespace engine = madara::knowledge;
namespace containers = engine::containers;

// create a knowledge base and make a container that points inside it
engine::KnowledgeBase knowledge;
containers::String name (".name", knowledge);

// set name to "John Smith"
name = "John Smith";

...

// read the value later
std::cout << "My name is " << *name << std::endl;

1.2 Vectors/Arrays

Array containers are facades into arrays of variable locations within the Knowledge Base, and provide O(1) access times to these elements. Unlike with instance variables, the name that is set for an array container is the prefix for variable locations instead of an actual variable name. MADARA currently supports the following vector/array containers:

1.2.1 Integer Vector/Array

Location: madara/knowledge/containers/IntegerVector.h

Documentation: IntegerVector

IntegerVector, also known as IntegerArray, is a container that provides interfaces for setting and getting an integer value from the Knowledge Base.

// setup includes and namespace alias for convenience
#include "madara/knowledge/containers/Integer_Vector.h"
namespace engine = madara::knowledge;
namespace containers = engine::containers;

// create a knowledge base and make a 10 element array that points inside it
engine::KnowledgeBase knowledge;
containers::IntegerVector my_array (".array", knowledge, 10);

// set index 1 of the array to true (1)
my_array.set (1);

// set index 5 of the array to 30
my_array.set (5, 30);

// set index 0 of the array to 17
my_array.set (0, 17);

...

// read the values of index 0, 1, and 5
std::cout << "my_array[0] is " << my_array[0] << std::endl;
std::cout << "my_array[1] is " << my_array[1] << std::endl;
std::cout << "my_array[5] is " << my_array[5] << std::endl;

1.2.2 Double Vector/Array

Location: madara/knowledge/containers/DoubleVector.h

Documentation: DoubleVector

DoubleVector, also known as DoubleArray, is a container that provides interfaces for setting and getting an array of double values within the Knowledge Base.

// setup includes and namespace alias for convenience
#include "madara/knowledge/containers/DoubleVector.h"
namespace engine = madara::knowledge;
namespace containers = engine::containers;

// create a knowledge base and make a 10 element array that points inside it
engine::KnowledgeBase knowledge;
containers::DoubleArray my_array (".array", knowledge, 10);

// set index 1 of the array to 6.5
my_array.set (1, 6.5);

// set index 5 of the array to 31.3
my_array.set (5, 31.3);

// set index 0 of the array to 15.2
my_array.set (0, 15.2);

...

// read the values of index 0, 1, and 5
std::cout << "my_array[0] is " << my_array[0] << std::endl;
std::cout << "my_array[1] is " << my_array[1] << std::endl;
std::cout << "my_array[5] is " << my_array[5] << std::endl;

1.2.3 String Vector/Array

Location: madara/knowledge/containers/StringVector.h

Documentation: StringVector

StringVector, also known as StringArray, is a container that provides interfaces for setting and getting an array of string values within the Knowledge Base.

// setup includes and namespace alias for convenience
#include "madara/knowledge/containers/StringVector.h"
namespace engine = madara::knowledge;
namespace containers = engine::containers;

// create a knowledge base and make a 10 element array that points inside it
engine::KnowledgeBase knowledge;
containers::StringArray my_array (".array", knowledge, 10);

// set the indices of the array to characters in Snow White
my_array.set (0, "Snow White");
my_array.set (1, "Doc");
my_array.set (2, "Dopey");
my_array.set (3, "Bashful");
my_array.set (4, "Grumpy");
my_array.set (5, "Sneezy");
my_array.set (6, "Sleepy");
my_array.set (7, "Happy");
my_array.set (8, "Huntsman");
my_array.set (9, "Queen");

...

// print out all elements of the array
for (unsigned int i 0; i < 10; ++i)
  std::cout << "array[" << i << "] is " << my_array[i] << std::endl;

1.2.4 Dynamically-typed Vector/Array

Location: madara/knowledge/containers/Vector.h

Documentation: Vector

Vector, also known as Array, is a container that provides interfaces for setting and getting an array of multi-typed values within the Knowledge Base.

// setup includes and namespace alias for convenience
#include "madara/knowledge/containers/Vector.h"
namespace engine = madara::knowledge;
namespace containers = engine::containers;

// create a knowledge base and make a 10 element array that points inside it
engine::KnowledgeBase knowledge;
containers::Array my_array ("some_array", knowledge, 10);

// the array contains five characters from Snow White and their ages
my_array.set (0, "Snow White");
my_array.set (1, "Doc");
my_array.set (2, "Dopey");
my_array.set (3, "Bashful");
my_array.set (4, "Grumpy");
my_array.set (5, 18);
my_array.set (6, 70);
my_array.set (7, 74);
my_array.set (8, 72);
my_array.set (9, 68);

...

// print out the people and their ages
for (unsigned int i 0; i < 5; ++i)
  std::cout << my_array[i] << "'s age is " << my_array[i + 5] << std::endl;

1.3 Map

Map containers are facades into string-keyed maps of variable locations within the Knowledge Base. Unlike with instance variables, the name that is set for a map container is the prefix for variable locations instead of an actual variable name, and access times are O(log m), where m is the number of items existing in the Map container.

Location: madara/knowledge/containers/Map.h

Documentation: Map

The Map class is one of the most versatile container classes available in MADARA. This class is likely to prove invaluable to developers wanting to emulate classes within the knowledge base.

// setup includes and namespace alias for convenience
#include "madara/knowledge/containers/Map.h"
namespace engine = madara::knowledge;
namespace containers = engine::containers;
typedef  engine::KnowledgeRecord::Integer  Integer;

// create a knowledge base and make a map
engine::KnowledgeBase knowledge;
containers::Map map ("profile.1", knowledge);

// create profile fields for a man named John Smith at profile.1
map.set ("name", "John Smith");
map.set ("age", Integer (30));
map.set ("funds", 300.50);
map.set ("photo1", "/images/profile/1/headshot.jpg");

// print the profile
std::cout << "Name: " << map["name"] << std::endl;
std::cout << "Age: " << map["age"] << std::endl;
std::cout << "funds: " << map["funds"] << std::endl;
std::cout << "Photo: " << map["photo1"] << std::endl;

1.4 Flexible Map

Flexible Maps are abstractions intended to support multi-dimensional arrays and maps. These containers override the [] operator for both integer-based index and string-based accesses. These operators return other Flexible Maps which can either be set to an arbitrary value (such as an integer, string, double, byte buffer, etc.) or subindexed with integers or strings.

Location: madara/knowledge_engine/containers/FlexMap.h

Documentation: FlexMap

FlexMap replaces the old VectorN class which was less intuitive and useful. FlexMap is O(1) for subindexing (e.g., [1][2]) and only incurs an O(log n), with n being the number of keys in the knowledge base, lookup on the first access of an element with an accessor or mutator function (e.g. to_string() or setting the value explicitly). Flex_Map essentially always assumes an index is not a variable lookup until you are setting or getting a value explicitly. The keys function is an O(n) operation that must look through all keys in the Knowledge_Base to find all versions with the prefix set in constructor or set_name function.

Though the subindexing is O(1), developers should keep in mind that each index is a concatenation of the container name with a delimiter (by default '.') and the index, which is essentially 2 concatenation operations.

FlexMap includes a to_container operation that can convert the FlexMap into any supported container for fast O(1) changes later to String, Integer, Double, etc.

// setup includes and namespace alias for convenience
#include "madara/knowledge/containers/FlexMap.h"
#include "madara/knowledge/containers/String.h"
namespace engine = madara::knowledge;
namespace containers = engine::containers;

// create a knowledge base and make a multi-dimensional array
engine::KnowledgeBase knowledge;
containers::FlexMap my_array ("cities", knowledge);
containers::String city_name;

// set location cities.1.2.3.4.name to "St. Louis"
my_array[1][2][3][4]["name"] = "St. Louis";

// set location cities.4.2.3.4.name to "San Francisco"
my_array[4][2][3][4]["name"] = "San Francisco";

// output the value at [1][2][3][4]
std::cout << "My current hometown is " << my_array[1][2][3][4]["name"].to_string () << std::endl;

// output the value at [4][2][3][4]
std::cout << "My current hometown is " << my_array[4][2][3][4]["name"].to_string () << std::endl;

// Flex_Map can create containers to its elements for faster access
may_array[1][2][3][4]["name"].to_container (city_name);
city_name = "Ontario";

std::cout << "St. Louis has been replaced by " << *city_name << std::endl;

1.5 Counter

Counters are abstractions intended to support aggregate counters across multiple participating agents. Counters provide a solution to the problem of the many writers problem for single variable increments/decrements. What a Counter does under the hood is give each agent a copy of the variable, and the agent only increments/decrements/sets their versions of the variable. When another agent wants to know, the aggregate count, they call to_integer() or to_record().

Location: madara/knowledge/containers/Counter.h

Documentation: Counter

Updating the Counter takes O(1) time. Retrieving a value from the Counter requires O(N) time, where N is the number of agents participating in the counting operation.

// setup includes and namespace alias for convenience
#include <iostream>
#include "madara/knowledge/KnowledgeBase.h"
#include "madara/knowledge/containers/Counter.h"
namespace engine = madara::knowledge;
namespace containers = engine::containers;

// create a knowledge base and make a multi-dimensional array
engine::KnowledgeBase knowledge;
containers::Counter entries ("entries", knowledge);

// resize our entries counter to have 2 variables (2 agents),
// with our id as 0 (the other agent would be 1)
entries.resize(0, 2);

// add to the number of entries that you are maintaining
++entries;
entries += 2;

// other agents across the network may have updated their entry in "entries"
std::cerr << "Total entries in network is : " << entries.to_integer() << std::endl;

2. Interaction with the Transport Layer

When creating containers, each constructor supports a madara::knowledge::KnowledgeUpdateSettings instance that provides control over the updating and retrieving of variables from the context. However, no function in the container classes directly interacts with the transport layer.

Container classes are intended to provide fast, efficient, and convenient facades into the Knowledge Base to support object-oriented programming. In order to send these updates to other reasoning agents in the network, functions will need to be called on the Knowledge_Base. In this section of the Wiki, we'll discuss how the containers are intended to be used and when interactions over the network occur.

2.1 The send_modifieds function

If the developer does not intend to use the evaluate or wait function on the Knowledge Base and intends to explicitly use MADARA containers, then the main option for communicating changes over the network is with the send_modifieds function. After all relevant changes have been made through the container classes, simply call send_modifieds.

To showcase this concept, we'll use a complete program that sends a profile over multicast.

#include "madara/knowledge/KnowledgeBase.h"
#include "madara/knowledge/containers/Map.h"

// setup convenience aliases
namespace engine = madara::knowledge;
namespace containers = engine::containers;
typedef  engine::KnowledgeRecord::Integer  Integer;

int main (int argc, char ** argv)
{
  // setup a Multicast transport at 239.255.0.1:4150
  madara::transport::QoSTransportSettings settings;
  settings.hosts.push_back ("239.255.0.1:4150");
  settings.type = madara::transport::MULTICAST;

  // use the transport constructor 
  engine::KnowledgeBase knowledge ("", settings);

  // lock the context from any external updates until we're done
  knowledge.lock ();

  // make changes to a 
  containers::Map map ("profile.1", knowledge);

  // create profile fields for a man named John Smith at profile.1
  map.set ("name", "John Smith");
  map.set ("age", Integer (30));
  map.set ("funds", 300.50);
  map.set ("photo1", "/images/profile/1/headshot.jpg");

  // unlock the context to allow the network to update our context again
  knowledge.unlock ();

  /**
   * send the changes we've made to profile.1 over the network
   * These will show up as updates to profile.1.name, profile.1.age,
   * profile.1.funds, and profile.1.photo1
   */ 
  knowledge.send_modifieds ();

  return 0;
}

2.2 Pairing containers with function calls from evaluate or wait

The more recommended way to use containers is within functions called from an evaluate or wait statement. There are several reasons for doing this:

  • Functions called from an evaluate are atomic and the context is already locked from outside updates
  • An evaluate call, by default, sends all modified variables over the attached network transports
  • MADARA containers allow for attaching a context through either the Variables or KnowledgeBase classes, so there is nothing gained by using containers outside of function calls from evaluate

In the next example, we create a function called set_profile that is called from an evaluate statement within the main function.

#include "madara/knowledge/KnowledgeBase.h"
#include "madara/knowledge/containers/Map.h"
#include "madara/utility/Utility.h"

// setup convenience aliases
namespace engine = madara::knowledge;
namespace containers = engine::containers;
typedef  engine::KnowledgeRecord::Integer  Integer;

// a Map container to reference profile information
containers::Map profile;

engine::Knowledge_Record
set_profile (engine::FunctionArguments & args, engine::Variables & vars)
{
  Madara::Knowledge_Record result;

  // set profile information for John Smith
  profile.set ("name", "John Smith");
  profile.set ("age", Integer (30));
  profile.set ("funds", 300.50);
  profile.set ("photo1", "/images/profile/1/headshot.jpg");  

  return result;
}

int main (int argc, char ** argv)
{
  // setup a Multicast transport at 239.255.0.1:4150
  madara::transport::QoSTransportSettings settings;
  settings.hosts.push_back ("239.255.0.1:4150");
  settings.type = madara::transport::MULTICAST;

  // use the transport constructor 
  madara::knowledge::KnowledgeBase knowledge ("", settings);

  // attach the profile Map container to the knowledge base
  profile.set_name ("profile.1", knowledge);

  // define the set_profile function within the knowledge base
  knowledge.define_function ("set_profile", set_profile);

  // call the set_profile function and immediately send updates
  knowledge.evaluate ("set_profile ()");

  /**
   * In a real application, we would probably have a control loop
   * that the evaluate is called in until some condition is met.
   *
   * Here we just sleep for a few seconds to make sure the transport
   * is given enough time to send out all of the updates
   **/
  madara::utility::sleep (3);

  return 0;
}

More Information

If you are looking for threading examples, there are several tests in the SVN checkout that extend examples in this guide. Examples can be found in the tests/threads directory of your checkout.

If you are looking for code examples and guides, your best bet would be to start with the tutorials (located in the tutorials directory of the MADARA root directory--see the README.txt. After that, there are many dozens of tests that showcase and profile the many functions, classes, and functionalities available to MADARA users.

Users may also browse the Library Documentation for all MADARA functions, classes, etc. and the Wiki pages on this website.

Guide Series

Video Tutorials

First Steps in MADARA

Covers installation and a Hello World program.

Intro to Networking

Covers creating a multicast transport and coordination and image sharing between multiple agents.

Related

Wiki: Checkpointing
Wiki: Encryption
Wiki: Home
Wiki: InteractingWithTheKnowledgeBase
Wiki: InteractingWithTheTransport
Wiki: KarlLanguage
Wiki: KnowledgeContainers
Wiki: MadaraArchitecture
Wiki: OptimizingKaRL
Wiki: WhereToUseMadara
Wiki: WorkingWithThreads

Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.