Welcome to Slimgrid wiki!

Slimgrid is a Java library for grid computations which is lighter than other ones (JPPF, Hadoop, ...). The main design goals are: minimalism, simplicity, pervasiveness. If you need to grab something which does not require you to comprehend massive and complex API's, do exhaustive configurations and installations, is robust and reliable, uses just one port for all management and communication, then SlimGrid is the right choice. The SlimGrid is built on top of the Apache's ZooKeeper library for coordination of distributed systems and Apache Curator awesome API simplification.

More info

• Architecture
• Installation instructions
• Hello World example

Simplicity

Here is an example how the Slimgrid can be used

Client side

// Create a client for the server
SgServer server = new SgServer("myServerIP:port", sessionTimeout, maxTaskDelay, scheduler);

//Submit a task and get the task ID which will identify task's result:
MyTask task = new MyTask( dataToCompute );
UUID taskId = server.submitTask(task, this);

// if need to submit a batch of tasks
List<UUID> taskIds = server.submitTask(taskBatch, this);

// Implement the task completion callback
public void processTaskResult(SgTaskResult<?> taskResult) {
    System.out.println( "Result of task ID " + res.getTaskId() +
                            " is: " + taskResult.getResult().toString());
}
// If no more tasks to compute, close the client:
server.close();

Code of the task

public class MyTask extends SgTask<MyInputData,MyResult> {
    public TaskHelloWorld(MyInputData data) {
        super(data);
    }

    @Override
    public void runTask() {     
        // get input data
        MyInputData input = this.getData();

        // do computations
        MyResult result = ...

        // send the result back
        this.setResult( result );
    }
}

Server configuration

Slimgrid uses as a server pure plain ZooKeeper server. Thus, there is nothing required to do more than to launch the ZooKeeper server. This is very straightforward. Basically unzip the file, configure the port and path for the server's data directory, launch.

Worker configuration

To configure worker, go to the congf directory and edit the node.properties file.

~~~~~~
# ------------------------------------
# Configuration file for the ZooKeeper's client as a computation node
## ------------------------------------

# ------------------------------------
# connection string in a form: server:port
server = localhost:2181

# ------------------------------------
# session timeout in miliseconds, if there no response for heart beats 
# from the server within the following time span, the server is considered be down
timeout = 30000

# ------------------------------------
# number of threads
threads = 4
~~~~~~

Robustness

Slimgrid is very robust, can easily detect node failure and automatically reschedule tasks. This feature makes it superior to JPPF which is unable to detect node's failure (unless very complicated configuration on both server and worker side, opening extra communication ports, ... hell of overhead).

Because the main server is a ZooKeeper server, user can very easily configure backup coordination servers which will take over if the primary server is down.

Pervasiveness

All what Slimgrid grid needs to run is just one open port on the coordination server. Client and all workers connect to this port to submit/receive tasks and results. In case there is a strict firewall, you con configure server to run on ports which are usually permitted (80, 8080, 8081). Communication is always established from client or worker to server. Server never initiates connection.

Many grid middlewares (such as JPPF) requires that all nodes (client, servers and workers) must have open at least one port. This can work in the small lab conditions but when you want to use computers out of the lab (at home, at work, at your girlfriend's parents, ...) it does not work due to firewalls and NAT.

Case studies

Particle Swarm Optimization

One of successful mission of the Slimgrid was to perform more than 140.000 tasks in the grid with 40+ computers located in three different countries (Norway, Germany, Canada) and behind various firewalls (University labs).

The goal of the project was to perform Particle Swarm Optimization of parameters of proposed self-organizing data collection protocol for Wireless Sensor Networks.

TEESer

Tool for Engineering Emergent Selforganizing bEhavioR
has provided a plugin to use Slimgrid to run various simulators in the grid.

UAV relay network

Used to perform thousands of simulations of a relay network maintained by UAVs in order to obtain statistically sound evidence of the algorithm performance as well as finding optimal parameters for the evaluated algorithm.

Project Members:

• Dalimir Orfanus (admin)