Open Source Linux Distributed Computing Software

Zipkin is a distributed tracing system. It helps gather timing data needed to troubleshoot latency problems in service architectures. Features include both the collection and lookup of this data. If you have a trace ID in a log file, you can jump directly to it. Otherwise, you can query based on attributes such as service, operation name, tags and duration. Some interesting data will be summarized for you, such as the percentage of time spent in a service, and whether or not operations failed. The Zipkin UI also presents a Dependency diagram showing how many traced requests went through each application. This can be helpful for identifying aggregate behavior including error paths or calls to deprecated services. Applications need to be “instrumented” to report trace data to Zipkin.

SocketTest - powerful and small software tool for socket testing. It can create both TCP and UDP client or server. It can be used to test any server or client that uses TCP or UDP protocol to communicate.

Datahike is a durable Datalog database powered by an efficient Datalog query engine. This project started as a port of DataScript to the hitchhiker-tree. All DataScript tests are passing, but we are still working on the internals. Having said this we consider Datahike usable for medium sized projects, since DataScript is very mature and deployed in many applications and the hitchhiker-tree implementation is heavily tested through generative testing. We are building on the two projects and the storage backends for the hitchhiker-tree through konserve. We would like to hear experience reports and are happy if you join us. We provide a small stable API for the JVM at the moment, but the on-disk schema is not fixed yet. We will provide a migration guide until we have reached a stable on-disk schema. Take a look at the ChangeLog before upgrading.

JPPF makes it easy to parallelize computationally intensive tasks and execute them on a Grid.

Joone is a neural net framework written in Java(tm). It's composed by a core engine, a GUI editor and a distributed training environment and can be extended by writing new modules to implement new algorithms or architectures starting from base component

A Java middleware API supporting broker-less, type+topic-based publish/subscribe and request/reply messaging for both intra- and inter-application, object-level communication. Broker-less: messages are transmitted directly between eBus applications. There is no message broker (i.e. server) in between. Type+topic addressing: eBus clients subscribe to a message class (type) and message topic, providing stronger typing than just topic-based subscriptions. Object-level communication: eBus delivers messages directly to client objects, not to a client process.

Project continued at github, see https://github.com/wolfc01/procexp/blob/master/README.md Graphical process explorer for Linux. Shows process information: process tree, TCP IP connections and graphical performance figures for processes. Aims to mimic Windows procexp from sysinternals, and aims to be more usable than top and ps, especially for advanced users. Audience for this tool: * Advanced system administrators trying to analyze on process level what is going on in a production server; * Software developers analyzing the troughput of their process: e.g. TCP throughput, memory usage, memory leaks; The most advanced feature is the monitoring of TCP/IP traffic figures, for each SINGLE process (actual connections and throughput). As far as known no other tool has this capability. In the process tree, use the right mouse button to monitor process details.

MX4J is an Open Source implementation of the Java Management Extensions technology, for both JSR 3 (JMX) and JSR 160 (JMX Remote API).

An extensible framework for distributed computing on P2P grids. We support peaceful free and open research and build an internet supercomputer. We render movies, solve Eternity puzzles, predict climate and improve a ~30 GHz cluster of clients.

DMUCS works with distcc to allow multiple users to share a gcc compilation farm, while guaranteeing that each compilation will be assigned to the highest-performing, available cpu in the compilation farm.

MARIE is a robotic development and integration environment focused on software reusability and exploitation of already available APIs and middlewares used in robotics. Its main purpose is to create a rapid-prototyping approach to robotics developments.

A general purpose implementation of the xmlrpc specification in C.

CloudI is an open-source private cloud computing framework for efficient, secure, and internal data processing. CloudI provides scaling for previously unscalable source code with efficient fault-tolerant execution of ATS, C/C++, Erlang/Elixir, Go, Haskell, Java, JavaScript/node.js, OCaml, Perl, PHP, Python, Ruby, or Rust services. The bare essentials for efficient fault-tolerant processing on a cloud!

MARF is a general cross-platform framework with a collection of algorithms for audio (voice, speech, and sound) and natural language text analysis and recognition along with sample applications (identification, NLP, etc.) of its use, implemented in Java.

OSE is a C++ library, with some Python wrappers, containing generic classes, as well as support for event driven systems, interprocess communications and a request/reply, publish/subscribe service agent framework with RPC over HTTP interface.

RedDwarf is a horizontally scalable application server for low latency apps such as online games, virtual worlds, and social networking applications. It is the official community fork for Project Darkstar, previously sponsored by Sun Microsystems.

The TclSOAP package provides a mechanism to bind Tcl command procedures to remote procedure calls using the Simple Object Access Protocol (SOAP) and XML-RPC over HTTP. Both client and server code is provided.

FLoM is a free open source distributed lock manager that can be used to synchronize shell commands, scripts and custom developed software. When used in a shell environment, FLoM manages process synchronization in the same way that "nice" manages process prioritization. Shell commands, scripts and custom programs can be synchronized inside a single system or in a network of IP connected systems. FLoM supports different types of abstract resources: simple, numeric, set, hierarchical and transactional sequence. libflom, the library shipped with FLoM, allows you to develop your own application using a simple C, C++, Java, Perl, PHP or Python client API. SSL/TLS security for network communication, X.509 certificates for peer to peer mutual authentication.

Project provides a set of concurrent building blocks (Java & C/C++) that can be used to develop parallel/multi-threaded applications. Components are grouped into 4 categories: 1.Data Structures 2. Parallel Patterns 3.Parallel functions 4.Atomics and STM

VJDBC (Virtual JDBC) is a JDBC type 3 driver which provides efficient tunneling of JDBC commands/results over different communication protocols (RMI, HTTP, HTTPS ...).

Fully integrated framework for building and deploying service-based networks (SOA / Microservices). Communication includes XML-RPC, REST and Web Services. Other features include service protection, admin scripts, metadata, query processes, autonomous, agent-based, or self-organising. Special to the system are its AI and text-processing features. In particular, there is an Autonomic Manager framework with a BPEL-style execution script. Sub-packages include text or query processing and a problem solver provides a range of algorithms. These can also be used separately, but advanced programming and configuration is required. A free All-in-One GUI allows you to manage services and function, and add your own modules. Example Apps are included and the images illustrate the functionality. The open source p2p server requires only the HTTP address and optional passwords. Small jar file sizes and mobile compatible.

IIOP.NET allows a seamless interoperation between .NET, CORBA and J2EE distributed objects. This is done by incorporating CORBA/IIOP support into .NET, leveraging the remoting framework.

FRODO is a Java platform to solve Distributed Constraint Satisfaction Problems (DisCSPs) and Optimization Problems (DCOPs). It provides implementations for a variety of algorithms, including DPOP (and its variants), ADOPT, SynchBB, DSA...

A modern secure, robust, multithreaded, exception aware, internationalisable, portable GUI toolkit library designed for mission-critical work in C++ and Python forked from the FOX library. Replicates the Qt API in many places.

Atomatrix is a multithreaded execution system of atomic matrices with a dynamic compiler and a network eviron. It is the next generation object-orientated environment. XML, MUD-OO CLI, GnucDNA support, windows and many subprojects are built-in.