Open Source Storage Software

Library and command line tools for XZ and LZMA compressed files

LZ4 is lossless compression algorithm, providing compression speed > 500 MB/s per core (>0.15 Bytes/cycle). It features an extremely fast decoder, with speed in multiple GB/s per core (~1 Byte/cycle). A high compression derivative, called LZ4_HC, is available, trading customizable CPU time for compression ratio. LZ4 library is provided as open-source software using a BSD license. This benchmark simulates simple "static content transfer" scenario such as OS Kernel compression or video game's static assets (text/images/tables/scripts/etc) which loading from Flash Memory / HDD / SSD. In this case, compression time is completely ignored. Because only content developers compress the data at once and usually they don't care about its computational cost. But they always care end user's experience a.k.a. "loading time" and bandwidth. Please pay attention to "LZ4HC -9" which is quite faster than other methods.

Zstandard is a fast compression algorithm, providing high compression ratios. It also offers a special mode for small data, called dictionary compression. The reference library offers a very wide range of speed / compression trade-off, and is backed by an extremely fast decoder (see benchmarks below). Zstandard library is provided as open source software using a BSD license. Its format is stable and published as IETF RFC 8478. The negative compression levels, specified with --fast=#, offer faster compression and decompression speed in exchange for some loss in compression ratio compared to level 1, as seen in the table above. Zstd can trade compression speed for stronger compression ratios. It is configurable by small increment. Decompression speed is preserved and remain roughly the same at all settings, a property shared by most LZ compression algorithms, such as zlib or lzma.

FileVerifier++ is a Windows utility for calculating hashes using a number of algorithms including CRC32, MD5, SHA-1, SHA-256/224/384/512, WHIRLPOOL, and RIPEMD-128/160/256/320. Supported hash file formats include MD5SUM .MD5, SFV, BSD CKSUM, and others.

Fsum Frontend is a files integrity checker. It can calculate 96 hash and checksum algorithms(CRC32, MD5, SHA1, SHA2, ADLER, DHA256, FORK256, ...). You can verify your files using a .sfv/.md5/.sha1/.sha2 file or create your own checksum file.

JDIFF is a program that outputs the differences between two binary files, either in binary format or in human readable format (detailed or summarized) and then allows to reconstruct the second file from the first one and the diff-file.

(Production / Stable) Command line tool to compute the 64 bit CRC of one or more files.

CADI is an implementation of the JPIP standard (JPEG2000 Interactive Protocol - Part 9). CADI includes a JPIP server, a JPIP proxy, and a JPIP Client. CADI is designed and programmed with the aim to provide a flexible framework to test and develop ne

minidjvu is a DjVu encoder for black-and-white images.

A slightly reduced strength bulk encryption. In exchange for the reduced strength, you get the ability to rsync the encrypted files, so that local changes in the plaintext file will result in (relatively) local changes to the cyphertext file.

Creates an index to DBF files using a B+ Tree; Cria um indice para um arquivo DBF usando uma arvore B+; Implementação em C; Baseada em Arquivo;

Genetic Algorithm File Fitter, GAFFitter for short, is a tool based on a genetic algorithm (GA) that tries to fit a collection of items, such as files/directories, into as few as possible volumes of a specific size (e.g. CDs or DVDs).

(New version 1.04 is released) Dynamic Arrays is a set of useful very fast classes for data manipulating in memory. Flexible memory control, functionality that standard containers do not have, fast operations (assembler implementation for x86 and x64 platforms). Available for Delphi all latest versions and for C++. Powerful Hash and Double Hash classes to work with pairs of values (key and value) and with values that have two keys (key1, key2, value). Give it a try and let me know how it works.

LZO is a compression library which is widely used around the world. The main problem with LZO is that it is absolutely not human readable. People have done crazy stuff to get LZO to run in their language. Usually it implies inline assembly or trying to execute data which actually contains machine code. This is sick. Whoever is responsible for this sorry situation ought to be ashamed. So I'm going to deobfuscate LZO and provide a ISC implementation of this algorithm in Python and C. In addition, I will provide a textual description of the algorithm so that it can be easily ported to any programming language. I expect a severe performance degradation, but I leave optimizing for speed to other people.

The OS561 operating system based around FORTH/Java. The OS is to run on a VHDL chip OpenHardware design called the Minon, but could become available for other platforms. The unique point of the design is a revolutionary data compression technology.

OpenDark is a free simplification of 'dark'. It's the first open BWT-DC compressor, which has a simple code, high speed and compression ratio. And it's waiting for your ideas! :)

SMSCompress is a project of investigation, development and implementation of different SMS compression techniques. Using these techniques, SMS of more than 160 characters can be sent at only one SMS, saving money to the user ;)

TCache is high performance key/value storage component. It can be easily plugged into any existing high-performance, distributed memory object caching systems such as MemcacheD or Dynamo, speeding up dynamic web applications by reducing database load

TER implements the CCSDS Recommendation for Image Data Compression (CCSDS 122.0-B-1). TER also adds new features to the Recommendation. TER is designed and programmed with the aim to provide a good basis to test and develop the CCSDS Recommendation.

antigift could be used for encryption files & folders on USB-flash or HDD partition. Works on Linux (including Android), Windows, Mac OS X, FreeBSD, DragonFly BSD and other on OSes. Do not required installation. Include WIPE tool for wiping free space and tool for recovery forgotten key. See documentation at http://sourceforge.net/p/antigift/wiki/Home/ antigift простой и кросс-платформенный инструмент для шифрования файлов и папок(с ccrypt в качестве основы) Подробности см. на страничке с русской документацией http://sourceforge.net/p/antigift/wiki/rus/

bzip2 is a free and open-source file compression program that uses the Burrows–Wheeler algorithm. It only compresses single files and is not a file archiver. It is developed and maintained by Julian Seward. Seward made the first public release of bzip2, version 0.15, in July 1996. The compressor's stability and popularity grew over the next several years, and Seward released version 1.0 in late 2000. Official website: https://www.sourceware.org/bzip2/ Source: https://sourceware.org/pub/bzip2/?C=M;O=D

jPasswordWallet strores your passwords in security. Find, modify or create quickly a password in the database. Organize your passwords as you like with creating categories and sub categories.

LZHW Command Line Lossless Compression Tool is a Windows command line tool used to compress and decompress files from and to any form, csv, excel etc without any dependencies or installations. Using an optimized algorithm (LZHW) developed from Lempel-Ziv, Huffman and LZ-Welch algorithms. The tool can work in parallel and most of its code is written in Cython, so it is pretty fast. It is based on python lzhw library. Full tool documentation can be found at: https://mnoorfawi.github.io/lzhw/6%20Using%20the%20lzhw%20command%20line%20tool/ While the documentation for the python library is at: https://mnoorfawi.github.io/lzhw/

Application level crossed-platform slab allocator implementation: fixed-allocating-time, automanual give memory back to os, address coloring to reduce bus-unbalance.

PileWorks provides the organizational structure for coordinating several different projects which approach some aspect of Pile Technology. PileWorks defines a set of interfaces and implements some basic infrastructure for Pile engines and agents.