Open Source Python Data Analytics Tools

PySchool is a free and open-source Python distribution intended primarily for students who learn Python and data analysis, but it can also used by scientists, engineering, and data scientists. It includes more than 150 Python packages (full edition) including numpy, pandas, scipy, sympy, keras, scikit-learn, matplotlib, seaborn, beautifulsoup4...

Apache Spark™ is a unified analytics engine for large-scale data processing. It provides high-level APIs in Scala, Java, Python, and R, and an optimized engine that supports general computation graphs for data analysis. It also supports a rich set of higher-level tools including Spark SQL for SQL and DataFrames, MLlib for machine learning, GraphX for graph processing, and Structured Streaming for stream processing. The SageMaker Spark Container is a Docker image used to run batch data processing workloads on Amazon SageMaker using the Apache Spark framework. The container images in this repository are used to build the pre-built container images that are used when running Spark jobs on Amazon SageMaker using the SageMaker Python SDK. The pre-built images are available in the Amazon Elastic Container Registry (Amazon ECR), and this repository serves as a reference for those wishing to build their own customized Spark containers for use in Amazon SageMaker.

Dash is a Python framework for building beautiful analytical web applications without any JavaScript. Built on top of Plotly.js, React and Flask, Dash easily achieves what an entire team of designers and engineers normally would. It ties modern UI controls and displays such as dropdown menus, sliders and graphs directly to your analytical Python code, and creates exceptional, interactive analytics apps. Dash apps are very lightweight, requiring only a limited number of lines of Python or R code; and every aesthetic element can be customized and rendered in the web. It’s also not just for dashboards. You have full control over the look and feel of your apps, so you can style them to look any way you want.

StreamAlert is a serverless, real-time data analysis framework that empowers you to ingest, analyze, and alert on data from any environment, using data sources and alerting logic you define. Computer security teams use StreamAlert to scan terabytes of log data every day for incident detection and response. Incoming log data will be classified and processed by the rules engine. Alerts are then sent to one or more outputs. Rules are written in Python; they can utilize any Python libraries or functions. Merge similar alerts and automatically promote new rules if they are not too noisy. Ingested logs and generated alerts can be retroactively searched for compliance and research. Serverless design is cheaper, easier to maintain, and scales to terabytes per day. Deployment is automated, simple, safe and repeatable for any AWS account. Secure by design, least-privilege execution, containerized analysis, and encrypted data storage.

VISPA is a novel development environment for high energy physics analyses, based on a combination of graphical and textual steering. The main goal of VISPA is to support users in prototyping, performing, and verifying a data analysis of any complexity.

Wooey is a simple web interface to run command line Python scripts. Think of it as an easy way to get your scripts up on the web for routine data analysis, file processing, or anything else. The project was inspired by how simply and powerfully sandman could expose users to a database and by how Gooey turns ArgumentParser-based command-line scripts into WxWidgets GUIs. Originally two separate projects (Django-based djangui by Chris Mitchell and Flask-based Wooey by Martin Fitzpatrick) it has been merged to combine our efforts. Enable the easy wrapping of any program in simple python instead of having to use language specific to existing tools such as Galaxy. Enable fellow lab members with no command line experience to utilize python scripts. Autodocument workflows for data analysis (simple model saving).

pyastrolib is a python library providing professional astronomers a robust set of tools for astronomical data analysis. This project will provide all the functionality as NASA's IDL Astronomy User's Library in addition to other planned features.

This simple Python API aims to provide a simple and easy to understand access to the discharge database of the GOLEM Tokamak. The scipy, numpy and matplotlib Python libraries are used for data analysis and plotting.