You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.adoc
+8-9Lines changed: 8 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,20 +11,20 @@ toc::[]
11
11
== Analytics
12
12
13
13
* https://spark.apache.org/[Apache Spark] - A unified analytics engine for large-scale data processing. Includes APIs in Scala, Java, Python (known as PySpark), and R (SparkR).
14
-
* https://flink.apache.org/[Apache Flink] - Stateful computations over data streams.
15
14
* https://beam.apache.org/[Apache Beam] - An open-source implementation of Google DataFlow. Provides capabilites of batch and streaming data processing jobs that run on any execution engine, including Spark, Flink, or its own DirectRunner. Supports multiple APIs in Java, Python, and Go.
15
+
* https://flink.apache.org/[Apache Flink] - Stateful computations over data streams.
16
16
17
17
== Business Intelligence
18
18
19
19
* https://superset.incubator.apache.org/[Apache Superset] - A modern, enterprise-ready business intelligence web application.
20
+
* https://gethue.com/[HUE] - The Hadoop User Interface. Similar to Superset, but interfaces between RDBMS, Hive, Impala, HBase, Spark, HDFS & S3, Oozie, Pig, YARN Job Explorer, and more. Offers an extensible Django environment for custom app integration.
20
21
* https://www.metabase.com/[Metabase] - An easy way for everyone in your company to ask questions and learn from data.
21
22
* https://redash.io/[Redash] - All the tools to unlock your data.
22
-
* https://gethue.com/[HUE] - The Hadoop User Interface. Similar to Superset, but interfaces between RDBMS, Hive, Impala, HBase, Spark, HDFS & S3, Oozie, Pig, YARN Job Explorer, and more. Offers an extensible Django environment for custom app integration.
23
23
24
24
== Change Data Capture
25
25
26
26
* https://debezium.io/[Debezium] - Change data capture for MySQL, Postgres, MongoDB, SQL Server and others.
27
-
* https://github.com/zendesk/maxwell[Maxwell] - Maxwell's daemon, a MySQL-to-JSON Kafka producer
27
+
* https://github.com/zendesk/maxwell[Maxwell] - Maxwell's daemon, a MySQL-to-JSON Kafka producer.
28
28
29
29
== Datastores
30
30
@@ -35,8 +35,8 @@ toc::[]
35
35
* https://pinot.apache.org/[Apache Pinot] - A realtime distributed OLAP datastore.
36
36
* https://clickhouse.tech/[ClickHouse] - Open Source distributed column-oriented DBMS.
37
37
* https://www.influxdata.com/[InfluxDB] - Purpose-Built Open Source Time Series Database.
38
-
* https://www.postgresql.org/[Postgres] - The World's Most Advanced Open Source Relational Database.
39
38
* https://min.io/[MinIO] - MinIO is a high performance, distributed object storage system and AWS S3 compatible.
39
+
* https://www.postgresql.org/[Postgres] - The World's Most Advanced Open Source Relational Database.
40
40
41
41
== Data Governance and Registries
42
42
@@ -47,10 +47,10 @@ toc::[]
47
47
48
48
== Data Virtualization
49
49
50
+
* https://drill.apache.org/[Apache Drill] - Schema-free SQL Query Engine for Hadoop, NoSQL and Cloud Storage.
50
51
* https://github.com/dremio/dremio-oss[Dremio] - A data lake engine. Provides an Apache Arrow-based query and acceleration engine together with the ability to create an IT-governed self-service layer for data scientists and analysts.
51
52
* http://teiid.io/[Teiid] - A relational abstraction of different information sources.
52
53
* https://prestodb.io/[Presto] - Distributed SQL Query Engine for Big Data.
53
-
* https://drill.apache.org/[Apache Drill] - Schema-free SQL Query Engine for Hadoop, NoSQL and Cloud Storage.
54
54
55
55
== Data Orchestration
56
56
* https://github.com/Alluxio/alluxio[Alluxio] - Scalable, multi-tiered distributed caching for HDFS, S3, Ceph, NFS, and related filestores. Provides integrations for SQL queries into a Catalog from Spark, Hive, and Presto.
@@ -64,9 +64,8 @@ toc::[]
64
64
* https://arrow.apache.org/[Apache Arrow] - A cross-language development platform for in-memory data. It specifies a standardized, language-independent, columnar memory format for flat and hierarchical data, organized for efficient analytic operations on modern hardware. It also provides computational libraries and zero-copy IPC and streaming messaging.
65
65
* https://capnproto.org/[Cap’n Proto] - A data interchange format and capability-based RPC system.
66
66
* https://google.github.io/flatbuffers/[FlatBuffers] - An efficient cross platform serialization library for C++, C#, C, Go, Java, JavaScript, Lobster, Lua, TypeScript, PHP, Python, and Rust.
* https://msgpack.org/index.html[MessagePack] - An efficient binary serialization format. It lets you exchange data among multiple languages like JSON.
* https://heron.incubator.apache.org/[Apache Heron] - The "direct successor of Apache Storm", built to be backwards compatible with Storm's topology API but with a wide array of architectural improvements.
96
96
* https://kafka.apache.org/documentation/streams/[Apache Kafka Streams] - A client library for building applications and microservices, where the input and output data are stored in Kafka.
97
97
* http://samza.apache.org/[Apache Samza] - A distributed stream processing framework.
98
98
* https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html[Apache Spark Structured Streaming] - A scalable and fault-tolerant stream processing engine built on the Spark SQL engine.
99
99
* http://storm.apache.org/[Apache Storm] - A distributed realtime computation system.
100
-
* https://heron.incubator.apache.org/[Apache Heron] - The "direct successor of Apache Storm", built to be backwards compatible with Storm's topology API but with a wide array of architectural improvements.
101
100
102
101
== Testing
103
102
@@ -107,9 +106,9 @@ toc::[]
107
106
108
107
* https://github.com/meirwah/awesome-workflow-engines[Awesome Workflow Engines] - A curated list of awesome open source workflow engines.
109
108
* https://airflow.apache.org/[Apache Airflow] - A platform created by community to programmatically author, schedule and monitor workflows.
110
-
* https://github.com/PrefectHQ/prefect/[Prefect] - A workflow management system designed for modern infrastructure.
111
109
* https://nifi.apache.org/[Apache NiFi] - Apache NiFi supports powerful and scalable directed graphs of data routing, transformation, and system mediation logic
112
110
* https://github.com/knime/[KNIME] - KNIME Analytics Platform offers a WYSIWYG Editor for Spark-based workflows, with over 2000+ integrations. Offers visualization and flow analytics in-place. KNIME Server is a commercially licensed component that adds additional features.
111
+
* https://github.com/PrefectHQ/prefect/[Prefect] - A workflow management system designed for modern infrastructure.
0 commit comments