Merge pull request duneanalytics#215 from duneanalytics/12-12-minor-tweaks-and-fixes

api examples, fantom chain added, tweaks and fixes T-22288

Author: cryptofreedman

docs/.hackathon-guide.md

@@ -15,7 +15,8 @@ To get started building on our API, [first create a Dune account here](https://d
 
 - Our [API Welcome page here](api/index.md) to learn about how our API works
 - Our [Quick Start Guides](api/quick-start/index.md) (for Python and JS) to learn how to setup your environment and start executing.
-- Our [API Reference](api/api-reference/index.md) section for specifics about our Endpoints
+- Our [API Reference](api/api-reference/index.md) section for specifics about our Endpoints.
+- [API Use Cases](api/quick-start/api-use-cases.md) page for inspiration!
 - Our [#dune-api Discord channel](https://discord.com/channels/757637422384283659/1019910980634939433) for questions and support!
 
 ## Picking a Programming Language

docs/getting-started/queries/images/parameters-overview-1.png

@@ -10,7 +10,7 @@ Dune V2 is an update to our database and query engine that includes:
 1. A scalable column oriented database (in contrast to PostgreSQL’s row oriented approach)
 2. Dune’s new query engines, Dune SQL and [Spark SQL](https://spark.apache.org/docs/latest/sql-programming-guide.html).
 
-For more on the why behind Dune V2, be sure to read the post from our CTO (to be published in the coming weeks)!
+For more on the why behind Dune V2, be sure to [read the post from our CTO](https://dune.com/blog/introducing-dune-sql)!
 
 ## New Database
 
@@ -67,6 +67,6 @@ Both also have well documented index of built in functions on their website:
 * [Spark - Spark SQL Language Reference](https://spark.apache.org/docs/latest/sql-programming-guide.html)
 * [Trino - Functions and Operators](https://trino.io/docs/current/functions.html)
 
-Our #dune-sql Discord channel is the best place to get help from our team and Wizard community when Google fails you.
+Our [#dune-sql Discord channel](https://discord.com/channels/757637422384283659/1051871389432422491) is the best place to get help from our team and Wizard community when Google fails you.
 
 As you come across issues or identify areas of improvement, please send us an email at [[email protected]](mailto:[email protected]) and we’ll work with you to update and optimize!

docs/getting-started/queries/images/parameters-overview-2.png

@@ -10,15 +10,53 @@ With Dune V2 we’re moving away from a [PostgreSQL](https://www.postgresql.org/
 1. Spark SQL - [Apache Spark](https://www.databricks.com/glossary/what-is-apache-spark) hosted on [Databricks](https://docs.databricks.com/getting-started/introduction/index.html).
 2. Dune SQL - A self hosted instance of [Trino](https://trino.io/)(**currently in alpha**). 
 
+As Dune usage has increased, our data access pattern has become quite clear: Dune writes data Once and wizard query it millions of times.
+
+In order to scale Dune to meet the demand for query execution, we need to decouple data storage from the computation of queries - something that can’t be done in PostgreSQL.
+
+As we experimented with solutions internally, we built a new system around the [Delta framework](https://delta.io/) and used Spark for parallel processing of that data.
+
+During experimentation, we began ingesting Solana data. Since solana.transactions is about 100x the size of ethereum.transactions, it was both a primary reason we needed to upgrade our systems (there’s no way Postgres could handle this) and the perfect test candidate.
+
+We saw some great wins in how quickly and flexibly we could write and store data but when it came time to start sharing access to Solana data we faced a dilemma.
+
+As we were focused on optimizing the ingesting and storing of this data, we did not spend a lot of time upgrading our infrastructure for querying that data.
+
+When we asked ourselves whether we should spend months figuring out how to improve our querying infrastructure or get Solana data in the hands of our community ASAP, we chose the latter - which meant shipping Dune V2 in beta with Spark SQL as our query engine.
+
+Now that we have a massive amount of usage data (compared to our internal testing), cracks have appeared that weren’t evident to us pre-launch.
+
+As things stand:
+
+* A range of queries that performed super well on v1, now take minutes on V2 (looking at you WHERE tx_hash = y). Part of this is clearly the tradeoff we’ve made in laying out our data in columnar form, as opposed to in rows, but that doesn’t fully explain the latency.
+* V2 (Beta) lacks support for user defined functions, views, tables, and it’s not easy for Dune to enable that functionality in a way that’s safe and helpful to users in Spark.
+* Because the Spark clusters are outsourced, we have trouble scaling them, and tuning them. It’s also not possible for us to improve performance or fix bugs without waiting for months for someone to [merge them](https://github.com/delta-io/delta/pull/1210).
+* We oftentimes find ourselves without visibility into the health of the query engine, which has lead to us being unable to resolve confusion for ourselves or the community when some queries have failed.
+
+As a result of these challenges, we’ve been internally researching how to fundamentally improve our V2 query experience and after a few breakthroughs, we’ve developed what Dune SQL, powered by a self-hosted instance of [Trino](https://trino.io/). 
+
+And as a step toward resolving some of the communication errors we made with the initial Spark SQL launch, we’re emphasizing that **Dune SQL is currently in alpha**.
+
+This means:
+
+* While we strongly believe Dune SQL/Trino is the long term solution for creating the best blockchain data querying experience in the world, it isn’t there yet - with your help we’ll break lots of things and use what we learn to continuously build a better system.
+* While we have a timeline for [Sunsetting Dune V1 (PostgreSQL)](../v1-sunsetting.md), we do not have one for fully transitioning from Spark to Dune SQL and will work with the community to decide when and how that transition will take place to move forward with [Speed](https://www.notion.so/Values-and-working-at-Dune-7efdcec2298a4913aaef8067b25820df#ffc480bf5c5e4e38a1c53d2fb9926e3e) and transparency.
+
+## Joining the SQL revolution
+
+We’re excited to have you join the SQL revolution with us as we make Dune SQL the absolute best blockchain querying experience around.
+
+To help out as an alpha tester, you’ll first need to create a separate account at [https://dev.dune.com/](https://dev.dune.com/).
+
+There you’ll find Dune SQL as an option in the Query Explorer, and right off the bat you should see some improvements over Spark SQL ([incomplete & cryptic error messages](https://dev.dune.com/queries/59553) or [0x strings](https://dev.dune.com/queries/59390), we’re looking at you).
+
 ## Syntax and operator differences
 
 The syntax and keyword operator differences between Postgres, Spark, and Dune SQL are quite minimal, however there are a few to be aware of.
 
 !!! warning
     **Dune SQL is still in alpha!** If you find any other changes in Spark or Dune SQL that are important to note, please feel free to [submit a PR to this docs page on GitHub](https://github.com/duneanalytics/docs/edit/master/docs/reference/dune-v2/query-engine.md) or let us know in #dune-sql.
 
-
-
 ### Syntax Comparison
 
 | <div style="width:290px">**Description**</div> | **V1 - PostgreSQL** | **V2 - Spark SQL** | **V2 - Dune SQL** |
@@ -60,14 +98,14 @@ select * from query_1234
 [Here is an example query](https://dune.com/queries/1746224) which queries [this query](https://dune.com/queries/1746191).
 The `queryId` is part of the URL of a query.
 
-Limitations:
+**Note**:
+
 - All output columns of the query being queried must be named. That is, you cannot query `select 1` or `select count(*) from ethereum.transactions`, but you can query `select 1 as v` and `select count(*) as total from ethereum.transactions`.
 - Parametrized queries are not supported.
 - Only public queries can be queried. Support for querying private queries will be added in the future.
 - Only saved queries can be queried.
 - Only queries written in Dune SQL can be queried.
 
-
 ## Other questions and feedback
 
 As ever, Google is a great friend in answering your SQL questions.
@@ -79,4 +117,6 @@ Both also have well documented index of built in functions on their website:
 * [Spark - Spark SQL Language Reference](https://spark.apache.org/docs/latest/sql-programming-guide.html)
 * [Trino - Functions and Operators](https://trino.io/docs/current/functions.html)
 
+Our [#dune-sql Discord channel](https://discord.com/channels/757637422384283659/1051871389432422491) is the best place to get help from our team and Wizard community when Google fails you.
+
 As you come across issues or identify areas of improvement, please send us an email at [[email protected]](mailto:[email protected]) and we’ll work with you to update and optimize!

docs/getting-started/queries/images/parameters-overview-3.png

@@ -24,6 +24,7 @@ Some data sets will have multiple blockchain icons - meaning data from each of t
 | ![arbitrum icon](images/arbitrum-icon.png) | Arbitrum blockchain Raw Data, Decoded Project, or Spell |
 | ![avalanche icon](images/avalanche-icon.png) | Avalanche C-Chain Raw Data, Decoded Project, or Spell |
 | ![goerli testnet icon](images/goerli-testnet-icon.png) | Ethereum Goerli Testnet Raw Data, Decoded Project, or Spell |
+| ![fantom icon](images/fantom-icon.png) | Fantom Raw Data, Decoded Project, or Spell |
 
 ## Dataset Icons
 

docs/reference/dune-v2/index.md

@@ -185,4 +185,10 @@ Working with the C-Chain on Dune works exactly like querying Ethereum mainnet da
 
 Created in September 2018 during ETHBerlin, [Goerli Testnet](https://goerli.net/) was the first proof-of-authority cross-client testnet, synching Parity Ethereum, Geth, Nethermind, Hyperledger Besu (formerly Pantheon), and EthereumJS.
 
-This is the perfect solution for dapp developers looking to get stats before you launch on mainnet!
+This is the perfect solution for dapp developers looking to get stats before you launch on mainnet!
+
+## Fantom
+
+[Fantom](https://fantom.foundation/) is a layer 1 blockchain offering smart contract functionality.
+
+It uses a Directed Acyclic Graph, which involves the seamless interaction of nodes in the network to ensure fast and secure transactions.