SQLite in 2018: A state of the art SQL dialect file:///C:/Users/stadimet/AppData/Local/Temp/bat/Message.

html

Window functions, filter clause, upsert, and more. SQLite got some cool new features in 2018! Read online

SQLite in 2018: A state of the art SQL dialect

SQLite is an undervalued database—some people even believe it is a toy
database that is not suitable for production use. In fact, SQLite is a very solid
database that can handle terabytes of data, but it doesn’t have a network layer.

SQLite is “just” a library, not a server. That, of course, makes it inappropriate

for some use cases, and at the same time it makes it the best fit for many other
use cases. Really…many other use cases. SQLite even claims to be the most
widely deployed and used database engine. I guess that is only possible
because SQLite is in the public domain. SQLite is your go-to solution
whenever you want to use SQL to store structured data in a file.

The SQL dialect of SQLite is also very strong. For example, it introduced the
with clause four years before MySQL did. Lately, it has even added window
functions—just five months after MySQL did the same.

This article covers the SQL enhancements that were added to SQLite in 2018,
i.e. the new SQL features introduced in versions 3.22.0 through 3.26.0.

Boolean Literals and Tests

SQLite has “fake” Boolean support: it accepts Boolean as a type name, but it
treats it as an alias for an integer (very much like MySQL does). The truth
values true and false are represented by the numerical values 1 and 0,
respectively (like in C).

Starting with release 3.23.0, SQLite understands the keywords true and
false as synonyms for the values 1 and 0 and supports the is [not]
true|false test. The keyword unknown is generally not supported. You can
use null instead because the values unknown and null are the same for
Boolean values.

The literals true and false can greatly improve the readability of values
and set clauses in insert and update statements.

1 of 8 3/12/2021, 5:49 PM
SQLite in 2018: A state of the art SQL dialect file:///C:/Users/stadimet/AppData/Local/Temp/bat/Message.html

The is [not] true|false test is useful because it has a different meaning

than the corresponding comparison operations:

WHERE c <> FALSE

vs.

WHERE c IS NOT FALSE

If c is the null value, the result of the condition c <> false is unknown. As
the where clause only accepts true values, but rejects false as well as
unknown values, it will remove those rows from the result.

In contrast, the result of c is not false is also true if c is the null value.
The second where clause will thus also accept rows where c has the null
value.

Another way to select the same rows is to accept the null case separately.

WHERE c <> FALSE

OR c IS NULL

This variant is longer and has some redundancy (c is mentioned twice). To

make a long story short, the is not false test can be used to avoid such
or … is null constructs. More about this in “Binary Decisions Based on
Three-Valued Results”.

On my Own Behalf
I make my living from training and other SQL related services.

In order to receive occasional e-mails about these services, opt-in to receive

marketing messages via e-mail.

The support of Boolean literals and Boolean tests in SQLite is now close to
that of other open source databases. The only gap is that SQLite does not
support the is [not] unknown test (use is [not] null instead).
Interestingly, these features are generally not available in the commercial
products shown below.

2 of 8 3/12/2021, 5:49 PM
SQLite in 2018: A state of the art SQL dialect file:///C:/Users/stadimet/AppData/Local/Temp/bat/Message.html

Window Functions
SQLite 3.25.0 introduces window functions. If you know window functions, you
also know that this is a big deal. If you don’t know window functions, learn how
to use them! This article is not the right place to explain window functions, but
trust me: it is the most important “modern” SQL feature at all.

SQLite’s support of the over clause is pretty close to that of other databases.
The only notable limitation is that range frames don’t support numeric or
interval distances (only current row and unbounded
preceding|following). This is the same limitation SQL Server has, and
PostgreSQL had at the time when SQLite 3.25.0 was released. In the
meantime, PostgreSQL 11 has lifted this limitation.

The set of window function supported by SQLite is pretty much at a state of the
art level. The main omissions (distinct in aggregates, width_bucket,
respect|ignore nulls, and from first|last) are also missing in

3 of 8 3/12/2021, 5:49 PM
SQLite in 2018: A state of the art SQL dialect file:///C:/Users/stadimet/AppData/Local/Temp/bat/Message.html

some other products.

Filter Clause
Even though the filter clause is just syntax sugar—you can easily use
case expressions for the same result—I think it’s essential syntax sugar
because it makes learning and understanding a lot easier.

Just look at the following select clauses. Which one is easier to understand?

SELECT SUM(revenue) total_revenue

, SUM(CASE WHEN product = 1
THEN revenue
END
) p1_revenue
...

4 of 8 3/12/2021, 5:49 PM
SQLite in 2018: A state of the art SQL dialect file:///C:/Users/stadimet/AppData/Local/Temp/bat/Message.html

vs.

SELECT SUM(revenue) total_revenue

, SUM(revenue) FILTER(WHERE product = 1) p1_revenue
...

This example pretty much summarizes what the filter clause does: it is a
suffix for aggregate functions that conditionally removes rows before the
aggregation. The pivot technique is the most common use case of the filter
clause. That includes transforming attributes from the entity-attribute-value
(EAV) model into columns. Learn more about it in “filter — Selective
Aggregates”.

SQLite 3.25.0 introduced the filter clause for aggregate functions that use
the over clause—not for aggregates that use group by. Unfortunately, that
means that you still cannot use filter for the above mentioned use cases in
SQLite. You have to revert to case expression as before. I truly hope that
changes soon.

Insert … on conflict (“Upsert”)

SQLite 3.24.0 introduced the so-called “upsert”: an insert statement that
allows graceful handling of primary key and unique constraint violations. You
can choose to either ignore these errors (on conflict … do nothing) or
update the existing rows (on conflict … do update …).

This is a proprietary SQL extensions, i.e. it is not part of the SQL standard and
therefore gray in the matrix below. However, SQLite uses the same syntax as
PostgreSQL for this feature. 0 The standard offers the merge statement for this
and other use cases.

Unlike PostgreSQL, SQLite has a problem with the following statement.

INSERT INTO target

SELECT *
FROM source
ON CONFLICT (id)
5 of 8 3/12/2021, 5:49 PM
SQLite in 2018: A state of the art SQL dialect file:///C:/Users/stadimet/AppData/Local/Temp/bat/Message.html

DO UPDATE SET val = excluded.val

According to the documentation, the problem is that “the parser does not know
if the token "ON" is part of a join constraint on the SELECT, or the beginning of
the upsert-clause.” This can be resolved by adding another clause to the
query, e.g. where true.

INSERT INTO target

SELECT *
FROM source
WHERE true
ON CONFLICT (id)
DO UPDATE SET val = excluded.val

Rename Column
Another proprietary feature that SQLite introduced is the ability to rename
columns in base tables. 1 The SQL standard does not offer such functionality. 2

SQLite follows the syntax commonly used by other products to rename

columns:

ALTER TABLE … RENAME COLUMN … TO …

6 of 8 3/12/2021, 5:49 PM
SQLite in 2018: A state of the art SQL dialect file:///C:/Users/stadimet/AppData/Local/Temp/bat/Message.html

Other News
Besides the SQL changes, there were also some API changes in SQLite in
2018. Please refer to the news section on sqlite.com for all details.

Coming Next on Modern-SQL.com

The next article about PostgreSQL 11 is already in the making. Follow modern-
sql.com via Twitter, e-mail or RSS to stay updated.

If you’d like to learn more about modern SQL, have a look at my training in
Vienna. Besides window functions (mentioned above) the training covers
recursion, indexing, and greatly improves your understanding of the basic SQL
concepts. The training is based on the current draft of my next book. Take a
look now!

About the Author

Markus Winand teaches efficient SQL—inhouse and
online. He minimizes the development time using
modern SQL and optimizes the runtime with smart
indexing. His book entitled SQL Performance Explained
has become standard reading.

Buy his Book on Amazon Hire Markus

The essence of SQL Markus offers SQL training
tuning in 200 pages and consulting for
developers working at
Buy on Amazon companies of any size.
(paperback only)
Learn more »
Paperback and PDF also
available at Markus’ store.

Footnotes
0 SQLite often follows the PostgreSQL syntax. Richard Hipp has referred to this as What Would PostgreSQL Do (WWPD).

1 Base tables are those tables created using create table. Columns in a derived table (e.g. result of select) can be renamed in the
select clause, the from clause, or via with.

2 As far as I know—a workaround could be an updatable view or generated columns.

7 of 8 3/12/2021, 5:49 PM
SQLite in 2018: A state of the art SQL dialect file:///C:/Users/stadimet/AppData/Local/Temp/bat/Message.html

Forward this newsletter

Unsubscribe from this newsletter • Unsubscribe from all newsletters

You received this message because you subscribed to my “ Modern SQL Blog ”. Would you like to manage your subscription?

Copyright © 2019 • Markus Winand • Maderspergerstr. 1-3/9/11 • 1160 Vienna • AUSTRIA • About

8 of 8 3/12/2021, 5:49 PM