Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .helper-scripts/llms/generate_llms_full.py
Original file line number Diff line number Diff line change
Expand Up @@ -712,7 +712,7 @@ def process_imports(self, content: str, current_file_path: Path) -> str:
print(f"Replaced {component_name} using default path: {default_path}")

# Remove or replace components that don't have clear partials
orphaned_components = ['Installation', 'Skip', 'OldCreateHypertable', 'PolicyVisualizerDownsampling', 'APIReference', 'Since2180']
orphaned_components = ['Installation', 'Skip', 'OldCreateHypertable', 'CreateHypertablePolicyNote', 'PolicyVisualizerDownsampling', 'APIReference', 'Since2180']
for component_name in orphaned_components:
# Handle both normal and spaced component tags
component_tags = [
Expand Down
5 changes: 2 additions & 3 deletions _partials/_create-hypertable-blockchain.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import OldCreateHypertable from "versionContent/_partials/_old-api-create-hypertable.mdx";
import CreateHypertablePolicyNote from "versionContent/_partials/_create-hypertable-columnstore-policy-note.mdx";
import HypertableIntro from "versionContent/_partials/_tutorials_hypertable_intro.mdx";

## Optimize time-series data using hypertables
Expand Down Expand Up @@ -31,13 +31,12 @@ import HypertableIntro from "versionContent/_partials/_tutorials_hypertable_intr
details JSONB
) WITH (
tsdb.hypertable,
tsdb.partition_column='time',
tsdb.segmentby='block_id',
tsdb.orderby='time DESC'
);
```

<OldCreateHypertable />
<CreateHypertablePolicyNote />

1. Create an index on the `hash` column to make queries for individual
transactions faster:
Expand Down
19 changes: 19 additions & 0 deletions _partials/_create-hypertable-columnstore-policy-note.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
When you create a $HYPERTABLE using [CREATE TABLE ... WITH ...][hypertable-create-table], the default partitioning
column is automatically the first column with a timestamp data type. Also, $TIMESCALE_DB creates a
[columnstore policy][add_columnstore_policy] that automatically converts your data to the $COLUMNSTORE after an interval
that matches the default [chunk_interval][create_table_arguments]. This columnar format enables fast scanning and
aggregation, optimizing performance for analytical workloads while also saving significant storage space. In the
$COLUMNSTORE conversion, $HYPERTABLE chunks are compressed by up to 98%, and organized for efficient, large-scale queries.

You can customize this policy later using [alter_job][alter_job_samples]. However, to change `after` or
`created_before`, the compression settings, or the $HYPERTABLE the policy is acting on, you must
[remove the columnstore policy][remove_columnstore_policy] and [add a new one][add_columnstore_policy].

You can also manually [convert chunks][convert_to_columnstore] in a $HYPERTABLE to the $COLUMNSTORE.

[add_columnstore_policy]: /api/:currentVersion:/hypercore/add_columnstore_policy/
[remove_columnstore_policy]: /api/:currentVersion:/hypercore/remove_columnstore_policy/
[create_table_arguments]: /api/:currentVersion:/hypertable/create_table/#arguments
[alter_job_samples]: /api/:currentVersion:/jobs-automation/alter_job/#samples
[convert_to_columnstore]: /api/:currentVersion:/hypercore/convert_to_columnstore/
[hypertable-create-table]: /api/:currentVersion:/hypertable/create_table/
7 changes: 3 additions & 4 deletions _partials/_create-hypertable-energy.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import OldCreateHypertable from "versionContent/_partials/_old-api-create-hypertable.mdx";
import CreateHypertablePolicyNote from "versionContent/_partials/_create-hypertable-columnstore-policy-note.mdx";
import HypertableIntro from "versionContent/_partials/_tutorials_hypertable_intro.mdx";

## Optimize time-series data in hypertables
Expand All @@ -15,12 +15,11 @@ import HypertableIntro from "versionContent/_partials/_tutorials_hypertable_intr
type_id integer not null,
value double precision not null
) WITH (
tsdb.hypertable,
tsdb.partition_column='time'
tsdb.hypertable
);
```

<OldCreateHypertable />
<CreateHypertablePolicyNote />

</Procedure>

Expand Down
6 changes: 2 additions & 4 deletions _partials/_create-hypertable-nyctaxis.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import OldCreateHypertable from "versionContent/_partials/_old-api-create-hypertable.mdx";
import CreateHypertablePolicyNote from "versionContent/_partials/_create-hypertable-columnstore-policy-note.mdx";


## Optimize time-series data in hypertables
Expand All @@ -15,7 +15,6 @@ same way. You use regular $PG tables for relational data.

1. **Create a $HYPERTABLE to store the taxi trip data**


```sql
CREATE TABLE "rides"(
vendor_id TEXT,
Expand All @@ -38,11 +37,10 @@ same way. You use regular $PG tables for relational data.
total_amount NUMERIC
) WITH (
tsdb.hypertable,
tsdb.partition_column='pickup_datetime',
tsdb.create_default_indexes=false
);
```
<OldCreateHypertable />
<CreateHypertablePolicyNote />

1. **Add another dimension to partition your $HYPERTABLE more efficiently**

Expand Down
5 changes: 2 additions & 3 deletions _partials/_create-hypertable-twelvedata-crypto.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import HypertableIntro from "versionContent/_partials/_tutorials_hypertable_intro.mdx";
import OldCreateHypertable from "versionContent/_partials/_old-api-create-hypertable.mdx";
import CreateHypertablePolicyNote from "versionContent/_partials/_create-hypertable-columnstore-policy-note.mdx";

## Optimize time-series data in a hypertable

Expand All @@ -25,12 +25,11 @@ import OldCreateHypertable from "versionContent/_partials/_old-api-create-hypert
day_volume NUMERIC
) WITH (
tsdb.hypertable,
tsdb.partition_column='time',
tsdb.segmentby='symbol',
tsdb.orderby='time DESC'
);
```
<OldCreateHypertable />
<CreateHypertablePolicyNote />

</Procedure>

Expand Down
7 changes: 3 additions & 4 deletions _partials/_create-hypertable-twelvedata-stocks.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import HypertableIntro from "versionContent/_partials/_tutorials_hypertable_intro.mdx";
import OldCreateHypertable from "versionContent/_partials/_old-api-create-hypertable.mdx";
import CreateHypertablePolicyNote from "versionContent/_partials/_create-hypertable-columnstore-policy-note.mdx";

## Optimize time-series data in hypertables

Expand All @@ -20,11 +20,10 @@ import OldCreateHypertable from "versionContent/_partials/_old-api-create-hypert
price DOUBLE PRECISION NULL,
day_volume INT NULL
) WITH (
tsdb.hypertable,
tsdb.partition_column='time'
tsdb.hypertable
);
```
<OldCreateHypertable />
<CreateHypertablePolicyNote />

1. **Create an index to support efficient queries**

Expand Down
7 changes: 3 additions & 4 deletions _partials/_create-hypertable.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import HypertableIntro from "versionContent/_partials/_tutorials_hypertable_intro.mdx";
import OldCreateHypertable from "versionContent/_partials/_old-api-create-hypertable.mdx";
import CreateHypertablePolicyNote from "versionContent/_partials/_create-hypertable-columnstore-policy-note.mdx";

<Procedure>

Expand All @@ -22,11 +22,10 @@ To create a hypertable:
price DOUBLE PRECISION NULL,
day_volume INT NULL
) WITH (
tsdb.hypertable,
tsdb.partition_column='time'
tsdb.hypertable
);
```
<OldCreateHypertable />
<CreateHypertablePolicyNote />

You see the result immediately:

Expand Down
10 changes: 4 additions & 6 deletions _partials/_dimensions_info.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import OldCreateHypertable from "versionContent/_partials/_old-api-create-hypertable.mdx";
import CreateHypertablePolicyNote from "versionContent/_partials/_create-hypertable-columnstore-policy-note.mdx";

### Dimension info

Expand Down Expand Up @@ -46,12 +46,11 @@ Create a by-range dimension builder. You can partition `by_range` on it's own.
temperature DOUBLE PRECISION NULL,
humidity DOUBLE PRECISION NULL
) WITH (
tsdb.hypertable,
tsdb.partition_column='time'
tsdb.hypertable
);
```

<OldCreateHypertable />
<CreateHypertablePolicyNote />

This is the default partition, you do not need to add it explicitly.

Expand Down Expand Up @@ -152,8 +151,7 @@ CREATE TABLE conditions (
temperature DOUBLE PRECISION NULL,
humidity DOUBLE PRECISION NULL
) WITH (
tsdb.hypertable,
tsdb.partition_column='time',
tsdb.hypertable
tsdb.chunk_interval='1 day'
);

Expand Down
2 changes: 2 additions & 0 deletions _partials/_hypercore-intro-short.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ transactional capabilities.

$HYPERCORE_CAP dynamically stores data in the most efficient format for its lifecycle:

![Move from rowstore to columstore in hypercore](https://assets.timescale.com/docs/images/hypercore_intro.svg )

* **Row-based storage for recent data**: the most recent chunk (and possibly more) is always stored in the $ROWSTORE,
ensuring fast inserts, updates, and low-latency single record queries. Additionally, row-based storage is used as a
writethrough for inserts and updates to columnar storage.
Expand Down
64 changes: 64 additions & 0 deletions _partials/_hypercore_create_hypertable_columnstore_policy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
import CreateHypertablePolicyNote from "versionContent/_partials/_create-hypertable-columnstore-policy-note.mdx";

1. **Enable $COLUMNSTORE on a $HYPERTABLE**

For [efficient queries][secondary-indexes], remember to `segmentby` the column you will
use most often to filter your data. For example:

* **$HYPERTABLE_CAPs**:

[Use `CREATE TABLE` for a $HYPERTABLE][hypertable-create-table]

```sql
CREATE TABLE crypto_ticks (
"time" TIMESTAMPTZ,
symbol TEXT,
price DOUBLE PRECISION,
day_volume NUMERIC
) WITH (
tsdb.hypertable,
tsdb.segmentby='symbol',
tsdb.orderby='time DESC'
);
```
<CreateHypertablePolicyNote />

* **$CAGG_CAPs**
1. [Use `ALTER MATERIALIZED VIEW` for a $CAGG][compression_continuous-aggregate]:
```sql
ALTER MATERIALIZED VIEW assets_candlestick_daily set (
timescaledb.enable_columnstore = true,
timescaledb.segmentby = 'symbol' );
```
Before you say `huh`, a $CAGG is a specialized $HYPERTABLE.

1. Add a policy to convert $CHUNKs to the $COLUMNSTORE at a specific time interval:

Create a [columnstore_policy][add_columnstore_policy] that automatically converts $CHUNKs in a $HYPERTABLE to
the $COLUMNSTORE at a specific time interval. For example:
``` sql
CALL add_columnstore_policy('assets_candlestick_daily', after => INTERVAL '1d');
```

$TIMESCALE_DB is optimized for fast updates on compressed data in the $COLUMNSTORE. To modify data in the
$COLUMNSTORE, use standard SQL.


[job]: /api/:currentVersion:/actions/add_job/
[alter_table_hypercore]: /api/:currentVersion:/hypercore/alter_table/
[compression_continuous-aggregate]: /api/:currentVersion:/continuous-aggregates/alter_materialized_view/
[convert_to_rowstore]: /api/:currentVersion:/hypercore/convert_to_rowstore/
[convert_to_columnstore]: /api/:currentVersion:/hypercore/convert_to_columnstore/
[informational-views]: /api/:currentVersion:/informational-views/jobs/
[add_columnstore_policy]: /api/:currentVersion:/hypercore/add_columnstore_policy/
[hypercore_workflow]: /api/:currentVersion:/hypercore/#hypercore-workflow
[alter_job]: /api/:currentVersion:/actions/alter_job/
[remove_columnstore_policy]: /api/:currentVersion:/hypercore/remove_columnstore_policy/
[in-console-editors]: /getting-started/:currentVersion:/run-queries-from-console/
[services-portal]: https://console.cloud.timescale.com/dashboard/services
[connect-using-psql]: /integrations/:currentVersion:/psql/#connect-to-your-service
[insert]: /use-timescale/:currentVersion:/write-data/insert/
[hypertables-section]: /use-timescale/:currentVersion:/hypertables/
[hypertable-create-table]: /api/:currentVersion:/hypertable/create_table/
[hypercore]: /use-timescale/:currentVersion:/hypercore/
[secondary-indexes]: /use-timescale/:currentVersion:/hypercore/secondary-indexes/
43 changes: 2 additions & 41 deletions _partials/_hypercore_policy_workflow.md
Original file line number Diff line number Diff line change
@@ -1,51 +1,12 @@
import OldCreateHypertable from "versionContent/_partials/_old-api-create-hypertable.mdx";
import CreateHypertableProcedure from "versionContent/_partials/_hypercore_create_hypertable_columnstore_policy.mdx";

<Procedure>

1. **Connect to your $SERVICE_LONG**

In [$CONSOLE][services-portal] open an [SQL editor][in-console-editors]. You can also connect to your $SERVICE_SHORT using [psql][connect-using-psql].

1. **Enable $COLUMNSTORE on a $HYPERTABLE**

Create a [$HYPERTABLE][hypertables-section] for your time-series data using [CREATE TABLE][hypertable-create-table].
For [efficient queries][secondary-indexes] on data in the columnstore, remember to `segmentby` the column you will
use most often to filter your data. For example:

* [Use `CREATE TABLE` for a $HYPERTABLE][hypertable-create-table]

```sql
CREATE TABLE crypto_ticks (
"time" TIMESTAMPTZ,
symbol TEXT,
price DOUBLE PRECISION,
day_volume NUMERIC
) WITH (
tsdb.hypertable,
tsdb.partition_column='time',
tsdb.segmentby='symbol',
tsdb.orderby='time DESC'
);
```
<OldCreateHypertable />

* [Use `ALTER MATERIALIZED VIEW` for a $CAGG][compression_continuous-aggregate]
```sql
ALTER MATERIALIZED VIEW assets_candlestick_daily set (
timescaledb.enable_columnstore = true,
timescaledb.segmentby = 'symbol' );
```
Before you say `huh`, a $CAGG is a specialized $HYPERTABLE.

1. **Add a policy to convert $CHUNKs to the $COLUMNSTORE at a specific time interval**

Create a [columnstore_policy][add_columnstore_policy] that automatically converts $CHUNKs in a $HYPERTABLE to the $COLUMNSTORE at a specific time interval. For example, convert yesterday's crypto trading data to the $COLUMNSTORE:
``` sql
CALL add_columnstore_policy('crypto_ticks', after => INTERVAL '1d');
```

$TIMESCALE_DB is optimized for fast updates on compressed data in the $COLUMNSTORE. To modify data in the
$COLUMNSTORE, use standard SQL.
<CreateHypertableProcedure />

1. **Check the $COLUMNSTORE policy**

Expand Down
5 changes: 2 additions & 3 deletions _partials/_import-data-iot.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import OldCreateHypertable from "versionContent/_partials/_old-api-create-hypertable.mdx";
import CreateHypertablePolicyNote from "versionContent/_partials/_create-hypertable-columnstore-policy-note.mdx";
import HypertableIntro from "versionContent/_partials/_tutorials_hypertable_intro.mdx";

<HypertableIntro />
Expand Down Expand Up @@ -38,12 +38,11 @@ import HypertableIntro from "versionContent/_partials/_tutorials_hypertable_intr
value double precision not null
) WITH (
tsdb.hypertable,
tsdb.partition_column='created',
tsdb.segmentby = 'type_id',
tsdb.orderby = 'created DESC'
);
```
<OldCreateHypertable />
<CreateHypertablePolicyNote />

1. Upload the dataset to your $SERVICE_SHORT
```sql
Expand Down
5 changes: 2 additions & 3 deletions _partials/_import-data-nyc-taxis.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import OldCreateHypertable from "versionContent/_partials/_old-api-create-hypertable.mdx";
import CreateHypertablePolicyNote from "versionContent/_partials/_create-hypertable-columnstore-policy-note.mdx";
import HypertableIntro from "versionContent/_partials/_tutorials_hypertable_intro.mdx";

<HypertableIntro />
Expand Down Expand Up @@ -53,13 +53,12 @@ import HypertableIntro from "versionContent/_partials/_tutorials_hypertable_intr
total_amount NUMERIC
) WITH (
tsdb.hypertable,
tsdb.partition_column='pickup_datetime',
tsdb.create_default_indexes=false,
tsdb.segmentby='vendor_id',
tsdb.orderby='pickup_datetime DESC'
);
```
<OldCreateHypertable />
<CreateHypertablePolicyNote />

1. Add another dimension to partition your $HYPERTABLE more efficiently:
```sql
Expand Down
10 changes: 9 additions & 1 deletion _partials/_old-api-create-hypertable.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,16 @@
For $TIMESCALE_DB v2.23.0 and higher, the table is automatically partitioned on the first column in the table with a
timestamp data type. For earlier versions, set `partition_column` to a time column.

If you are self-hosting $TIMESCALE_DB v2.20.0 to v2.22.1, to convert your data to the $COLUMNSTORE after a specific time
interval, you have to call [add_columnstore_policy] after you call [CREATE TABLE][hypertable-create-table]

If you are self-hosting $TIMESCALE_DB v2.19.3 and below, create a [$PG relational table][pg-create-table],
then convert it using [create_hypertable][create_hypertable]. You then enable $HYPERCORE with a call
to [ALTER TABLE][alter_table_hypercore].


[pg-create-table]: https://www.postgresql.org/docs/current/sql-createtable.html
[create_hypertable]: /api/:currentVersion:/hypertable/create_hypertable/
[alter_table_hypercore]: /api/:currentVersion:/hypercore/alter_table/
[add_columnstore_policy]: /api/:currentVersion:/hypercore/add_columnstore_policy/
[hypertable-create-table]: /api/:currentVersion:/hypertable/create_table/
[chunk_interval]: /api/:currentVersion:/hypertable/set_chunk_time_interval/
Loading