Skip to content

Conversation

@JamesGuthrie
Copy link
Member

Description

Add warning about SERIAL types and dual-write

@github-actions
Copy link

Allow 10 minutes from last push for the staging site to build. If the link doesn't work, try using incognito mode instead. For internal reviewers, check web-documentation repo actions for staging build status. Link to build for this PR: http://docs-dev.timescale.com/docs-jg-dual-write-serial

Copy link
Contributor

@billy-the-fish billy-the-fish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice doc. Couple of small language updates for you.


Should your time-series data have foreign-key references into a plain table,
you must ensure that your application correctly maintains the foreign key
relations. Beware that if the referenced column is a `*SERIAL` type, the same
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you change Beware that if to simple If please.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

you must ensure that your application correctly maintains the foreign key
relations. Beware that if the referenced column is a `*SERIAL` type, the same
row inserted into the source and target _may not_ obtain the same autogenerated
id. If this happens, the data backfilled from the source to the target will be
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Always use the present tense:

from the source to the target is

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

internally corrupt. In the best case it causes a foreign key violation, in the
worst case, the foreign key relation constraint is maintained, but the data
references the wrong foreign key. If reading this has made you worried,
consider using [live migration] instead.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If reading this has made you worried, consider using [live migration] instead.

To avoid these issues, best practice is to follow [live migration][live-migration].

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

@billy-the-fish billy-the-fish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@JamesGuthrie JamesGuthrie force-pushed the jg/dual-write-serial branch from 8b93a85 to 12b3517 Compare April 25, 2024 13:48
@JamesGuthrie JamesGuthrie merged commit 39318f4 into latest Apr 25, 2024
@JamesGuthrie JamesGuthrie deleted the jg/dual-write-serial branch April 25, 2024 13:51
target database, and what mechanism you want to or can build to recover from
such a failure.

Should your time-series data have foreign-key references into a plain table,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fail to see when something like this could happen. When the non-TS data is migrated then all the IDs that are going to be referenced by the backfilled data should be already present in TARGET. No?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the customer is only dual-writing timeseries data, yes. If they intend to dual-write both the referenced data and the timeseries data then this can happen.

billy-the-fish added a commit that referenced this pull request May 10, 2024
* Merging up stubs into main documents, editing, formatting

* Oops typo

* Update for linter

Signed-off-by: Iain Cox <[email protected]>

* Update administration.md

Signed-off-by: Iain Cox <[email protected]>

* fix: update navigation and links for page merge.

* fix: clean up de-chunking.

* fix: clean up de-chunking.

* feat: unite stubs.

* feat: unite stubs.

* Fix typo (#3052)

Signed-off-by: Florian Mt <[email protected]>
Co-authored-by: Iain Cox <[email protected]>

* Fix typo in troubleshooting section (#3073)

* Update live-migration to v0.0.9 (#3071)

Co-authored-by: Iain Cox <[email protected]>

* Copy Nit (#3078)

Signed-off-by: Austin Lai <[email protected]>

* Update import-csv.md (#3080)

Make time column references consistent

Signed-off-by: Solar Olugebefola <[email protected]>

* Update new about compression (#3012)

* live-migration: Add docs changes for v0.0.10 (#3091)

Signed-off-by: Arunprasad Rajkumar <[email protected]>

* Add Google CA (#3090)

We've added a new CA provider to our cloud, reflecting this fact in the
docs.

* Update ruby.md (#3065)

* Update ruby.md 

Modify references to Timescale/TimescaleDB/Timescale Cloud

Signed-off-by: Solar Olugebefola <[email protected]>

* fix: update TOC links to match headings.

* fix: update TOC links to match headings.

* fix: correct product name.

---------

Signed-off-by: Solar Olugebefola <[email protected]>
Co-authored-by: Iain Cox <[email protected]>

* live-migration: Add docs changes for v0.0.11 (#3096)

Signed-off-by: Arunprasad Rajkumar <[email protected]>

* Move multinode migration under playbooks (#3110)

* 3113 docs rfc add new editor (#3115)

* First draft

* Complete article, add images

* Add a screenshot of PopSQL

* Update popsql_button_in_console

Highlighted the button

* Wrap text

* feat: update PopSQL page, get started and update sections with PopSQL information.

* feat: link to images on S3.

* fix: language cleanup.

* Apply suggestions from code review

Co-authored-by: sremertimescale <[email protected]>
Signed-off-by: Iain Cox <[email protected]>

* fix: language cleanup after review.

* fix: another language review.

* fix: updates following review by Ramon.

* Update use-timescale/popsql.md

Co-authored-by: Ramon Guiu <[email protected]>
Signed-off-by: Iain Cox <[email protected]>

* Update use-timescale/query-data/about-query-data.md

Co-authored-by: Ramon Guiu <[email protected]>
Signed-off-by: Iain Cox <[email protected]>

* fix: add contact information to add more seats.

---------

Signed-off-by: Iain Cox <[email protected]>
Co-authored-by: Rahil Sondhi <[email protected]>
Co-authored-by: sremertimescale <[email protected]>
Co-authored-by: Ramon Guiu <[email protected]>

* fix: numbering and full stop. (#3120)

* missing word in add_reorder_policy (#3106)

Signed-off-by: Sam Debruyn <[email protected]>
Co-authored-by: Iain Cox <[email protected]>

* Restrict usage of time_bucket_ng in CAggs (#3112)

The function time_bucket_ng is deprecated and should not be used in new
CAggs. 2.15.0 will also block the usage of this function.

* fix: broken links. (#3123)

* fix: broken links.

* fix: broken links.

* Update CAgg add_continuous_aggregate_policy documentation (#3128)

* fix: deprecate distributed hypertables from the API reference. (#3134)

* fix: update API reference for set_integer_now_func. (#3135)

* fix: update API reference for set_integer_now_func.

Co-authored-by: Jônatas Davi Paganini <[email protected]>
Signed-off-by: Iain Cox <[email protected]>

* Add compression settings information views (#3142)

* fix: using the public API to check the  chunk time interval for a hypertable. (#3141)

* 3144 docs rfc be more clear about tiered storage being on timescale not on self hosted (#3145)

* feat: update tiered storage overview

* feat: update tiered storage overview

* Update alter_job.md (#3103)

Added missing information in alter_job options

Signed-off-by: Fabrízio de Royes Mello <[email protected]>
Co-authored-by: Iain Cox <[email protected]>

* fix: some broken links. A couple of text updates. (#3151)

* feat: FAQ for live migration (#3139)

Signed-off-by: Harkishen-Singh <[email protected]>

* chore: update live-migration to v0.0.13 (#3150)

Signed-off-by: Harkishen-Singh <[email protected]>

* Add pgstattuple to list of available extensions (#3154)

* fix: clarify that each service hosts a single service only. (#3161)

* fix: change link in Migrate the entire database at once to the live migration page. (#3159)

* Add warning about SERIAL types and dual-write (#3162)

* Address possible installation issues regarding psycopg2 (#3165)

* Address possible installation issues regarding psycopg2

* Address possible installation issues regarding psycopg2

* chore: few language updates for the install procedure.

---------

Co-authored-by: billy-the-fish <[email protected]>

* Add troubleshooting note about partial continuous aggregates (#3164)


Signed-off-by: James Guthrie <[email protected]>
Co-authored-by: billy-the-fish <[email protected]>

* 2829 obsolete restriction on replication factor 1 still in the docs 1 (#3129)

* fix: update create_distributed_hypertable

* Recommend live-migration instead of dual-writes (#3173)

* Refer to live-migration 0.0.14 (#3179)

Signed-off-by: Arunprasad Rajkumar <[email protected]>

* feat: add version downgrade for live-migration FAQ (#3163)

Signed-off-by: Harkishen-Singh <[email protected]>

This commit adds a FAQ question that helps users to downgrade
TimescaleDB version on their target database to match with the
source database. This is useful when the source database (MST) does not
have the TimescaleDB version that the target database is installed
with.

* Make some changes to refresh cagg note (#3146)

* Make some changes to refresh cagg note

This fixes one typo and changes a few other words.

Signed-off-by: RobAtticus <[email protected]>

* chore: updates after review.

---------

Signed-off-by: RobAtticus <[email protected]>
Co-authored-by: Iain <[email protected]>

* Add role based auth to CloudWatch exporter doc (#3156)

* Add role based auth to CloudWatch exporter doc

Role based authentication is added for CloudWatch exporter that allows
to authenticate with users' AWS accounts and we need to document
configuration that needs to be done on the user side to make it work.

* add provider URLs and role example

* Clarify what the URL serves for

* chore: first go updating the configuration procedure.

* chore: updates after first review.

* chore: updates after first review.

* Update use-timescale/metrics-logging/integrations.md

Nice.

Co-authored-by: Danil Zhigalin <[email protected]>
Signed-off-by: Iain Cox <[email protected]>

* fix: update metatags for MDX.

* Replace trust policy to a role

AWS docs refer to this as a role. Make sure we are talking about the
same thing in our docs.

* Update use-timescale/metrics-logging/integrations.md

Co-authored-by: Nicolas Nosenzo <[email protected]>
Signed-off-by: Iain Cox <[email protected]>

---------

Signed-off-by: Iain Cox <[email protected]>
Co-authored-by: niconosenzo <[email protected]>
Co-authored-by: billy-the-fish <[email protected]>
Co-authored-by: Iain Cox <[email protected]>

* fix: document actual behavior of add_continuous_aggregate_policy when using if_not_exists (#3037)

Signed-off-by: dtext <[email protected]>
Co-authored-by: Iain Cox <[email protected]>

* fix: add verb to sentence. (#3131)

* fix: add verb to sentence.
* fix: update after review.

* fix: update navigation and links for page merge.

* fix: clean up de-chunking.

* chore: migrate changes in future versions back to this PR.

* fix: my fatal weakness.

* fix: one broken link fixed.

* fix: more link fixing after manual check.

---------

Signed-off-by: Iain Cox <[email protected]>
Signed-off-by: Florian Mt <[email protected]>
Signed-off-by: Austin Lai <[email protected]>
Signed-off-by: Solar Olugebefola <[email protected]>
Signed-off-by: Arunprasad Rajkumar <[email protected]>
Signed-off-by: Sam Debruyn <[email protected]>
Signed-off-by: Iain Cox <[email protected]>
Signed-off-by: Fabrízio de Royes Mello <[email protected]>
Signed-off-by: Harkishen-Singh <[email protected]>
Signed-off-by: James Guthrie <[email protected]>
Signed-off-by: RobAtticus <[email protected]>
Signed-off-by: dtext <[email protected]>
Co-authored-by: Jay Allen <[email protected]>
Co-authored-by: James Sewell <[email protected]>
Co-authored-by: Florian Mt <[email protected]>
Co-authored-by: Ante Kresic <[email protected]>
Co-authored-by: James Guthrie <[email protected]>
Co-authored-by: Austin Lai <[email protected]>
Co-authored-by: Solar Olugebefola <[email protected]>
Co-authored-by: Arunprasad Rajkumar <[email protected]>
Co-authored-by: Danil Zhigalin <[email protected]>
Co-authored-by: Konstantinos Gkiokas <[email protected]>
Co-authored-by: Rahil Sondhi <[email protected]>
Co-authored-by: sremertimescale <[email protected]>
Co-authored-by: Ramon Guiu <[email protected]>
Co-authored-by: Sam Debruyn <[email protected]>
Co-authored-by: Jan Nidzwetzki <[email protected]>
Co-authored-by: Jônatas Davi Paganini <[email protected]>
Co-authored-by: Fabrízio de Royes Mello <[email protected]>
Co-authored-by: Harkishen Singh <[email protected]>
Co-authored-by: John Pruitt <[email protected]>
Co-authored-by: alejandrodnm <[email protected]>
Co-authored-by: RobAtticus <[email protected]>
Co-authored-by: niconosenzo <[email protected]>
Co-authored-by: dtext <[email protected]>
billy-the-fish added a commit that referenced this pull request Aug 12, 2024
* Merging up stubs into main documents, editing, formatting

* Oops typo

* Update for linter

Signed-off-by: Iain Cox <[email protected]>

* Update administration.md

Signed-off-by: Iain Cox <[email protected]>

* fix: update navigation and links for page merge.

* fix: clean up de-chunking.

* fix: clean up de-chunking.

* feat: unite stubs.

* feat: unite stubs.

* Fix typo (#3052)

Signed-off-by: Florian Mt <[email protected]>
Co-authored-by: Iain Cox <[email protected]>

* Fix typo in troubleshooting section (#3073)

* Update live-migration to v0.0.9 (#3071)

Co-authored-by: Iain Cox <[email protected]>

* Copy Nit (#3078)

Signed-off-by: Austin Lai <[email protected]>

* Update import-csv.md (#3080)

Make time column references consistent

Signed-off-by: Solar Olugebefola <[email protected]>

* Update new about compression (#3012)

* live-migration: Add docs changes for v0.0.10 (#3091)

Signed-off-by: Arunprasad Rajkumar <[email protected]>

* Add Google CA (#3090)

We've added a new CA provider to our cloud, reflecting this fact in the
docs.

* Update ruby.md (#3065)

* Update ruby.md 

Modify references to Timescale/TimescaleDB/Timescale Cloud

Signed-off-by: Solar Olugebefola <[email protected]>

* fix: update TOC links to match headings.

* fix: update TOC links to match headings.

* fix: correct product name.

---------

Signed-off-by: Solar Olugebefola <[email protected]>
Co-authored-by: Iain Cox <[email protected]>

* live-migration: Add docs changes for v0.0.11 (#3096)

Signed-off-by: Arunprasad Rajkumar <[email protected]>

* Move multinode migration under playbooks (#3110)

* 3113 docs rfc add new editor (#3115)

* First draft

* Complete article, add images

* Add a screenshot of PopSQL

* Update popsql_button_in_console

Highlighted the button

* Wrap text

* feat: update PopSQL page, get started and update sections with PopSQL information.

* feat: link to images on S3.

* fix: language cleanup.

* Apply suggestions from code review

Co-authored-by: sremertimescale <[email protected]>
Signed-off-by: Iain Cox <[email protected]>

* fix: language cleanup after review.

* fix: another language review.

* fix: updates following review by Ramon.

* Update use-timescale/popsql.md

Co-authored-by: Ramon Guiu <[email protected]>
Signed-off-by: Iain Cox <[email protected]>

* Update use-timescale/query-data/about-query-data.md

Co-authored-by: Ramon Guiu <[email protected]>
Signed-off-by: Iain Cox <[email protected]>

* fix: add contact information to add more seats.

---------

Signed-off-by: Iain Cox <[email protected]>
Co-authored-by: Rahil Sondhi <[email protected]>
Co-authored-by: sremertimescale <[email protected]>
Co-authored-by: Ramon Guiu <[email protected]>

* fix: numbering and full stop. (#3120)

* missing word in add_reorder_policy (#3106)

Signed-off-by: Sam Debruyn <[email protected]>
Co-authored-by: Iain Cox <[email protected]>

* Restrict usage of time_bucket_ng in CAggs (#3112)

The function time_bucket_ng is deprecated and should not be used in new
CAggs. 2.15.0 will also block the usage of this function.

* fix: broken links. (#3123)

* fix: broken links.

* fix: broken links.

* Update CAgg add_continuous_aggregate_policy documentation (#3128)

* fix: deprecate distributed hypertables from the API reference. (#3134)

* fix: update API reference for set_integer_now_func. (#3135)

* fix: update API reference for set_integer_now_func.

Co-authored-by: Jônatas Davi Paganini <[email protected]>
Signed-off-by: Iain Cox <[email protected]>

* Add compression settings information views (#3142)

* fix: using the public API to check the  chunk time interval for a hypertable. (#3141)

* 3144 docs rfc be more clear about tiered storage being on timescale not on self hosted (#3145)

* feat: update tiered storage overview

* feat: update tiered storage overview

* Update alter_job.md (#3103)

Added missing information in alter_job options

Signed-off-by: Fabrízio de Royes Mello <[email protected]>
Co-authored-by: Iain Cox <[email protected]>

* fix: some broken links. A couple of text updates. (#3151)

* feat: FAQ for live migration (#3139)

Signed-off-by: Harkishen-Singh <[email protected]>

* chore: update live-migration to v0.0.13 (#3150)

Signed-off-by: Harkishen-Singh <[email protected]>

* Add pgstattuple to list of available extensions (#3154)

* fix: clarify that each service hosts a single service only. (#3161)

* fix: change link in Migrate the entire database at once to the live migration page. (#3159)

* Add warning about SERIAL types and dual-write (#3162)

* Address possible installation issues regarding psycopg2 (#3165)

* Address possible installation issues regarding psycopg2

* Address possible installation issues regarding psycopg2

* chore: few language updates for the install procedure.

---------

Co-authored-by: billy-the-fish <[email protected]>

* Add troubleshooting note about partial continuous aggregates (#3164)


Signed-off-by: James Guthrie <[email protected]>
Co-authored-by: billy-the-fish <[email protected]>

* 2829 obsolete restriction on replication factor 1 still in the docs 1 (#3129)

* fix: update create_distributed_hypertable

* Recommend live-migration instead of dual-writes (#3173)

* Refer to live-migration 0.0.14 (#3179)

Signed-off-by: Arunprasad Rajkumar <[email protected]>

* feat: add version downgrade for live-migration FAQ (#3163)

Signed-off-by: Harkishen-Singh <[email protected]>

This commit adds a FAQ question that helps users to downgrade
TimescaleDB version on their target database to match with the
source database. This is useful when the source database (MST) does not
have the TimescaleDB version that the target database is installed
with.

* Make some changes to refresh cagg note (#3146)

* Make some changes to refresh cagg note

This fixes one typo and changes a few other words.

Signed-off-by: RobAtticus <[email protected]>

* chore: updates after review.

---------

Signed-off-by: RobAtticus <[email protected]>
Co-authored-by: Iain <[email protected]>

* Add role based auth to CloudWatch exporter doc (#3156)

* Add role based auth to CloudWatch exporter doc

Role based authentication is added for CloudWatch exporter that allows
to authenticate with users' AWS accounts and we need to document
configuration that needs to be done on the user side to make it work.

* add provider URLs and role example

* Clarify what the URL serves for

* chore: first go updating the configuration procedure.

* chore: updates after first review.

* chore: updates after first review.

* Update use-timescale/metrics-logging/integrations.md

Nice.

Co-authored-by: Danil Zhigalin <[email protected]>
Signed-off-by: Iain Cox <[email protected]>

* fix: update metatags for MDX.

* Replace trust policy to a role

AWS docs refer to this as a role. Make sure we are talking about the
same thing in our docs.

* Update use-timescale/metrics-logging/integrations.md

Co-authored-by: Nicolas Nosenzo <[email protected]>
Signed-off-by: Iain Cox <[email protected]>

---------

Signed-off-by: Iain Cox <[email protected]>
Co-authored-by: niconosenzo <[email protected]>
Co-authored-by: billy-the-fish <[email protected]>
Co-authored-by: Iain Cox <[email protected]>

* fix: document actual behavior of add_continuous_aggregate_policy when using if_not_exists (#3037)

Signed-off-by: dtext <[email protected]>
Co-authored-by: Iain Cox <[email protected]>

* fix: add verb to sentence. (#3131)

* fix: add verb to sentence.
* fix: update after review.

* fix: update navigation and links for page merge.

* fix: clean up de-chunking.

* chore: migrate changes in future versions back to this PR.

* fix: my fatal weakness.

* fix: one broken link fixed.

* fix: more link fixing after manual check.

---------

Signed-off-by: Iain Cox <[email protected]>
Signed-off-by: Florian Mt <[email protected]>
Signed-off-by: Austin Lai <[email protected]>
Signed-off-by: Solar Olugebefola <[email protected]>
Signed-off-by: Arunprasad Rajkumar <[email protected]>
Signed-off-by: Sam Debruyn <[email protected]>
Signed-off-by: Iain Cox <[email protected]>
Signed-off-by: Fabrízio de Royes Mello <[email protected]>
Signed-off-by: Harkishen-Singh <[email protected]>
Signed-off-by: James Guthrie <[email protected]>
Signed-off-by: RobAtticus <[email protected]>
Signed-off-by: dtext <[email protected]>
Co-authored-by: Jay Allen <[email protected]>
Co-authored-by: James Sewell <[email protected]>
Co-authored-by: Florian Mt <[email protected]>
Co-authored-by: Ante Kresic <[email protected]>
Co-authored-by: James Guthrie <[email protected]>
Co-authored-by: Austin Lai <[email protected]>
Co-authored-by: Solar Olugebefola <[email protected]>
Co-authored-by: Arunprasad Rajkumar <[email protected]>
Co-authored-by: Danil Zhigalin <[email protected]>
Co-authored-by: Konstantinos Gkiokas <[email protected]>
Co-authored-by: Rahil Sondhi <[email protected]>
Co-authored-by: sremertimescale <[email protected]>
Co-authored-by: Ramon Guiu <[email protected]>
Co-authored-by: Sam Debruyn <[email protected]>
Co-authored-by: Jan Nidzwetzki <[email protected]>
Co-authored-by: Jônatas Davi Paganini <[email protected]>
Co-authored-by: Fabrízio de Royes Mello <[email protected]>
Co-authored-by: Harkishen Singh <[email protected]>
Co-authored-by: John Pruitt <[email protected]>
Co-authored-by: alejandrodnm <[email protected]>
Co-authored-by: RobAtticus <[email protected]>
Co-authored-by: niconosenzo <[email protected]>
Co-authored-by: dtext <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants