Skip to content

[Enhancement] add support for restore to ccr #303

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 358 commits into
base: dev
Choose a base branch
from
Open

[Enhancement] add support for restore to ccr #303

wants to merge 358 commits into from

Conversation

Vallishp
Copy link
Contributor

@Vallishp Vallishp commented Dec 11, 2024 •
edited
Loading

add support for restore to ccr
doris PR apache/doris#45162

  1. DBSync
    1. If restore involves only one table. Then do partial snapshot to that table.
    2. If restore involves multiple tables, then do full snapshot on that db.
  2. TableSync
    1. If restore involves table, then each table needs full sync.

@Vallishp Vallishp mentioned this pull request Dec 12, 2024
Open
16 tasks
w41ter and others added 28 commits January 11, 2025 08:11
@w41ter @Vallishp
Fix case db_sync by send TEST_* query to target (selectdb#52)
0c9303e
@w41ter @Vallishp
Fix case db_sync by send TEST_* query to target (selectdb#55)
27e3769
@lsy3993 @Vallishp
fix sync db error when table name is keyword (selectdb#53)
483833d
@lsy3993 @Vallishp
add option to start pprof server (selectdb#51)
b9b9645
@lsy3993 @Vallishp
skip view when set binlog.enable (selectdb#50)
9b90f28
@ButterBright @w41ter @Vallishp
Make the constants configurable (selectdb#49)
4e93f0f 
Co-authored-by: walter <[email protected]>
@lsy3993 @Vallishp
fix start helper parameter (selectdb#57)
daf36c8 
* fix start helper parameter
* add trace
@w41ter @Vallishp
Add golang workflows
19ea40f
@w41ter @Vallishp
Drop the table if restore is cancelled by signature not matched (sele…
98dd622 
…ctdb#58)
@w41ter @Vallishp
Update changelog after 2.0.3.8
b664229
@w41ter @Vallishp
Fix bloomfilter index case by wait add index finish (selectdb#56)
7b79d5b
@w41ter @Vallishp
Fix materialized view
1753338
@lsy3993 @Vallishp
fix table key word (selectdb#60)
40904a6
@lsy3993 @Vallishp
fix syntax of auto partition in db sync test (selectdb#61)
420c7e1
@w41ter @Vallishp
Add tarball target
b4f8b38
@w41ter @Vallishp
Release 2.0.3.9
fde26c0
@w41ter @Vallishp
Add LICENSE (selectdb#63)
74262cd
@w41ter @Vallishp
Save key time point for ccr job progress (selectdb#64)
a661efc
@w41ter @Vallishp
Fix keyword name in ADD PARTITION (selectdb#65)
3e18d2c
@w41ter @Vallishp
Fix drop table with keyword (selectdb#67)
510fc4e
@w41ter @Vallishp
Fix drop table keyword name if table schema conflicts (selectdb#68)
5662c81
@w41ter @Vallishp
Fix test keyword name test (selectdb#69)
852a5a2
@lsy3993 @w41ter @Vallishp
create a handler to view job details (selectdb#66)
0fc2e5a 
Co-authored-by: walter <[email protected]>
@w41ter @Vallishp
Fix test inverted index (selectdb#70)
e4e6559
@w41ter @Vallishp
Fix keyword name in job.go (selectdb#71)
211ba02
@lsy3993 @Vallishp
create a handler can get job progress (selectdb#72)
dea2407
@w41ter @Vallishp
Avoid restart full sync when network interrupt or process crash (sele…
ea69bb7 
…ctdb#75)
@w41ter @Vallishp
Update CHANGELOG for 2.1.3
a4c3ce9
wyxxxcat and others added 29 commits January 11, 2025 08:11
@wyxxxcat @Vallishp
Fix incorrect test (selectdb#265)
f1ccabe
@w41ter @Vallishp
Support rename partition/rollup (selectdb#268)
57efa0b
@w41ter @Vallishp
Support add/drop rollup binlogs (selectdb#269)
367b64a
@w41ter @Vallishp
Fix modify view info/modify comment binlog in barrier log (selectdb#270)
300ffae
@w41ter @Vallishp
Add rollup schema change cases (selectdb#272)
1f831f0
@w41ter @Vallishp
Fix alter view def (selectdb#273)
5b0725a 
1. Add view to table mapping
2. Remove database prefix in inline view def
@wyxxxcat @Vallishp
[improve] API job_progress shouldn't returns the data field of job pr…
4051aa0 
…ogress (selectdb#271)
@wyxxxcat @Vallishp
Supplement table property compaction_policy test (selectdb#274)
37b2cf2
@w41ter @Vallishp
Fix restore with view (selectdb#275)
a507507
@w41ter @Vallishp
fix test_ds_mv_basic/test_ds_view_basic (selectdb#276)
904ec71
@w41ter @Vallishp
Fix test_ds_view_basic (selectdb#277)
641035b
@w41ter @Vallishp
Support replace table in table sync (selectdb#279)
3187cbd
@w41ter @Vallishp
Update CHANGELOG.md (selectdb#281)
c276db3
@w41ter @Vallishp
Retry handle upsert binlog, to fix meta error (selectdb#282)
98db57a
@w41ter @Vallishp
Fix wrong infinity partition key in create table sql (selectdb#285)
ffa995a
@w41ter @Vallishp
Support create table with agg_state (selectdb#286)
7efc7ab
@w41ter @Vallishp
Fix test_ds_tbl_res_agg_state (selectdb#287)
1638bac
@w41ter @Vallishp
Support private IP and public IP (selectdb#288)
4d72f8c
@w41ter @Vallishp
Update operations.md (selectdb#289)
4a1ea02
@wyxxxcat @Vallishp
Add alter table property for ds/ts test (selectdb#264)
a910cf5
@lsy3993 @Vallishp
Fix spelling mistake (selectdb#291)
49a8ca4
@wyxxxcat @Vallishp
Add db incremental sync test for table property (selectdb#280)
1d97afe
@w41ter @Vallishp
Support add/drop multi inverted indexes (selectdb#296)
7d49241
@w41ter @Vallishp
Fix fullsync commit seq with views (selectdb#297)
a56d056
@wyxxxcat @Vallishp
[TEST] fix error test for ccr (selectdb#300)
c6f99a1
@w41ter @Vallishp
Fix be rpc leaks in rpc factory (selectdb#299)
97de4e3 
According to the golang document, the KeyType of a map
may be any type that is comparable:

1. Two pointer values are equal if they point to the same
variable or if both have value nil
2. Two struct values are equal if their corresponding non-
blank field values are equal.
@Vallishp
[feat](binlog) Add Support recover binlog (selectdb#284)
a110d63
@w41ter @Vallishp
Fix format (selectdb#302)
8955600
@Vallishp
[Enhancement] add support for restore to ccr
b18ce73
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@Vallishp @w41ter @lsy3993 @ButterBright @XueYuhai @smallx @wyxxxcat @xiaoming12306