Import wiredtiger: d2a6f8ba360b1c36405666ff471352320c4583fb from branch mongodb-master

ref: dcb3808c92..d2a6f8ba36
for: 5.1.0

WT-7007       Backup architecture guide page

Author: alisonfel

src/third_party/wiredtiger/dist/docs_data.py

@@ -11,6 +11,9 @@ def __init__(self, doxygen_name, data_structures, files):
 # List of all architecture subsections
 ##########################################
 arch_doc_pages = [
+    ArchDocPage('arch-backup',
+        ['WT_CURSOR_BACKUP'],
+        ['src/cursor/cur_backup.c', 'src/cursor/cur_backup_incr.c']),
     ArchDocPage('arch-block',
         ['WT_BLOCK', 'WT_BLOCK_CKPT', 'WT_BLOCK_DESC', 'WT_BLOCK_HEADER',
          'WT_BM', 'WT_EXTLIST'],

src/third_party/wiredtiger/import.data

@@ -2,5 +2,5 @@
     "vendor": "wiredtiger",
     "github": "wiredtiger/wiredtiger.git",
     "branch": "mongodb-master",
-    "commit": "dcb3808c928c50c20e8ea46bb44609986d58c75d"
+    "commit": "d2a6f8ba360b1c36405666ff471352320c4583fb"
 }

src/third_party/wiredtiger/src/docs/arch-backup.dox

@@ -0,0 +1,69 @@
+/*! @arch_page arch-backup Backup
+
+# Overview #
+
+Backup in WiredTiger can be performed using the \c "backup:" cursor. WiredTiger
+supports four types of backups.
+
+1. Full database
+2. Block-based incremental backup
+3. Log-based incremental backup
+4. Target backup
+
+The following page describes how backup works inside the WiredTiger. Please refer to
+@ref backup for details on how to use each of these types of backups.
+
+# Full database #
+
+When the application opens the backup cursor, internally WiredTiger generates a list of all
+files in the database that are necessary for the backup. WiredTiger takes both the checkpoint
+and schema locks to block database file modifications while generating the list files. There
+is a contract that all files that exist at the time the backup starts exist for the entire time
+the backup cursor is open. This contract applies to all files in the database, not just files
+that are included in the generated list.
+
+WiredTiger log files are also part of the generated file list for backup. There is a contract
+that the log files do not contain or make visible operations that are added to the log after
+the backup started that could have adverse affects during recovery and restore. Therefore
+WiredTiger forces a log file switch when opening the backup cursor. To fulfill the earlier
+contract that all files must exist during backup, WiredTiger does not use any pre-allocated
+log files nor does it archive or remove old log files during the time the backup cursor is open.
+
+Once the backup is opened successfully, any checkpoints that exist on the database before
+backup starts must be retained until the backup cursor is closed. All the newer checkpoints
+that are created during the backup in progress are cleaned whenever a newer checkpoint is
+created.
+
+# Block-based incremental backup #
+
+Block-based incremental backup is performed by tracking the modified blocks in the checkpoint.
+Whenever a new checkpoint occurs on a file, any new or modified blocks in this checkpoint are
+recorded as a bit string. This bit string information is updated with every new checkpoint.
+
+Whenever the incremental backup cursor is opened to perform the backup, WiredTiger returns all
+the file offsets and sizes that are modified from the previous source backup identifier.
+WiredTiger returns the full file information to be copied for all the new files that are created,
+renamed or imported into the database after the incremental backup cursor is opened. Refer to
+@ref backup_incremental-block for more information on how to use the block-based incremental backup.
+
+# Log-based incremental backup #
+
+When the backup cursor is opened with the \c target configuration string as \c "target=(\"log:\\")"
+the log-based incremental backup is performed by adding all the existing log files in the database
+to the list of files that needs to be copied. Applications wanting to use log files for incremental
+backup must first disable automatic log file removal using the \c log=(archive=false) configuration
+to ::wiredtiger_open. By default, WiredTiger automatically removes log files no longer required
+for recovery. Refer to @ref backup_incremental for more information on how to use the log-based
+incremental backup.
+
+# Target backup #
+
+When the backup cursor is opened with the \c target configuration string as \c "target=",
+WiredTiger internally generates a list of targeted files instead of all files like the full backup.
+If the targeted object is a table, all the indexes and column groups of the table are also backed up.
+
+Note: There can only be one backup cursor open at a time unless using duplicate backup
+cursors. The duplicate backup cursors are used for catching up with the log files or with
+block-based incremental backup. Please refer to @ref backup_duplicate for more details.
+
+*/

src/third_party/wiredtiger/src/docs/arch-index.dox

@@ -123,6 +123,10 @@ wt_file -[hidden]right-> log_file
 
 We go into some detail for some of the internal components.
 
+@subpage arch-backup
+
+Hot backup uses a type of cursor to backup the database.
+
 @subpage arch-block
 
 The Block Manager manages the reading and writing of disk blocks.

src/third_party/wiredtiger/src/docs/backup.dox

@@ -77,6 +77,8 @@ the backup is available in the backup when that log file is included in the
 list of files. WiredTiger offers a mechanism to gather additional log files that
 may be created during the backup.
 
+@section backup_duplicate Duplicate backup cursors
+
 Since backups can take a long time, it may be desirable to catch up at the
 end of a backup with the log files so that operations that occurred during
 backup can be recovered. WiredTiger provides the ability to open a duplicate