v1.10

Document

How to Start SequoiaDB

Basic Operation

Data Model

Database Management

Aggregation

Index

Replication

Cluster

Connector

Reference

Development

Release Note

FAQ

  • Content
  • Comments
  • History

Backup & Recovery Using SequoiaDB Tools

Detail and examples of backup and recovery of SequoiaDB.

Backup Operations

The backup of SequoiaDB is executed on the sdb client with specific schema. Related functions include: db.backupOffline(), db.listBackup(), db.removeBackup(). Off-line backup is supported, but add, update, delete actions will be suspend during the backup process, only query is available.

There are two types of back up:

  • Full Backup: back up the whole database with data, settings and logs.
  • Incremental Backup: back up the new updated data, log or settings after the last backup.

Backup Instructions:

  1. Confirm that there is no current add, update, delete or other alteration actions before backup operation.

  2. Enter sdb client:
    # /opt/sequoiadb/bin/sdb
  3. Connect coord nodes:
    > var db = new Sdb('htest1', 11810)
  4. Check whether there is any backup:

    Check all the data in the database cluster:
    > db.list(4)
    {
      "Name": "foo.bar"
    }
    Return 1 row(s).
    Takes 0.2882s.
    > db.foo.bar.count()
    2880000
    Takes 0.2586s.
    > db.listBackup()
    Return 0 row(s).
    Takes 0.6483s.
  5. Execute database backup.

    The cluster backup is executing by groups, so we must set the group first. Following action is to backup a CatalogNode, and will check when backup complete. The backup file is saved at host htest1, under /opt/sequoiadb/database/cata/30000/bakfile path.
    > db.backupOffline({Name:"catalogBackup",GroupName:"SYSCatalogGroup"})
    Takes 1.762381s.
    > db.listBackup()
    {
          "Name": "catalogBackup",
          "NodeName": "htest1:30000",
          "GroupName": "SYSCatalogGroup",
          "EnsureInc": false,
          "BeginLSNOffset": 0,
          "EndLSNOffset": 9716,
          "StartTime": "2014-07-29-22:34:40",
          "HasError": false
    }
    Return 1 row(s).
    Takes 0.7994s.
    DataNodes are using the same backup type, the file is saved at host htest3, under /opt/sequoiadb/database/data/41000/bakfile/ content.
    > db.backupOffline({Name:"datagroupG1Backup",GroupName:"g1"})
    Takes 12.85741s.  
    > db.listBackup({Name:"datagroupG1Backup",GroupName:"g1"})
    {
          "Name": "datagroupG1Backup",
          "NodeName": "htest3:41000",
          "GroupName": "g1",
          "EnsureInc": false,
          "BeginLSNOffset": 0,
          "EndLSNOffset": 460800544,
          "StartTime": "2014-07-29-22:41:40",
          "HasError": false
    }
    Note: If you want to backup the whole database, you don’t need to set any parameters. Then it will backup the whole database including CatalogNodes and DataNodes. The backup files are saved at the local main node’s /opt/sequoiadb/database/../../dialog/bakfile cotent.
    > db.backupOffline()
  6. See the backup file:
    > db.listBackup()
    {
      "Name": "catalogBackup",
      "NodeName": "htest1:30000",
      "GroupName": "SYSCatalogGroup",
      "EnsureInc": false,
      "BeginLSNOffset": 0,
      "EndLSNOffset": 9716,
      "StartTime": "2014-07-29-22:34:40",
      "HasError": false
    }
    {
      "Name": "datagroupG1Backup",
      "NodeName": "htest3:41000",
      "GroupName": "g1",
      "EnsureInc": false,
      "BeginLSNOffset": 0,
      "EndLSNOffset": 460800544,
      "StartTime": "2014-07-29-22:41:40",
      "HasError": false
    }
    Return2 row(s).
    Takes 0.16428s.

    If not set the path, the database will be automatically saved to the main node's cotent bakfile. If saved in other specific content, queries need the content for a backup file.

Data Recovery

Using backup files to recovery data of a data shred, the data shred must be stopped from any current actions. The recovery process will clear all the record and logs in the data node, then copy the data and others from the backup files to data node. The recovery tools are at content /opt/sequoiadb/bin/sdbrestore, Use command ./sdbrestore --help to see all the parameters.

Recovery Processes:

  1. Stop any group service process:

    It's better to stop database service with sdb client, then we don't need to access the node's host to stop it. (Warning: If the group you are going to stop has a catalog Node, it must be stopped last.)
    sdbadmin@htest1:~$ /opt/sequoiadb/bin/sdb
    > var db = new Sdb('localhost',11810)
    > datarg = db.getRG('g1')
    > datarg.stop()
  2. Copy the database backup to the shared cotent:

    Because SequoiaDB can only recover one host at once, we need to copy the backup file to the shared content where the cluster at. Like /mnt/xiaojun/BACKUP then :
    sdbadmin@htest1:/opt/sequoiadb$ cp -r /opt/sequoiadb/database/cata/30000/bakfile/ /mnt/xiaojun/BACKUP/bakfile.30000.htest1
    sdbadmin@htest3:/opt/sequoiadb$ cp -r /opt/sequoiadb/database/data/41000/bakfile/ /mnt/xiaojun/BACKUP/bakfile.41000.htest3

    Copying backup file to the shared content will only share the same shared content, not need to copy the backup file to each host when recovering.

  3. Data recovery steps:

    The recovery tools are under SequoiaDB's bin content /opt/sequoiadb/bin .Use sdb shell to access the datanode to execute the revocery. When recovering, more than half of the nodes in group must be recovered, you may better recover all the nodes. The cluster shown below consists of 2 groups, a CatalogGroup 'SYSCatalogGroup' and a DataGroup 'G1', deployed on 3 hosts respectively: htest1/htest2/htest3. Recovering the node need to access the host, like htest1, we need to do following commands:
    sdbadmin@htest1:/opt/sequoiadb/bin$ ./sdbrestore -p /mnt/xiaojun/BACKUP/bakfile.30000.htest1/ -n catalogBackup --dbpath 
    > /opt/sequoiadb/database/cata/30000/ --confpath /opt/sequoiadb/conf/local/30000/ --svcname 30000
    2014-07-29-23.09.19.382036               Level:EVENT
    PID:9556                                 TID:9556
    Function:pmdRestoreThreadMain            Line:491
    File:SequoiaDB/engine/pmd/sdbrestore.cpp
    Message:
    Start sdbrestore [Ver: 1.8, Release: 13673, Build: 2014-07-22-00.11.23(Debug)]...
                       .
                       .
                       .
    *****************************************************
    Restore succeed!
    *****************************************************

    Then do the same to htest2 and htest3.

    After access the CataLog Nodes, now we can recover the data of G1, similar to CatalogNodes, we need to recover all 3 hosts:
    sdbadmin@htest1:/opt/sequoiadb/bin$ ./sdbrestore -p /mnt/xiaojun/BACKUP/bakfile.41000.htest3/ -n datagroupG1Backup --dbpath 
    > /opt/sequoiadb/database/data/41000/ --confpath /opt/sequoiadb/conf/local/41000/ --svcname 41000
    2014-07-29-23.18.08.966708               Level:EVENT
    PID:9590                                 TID:9590
    Function:pmdRestoreThreadMain            Line:491
    File:SequoiaDB/engine/pmd/sdbrestore.cpp
    Message:
    Start sdbrestore [Ver: 1.8, Release: 13673, Build: 2014-07-22-00.11.23(Debug)]...
                       .
                       .
                       .
    *****************************************************
    Restore succeed!
    *****************************************************
  4. Check if the recovery succeed:

    Use sdb client to check if the result is right:
    sdbadmin@htest1:~$ /opt/sequoiadb/bin/sdb
    > var db = new Sdb('htest1',11810)
    Takes 0.8073s.
    > db.list(4)
    {
      "Name": "foo.bar"
    }
    Return 1 row(s).
    Takes 0.47547s.
    > db.foo.bar.count()
    2880000
    Takes 0.164788s.
    > db.foo.bar.find().limit(1)
    {
      "_id": 4000000,
      "id": 4000000,
      "ID": "http://www.sequoiadb.com",
      "change": "ccc",
      "info": "email:contact@sequoiadb.com"
    }
    Return 1 row(s).
    Takes 0.28824s.
    > db.foo.bar.insert({testing:"testingBackup"})
    Takes 0.296942s.
    > db.foo.bar.find({testing:"testingBackup"})
    {
      "_id": {
        "$oid": "53d7c4f9c6e6fab31f000000"
      },
      "testing": "testingBackup"
    }
    Return 1 row(s).
    Takes 4.311792s.

    Then we may confirm the recovery is successful in the end.

please login to comment.
Latest Comment
  • 2015-02-14

About Us

SequoiaDB is a financial-level distributed database vendor and is the first Chinese database listed in Gartner’s Magic Quadrant OPDBMS report. SequoiaDB has recently released version 3.0.
SequoiaDB is now penetrating the vertical sector Financial Industry quickly and had more than 50 banking clients and hundreds of enterprise customers in industries including government, telecommunication, Internet and IoT.

Beijing:
Tower R, No.8 North Star East Road, Chaoyang District, Beijing,China
Guangzhou:
Tower A, No.22 Qinglan Street, Panyu District, Guangzhou,China
Shenzhen:
Tsing Hua Tech Park, Nanshan District, Shenzhen,China
Tel:400-8038-339
E-mail:contact@sequoiadb.com