Activities

April 2017
M T W T F S S
« Feb    
 12
3456789
10111213141516
17181920212223
24252627282930

MySQL Enterprise Backup (MEB) : How to Backup and Restore a DB server

We are maintaining a huge MySQL database server which having more than 3TB data. It’s very had for us to backup, move and restore those databases from one host to another or once network to another due to size constrains. Moreover any of these activities would be involved lots of time and work precision during any of db restoration and backup.

After a Googling it is identify that MySQL Enterprise version have all those features to fast backup and restore with better performance. But it is not freely available to download :-p . It is identified that their backup involve the following facts

Note: It is not possible to take single database backup/restore using MEB tool(did not tried :-))

MEB Backup Logic

a. It’s possible to backup Online without having any table lock or performance change.But recommending it to execute on off business hours.
b. MySQL Enterprise Backup (MEB) compress the backup in .bz format (This is optional and we’ve chosen this since we have TB storage) and store it on –backup-dir location. Hence the backup size will be noticeable less than the actual data storage and portable.

MEB Restore Logic

a. Apply-log and uncompress option will decompress the data files in backup location and make it ready for further action. This will decompress the files in backup location without over writing it. But I found that huge compressed files will be deleted once it is decompressed.

b. Copy backup option will be copy the actual data files from backup location those are decompressed and move it to “–datadir” path which mention in the my.cnf file.

MEB backup involves three steps if we compress the backup.

1. How do I backup the MySQL server using MEB

  /opt/meb-3.8/bin/mysqlbackup -u root --backup-dir="/storage/dbbackup/backup-`date +%F`/" --compress --compress-level=1   --show-progress backup --socket=/var/lib/mysql/mysql.sock --show-progress=file:/tmp/mebProgress`date +%Y-%m-%d-%H-%M`.log     2>&1 | tee /tmp/mysqlbackupFull.`date +%Y-%m-%d-%H-%M`.log

–backup-dir : Location of backup files to be stored
–socket : Will identify the running sever and collect the mysql data location to be backed up.
/tmp/mysqlbackupFull and –show-progress=file : These options will capture all the MEB output for identifying the progress. We uses a screen session from Linux box to trigger the backup as it will take 6-8 hours to complete.

170421 00:31:53 mysqlbackup: INFO: Copying the database directory 'Docs'
170421 00:31:53 mysqlbackup: INFO: Copying the database directory 'mysql'
170421 00:31:53 mysqlbackup: INFO: Copying the database directory 'performance_schema'
170421 00:31:53 mysqlbackup: INFO: Copying the database directory 'test'
170421 00:31:53 mysqlbackup: INFO: Copying the database directory 'test1'
170421 00:31:53 mysqlbackup: INFO: Copying the database directory 'tests'
170421 00:31:53 mysqlbackup: INFO: Completing the copy of all non-innodb files.
170421 00:31:54 mysqlbackup: INFO: Progress: 20 of 20 MB; state: Copying metadata to image; compression: -
170421 00:31:54 mysqlbackup: INFO: A copied database page was modified at 5312129472.
          (This is the highest lsn found on page)
          Scanned log up to lsn 5312132652.
          Was able to parse the log up to lsn 5312132652.
          Maximum page number for a log record 428
170421 00:31:54 mysqlbackup: INFO: All tables unlocked
170421 00:31:54 mysqlbackup: INFO: All MySQL tables were locked for 0.914 seconds.
170421 00:31:54 mysqlbackup: INFO: Progress: 20 of 20 MB; state: Completed; compression: -
170421 00:31:54 mysqlbackup: INFO: Compressed 18 MB of data files to 1092 kbytes (compression 94.08%).
170421 00:31:54 mysqlbackup: INFO: Compress Backup operation completed successfully.
170421 00:31:54 mysqlbackup: INFO: Backup created in directory '/var/dbbackup/backup-2017-04-21/'

-------------------------------------------------------------
   Parameters Summary
-------------------------------------------------------------
   Start LSN                  : 5312129024
   End LSN                    : 5312132652
-------------------------------------------------------------

mysqlbackup completed OK!

2. How to Decompress the backup

/opt/meb-3.8/bin/mysqlbackup --backup-dir="/var/dbbackup/backup-2017-04-20" --uncompress apply-log  2>&1 | tee /tmp/Extracing_backup.`date +%Y-%m-%d-%H-%M`.log

–backup-dir : backup location which MEB push the backup earlier.

 mysqlbackup: INFO: Creating 1 buffers each of size 65678.
170421 00:48:02 mysqlbackup: INFO: Apply-log operation starts with following threads
                1 read-threads    1 process-threads
170421 00:48:02 mysqlbackup: INFO: ibbackup_logfile's creation parameters:
          start lsn 5312129024, end lsn 5312132652,
          start checkpoint 5312129472.
InnoDB: Doing recovery: scanned up to log sequence number 5312132652
 mysqlbackup: INFO: InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 Setting log file size to 536870912
InnoDB: Progress in MB: 100 200 300 400 500
Setting log file size to 536870912
InnoDB: Progress in MB: 100 200 300 400 500
170421 00:48:05 mysqlbackup: INFO: We were able to parse ibbackup_logfile up to
          lsn 5312132652.
170421 00:48:05 mysqlbackup: INFO: The first data file is '/var/dbbackup/backup-2017-04-21/datadir/ibdata1'
          and the new created log files are at '/var/dbbackup/backup-2017-04-21/datadir'
170421 00:48:05 mysqlbackup: INFO: Apply-log operation completed successfully.
170421 00:48:05 mysqlbackup: INFO: Full backup prepared for recovery successfully.

mysqlbackup completed OK! with 1 warnings

3. How do I restore the MEB backup

Restoration involves lots of steps. First we need to stop the MySQL service and remove the data directory. Pls take a copy of mysql database and other tables if you required. Mysql db backup can help to retain the existing permission if you are using same db names on target location.

a. Stop the Mysql service after taking the necessary backup.
b. Delete/move the data folder path shown at ‘–datadir’ variable in my.cnf file.
c. Apply the below command and change the path –datadir with your location.

/opt/meb-3.8/bin/mysqlbackup --backup-dir="/var/dbbackup/backup-2017-04-20"  copy-back  --datadir=/var/lib/mysql --show-progress=file:/tmp/mebProgress`date +%Y-%m-%d-%H-%M`​.log     2>&1 | tee /tmp/mysqlrestore.`date +%Y-%m-%d-%H-%M`.log
[root@ ~]# /opt/meb-3.8/bin/mysqlbackup --backup-dir="/var/dbbackup/backup-2017-04-21"  copy-back  --datadir=/var/lib/mysql --show-progress=file:/tmp/mebProgress`date +%Y-%m-%d-%H-%M`​.log     2>&1 | tee /tmp/mysqlrestore.`date +%Y-%m-%d-%H-%M`.log
MySQL Enterprise Backup version 3.8.2 [2013/06/18]
Copyright (c) 2003, 2012, Oracle and/or its affiliates. All Rights Reserved.

 mysqlbackup: INFO: Starting with following command line ...
 /opt/meb-3.8/bin/mysqlbackup
        --backup-dir=/var/dbbackup/backup-2017-04-21 copy-back
        --datadir=/var/lib/mysql
        --show-progress=file:/tmp/mebProgress2017-04-21-00-49​.log

IMPORTANT: Please check that mysqlbackup run completes successfully.
           At the end of a successful 'copy-back' run mysqlbackup
           prints "mysqlbackup completed OK!".

--------------------------------------------------------------------
                       Server Repository Options:
--------------------------------------------------------------------
  datadir = /var/lib/mysql
  innodb_data_home_dir = /var/lib/mysql
  innodb_data_file_path = ibdata1:10M:autoextend
  innodb_log_group_home_dir = /var/lib/mysql
  innodb_log_files_in_group = 2
  innodb_log_file_size = 512M
  innodb_page_size = Null
  innodb_checksum_algorithm = innodb

--------------------------------------------------------------------
                       Backup Config Options:
--------------------------------------------------------------------
  datadir = /var/dbbackup/backup-2017-04-21/datadir
  innodb_data_home_dir = /var/dbbackup/backup-2017-04-21/datadir
  innodb_data_file_path = ibdata1:10M:autoextend
  innodb_log_group_home_dir = /var/dbbackup/backup-2017-04-21/datadir
  innodb_log_files_in_group = 2
  innodb_log_file_size = 536870912
  innodb_page_size = 16384
  innodb_checksum_algorithm = innodb

 mysqlbackup: INFO: Uses posix_fadvise() for performance optimization.
 mysqlbackup: INFO: Creating 14 buffers each of size 16777216.
170421 00:49:50 mysqlbackup: INFO: Copy-back operation starts with following threads
                1 read-threads    1 write-threads
170421 00:49:50 mysqlbackup: INFO: Copying the log file 'ib_logfile0'
170421 00:49:50 mysqlbackup: INFO: Copying the log file 'ib_logfile1'
170421 00:49:50 mysqlbackup: INFO: Copying /var/dbbackup/backup-2017-04-21/datadir/ibdata1.
170421 00:49:50 mysqlbackup: INFO: Copying /var/dbbackup/backup-2017-04-21/datadir/fmi/Document.ibd.
170421 00:49:50 mysqlbackup: INFO: Copying /var/dbbackup/backup-2017-04-21/datadir/fmiDocument/DocumentContentTypes.ibd.
170421 00:49:50 mysqlbackup: INFO: Copying /var/dbbackup/backup-2017-04-21/datadir/fmi/adsdsy.ibd.
170421 00:49:50 mysqlbackup: INFO: Copying /var/dbbackup/backup-2017-04-21/datadir/testasd/test.ibd.
170421 00:49:50 mysqlbackup: INFO: Copying the database directory 'fmiDocument'
170421 00:49:50 mysqlbackup: INFO: Copying the database directory 'mysql'
170421 00:49:50 mysqlbackup: INFO: Copying the database directory 'performance_schema'
170421 00:49:50 mysqlbackup: INFO: Copying the database directory 'test'
170421 00:49:50 mysqlbackup: INFO: Copying the database directory 'test1'
170421 00:49:50 mysqlbackup: INFO: Copying the database directory 'testDss'
170421 00:49:50 mysqlbackup: INFO: Completing the copy of all non-innodb files.
170421 00:49:50 mysqlbackup: INFO: Copying the log file 'ib_logfile0'
170421 00:49:51 mysqlbackup: INFO: Copying the log file 'ib_logfile1'
170421 00:49:53 mysqlbackup: INFO: Copy-back operation completed successfully.
170421 00:49:53 mysqlbackup: INFO: Finished copying backup files to '/var/lib/mysql'
mysqlbackup completed OK!
[root@ ~]#

Now you would need to grant the permission to MySQL data folder and then start it.

170421 00:49:53 mysqlbackup: INFO: Finished copying backup files to '/var/lib/mysql'
mysqlbackup completed OK!
[root@fina-peak-qa01 ~]# chown -R mysql.mysql /var/lib/mysql
[root@fina-peak-qa01 ~]# chmod -R 770 /var/lib/mysql
[root@fina-peak-qa01 ~]# service mysql start
Starting MySQL.. SUCCESS!
[root@fina-peak-qa01 ~]#

############################# Pls note MEB IS NOT FREELY DOWNLOADABLE ###############################

1 comment to MySQL Enterprise Backup (MEB) : How to Backup and Restore a DB server

  • mixmox

    about MEB
    i need more information about this transfer,
    the company accept to pay for enterprise version?
    10,000$ for this product? very expensive !

    there is no way to transfer huge database like this? just buy enterprise version ?

Leave a Reply

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>