What is gbak?
1. Mastering Backups with Gbak
1.1 The simplest Firebird backup with gbak command
1.2. Local backup with gbak which can be done online on Windows
1.3. Backup with gbak with TCP/IP connection string
1.4. Faster backup with gbak with Service Manager
1.5. The fastest backup with gbak with Service Manager and inhibited garbage collection
1.6. Backup to the network share or network location
1.7. Simple backup from the remote server to local machine
1.8. Faster backup from a remote server to local machine with Service Manager
1.9. Backup Firebird database on the remote server to the same remote server using Service Manager
1.10. Backup Firebird database 6x faster with HQbird
2. Restore with Gbak tool
2.1. The simplest restore command
2.2. Restore with localhost connection string
2.3. Restore with XNET on Windows
2.4. Faster restore with Service Manager
2.5. Non-recommended switch
2.6. Restore database using alias
2.7. Restore local backup to the remote server
2.8. Restore the local backup to the remote server with Service Manager
2.9. Restore extremely long tables
3. Tweaking and logging backup and restore processes
3.1. Gbak with verbose output
3.2. Add performance statistics to verbose output
3.3. Exclude tables from the backup and/or from the restore
3.4. Fetch password for backup or restore from the file
4. One-step backup-restore
5. Performance summary
Very Frequently Asked Question About VM and Firebird Backups
Appendix A. Errors during backup/restore
mkdir /db chown firebird -R /db
gbak -b c:\Data\test1.fdb C:\data\backup1.fbk -user SYSDBA -pass masterkeyLinux
gbak -b /db/test1.fdb /db/backup1.fbk -user SYSDBA -pass masterkeyIn this example, gbak tool accesses database file using local or embedded access.
gbak -b xnet://c:\Data\test1.fdb C:\data\backup1.fbk -user SYSDBA -pass masterkeyFor Firebird 2.5, we can use a local connection string, and it will use XNET too:
gbak -b c:\Data\test1.fdb C:\data\backup1.fbk -user SYSDBA -pass masterkeyOn Linux, Firebird doesn't support a specific local protocol like XNET on Windows, so it is necessary to use TCP/IP connection string (see section 1.3).
gbak -b localhost:c:\Data\test1.fdb C:\data\backup1.fbk -user SYSDBA -pass masterkeyLinux
gbak -b localhost:/db/test1.fdb /db/backup1.fbk -user SYSDBA -pass masterkeyIn this case, by specifying localhost: at the beginning of the database path, the connection is done through the network subsystem of Firebird.
gbak -b localhost/3051:c:\Data\test1.fdb C:\data\backup1.fbk -user SYSDBA -pass masterkeyLinux
gbak -b localhost/3051:/db/test1.fdb /db/backup1.fbk -user SYSDBA -pass masterkeyTime to backup: 182 seconds
gbak -b -se localhost:service_mgr c:\Data\test1.fdb c:\Data\backup1.fbk -user SYSDBA -pass masterkeyLinux
gbak -b -se localhost:service_mgr /db/test1.fdb /db/backup1.fbk -user SYSDBA -pass masterkeyThis command uses switch -service to specify that we want to use Service Manager of Firebird instance on port 3050 to perform the backup.
In this case, the backup will be performed directly inside the Firebird process (it has a copy of gbak code), and since the in-process communication is much faster, the backup will be significantly faster in this case.
If Firebird is running on the non-standard (e.g., 3051), the command can look like:
gbak -b -se localhost/3051:service_mgr c:\Data\test1.fdb c:\Data\backup1.fbk -user SYSDBA -pass masterkeyNote: There is a significant limitation in Firebird 2.5 and Firebird 3.0.0-3.0.5 (removed only in 3.0.6): the command line (all parameters and paths for the database and for the backup) must be less than 256 symbols.
If you hit this limit, for example, due to long paths of database and backup, you can declare an alias for the database in databases.conf (3.0 and higher) or aliases.conf (2.5):
mydb1=/db/test1.fdb #linuxand then use it in our command:
gbak -b -se localhost:service_mgr mydb1 c:\Data\backup1.fbk -user SYSDBA -pass masterkeyLinux
gbak -b -se localhost:service_mgr mydb1 /db/backup1.fbk -user SYSDBA -pass masterkey
Time to backup: 115 seconds
-G(ARBAGE_COLLECT) inhibit garbage collectionSo, the backup command will be the following
gbak -b -se localhost:service_mgr -g mydb1 c:\Data\backup1.fbk -user SYSDBA -pass masterkeyLinux
gbak -b -se localhost:service_mgr -g mydb1 /db/backup1.fbk -user SYSDBA -pass masterkey
Switch -g forces Firebird engine to disable garbage collection for the backup process in the database file.
It does not mean that garbage record versions will be stored in the backup file, it means that the server will not try to clean existing garbage in the database during backup, and backup will be faster.
We strongly recommend using this switch, because we believe that garbage collection and associated cleaning should be done by sweep (gfix -sweep or autosweep), so better do not consider gbak as any kind of alternative of the sweep.
Time to backup: 105 seconds
The often confusion of new Firebird users: manual backup (when you start command from a command prompt), with simple gbak -b, to the network share works fine, but the fast version of gbak with -se localhost:service_mgr, does not work.
The reason is that Firebird on Windows runs under LocalSystem account, which does not have access to the network locations (unless these network shares have configured access for the group “Everyone”, but this is very very dangerous in our era of ransomware).
The solution is to run Firebird service on Windows under the account with enough rights to access the network share, and, simultaneously, enough rights to access local database files and system files in C:\ProgramData\Firebird. Also, a good idea is to configure parameter RestrictAccess in firebird.conf.
Since Firebird on Linux runs under account "firebird", mount network share with mapping to user “firebird”, so Firebird service will be able to access the network location in the same way as a local drive.
gbak -b -user SYSDBA -pass masterkey 192.168.0.108:/db/test1.fdb c:\data\remotebackup1.fbkTime to backup: 568 seconds
gbak -b -se 192.168.0.108:service_mgr -user SYSDBA -pass masterkey /db/test1.fdb stdout > C:\Data\remoteback1.fbkIt uses Service Manager to do the backup on the remote server, but the output is being sent to the stdout pipe, and then redirected to the local file.
gbak -b -se 192.168.0.108:service_mgr -user SYSDBA -pass masterkey /db/test1.fdb /db/back12.fbkThis command through Service Manager invokes backup on the remote server, with the instruction to store the backup file at the same network server too.
gbak -b -par 8 -se localhost:service_mgr -g C:\Data\testbigdb1.fdb c:\Data\backup1.fbk -user SYSDBA -pass masterkeyAs you can see, there is a new parameter -par 8, which makes gbak to use 8 threads to create a backup.
Let’s assume the file is in C:\Data\backup1.fbk in case of Windows, or /db/backup1.fbk in case of Linux.
gbak -c C:\Data\backup1.fbk C:\data\new1.fdb -user SYSDBA -pass masterkeyOn Linux
gbak -c /db/backup1.fbk /db/new1.fdb -user SYSDBA -pass masterkeyFirst of all, please note that gbak -c does not overwrite the database file, and if there exists file C:\data\new1.fdb or /db/new1.fdb, gbak will return an error saying that the database already exists.
Then, this command is actually working very differently on 2.5/3.0+ and Windows/Linux.
On Linux, this command will use embedded access to the created database (if you did not change in firebird.conf the order of Firebird providers, of course) both for 3.0 and 2.5.
On Windows, on Firebird 3.0 with default order of providers, it will be embedded access, on 2.5 – XNET.
Then, this command creates a file with the rights of the user who started gbak, it is especially important on Linux – if you run such gbak under root, the owner of the database file will be root, and Firebird process, which runs under user “firebird”, will be unable to access the restored file.
Note for Linux users
Many people, in order to “fix” the ownership, apply permission for everybody to access the restored database, i.e., something like “chmod 777 database”, but this is very insecure, the proper way is to change the owner of the database to firebird, with the following command
chown firebird /db/new1.fdb
In general, this command is good enough for the simple restore of non-production databases (used for testing or in development).
Time to restore: 275 seconds
gbak -c C:\Data\backup1.fbk localhost:C:\data\new1.fdb -user SYSDBA -pass masterkey
gbak -c /db/backup1.fbk localhost:/db/new1.fdb -user SYSDBA -pass masterkey
gbak -c C:\Data\backup1.fbk localhost/3051:C:\data\new1.fdb -user SYSDBA -pass masterkeyLinux
gbak -c /db/backup1.fbk localhost/3051:/db/new1.fdb -user SYSDBA -pass masterkeyTime to restore: 1225 seconds
gbak -c C:\Data\backup1.fbk xnet://C:\Data\New2.fdb -user SYSDBA -pass masterkeyOn Firebird 2.5 on Windows, XNET access will be used with the simple command line (if there is only one Firebird instance is running):
gbak -c C:\Data\backup1.fbk C:\data\new1.fdb -user SYSDBA -pass masterkeyTime to restore: 585 seconds
gbak -c -se localhost:service_mgr C:\Data\backup1.fbk C:\data\new1.fdb -user SYSDBA -pass masterkeyLinux
gbak -c -se localhost:service_mgr /db/backup1.fbk localhost:/db/new1.fdb -user SYSDBA -pass masterkeyWith switch -se, we invoke Service Manager on the localhost address and instruct it to perform restore code inside the Firebird engine.
When restore is done by Service Manager, the created database file will be owned by the account of running Firebird instance (process) – it is "firebird" on Linux, and LocalSystem on Windows.
Time to restore: 244 seconds
-R(ECREATE_DATABASE) [O(VERWRITE)] create (or replace if OVERWRITE used) database from backup file (restore)
in order to force replacing of the existing database with the new one.
In our experience, this switch greatly increases the chances to accidentally overwrite the production database.
We strongly recommend restoring the database every time with the new name and rename it, as well as delete the old database, explicitly.
We even will not provide the example of the command with this switch.
For example, we have the following declaration
restdb=c:\Data\newrest1.fdb #Windows restdb=/db/newrest1.fdb #LinuxSo we can run the following command to restore the backup to the path, specified by the alias
gbak -c -se localhost:service_mgr C:\Data\backup1.fbk restdb -user SYSDBA -pass masterkeyLinux
gbak -c -se localhost:service_mgr /db/backup1.fbk restdb -user SYSDBA -pass masterkey
In this example, we restore the backup file stored on Windows, to the Linux server (its IP address 22.214.171.124):
gbak -c C:\Data\backup1.fbk 192.168.0.108:/db/newdb1.fdb -user SYSDBA -pass masterkeyTime to restore: 7009 seconds
gbak -c -se 192.168.0.108:service_mgr -user SYSDBA -pass masterkey stdin /db/new3.fdb < C:\Data\backup1.fbkThis command invokes restore on the remote server with the standard input stdin as the source of backup – and supplies the input using the < C:\Data\backup1.fbk portion of the command.
Looks a bit tricky? But it is an easy way to 10x increase of the gbak performance to restore to the remote server!
Time to restore: 450 seconds
gbak -c -se localhost:service_mgr -one C:\Data\backup1.fbk C:\data\new44.fdb -user SYSDBA -pass masterkey
gbak -b -se localhost/3050:service_mgr -g mydb1 c:\Data\backup1.fbk –v -user SYSDBA -pass masterkeyAs a result, there will be more details. The minor, but annoying problem that printing output to the console can make verbose backup significantly slower than the silent variant, so the good idea will be to save the log to the file with switch -y logfile:
gbak -b -se localhost/3050:service_mgr -g -v mydb1 c:\Data\backup1.fbk -user SYSDBA -pass masterkey -y C:\data\backuplog1.txt
Note: gbak will not overwrite the existing log file! If you already have C:\data\backuplog1.txt in this example, the backup will raise the error (see #3 in Appendix A).
Note 2: there is option -verbint to control the interval to report the number of processed records during backup or restore.
gbak: writing data for table COUNTRY gbak:16 records writtenfor every table and other database objects.
For this, it is necessary to use switch -st(atistics):
-ST(ATISTICS) TDRW show statistics: T time from start D delta time R page reads W page writes
gbak -b -se localhost/3050:service_mgr -g -v mydb1 -st tdrw c:\Data\backup1.fbk -user SYSDBA -pass masterkey -y C:\data\backuplog1.txtWhen applied, it will add to the log the following columns:
gbak: time delta reads writesso we will be able to see time and IO spent on each line.
In the example below, we exclude data from tables COUNTRY and JOB from backup:
gbak -b -se localhost/3050:service_mgr -g -v mydb1 -SKIP_D ‘(COUNTRY|JOB)’ c:\Data\backup1.fbk -user SYSDBA -pass masterkey -y C:\data\backuplog1.txtAnd, in the example below, we exclude table CLIENT from the restore:
gbak -c -se localhost:service_mgr C:\Data\backup1.fbk C:\data\new33.fdb -user SYSDBA -pass masterkey -SKIP_D "CLIENT"Please note that parameter to SKIP_DATA must be transmitted as the single parameter, so it must be in quotes!
SQL> SELECT RDB$RELATION_NAME FROM RDB$RELATIONS WHERE TRIM(RDB$RELATION_NAME) SIMILAR TO '(COUNTRY|JOB)'; RDB$RELATION_NAME =============================== COUNTRY JOBPlease note, that tables from the backup or restore will be excluded regardless of the existing constraints (Foreign Keys), so, if you did not plan such exclusion carefully, it is very easy to receive the error “Cannot commit foreign key index” during the restore process.
Let’s create the file with the password in C:\Data\passfile.txt and use it (here we use a very simple embedded variant, of course, the switch will work with Service Manager too):
gbak -b c:\Data\test1.fdb c:\Data\backup5.fbk -user SYSDBA -fetch C:\Data\passfile.txtThere are 2 practical benefits:
In this case, it is possible to perform backup-restore with the single command, using standard input and output as sources for the appropriate commands, to bypass creating of the intermediate backup file, reduce requirements for the free space, and speed up the process.
The command is the following:
gbak -b -se localhost:service_mgr -g -user SYSDBA -password masterkey C:\Data\test1.fdb stdout | gbak -c -se localhost:service_mgr -user SYSDBA -password masterkey stdin C:\Data\new10.fdbIn essence, here we do 2 commands, united by symbol |,
gbak -b -se localhost:service_mgr -g -user SYSDBA -password masterkey C:\Data\test1.fdb stdoutand the second, for restore from stdin
gbak -c -se localhost:service_mgr -user SYSDBA -password masterkey stdin C:\Data\new10.fdbThis command is the fastest way to do backup-restore on the same Firebird instance.
Please note: for converting databases with one-step backup-restore from 2.5 to 3.0 it is necessary to use 2 instances of Firebird, see details here.
Or, I do backup the complete image of Virtual Machine, why should I bother about backup of Firebird database?
The answer is here.
1) An attempt to run gbak without parameters, or with the non-owner user of the database/non-SYSDBA will lead to the following error:
gbak: ERROR:Unable to perform operation. You must be either SYSDBA or owner of the database gbak:Exiting before completion due to errors
2) If you specify the wrong password, there will be the following error:
gbak: ERROR:Your user name and password are not defined. Ask your database administrator to set up a Firebird login. gbak:Exiting before completion due to errors
3) Error occurs when the existing file is specified as a verbose log destination
gbak: ERROR:cannot open status and error output file C:\data\backuplog1.txt gbak: ERROR: Exiting before completion due to errors gbak:Exiting before completion due to errors
4) Error occurs if the existing database is specified in the gbak restore command as a destination
gbak: ERROR:database C:\data\new1.fdb already exists. To replace it, use the -REP switch gbak:Exiting before completion due to errors5) Error occurs when gbak tries to write a backup to the location where it does not have sufficient rights to write.
gbak: ERROR:cannot open file /db/test1.fbk gbak:Exiting before completion due to errors
6) When gbak tries to access the file without permission to do it – for example, the file has another owner than user “firebird” on Linux
gbak: ERROR:no permission for read-write access to database /db/test1.fdb gbak: ERROR: IProvider::attachDatabase failed when loading mapping cache gbak:Exiting before completion due to errors7) Attempt to use verbose output
C:\HQbird\Firebird30>gbak -se 192.168.0.108:service_mgr -v -st tdrw -user SYSDBA -pass masterkey /db/test1.fdb stdout > c:\data\rembackup2.fbk gbak: ERROR:standard output is not supported when using split operation or in verbose mode gbak: ERROR: Exiting before completion due to errors gbak:Exiting before completion due to errors8) Attempt to do backup with Service Manager on the remote server with enabled verbose and storing to the log file.
C:\HQbird\Firebird30>gbak -se 192.168.0.108:service_mgr -v -st tdrw -y lg1.txt -user SYSDBA -pass masterkey /db/test1.f db stdout > c:\data\rembackup2.fbk gbak: ERROR:Invalid clumplet buffer structure: string length doesn't match with clumplet gbak:Exiting before completion due to errors9) Error in one-step backup-restore when backup fails for some reason:
gbak: ERROR:No request from user for stdin data gbak:Exiting before completion due to errors10) If you are trying to pass non-backup to the gbak
gbak: ERROR:unavailable database gbak:Exiting before completion due to errors gbak: ERROR:expected backup description record gbak:Exiting before completion due to errors11) Backup of a corrupted database file with the wrong page will report the following error (number and database file will differ, of course)
gbak: ERROR:database file appears corrupt (E:\DATABASE1.FDB) gbak: ERROR: wrong page type gbak: ERROR: page 9294588 is of wrong type (expected 8, found 0) gbak: ERROR:gds_$get_segment failed gbak:Exiting before completion due to errors gbak: ERROR:Unexpected I/O error while reading from backup file gbak:Exiting before completion due to errors