Skip to main content

Hubble Accelerator Backup and Recovery Strategies

Backing up the Accelerator

Data on the Accelerator which has not been replicated using Qlik Replicate can be back up as described in this topic. The replicated data is not backed up as the replication process will be used to re-import that data.

Storage of Backup Archives

Backup archives are written to the location /import/dbbackup on the Accelerator. This allows the NFS mount feature (see How to Create an NFS mount to a Remote NFS Server on the Accelerator) to be used to redirect the archives onto other network storage devices.

Note: Do not rely on storing the backup files on the same physical server as the Accelerator. When configuring the NFS mount, use dbbackup as the {loca/Jolder_name}.

Note: This NFS mount must be configured before a backup is run. If the folder /import/dbbackup does not exist then any backup operations will fail.

Manually Triggering a Backup

Note: The backup runs for the schema called hubble on the Accelerator ignoring the schema used on the profile.

The backup process is triggered from the Administration tool by right-clicking on a profile configured to use an Accelerator for its data connection.

On clicking Yes, the following message is presented.

If there was an error starting the backup (for example, the Accelerator could not be reached) then this will be shown to the user as an error message.

Any errors which occur during the backup process (for example, insufficient space to store the resulting file) will not be reported back to the user. Instead these will be logged to the file /tmp/ybapi.log on the Accelerator.

Using the REST API to Trigger a Backup

The backup process has a public REST API which can be called using a scheduling tool which allows a URL to be called using the POST Method. The API URL is called using the accelerator server hostname, for example, http://hostname:5000/api/.

The API service can be tested by entering the API URL into a browser, the following should be returned.

REST API Details

The three backup operations that can be performed from the API are, starting a job, checking on its status, and canceling it.

Execute

  • Verb: POST

  • URL:  http://hostname:5000/api/Backup/Execute

  • Parameters: None

  • Result: Status JSON (see below)

    Starts a new backup job and assigns it an ID that can be used for the following calls. The backup process will continue running after the HTTP call has completed.

Status

  • Verb: GET

  • URL:  http://hostname:5000/api/Backup/Status

  • Parameters: 'id' - The ID from a previous status JSON result.

  • Result: Status JSON (see below)

    Returns the status of the backup job, based on the ID from a previous Execute operation.

Cancel

  • Verb: POST

  • URL:  http://hostname:5000/api/Backup/Cancel

  • Parameters: 'id' - The ID from a previous status JSON result.

  • Result: Status JSON (see below)

    Attempts to cancel the backup job identified by the ID from a previous Execute operation.

StatusJSON
{
"Id":  "b61a96c2134f4a59b91188c94e24aa57",
"Status":  "Queued",
"Message": "Job was queued for    execution."
}

The result of all three calls is a JSON object in the same format and with the same three elements:

  • Id: The ID of the job. For Execute this will be the ID that must be saved for future information. For Status or Cancel calls this will echo back the ID supplied to the request.

  • Status: There are six possible values for this field:

    • Success - The backup job was completed successfully

    • Queued - The backup job has been queued and is waiting to be processed

    • Executing - The backup job is being executed.

    • Canceled - The backup job has been canceled. This is unlikely to be received in practice as canceled jobs are removed from the queue once they have completed.

    • LimitReached - An internal limit (for example, number of concurrent jobs) has been reached that meant that the job could not be started.

    • Error - An internal error has occurred. As well as the message, further information will be available in the accelerator APl's internal log file.

  • Message: A human-readable explanation of the status. This may provide more information, especially in the case of an error.

Scheduling calls to the Accelerator Backup REST API

You may also already possess scheduling software which is capable of calling REST APls directly. If not, below are two example approaches to scheduling the backup using the API. There is widespread guidance online regarding the use of these tools - Hubble cannot provide specific support for them.

Windows Task Scheduler and PowerShell

You can schedule a PowerShell script to invoke the backup API using Windows Task Scheduler. The script should look like the following:

Invoke-RestMethod -Uri "http://<hostname>:5000/api/Backup/Execute" -Method Post

where you replace <hostname> with the host name or IP address of your accelerator. To learn how to schedule this see for example, https://social.technet.microsoft.com/wiki/contents/articles/ 38580.configure-to-run-a-powershell-script-into-task-scheduler.aspx.

To run unsigned PowerShell scripts, you may need to relax the 'execution policy' on the server, or override it on the PowerShell command line. See for example https://superuser.com/guestions/106360/ how-to-enable-execution-of-powershell-scripts.

Linux Cron and Curl

You can schedule the Linux command 'curl' using the scheduler 'cron'. The command you need to schedule should look like the following:

curl -X POST -d "" http://<hostname>:5000/api/Backup/Execute

where you replace <hostname> with the host name or IP address of your accelerator. Depending on your Linux distribution, you might have to use 'wget' instead of 'curl'.

Cron is not available within an accelerator deployed as a VMWare or AWS image, but you can schedule the task on any other Linux server. If you have deployed the accelerator as a Docker container, then you can use cron in the Docker host server which you supplied.

Troubleshooting

  1. Check the log file /tmp/ybapi. log on the Accelerator for error messages:

    1. Job with a given ID was not found. - Either the Cancel or Status API commands were executed with a Job ID that is either invalid or has completed.

    2. Unable to move file: {Error}-Several steps involve moving files, one of these failed. Examples are moving the replication backup file to a temporary folder, or moving the backup file to / import/dbbackup.

    3. An error occurred while executing system command: {Command}. Error: {Error}- While we ask Qlik or Vector to perform a backup their data an error could occur.

    4. Not enough space is left in '{Location}': minimum required {xx}MB, space left {xx}MB. - Both the Accelerator temporary directory and / import/dbbackup must have a minimum amount of space available. By default this is 2GB.

    5. Unable to determine the drive of path '{Location}'- While checking for available space there was an error determining the drive to check.

    6. Directory '{Location}' does not exist. - The directory does not exist.

    7. File {Location} does not exist. - The file does not exist. This may occur because of an internal error that prevents a file we're trying to move from being created.

  2. Restart the service by connecting to the accelerator using an ssh client like putty and run /etc/ init.d/S70ybapplianceapi restart.

Was this article helpful?

We're sorry to hear that.