Configure disaster recovery for ArcGIS Enterprise

You can replicate your ArcGIS Enterprise deployment to a disconnected standby deployment. The standby deployment can be brought online and made your active deployment if your primary deployment fails.

Because the standby deployment is not actively connected to your primary deployment, you can set up the standby deployment in another building or city to guard against downtime due to such occurrences as localized natural disasters or power outages. Keep in mind, though, that the standby deployment must be accessible from your clients.

Export the components of your primary ArcGIS Enterprise deployment to a compressed file using the webgisdr tool, and use the compressed file to set up and maintain the standby deployment.

Before you configure disaster recovery, be sure you read Disaster recovery and replication.

Export ArcGIS Enterprise

Use the webgisdr utility with the export operation and a properties file to export your primary ArcGIS Enterprise deployment. The exported file includes your portal, hosting server, relational and tile cache data stores, and federated servers. The file size and time it takes to create it vary depending on how many items are in your portal, the number and type of hosted web layers you have, how many federated servers you have, and how many ArcGIS Server machines are in your hosting and federated servers.

You can find the webgisdr utility in the Portal for ArcGIS tools directory. The following criteria must be met to use this utility:

• The shared directory you specify must be large enough to contain the exported file. Do a test export of your ArcGIS Enterprise deployment to get an idea how large the file will be, decide how many files you intend to keep in the shared directory, and size the directory accordingly.
• The name, user ID, and group ID of the account that was used to install each software component in ArcGIS Enterprise on both the primary and standby deployments, and the account used to run the webgisdr tool must be the same and must have read and write privileges on the shared directory.

Note:

If your portal content is stored in the file system, real-time antivirus scanning can impact the performance of the backup. Consider excluding the portal content directory and your backup SHARED_LOCATION and BACKUP_LOCATION directories from your antivirus scanner.

Follow these steps to export your primary ArcGIS Enterprise deployment:

1. Make a copy of the template properties file. You can save the copy in the same directory as the template or to a new directory.

   The template properties file—webgisdr.properties—is installed in /arcgis/portal/tools/webgisdr by default.

   In this example, the copy of the file is named mywebgis.properties and saved in /home/ags/arcgis/portal/propfiles.

2. Open the copy of the properties file and edit it to include information specific to your site.

   • PORTAL_ADMIN_URL = <portal admin directory URL>

     Specify the URL of the Portal for ArcGIS admin directory. Use the format https://portalhostname.domain.com:7443/arcgis, where portalhostname.domain.com is the fully qualified name of the machine where Portal for ArcGIS is installed.

   • PORTAL_ADMIN_USERNAME = <portal administrator user name>

     Specify the user name of a portal member assigned to the Administrator role.

   • PORTAL_ADMIN_PASSWORD = <portal administrator password>

     Specify the password of the portal administrator account.

   • PORTAL_ADMIN_PASSWORD_ENCRYPTED = <true | false>

     Set this option to false the first time you populate the file with your administrator password. When you save the file, your password is encrypted and the value for PORTAL_ADMIN_PASSWORD_ENCRYPTED is set to true to indicate the password has been encrypted. If you need to change the password in the future, set PORTAL_ADMIN_PASSWORD_ENCRYPTED = false, provide your new administrator password, and save the file.

   • BACKUP_RESTORE_MODE = <backup | full | incremental>

     Introduced at 10.9, the backup mode can be used when limited hard drive space is available on the portal machine or when backups are not regularly scheduled. When using this mode, the default size limit is enforced on internal transaction logs, preventing them from filling up your hard drive. If you only create full backups, you can set this property to full. To decrease the size of backups, though, you can create two different properties files—one with BACKUP_RESTORE_MODE = full, which you would use occasionally to create full backups, and one with BACKUP_RESTORE_MODE = incremental to generate incremental backups between full backups. Each incremental backup contains all changes since the last full backup.

   • SHARED_LOCATION = <location for backup files>

     Designate a shared location in which to create a temporary copy of the backup files for ArcGIS Server, Portal for ArcGIS, and ArcGIS Data Store. The account that installed ArcGIS Server, Portal for ArcGIS, and ArcGIS Data Store and the account running the webgisdr utility must have read and write privileges on this location. Be sure the location is large enough to hold the backup files for each component. Though the files are compressed, they can be quite large, depending on the amount and type of data you have. If you have hosted scene layers and caches and set INCLUDE_SCENE_TILE_CACHES to true, the ArcGIS Data Store backup file can be especially large.

     Ensure any backslashes in the path are escaped. For example, C:\backupswould be entered as C:\\backups and \fileserver\backups would be entered as \\\\fileserver\\backups.
   • #TOKEN_EXPIRATION_MINUTES=<integer>

     By default, restores of site backups use a token with an expiration time of 60 minutes. If a restoration of this backup takes longer than 60 minutes, the token will time out and cause the restore operation to fail. If you uncomment this line, you can define a custom token expiration time in minutes.

   • INCLUDE_SCENE_TILE_CACHES = <true | false>

     If you publish hosted scene layers to your portal and want to include the scene cache data in the backup, set INCLUDE_SCENE_TILE_CACHES to true. Be aware that, if set to true, all scene cache data is included in the backup, not just the new cache data created since your last backup. If you know that no new scene caches have been created since your last backup or if you do not publish scenes to your portal, you can set INCLUDE_SCENE_TILE_CACHES to false.

   • BACKUP_STORE_PROVIDER = {FileSystem | AmazonS3 | AzureBlob}

     Define whether you want to save the webgissite backup file on the file system or on cloud storage.

     If you want to store your ArcGIS Enterprise backup in a file share location on-premises, specify FileSystem, and the webgisdr utility will store your backup files in the file share you specify for BACKUP_LOCATION.

     If you want to store your ArcGIS Enterprise backups in an Amazon Simple Storage Service (S3) or an S3 compatible bucket storage location, create one specifically for storing backup files generated by the webgisdr utility, and update the S3 properties with the information needed to access your bucket.

     If you want to store your ArcGIS Enterprise backups in Microsoft Azure Blob storage, create an Azure Blob storage container specifically for storing backup files generated by the webgisdr utility, and update the Azure properties with the information needed to access your container.

     Note:

     If your ArcGIS Enterprise deployment is on-premises but your backups are stored in the cloud, creating backups and restoring will take longer than it would if you stored your backups in an on-premises file share.

   • BACKUP_LOCATION = <location of on-premises backup file>

     Designate the location for the webgissite backup file.

     Ensure any backslashes in the path are escaped. For example, C:\backupswould be entered as C:\\backups and \\fileserver\backups would be entered as \\\\fileserver\\backups.

     Backup files are initially stored in the SHARED_LOCATION but are copied to the BACKUP_LOCATION. Be sure the location is large enough to hold the backup file. Although the file is compressed, it can be quite large depending on the amount and type of data you have. The account that runs the webgisdr utility must have read and write privileges on the BACKUP_LOCATION.

     If you set BACKUP_LOCATION to a folder, the disaster recovery tool imports the latest backup file available in that folder. If you set BACKUP_LOCATION to a specific backup file in the folder, that file is imported by the disaster recovery tool.

     Note:

     During an export, if it is taking a long time to package the backup, consider setting the BACKUP_LOCATION to a local path. You can then copy the finished package to its intended location. Make sure there is enough space on the local drive to store the backup temporarily.

   • If you set BACKUP_STORE_PROVIDER to AmazonS3, provide information for the following properties:
     • S3_ACCESSKEY= <The access key for your Amazon Web Services (AWS) account>

       You only need to set this if you set S3_CREDENTIAL_TYPE=accessKeys. The IAM user represented by this access key must have read and write access to the S3 bucket you specify with the S3_BUCKET property.

     • S3_SECRETKEY= <The secret key for your AWS account>

       Specify the secret key associated with the access key for your AWS account. This is used only if you set S3_CREDENTIAL_TYPE to accessKeys.

     • S3_ENCRYPTED = false

       Leave this set to false. If you have S3_CREDENTIAL_TYPE set to accessKeys when you run the webgisdr utility, the utility will encrypt the access key and secret key and set S3_ENCRYPTED to the encrypted access key value.

     • S3_BUCKET= <name of the S3 bucket>

       This is the name of the Amazon S3 bucket in which you want to store your ArcGIS Enterprise backup file.

     • S3_CREDENTIAL_TYPE= <IAMRole | accessKeys

       Set S3_CREDENTIAL_TYPE to IAMRole if you will access the S3_BUCKET using an AWS IAM role. Set it to accessKeys if you will access the S3_BUCKET using an AWS IAM user via an access key. The IAM role or user must have read and write access to the S3 bucket you specify with the S3_BUCKET property.

     • S3_REGION= <AWS region in which you created the S3 bucket>
     • S3_BACKUP_NAME = <backup file name>

       This property is only used when you restore an ArcGIS Enterprise deployment. Specify the name of the backup file you want to restore. If you do not specify a backup file name, the latest backup file is restored.

   • If your ArcGIS Enterprise deployment is running on AWS and your portal content directory is stored in Amazon S3, you need a backup bucket for the content directory. This should not be the same bucket that the portal itself is in; otherwise, a duplication of content will occur each time you perform a backup. Create a backup bucket in S3 and set PORTAL_BACKUP_S3_BUCKET in the properties file to the name of the bucket and set PORTAL_BACKUP_S3_REGION to the AWS region in which you created the S3 bucket. See Configure highly available ArcGIS Enterprise with AWS storage services in the ArcGIS Enterprise on Amazon Web Services help for more information on this type of implementation.

     Legacy:

     Prior to ArcGIS Enterprise 10.5.1, the PORTAL_BACKUP_S3_BUCKET property was BACKUP_BUCKET_NAME.

   • If you set BACKUP_STORE_PROVIDER to AzureBlob, provide information for the following properties:
     • AZURE_BLOB_ACCOUNT_NAME = <Azure storage account>

       Specify the Azure Blob storage account name.

       Tip:

       You specified this storage account name when you chose to Use Azure Cloud Storage for the configuration and content store on the Deployment Options panel of Cloud Builder.

     • AZURE_BLOB_ACCOUNT_KEY = <account key>

       Specify the account key associated with your Azure Blob storage account.

     • AZURE_BLOB_ACCOUNT_KEY_ENCRYPTED = false

       Specify false when you first add the ACCOUNT_KEY values to the file. When you save the webgisdr.properties file, the key will be encrypted and the tool will set these properties to true.

     • AZURE_BLOB_CONTAINER_NAME = <Name of your Azure Blob storage container>

       Specify the Blob container name.

     • AZURE_BLOB_ACCOUNT_ENDPOINT_SUFFIX = <Blob storage account endpoint>

       Specify the Blob service storage account endpoint.

     • (Optional) AZURE_BLOB_ENDPOINT_URL = <Blob service URL>

       Specify the Blob service endpoint URL if you are using a custom blob storage endpoint. If you aren't using a custom endpoint, don't uncomment this line. The default URL is in the format https://<BLOB_ACCOUNT_NAME>.blob.core.windows.net.

   • You can also store your portal content directory in an Azure Blob storage container. If you are doing so, you need a backup bucket for the content directory in Azure. This should not be the same bucket that the portal itself is in; otherwise, a duplication of content will occur each time you perform a backup. The set of properties beginning with PORTAL_BACKUP_BLOB defines the account used for the portal content directory. The values for these properties should match those set in the AZURE_BLOB section above; the definitions for that section apply here as well.
     • PORTAL_BACKUP_BLOB_ACCOUNT_NAME = <Azure storage account>
     • PORTAL_BACKUP_BLOB_ACCOUNT_KEY = <account key>
     • PORTAL_BACKUP_BLOB_ACCOUNT_KEY_ENCRYPTED = false
     • PORTAL_BACKUP_BLOB_CONTAINER_NAME = <Name of your Azure Blob storage container>
     • PORTAL_BACKUP_BLOB_ACCOUNT_ENDPOINT_SUFFIX = <Blob storage account endpoint>
     • (Optional) PORTAL_BACKUP_BLOB_ENDPOINT_URL = <custom Blob service URL>
   • IS_PORTAL_PKI_AUTHENTICATED = <true | false>

     If you have configured a public key infrastructure (PKI) to secure access to your portal, set this value to true.

     Learn more about portal authentication options

     If set to true, provide values for the following additional properties:

     • PORTAL_CLIENT_CERTIFICATE_FILE_PATH = <full certificate file path>

       Specify the full file path of the portal's PKI certificate in PKCS12 format.

     • PORTAL_CLIENT_CERTIFICATE_PASSWORD = <password for certificate>

       Specify the corresponding password for the PKI certificate.

     • PORTAL_CLIENT_CERTIFICATE_PASSWORD_ENCRYPTED = false

       When you first enter the value for the certificate password, leave this value as false. When you first run the tool after saving this value, the tool will encrypt the password and change this property to true.

   In this example, a backup of the portal at URL https://portalhostname.domain.com:7443/arcgis, plus the services and settings of its hosting and federated servers, and the hosted feature layer data stored in the ArcGIS Data Store relational data store will be initially output to files in \\\\myserver\\tempbackups. The separate files will be compressed into a single backup file (entbackup) and copied to \\mybuserver\\wgbackups. Scene layer caches from the ArcGIS Data Store tile cache data store will not be included because INCLUDE_SCENE_TILE_CACHES is set to false.

   PORTAL_ADMIN_URL = https://portalhostname.domain.com:7443/arcgis
   PORTAL_ADMIN_USERNAME = admin
   PORTAL_ADMIN_PASSWORD = Th3.Ad.Pass
   PORTAL_ADMIN_PASSWORD_ENCRYPTED = false
   
   BACKUP_RESTORE_MODE = backup
   SHARED_LOCATION = \\\\myserver\\tempbackups
   INCLUDE_SCENE_TILE_CACHES = false
   
   BACKUP_STORE_PROVIDER = FileSystem
   BACKUP_LOCATION = \\\\mybuserver\\wgbackups\\entbackup

   In this example, a backup of the portal at URL https://portalonaws.mydomain.com:7443/gis, its content directory (stored in S3 bucket portalcontent), the services and settings of the portal's hosting and federated servers, and the data stored in the relational and tile cache data stores will be output to the S3 bucket entbackups in the AWS region eu-west-1. The backup file name is fullbackup06June. An IAM role is used to access the AWS account.

   PORTAL_ADMIN_URL = https://portalonaws.mydomain.com:7443/arcgis
   PORTAL_ADMIN_USERNAME = entadmin
   PORTAL_ADMIN_PASSWORD = p0rtl.a.p
   PORTAL_ADMIN_PASSWORD_ENCRYPTED = false
   
   BACKUP_RESTORE_MODE = backup
   SHARED_LOCATION = \\fileserver\backups
   INCLUDE_SCENE_TILE_CACHES = true
   BACKUP_STORE_PROVIDER = AmazonS3
   
   S3_BUCKET = entbackups
   S3_CREDENTIAL_TYPE = IAMRole
   S3_REGION = eu-west-1
   S3_BACKUP_NAME = fullbackup06June
   
   PORTAL_BACKUP_S3_BUCKET = portalcontent
   PORTAL_BACKUP_S3_REGION =eu-west-1

   This example is for Microsoft Azure Blob storage. A backup of the portal at URL https://portalonazure.domain.com:7443/arcgis, its content directory, the services and settings of the portal's hosting and federated servers, and the data stored in the relational and tile cache data stores will be output to the Azure blob drbackups.

   PORTAL_ADMIN_URL = https://portalonazure.domain.com:7443/arcgis
   PORTAL_ADMIN_USERNAME = entadmin
   PORTAL_ADMIN_PASSWORD = p0rtl.a.p
   
   BACKUP_RESTORE_MODE = backup
   SHARED_LOCATION = \\fileserver\backups
   BACKUP_STORE_PROVIDER = AzureBlob
   
   AZURE_BLOB_ACCOUNT_NAME = entbackups
   AZURE_BLOB_ACCOUNT_KEY = <key>
   AZURE_BLOB_ACCOUNT_KEY_ENCRYPTED =false
   AZURE_BLOB_ACCOUNT_ENDPOINT_SUFFIX = core.windows.net
   AZURE_BLOB_CONTAINER_NAME = drbackups
   
   PORTAL_BACKUP_BLOB_ACCOUNT_NAME = entbackups
   PORTAL_BACKUP_BLOB_ACCOUNT_KEY = <key>
   PORTAL_BACKUP_BLOB_ACCOUNT_KEY_ENCRYPTED = false
   PORTAL_BACKUP_BLOB_ACCOUNT_ENDPOINT_SUFFIX = core.windows.net
   PORTAL_BACKUP_BLOB_CONTAINER_NAME = portalcontent

3. Save the properties file.

   The PORTAL_ADMIN_PASSWORD value will be encrypted in the file and PORTAL_ADMIN_PASSWORD_ENCRYPTED is set to true once you run the webgisdr utility with this file.

4. Open a command shell, change directories to the location of the webgisdr utility, and run the utility with the export option.

   In this example, the properties file (mywebgis.properties) was saved in the user-created directory /home/ags/arcgis/portal/propfiles.

   webgisdr --export --file /home/ags/arcgis/portal/propfiles/mywebgis.properties

5. You can also add a second command, such as a validation script, to run after the webgisdr tool completes. For example:

   > call webgisdr.bat -i -f webgisdr.properties
   > IF %ERRORLEVEL% EQU 0 <your validation script file>

When the backup completes, the file is created relative to the value set for the BACKUP_STORE_PROVIDER. If you set the BACKUP_STORE_PROVIDER to FileSystem, the backup resides in the location you specified for the BACKUP_LOCATION in the properties file. If you set the BACKUP_STORE_PROVIDER to AmazonS3 or AzureBlob, it is copied to the respective cloud storage location.

The name of the file is <timestamp>.webgissite.

Move the export file

If the export file was created in a location that the standby deployment cannot access, move the backup file to a location that the webgisdr tool can access. Make a copy of the properties file specifically to be used for importing, and update the BACKUP_LOCATION value in the new properties file to reflect the new location of the export file. If the BACKUP_LOCATION is directory, the tool will extract the latest backup within the directory into the SHARED_LOCATION. If the BACKUP_LOCATION is set to a specific backup file, the tool will extract that backup into the SHARED_LOCATION.

Ensure the machines in the standby deployment can access the SHARED_LOCATION path.

Ensure that the account that runs the webgisdr tool has at least read access to the BACKUP_LOCATION and write access to the SHARED_LOCATION. Also be sure that the account that installed ArcGIS Server, Portal for ArcGIS, and ArcGIS Data Store has read access to the SHARED_LOCATION.

For the purposes of this workflow example, the copy of the properties file is named toimport.properties.

Install components for standby deployment

Before you import the ArcGIS Enterprise deployment to the standby for the first time, you need to install and configure all software components (ArcGIS Server, Portal for ArcGIS, ArcGIS Web Adaptor, and ArcGIS Data Store) on the standby machines. The URL used to access the portal (either the load balancer URL or web adaptor URL) and the ArcGIS Server services URL must be the same for the standby deployment as they are for your primary deployment.

For example, if the URL to access the portal on the primary deployment is https://myportalslb.organization.com/portal and the services URL for the GIS Server on the primary deployment is https://myserverslb.organization.com/server, the standby deployment must use the same URLs.

Additional ArcGIS licenses are not required for the standby deployment because it is not actively accessed; you only make it the active deployment if the primary fails.

Import to standby deployment

Once you have an empty deployment on the standby machines, import the content from your primary deployment.

If your primary ArcGIS Enterprise deployment is highly available, the webgisdr tool maintains high-availability settings for the GIS Server site and the ArcGIS Data Store relational and tile cache data stores when you import to the standby deployment. For the portal, the webgisdr tool unregisters the standby portal machine, restores the portal to the primary machine, and reregisters the standby machine when imported.

1. Open a command shell on the Portal for ArcGIS machine of the standby deployment, change directories to the location of the webgisdr utility, and run the utility with the import option.

   In this example, the properties file (toimport.properties) was saved in the user-created directory /home/ags/arcgis/portal/propfiles.

   webgisdr --import --file /home/ags/arcgis/portal/propfiles/toimport.properties

   Note that the property file does not have to be in the same directory you saved it in when first created. If you do move it, specify the new directory location with the --file parameter.

Maintain standby deployment

To minimize data loss, export from the primary and import to the standby deployment as frequently as possible. How often you can do this depends on how long it takes to export and import your deployment, and whether you can script the export and import processes without the need for manual intervention (for example, you may need to physically copy files from one location to another).

Switch to standby when primary fails

As discussed in Disaster recovery and replication, your IT department can bring the standby deployment online if your primary deployment fails.