Create a data store

You can use the Data Store Configuration wizard to create relational, tile cache, and spatiotemporal big data stores and join them to a GIS Server site.

If you do not have access to a web browser or if you need to create an object store, run the configuredatastore utility on the machine where you installed ArcGIS Data Store.

Considerations before creating a data store

Be aware of the following before you create a data store.

Disk space and memory

Consider the amount of disk space and memory needed on the machine where you will create a data store.

The following are examples of how a data store can use large amounts of disk space:

• Relational data stores can store data for thousands of hosted feature layers, which can take up several gigabytes of disk space.
• Caches for 3D data can be multiple gigabytes in size or even terabytes in size, depending on the area included in the scene layer and the complexity of the data.
• If you archive data that is continually coming in from a stream layer or you track locations for numerous field workers, the spatiotemporal big data store can fill up very quickly.
• To take full advantage of cached query responses to improve drawing performance for feature layers, the caches must persist for as long as possible in the object store. This means disk space on the object store can fill up quickly. To facilitate storage of query response caches for longer periods of time, use a single machine for the object store, and ensure that the single machine contains large amounts of free disk space. How much disk space you need depends on how many layers are enabled for caching, how many unique queries are made on those layers, and how long the queries persist. (Using a cluster of object stores has not been shown to provide as much improvement in drawing performance as using a single machine with large amounts of free disk space.)

In all cases, it is important to monitor the data store machines and add disk space as necessary.

You also need to ensure that the machines have plenty of memory available. For example, spatiotemporal big data stores consume approximately 40 percent of available memory on the machine when initially configured. Relational data stores also preallocate memory to ensure there is enough to process data. High-volume traffic on hosted feature and scene layers can also consume a considerable amount of memory.

For these reasons, Esri recommends that you configure your spatiotemporal big data store on machines separate from the relational and tile cache data stores and, whenever possible, configure the relational data store and tile cache data store on machines separate from one another. Also configure the object store on a machine separate from any other data store. If you do configure more than one type of data store on the same machine, use robust machines with plenty of memory and disk space.

In all cases, monitor the data store logs for messages about disk space and memory. If your machine runs out of either resource, your data store will stop functioning.

See ArcGIS Data Store system requirements for information on the minimum disk space and memory needed to create each type of data store, and add to that the estimated size of the data or caches you intend to store in each.

Multiple NIC cards or DNS entries

If you installed ArcGIS Data Store on a machine that contains more than one network interface controller (NIC) card or multiple DNS entries (also called records), identify which IP address and host name the data store will use for network communication. Before you create a data store on the machine, determine which IP address can be used by other machines on your network to communicate with the ArcGIS Data Store machine, open the ArcGIS Data Store hostidentifier.properties file, and add the IP address to the file.

1. Determine which IP address to use. Open a command prompt on different machines on the same network and ping the ArcGIS Data Store machine.
2. Log in to the machine where ArcGIS Data Store is installed and open hostidentifier.properties in a text editor. Log in using the same account used when ArcGIS Data Store was installed.

   The file is installed in framework\etc of the ArcGIS Data Store installation directory. By default, that location is /home/ags/arcgis/datastore/framework/etc.

3. Remove the comment mark (#) from the hostidentifier property, and set the property to the IP address of the NIC card or DNS entry you want the data store to use for network communication.

   In this example, the IP address used to communicate with the machine is 12.34.56.789.

   hostidentifier=12.34.56.789

4. Save and close the hostidentifier.properties file.
5. Restart the ArcGIS Data Store service on that machine.

If necessary, follow the steps to configure other ArcGIS Enterprise software components on machines with multiple NIC cards or DNS entries:

• ArcGIS Server
• Portal for ArcGIS

The hosting server

An ArcGIS Enterprise deployment includes one GIS Server site to act as its hosting server. At minimum, the hosting server must be configured with a relational data store created through ArcGIS Data Store. This relational data store holds the data used by hosted feature layers created when you add files such as comma-separated values (CSV) files, zipped shapefiles, Microsoft Excel files, or zipped file geodatabases to your organization and publish. It also allows portal users to add CSV files directly to Map Viewer.

If the GIS Server site you intend to use as the hosting server has an existing enterprise geodatabase registered as its managed database, follow the instructions in Change the managed database to an ArcGIS Data Store relational data store to reconfigure the hosting server with a relational data store.

Create a data store using the Data Store Configuration wizard

If you run the installation wizard, the Data Store Configuration wizard automatically opens in your default web browser. Alternatively, you can open the wizard from the computer's shortcut menu or type the URL in a browser. The Data Store Configuration wizard URL is in the format https://datastoremachine.domain.com:2443/arcgis/datastore.

1. Specify the URL of the ArcGIS Server site for which you want to create and register a data store in the format https://gisserver.domain.com:6443. Provide the URL in this format even if you have a Web Adaptor configured with the site. If you have a multiple-machine site, you can use the URL pointing to any of the machines.
2. Type the user name and password of an ArcGIS Server administrator and click Next.

   The user must be a built-in user.

3. Choose the type of data store to create on this machine and click Next.

   While it's possible to create more than one type of data store on the same machine, Esri recommends that you do not do this because the data stores will compete for memory resources, which can lead to poor performance.

4. Type a location for the ArcGIS Data Store directory.

   This directory contains the data store files and the relational data store backup directory (if you created a relational data store).

   Note:

   The directory path cannot contain a space when you specify an ArcGIS Data Store directory for a tile cache or spatiotemporal big data store.

   You cannot use a UNC path when specifying the data store file directory.

   Keep data store files on the same machine where you install ArcGIS Data Store.

   After you create a relational data store, configure a mapped network drive on a separate machine and specify it as the backup directory for your relational data store. You can use a UNC path for the backup directory you create on a machine that is separate from your ArcGIS Data Store installation. See Manage data store backups for more information.

5. Click Next.
6. Review the information on the Configuration Summary dialog box. If it is correct, click Finish. If changes are needed, click Back and make corrections.

   The first relational data store you register with a GIS Server site is the primary machine. If you install ArcGIS Data Store on a second machine and create another relational data store registered to the same GIS Server site, it will be a standby machine. This is also true for tile cache data stores running in primary-standby mode.

   When the setup completes successfully, you have a data store machine.

   If there were errors with the setup, click Next on the Configuration Summary dialog box to reopen the Data Store Configuration wizard so you can provide different information. See Troubleshoot ArcGIS Data Store for causes and resolutions for common setup errors.

Create a data store using the configuredatastore utility

You can use the configuredatastore command utility to create a relational, tile cache, spatiotemporal big data store or object store and register it with your GIS Server site. The utility is installed in <ArcGIS Data Store installation directory>\arcgis\datastore\tools.

Note:

If you script the creation of multiple spatiotemporal big data store machines, one spatiotemporal big data store machine must be manually configured with the GIS Server before you can script the creation of additional spatiotemporal big data store machines. Include wait times in your script to be sure the additional spatiotemporal big data store machines are not added at the same time.

1. Open a command shell.
2. Run the configuredatastore.sh file and specify the GIS Server URL with which you want to register the data store, the user name and password of an ArcGIS Server administrator, the path to use for the ArcGIS Data Store data directory, and the type of data store you want to create.

   In the following examples, the GIS Server URL is https://myserver.domain.com:6443/arcgis/admin, the administrator user name and password are siteadmin and T1n@sp, and the data directory is /home/dsuser/data.

   In this example, a relational data store is created:

   ./configuredatastore.sh https://myserver.domain.com:6443/arcgis/admin siteadmin T1n@sp /home/dsuser/data --stores relational

   In this example, a spatiotemporal big data store is created:

   ./configuredatastore.sh https://myserver.domain.com:6443/arcgis/admin siteadmin T1n@sp /temporal/data --stores spatiotemporal

   When you create a tile cache data store, you can use the --mode operation to specify the deployment mode: either primary-standby (the default) or cluster mode. If you do not specify the --mode operation, the tile cache data store is created in primary-standby mode.

   In this example, a tile cache data store is created in cluster mode on a separate machine but is registered with the same hosting server as the relational data store created in the last example:

   ./configuredatastore.sh https://myserver.domain.com:6443/arcgis/admin siteadmin T1n@sp /home/dsuser/scenedata --stores tileCache --mode cluster

   The --mode operation is also used to specify the deployment mode of an object store. By default, the object store is deployed as a single machine deployment (in primary-standby mode). A single machine object store with a single large disk will provide the best performance. If you require the object store to be highly available, you can deploy a four machine cluster. To do that, specify --mode cluster when creating an object store. In addition, when creating an object store, use the --machines option to specify the machine name for a single machine object store or a comma separated list of machines for cluster mode. You must also use the data-loc option to specify the directory for a single machine or a comma-separated list of nearly identical directories (for cluster mode) where caches will be stored.

   In this example, a cluster of four machines are deployed as an object store. The data location directories are in the same location on each machine.

   ./configuredatastore.sh https://myserver.domain.com:6443/arcgis/admin siteadmin T1n@sp /home/dsuser/ftcacheds 
   --stores object --mode cluster --machines objectstore1,objectstore2,objectstore3,objectstore4 
   --data-loc /fscache/mycaches1,/fscache/mycaches2,/fscache/mycaches3,/fscache/mycaches4

   Note:

   The directory path cannot contain a space when you specify an ArcGIS Data Store directory for a tile cache or spatiotemporal big data store or an object store.

   To create more than one type of data store on the same machine using the same data directory, specify the values separated by a comma. For example, you can create a relational and tile cache data store on the same machine by specifying relational,tileCache with the --stores operation. However, Esri recommends that you do not put more than one data store on the same machine because the data stores compete with one another for resources such as memory, which can negatively affect performance.

Next steps

If you require more than one type of data store, create additional data store types. While it's possible to create another type of data store on the same machine, instead install and configure additional data store types on separate machines.

If you require more than one machine for each type of data store, add a machine to your relational data store or tile cache data store running in primary-standby mode, add at least two machines to your spatiotemporal big data store or tile cache data store running in cluster mode, or add at least three machines, following a specific naming convention, to an object store.

Ensure that the GIS Server site with which you registered your data store has been configured as your portal's hosting server.

Also, configure a secure location for data store backups.