GeoAnalytics health check—ArcGIS GeoAnalytics Server | Documentation for ArcGIS Enterprise

The health check operation indicates the status of compute resources and jobs within the GeoAnalytics Server compute platform, the underlying framework that allows GeoAnalytics Server to distribute analysis across multiple GeoAnalytics Server cores and machines.

The operation, which can be used for troubleshooting and monitoring, is available to the server or portal administrator and is accessed by logging in to your ArcGIS Server Administrator Directory using the URL format https://gis_geoanalytics_server.domain.com:6443/arcgis/admin. To open the health check, start from your GeoAnalytics Server site and click System > Platform Services > Compute Platform > Health.

Use the health check operation to do the following:

Verify and identify GeoAnalytics Server machines that are being used for analysis.
Verify that the allotted memory and cores are being used when running analysis (or running a tool).
Check the amount of cores or memory available on the GeoAnalytics Server machines. This can be helpful when using the GeoAnalytics Server settings.
Learn more about GeoAnalytics Server settings
Find the jobID of a GeoAnalytics Server job to view the REST endpoint.
Troubleshoot and resolve error messages.
Note:
If you receive the error Unable to initialize distributed compute platform service, complete the health check operation and check the status of the worker nodes. Once you have checked the status of the nodes, complete the steps listed in Common GeoAnalytics Server problems and solutions.

Note:

The GeoAnalytics Server health check is also available through ArcGIS Monitor.

Learn more about ArcGIS Monitor

The compute platform health check uses terminology from the underlying framework that distributes the analysis. Terms and concepts used throughout this documentation are outlined in the following table:


Term	Description
Node	A server machine.
Worker	A machine used for computation. The worker machine receives instructions from the primary machine and executes them. There can be one or more worker machines to a primary machine.
Primary	The server that oversees the worker nodes. A server can be both a primary node and worker node. In Spark this is referred to as master.
Job	A job refers to running a GeoAnalytics Server tool.

The health check is composed of keys that are either a single value or an array. When a GeoAnalytics Server job has been run and completed, the health check has the following keys for each GeoAnalytics Server machine in the site:

{
  "<geoanalytics server machine name>":{
  {.. url ..},
  {.. workers ..},
  {.. aliveworkers ..},
  {.. cores ..},
  {.. coresused ..},
  {.. memory ..},
  {.. memoryused ..},
  {.. activeapps ..},
  {.. completedapps ..},
  {.. activedrivers ..},
  {.. completeddrivers ..},
  {.. status ..}
}

When there is more than one machine in a site, there will be multiple geoanalytics service machine name keys. Only the primary node machine will contain resource and job information. The worker node machines will contain only null and 0 values. Within each top-level server object, the following keys contain an array of values: workers, activeapps, completedapps, and activedrivers. The top-level keys' url, cores, coresused, memory, memoryused, and status have a single value.

Each key is explored in more detail using the following examples:

Example 1: A one-machine GeoAnalytics Server site.
- Each machine has 32 GB (32768 MB) of RAM and 8 CPU cores.
- The system properties is set to use the maximum of 90 percent of available RAM and cores.
  - The GeoAnalytics Server will use 29491 MB of RAM per machine. This was calculated using the following equation:
```
 floor(32768 x .90) = 29491 
```
  - The GeoAnalytics Server will use 7 cores. This was calculated using the following equation:
```
floor(8 cores x 1 machines x .90) = 7 
```
Example 2: A three-machine GeoAnalytics Server site.
- Each machine has 16 GB (16384 MB) of RAM and 4 CPU cores.
- The system properties use the default of a maximum of 80 percent of available RAM and cores.
  - The GeoAnalytics Server will use 13107 MB of RAM per machine. This was calculated using the following equation:
```
 floor(16384 x .80)= 13107
```
  - The GeoAnalytics Server will use 9 cores across all machines. This was calculated using the following equation:
```
floor(4 cores x 3 machines x .80) = 9 
```

Site information and resources

The url, cores, coresused, memory, memoryused, and status keys are described in this section. If you have more than one machine in your GeoAnalytics Server site, there will be multiple url keys. One of the url keys will be populated with values as shown in the following examples. All other url keys will have values of 0, and no workers will be listed. The URL with populated values is the primary node; the other URLs are machines that are represented as workers and are not the primary node. The activedrivers key is always an empty array.

Example 1—One-machine site Example 2—Three-machine site

Example 1—One-machine site	Example 2—Three-machine site
`{ "machine1.example-esri.com": { "url": "spark://machine1.example-esri.com:7077", "workers": [ { ...worker1... } ], "aliveworkers": 1, "cores": 7, "coresused": 7, "memory": 29491, "memoryused": 29491, "activeapps": [ ... ], "completedapps": [ ... ], "activedrivers": [], "completeddrivers": [], "status": "ALIVE" } }`	{ "machine1.example-esri.com": { "url": "spark://machine1.example-esri.com:7077", "workers": [ { ...worker1... }, { ...worker2... }, { ...worker3... } ], "aliveworkers": 3, "cores": 9, "coresused": 9, "memory": 13107, "memoryused": 13107, "activeapps": [ ... ], "completedapps": [ ... ], "activedrivers": [], "completeddrivers": [], "status": "ALIVE" }, "machine2.example-esri.com": { "url": "spark://machine2.example-esri.com:7077", "workers": [], "aliveworkers": 0, "cores": 0, "coresused": 0, "memory": 0, "memoryused": 0, "activeapps": [], "completedapps": [], "activedrivers": [], "completeddrivers": [], "status": "STANDBY" }, "machine3.example-esri.com": { "url": "spark://machine3.example-esri.com:7077", "workers": [], "aliveworkers": 0, "cores": 0, "coresused": 0, "memory": 0, "memoryused": 0, "activeapps": [], "completedapps": [], "activedrivers": [], "completeddrivers": [], "status": "STANDBY" } }

{
   "machine1.example-esri.com": {
       "url": "spark://machine1.example-esri.com:7077",
       "workers": [
           {
               ...worker1...
           }
       ],
       "aliveworkers": 1,
       "cores": 7,
       "coresused": 7,
       "memory": 29491,
       "memoryused": 29491,
       "activeapps": [ ... ],
       "completedapps": [ ... ],
       "activedrivers": [],
       "completeddrivers": [],
       "status": "ALIVE"
   }
}

{
 "machine1.example-esri.com": {
       "url": "spark://machine1.example-esri.com:7077",
       "workers": [
           {
               ...worker1...
           },
           {
               ...worker2...
           },
           {
               ...worker3...
           }
       ],
       "aliveworkers": 3,
       "cores": 9,
       "coresused": 9,
       "memory": 13107,
       "memoryused": 13107,
       "activeapps": [ ... ],
       "completedapps": [ ... ],
       "activedrivers": [],
       "completeddrivers": [],
       "status": "ALIVE"
   },
   "machine2.example-esri.com": {
       "url": "spark://machine2.example-esri.com:7077",
       "workers": [],
       "aliveworkers": 0,
       "cores": 0,
       "coresused": 0,
       "memory": 0,
       "memoryused": 0,
       "activeapps": [],
       "completedapps": [],
       "activedrivers": [],
       "completeddrivers": [],
       "status": "STANDBY"
   },
   "machine3.example-esri.com": {
       "url": "spark://machine3.example-esri.com:7077",
       "workers": [],
       "aliveworkers": 0,
       "cores": 0,
       "coresused": 0,
       "memory": 0,
       "memoryused": 0,
       "activeapps": [],
       "completedapps": [],
       "activedrivers": [],
       "completeddrivers": [],
       "status": "STANDBY"
   }
}

Description

url—The URL of the GeoAnalytics Server machine with the Spark prefix and Spark port. Only one URL can be listed per machine.
cores—The total number of cores available to the GeoAnalytics Server across all GeoAnalytics Server machines in the site. This represents the total available cores multiplied by the percentageMaxAllowedComputeCores setting, which is 80 percent by default.
Learn more about setting the percentageMaxAllowedComputeCores.
coresused—The total number of cores currently being used in GeoAnalytics Server analysis across all machines in the GeoAnalytics Server site.
memory—The total amount of memory (MB) available to the GeoAnalytics Server across all GeoAnalytics Server machines in the site. This represents the amount of memory on each machine multiplied by the percentageMaxAllowedComputeMemoryand the number of machines.
memoryused—The total amount of memory currently being used in GeoAnalytics Server analysis across all machines in the GeoAnalytics Server site.
activedrivers—Always an empty array.

Workers

The workers key identifies the machines and information as part of the GeoAnalytics Server site.

Example 1—One-machine site Example 2—Three-machine site

Example 1—One-machine site	Example 2—Three-machine site
`{ "machine1.example-esri.com": { "url": "spark://machine1.example-esri.com:7077", "workers": [ { "id": "worker-20180302114515-10.33.9.162-56544", "host": "10.33.9.162", "port": 56544, "cores": 3, "coresused": 3, "coresfree": 0, "memory": 29491, "memoryused": 29491, "memoryfree": 0, "state": "ALIVE", "lastheartbeat": 1520469707880, "lastheartbeatString": "2018-03-07T04:41:47,880", "hostname": "machine1.example-esri.com" } ], ... } } }`	{ "machine1.example-esri.com": { "url": "machine1.example-esri.com:7077", "workers": [ { "id": "worker-20180302114515-10.33.9.162-56544", "host": "10.33.9.162", "port": 56544, "cores": 3, "coresused": 3, "coresfree": 0, "memory": 13107, "memoryused": 13107, "memoryfree": 0, "state": "ALIVE", "lastheartbeat": 1520469707880, "lastheartbeatString": "2018-03-07T04:41:47,880", "hostname": "machine1.example-esri.com" }, { "id": "worker-20180302114111-10.33.10.149-56543", "host": "10.33.10.149", "port": 56543, "cores": 3, "coresused": 3, "coresfree": 0, "memory": 13107, "memoryused": 13107, "memoryfree": 0, "state": "ALIVE", "lastheartbeat": 1520469712961, "lastheartbeatString": "2018-03-07T04:41:52,961", "hostname": "machine2.example-esri.com" }, { "id": "worker-20180302113155-10.33.9.165-56540", "host": "10.33.9.165", "port": 56540, "cores": 3, "coresused": 3, "coresfree": 0, "memory": 13107, "memoryused": 13107, "memoryfree": 0, "state": "ALIVE", "lastheartbeat": 1520469712874, "lastheartbeatString": "2018-03-07T04:41:52,874", "hostname": "machine3.example-esri.com" } ], ... }, "machine2.example-esri.com": {...}, "machine3.example-esri.com": {...} } }

{
   "machine1.example-esri.com": {
       "url": "spark://machine1.example-esri.com:7077",
       "workers": [
           {
               "id": "worker-20180302114515-10.33.9.162-56544",
               "host": "10.33.9.162",
               "port": 56544,
               "cores": 3,
               "coresused": 3,
               "coresfree": 0,
               "memory": 29491,
               "memoryused": 29491,
               "memoryfree": 0,
               "state": "ALIVE",
               "lastheartbeat": 1520469707880,
               "lastheartbeatString": "2018-03-07T04:41:47,880",
               "hostname": "machine1.example-esri.com"
           }
       ],
    ...
    }
   }
}

{
   "machine1.example-esri.com": {
       "url": "machine1.example-esri.com:7077",
       "workers": [
           {
               "id": "worker-20180302114515-10.33.9.162-56544",
               "host": "10.33.9.162",
               "port": 56544,
               "cores": 3,
               "coresused": 3,
               "coresfree": 0,
               "memory": 13107,
               "memoryused": 13107,
               "memoryfree": 0,
               "state": "ALIVE",
               "lastheartbeat": 1520469707880,
               "lastheartbeatString": "2018-03-07T04:41:47,880",
               "hostname": "machine1.example-esri.com"
           },
           {
               "id": "worker-20180302114111-10.33.10.149-56543",
               "host": "10.33.10.149",
               "port": 56543,
               "cores": 3,
               "coresused": 3,
               "coresfree": 0,
               "memory": 13107,
               "memoryused": 13107,
               "memoryfree": 0,
               "state": "ALIVE",
               "lastheartbeat": 1520469712961,
               "lastheartbeatString": "2018-03-07T04:41:52,961",
               "hostname": "machine2.example-esri.com"
           },
           {
               "id": "worker-20180302113155-10.33.9.165-56540",
               "host": "10.33.9.165",
               "port": 56540,
               "cores": 3,
               "coresused": 3,
               "coresfree": 0,
               "memory": 13107,
               "memoryused": 13107,
               "memoryfree": 0,
               "state": "ALIVE",
               "lastheartbeat": 1520469712874,
               "lastheartbeatString": "2018-03-07T04:41:52,874",
               "hostname": "machine3.example-esri.com"
           }
       ],
    ...
    },    
   "machine2.example-esri.com": {...},
   "machine3.example-esri.com": {...}
   }
}

Description

id—A unique ID for the worker.
host—The host address of the machine.
port—The port on the machine used for GeoAnalytics Server analysis. Each worker is assigned its own port. It is required that these ports are opened for communication.
cores—The number of cores allocated to GeoAnalytics Server analysis on the machine. This is set using GeoAnalytics Server settings and represents the percentageMaxAllowedComputeCores setting.
coresused—The number of cores currently used for GeoAnalytics Server analysis. If there are any GeoAnalytics Tools running, this value is expected to be greater than 0.
coresfree—The number of cores that are currently free and can be immediately used for GeoAnalytics Server analysis on this machine. If this number is equal to cores on all machines, no GeoAnalytics Tools are currently running.
memory—The amount of memory (MB) allocated to GeoAnalytics Server analysis on the machine. This is set using GeoAnalytics Server settings and represents the percentageMaxAllowedComputeMemory setting.
memoryused—The amount of memory (MB) currently used for GeoAnalytics Server analysis by the specified machine.
memoryfree—The amount of memory (MB) that is currently free and can be used immediately for GeoAnalytics Server analysis on this machine. The amount of memoryfree must be greater or equal to the amount set in the server setting on each machine for GeoAnalytics Tools to run.
state—The state of the current machine. ALIVE is on, ready to run a job or running one. STANDBY switches to ALIVE when a job is started. DEAD may have an error, and the GeoAnalytics Server should be restarted. UNKNOWN may have an error, and the GeoAnalytics Server should be restarted. If you cannot get a machine to switch to ALIVE or STANDBY from DEAD or UNKNOWN, contact Esri Support.
lastheartbeat—The last time the current machine contacted the primary node in milliseconds from epoch. This is calculated in the local time of the machine.
lastheartbeatString—The last time the current machine contacted the primary node in string format. This is calculated in the local time of the machine.
hostname—The hostname of the worker machine.

Completed apps

The completedapps key identifies the completed GeoAnalytics Server tools and information. This includes tools that successfully completed and tools that resulted in a failure. The completed apps information is cleared when GeoAnalytics Server is restarted.

Example 1—One-machine site Example 2—Three-machine site

Example 1—One-machine site	Example 2—Three-machine site
`{ "machine1.example-esri.com": { "url": "spark://machine1.example-esri.com:7077", "workers": [...], ..., "activeapps": [...], "completedapps": [ { "starttime": 1520469691617, "id": "app-20180307164131-0008", "name": "ArcGIS GeoAnalytics (jcb8bda25a3f14dc7835dca0b3267b42c)", "cores": 7, "user": "agstest", "memoryperslave": 29491, "submitdate": "Wed Mar 07 16:41:31 PST 2018", "state": "FINISHED", "duration": 21947 } ], "activedrivers": [], "completeddrivers": [], "status": "ALIVE" } }`	{ "machine1.example-esri.com": { "url": "spark://machine1.example-esri.com:7077", "workers": [...], ..., "activeapps": [...], "completedapps": [ { "starttime": 1520469691617, "id": "app-20180307164131-0008", "name": "ArcGIS GeoAnalytics (jcb8bda25a3f14dc7835dca0b3267b42c)", "cores": 9, "user": "agstest", "memoryperslave": 13107, "submitdate": "Wed Mar 07 16:41:31 PST 2018", "state": "FINISHED", "duration": 21947 } ], "activedrivers": [], "completeddrivers": [], "status": "ALIVE" }, "machine2.example-esri.com": {...}, "machine3.example-esri.com": {...} }

{
   "machine1.example-esri.com": {
       "url": "spark://machine1.example-esri.com:7077",
       "workers": [...],
       ...,
       "activeapps": [...],
       "completedapps": [
           {
               "starttime": 1520469691617,
               "id": "app-20180307164131-0008",
               "name": "ArcGIS GeoAnalytics (jcb8bda25a3f14dc7835dca0b3267b42c)",
               "cores": 7,
               "user": "agstest",
               "memoryperslave": 29491,
               "submitdate": "Wed Mar 07 16:41:31 PST 2018",
               "state": "FINISHED",
               "duration": 21947
           }
       ],
       "activedrivers": [],
       "completeddrivers": [],
       "status": "ALIVE"
   }
}

{
   "machine1.example-esri.com": {
       "url": "spark://machine1.example-esri.com:7077",
       "workers": [...],
       ...,
       "activeapps": [...],
       "completedapps": [
           {
               "starttime": 1520469691617,
               "id": "app-20180307164131-0008",
               "name": "ArcGIS GeoAnalytics (jcb8bda25a3f14dc7835dca0b3267b42c)",
               "cores": 9,
               "user": "agstest",
               "memoryperslave": 13107,
               "submitdate": "Wed Mar 07 16:41:31 PST 2018",
               "state": "FINISHED",
               "duration": 21947
           }
       ],
       "activedrivers": [],
       "completeddrivers": [],
       "status": "ALIVE"
   },
   "machine2.example-esri.com": {...},
   "machine3.example-esri.com": {...}
}

Description

starttime—The time the GeoAnalytics Tools job started running in milliseconds from epoch.
id—A unique ID for the job.
name—The name of the job. This is always in the format ArcGIS GeoAnalytics (<jobID>). The jobID is the same jobID used at the REST endpoint when running a job.
cores—The total number of cores used for analysis across all machines.
user—The user who ran the job. This is the user who set up ArcGIS Server and will always be the same on your site. It is not the portal member that ran the job.
memoryperslave—The amount of memory used on each machine.
submitdate—The time the GeoAnalytics Tools job started running in a readable text format.
state—The state of the current job. FINISHED means that the job has been completed. The job may have completed successfully or resulted in an error.
duration—The total duration of the job in milliseconds.

Active apps

The activeapps key identifies the GeoAnalytics Tools that are currently running and their information. Once a job has been completed, the information is moved to the completedapps object.

Example 1 — One-machine site Example—Three-machine site

Example 1 — One-machine site	Example—Three-machine site
`{ "machine1.example-esri.com": { "url": "spark://machine1.example-esri.com:7077", "workers": [...], ..., "activeapps": [ { "starttime": 1520464610794, "id": "app-20180307151650-0007", "name": "ArcGIS GeoAnalytics (j4b986eb5c603426fae7abe8e9d5d7afa)", "cores": 7, "user": "agstest", "memoryperslave": 29491, "submitdate": "Wed Mar 07 15:16:50 PST 2018", "state": "RUNNING", "duration": 5105025 } ], "completedapps": [...] } }`	`{ "machine1.example-esri.com": { "url": "spark://machine1.example-esri.com:7077", "workers": [...], ..., "activeapps": [ { "starttime": 1520464610794, "id": "app-20180307151650-0007", "name": "ArcGIS GeoAnalytics (j4b986eb5c603426fae7abe8e9d5d7afa)", "cores": 9, "user": "agstest", "memoryperslave": 13107, "submitdate": "Wed Mar 07 15:16:50 PST 2018", "state": "RUNNING", "duration": 5105025 } ], "completedapps": [...] }, "machine2.example-esri.com": {...}, "machine3.example-esri.com": {...} }`

{
   "machine1.example-esri.com": {
       "url": "spark://machine1.example-esri.com:7077",
       "workers": [...],
       ...,
       "activeapps": [
           {
               "starttime": 1520464610794,
               "id": "app-20180307151650-0007",
               "name": "ArcGIS GeoAnalytics (j4b986eb5c603426fae7abe8e9d5d7afa)",
               "cores": 7,
               "user": "agstest",
               "memoryperslave": 29491,
               "submitdate": "Wed Mar 07 15:16:50 PST 2018",
               "state": "RUNNING",
               "duration": 5105025
           }
       ],
       "completedapps": [...]
   }
}

{
   "machine1.example-esri.com": {
       "url": "spark://machine1.example-esri.com:7077",
       "workers": [...],
       ...,
       "activeapps": [
           {
               "starttime": 1520464610794,
               "id": "app-20180307151650-0007",
               "name": "ArcGIS GeoAnalytics (j4b986eb5c603426fae7abe8e9d5d7afa)",
               "cores": 9,
               "user": "agstest",
               "memoryperslave": 13107,
               "submitdate": "Wed Mar 07 15:16:50 PST 2018",
               "state": "RUNNING",
               "duration": 5105025
           }
       ],
       "completedapps": [...]
   },
   "machine2.example-esri.com": {...},
   "machine3.example-esri.com": {...}
}

Description

starttime—The time the GeoAnalytics Tools started running in milliseconds from epoch.
id—A unique ID for the job.
name—The name of the job. This is always in the format ArcGIS GeoAnalytics (<jobID>). The jobID is the same job ID used at the REST endpoint when running a job.
cores—The total number of cores used for analysis across all machines.
user—The user who ran the job. This is the user who set up ArcGIS Server and will always be the same on your site. It is not the portal member who ran the job.
memoryperslave—The amount of memory used on each machine.
submitdate—The time the GeoAnalytics Tools started running in a readable text format.
state—The state of the current job. RUNNING means the job has started. WAITING means the job will wait for one minute to determine if there are enough resources. If there aren't enough resources within a minute, the job will cancel. If there are resources, it will start RUNNING.
duration—The current duration of the job in milliseconds.

Feedback on this topic?

Note:

Note:

Site information and resources

Workers

Completed apps

Active apps

In this topic