Troubleshoot ACS error messages

The table lists common ACS API error messages and suggested troubleshooting steps you can take to investigate and resolve these issues.

Feature Error message Troubleshooting steps
All ACS endpoints 401-unauthorized: call not properly authenticatedPossible causes:
  • JWT token has expired or is not valid yet. Check the Expires On and Not Before token fields. If token has expired, create a new valid token and try again.
  • Make sure to specify the JWT token value and not the token-id value in the Auth header.
  • If attempting to target a search head, the token may be valid, but not for the target search head. When targeting a search head, you must authenticate with a JWT token created on that specific search head. You can create such a token using the search head prefix with the ACS tokens endpoint. For more information, see Targeting a search head.
401-unauthorized","message":"failed to parse tokenPossible causes:
  • Incomplete or invalid JWT token specified in Auth header.
  • Make sure to specify the JWT token value and not the token-id value in the Auth header.
403-forbidden: operation not supported for splunk stack, please refer to the API documentation for limitations and more detailCheck ACS feature compatibility with your Splunk Cloud Platform deployment Experience. See ACS compatibility matrix.
403-forbidden: insufficient permission(s) to perform this operationCheck RBAC table to ensure the user has the right capability(s) to access the endpoint. See Required ACS capabilties.
403-forbidden: user does not have the required capability: [x]Check RBAC table to ensure the user has the right capability(s) to access the endpoint. See Required ACS capabilties.
403-forbidden: user does not have one of the required roles: [x1, x2]User must have one of the listed roles to access the endpoint.
404-stack-not-foundMake sure stack name is correct. Each Splunk Cloud Platform deployment is identified by the stack-name, which is the prefix of the deployment's URL. For example, if your deployment's URL is "https://my-company-name.splunkcloud.com" the stack-name is "my-company-name".
409-object-already-existsAn object with the same name already exists. To update the object, use PATCH or PUT based on the specific ACS API feature.
500-internal-server-errorMake sure your deployment has one or more separate search heads or a search head cluster. ACS is not supported on single instance deployments. 
503-service-unavailable
"message": "service temporarily unavailable, please try again later."
Multiple concurrent requests can cause the service to become unavailable. Retry your request at a later time.
424-failed-dependency: A deployment task is still in progress. Please try again later.On Classic Experience, if a previous POST/PUT/DELETE/PATCH request is in progress, subsequent requests cannot be made. Retry your request when previous request completes. See Retry failed operations in Splunk Cloud Platform.
Configure IP allow lists400-bad-request cannot remove all access to stack's <feature>This check prevents users from removing all subnets for a certain feature. It ensures there's at least one subnet, preventing users from cutting off all access to their stack for ingestion.
400-bad-request missing access feature parameterYou must specify the {feature} parameter with the ACS request, for example search-api, hec,s2s, and so on. For more information, see Determine IP allow list use case. 
400-bad-request 
"message": "SH allowlist count of 242 exceeds maximum allowed count of 230"
Make sure the aggregate number of IP subnet rules for the IP allowlist feature group (search heads, indexers, IDMs, or single instance) does not exceed 230. For more information, see IP allow list behavior and IP subnet limits.
400-bad-request subnet overlaps splunk's reserved IP blocksRefer to Splunk Cloud Platform reserved IP block:

"54.147.174.149/32"

"3.210.45.80/32"

"50.19.33.107/32"

"18.208.12.214/32"

"35.168.6.234/32"

"54.227.109.89/32"

400-bad-request cannot delete splunk's reserved subnet
Configure outbound ports400-bad-request invalid port number provided. Please retry with a port number within range 1-65535Determine correct port number and try again with a port number between 1-65535. Not including ports 443, 9997, 2525. 
400-bad-request
"message": "outbound rules for port 443 may not be added."
Export apps503: App export feature is temporarily unavailableIndicates that app export endpoint has been temporarily deactivated due to a known issue that can cause credential information to be overwritten.
403-forbidden","message":"app export operation not supported for this splunk version.The specific version of Splunk Cloud Platform you are using does not support ACS app export. To request an upgrade to a version that supports ACS app export, contact Splunk Support. To find the current Splunk Cloud Platform version, in Splunk Web, select Support & Services, then select About.
Manage app permissions404-app-not-found: {app} app not foundUser inputted a nonexistent app or user's role does not have read permission.
400-bad-request: Read or write permissions list may not be empty.User inputted an empty array as one of the arguments to PATCH. 
400-bad-request: invalid role provided.
<name of offending role> role does not exist.
User inputted an invalid role as an argument in PATCH. Role may be an existing role that the user does not have access to.
400-bad-request: Invalid roles set. * cannot include other roles.User inputted *, meaning all roles, alongside other roles to PATCH. Remove the duplicate roles.
400-bad-request: Invalid role set. Roles with write permission must have read permission.User inputted *, meaning all roles, alongside other roles to PATCH. Remove the duplicate roles.
Manage HTTP Event Collector (HEC) tokens404-hec-not-foundIf returned by GET request after creating new hec-token, token creation is still in progress. It might take a few minutes for hec-token creation process to complete. Repeat GET request until status code is 200. 
424-failed-dependency
"message": "Cannot update HEC tokens because the previous deployment task has failed. Retry the latest deployment task using the following ACS API retry endpoint.
If returned after running an HEC token management operation, send a POST request to the deployment/retry endpoint to retry the failed operation. See Retry failed operations in Splunk Cloud Platform.
Manage indexes403-forbidden internal index names cannot be usedRefer to list of internal indexes:

"Main"

"History"

"Splunklogger"

"Summary"

"lastchanceindex"

403-forbidden internal index names cannot be deleted
400-bad-request internal index names are not allowed for defaultIndex
404-index-not-foundIf returned by GET request after creating new index, index creation is still in progress. It might take a few minutes for the index creation process to complete. Repeat GET request until status code is 200.
Manage limits.conf configurations403-forbidden - not allowed to access stanzaSpecified limits.conf stanza is not supported by ACS. For more information on limits.conf error messages, see Manage limits.conf configurations.
Manage private apps400-bad-request This app has failed AppInspect validationPossible causes:
  • Appinspect validation failures. Verify the failure count in the Appinspect report.
  • Appinspect version 3.3.0 and higher adds a check_package_compression validation check. Make sure to use the -z flag to compress the file when tarring an app package. For example: tar -zcvf acs_test_app.
  • Incorrect Appinspect tags. Make sure to specify the correct tags, as follows:
    • For Victoria Experience:
      • Version 8.2.2111 and lower, correct tags are 'included_tags="cloud,self-service"'.
      • Version 8.2.2112-8.2203, correct tag is 'included_tags="private_app"'.
      • Version 9.0.2205 and higher, no tag required when sending POST request to app/validate endpoint. But make sure to specify query parameter ?included_tag=private_victoria, when sending GET requests to the app/validate/status/{request_id} and app/report/{request_id} endpoints. See Validate your private app (Victoria Experience).
    • For Classic Experience:
      • Version 8.2.2203 and lower, the correct tag is 'included_tags="private_app"'.
      • Version 9.0.2205 and higher, no tag required when sending POST requests to app/validate endpoint. But make sure to specify query parameter ?included_tag=private_classic, when sending GET requests to the app/validate/status/{request_id} and app/report/{request_id} endpoints. See Validate your private app (Classic Experience).
  • Splunk Cloud Platform versions prior to 8.2.2111 might have manual checks failing Appinspect validation.
  • Check if the user doesn't override value of private_app_vetting_global in server.conf
  • Make sure the App-inspect token is not expired. It expires in about ~1 day. Try with a fresh token.
  • If using Github private-app-demo or ACS CLI, Victoria stack must be on version 8.2.2112 or highger.
400-bad-request "message":"AppUpload_InspectValidation: Unable to retrieve AppInspect validation result. Try again later.Possible causes:
  • Incorrect Appinspect tags. Make sure to specify the correct tags, as follows:
    • For Victoria Experience:
      • Version 8.2.2111 and lower, correct tags are 'included_tags="cloud,self-service"'
      • Version 8.2.2112-8.2203, correct tag is 'included_tags="private_app"'.
      • Version 9.0.2205 and higher, no tag required when sending POST request to app/validate endpoint. But make sure to specify query parameter ?included_tag=private_victoria, when sending GET requests to the app/validate/status/{request_id} and app/report/{request_id} endpoints. See Validate your private app (Victoria Experience).
    • For Classic Experience:
      • Version 8.2.2203 and lower, the correct tag is 'included_tags="private_app"'.
      • Version 9.0.2205 and higher, no tag required when sending POST request to app/validate endpoint. But make sure to specify query parameter ?included_tag=private_classic, when sending GET requests to the app/validate/status/{request_id} and app/report/{request_id} endpoints. See Validate your private app (Classic Experience).
400-bad-request Extract app information from the package failedPossible causes:
  • Occurs when your file is not added to the app package or you specify an incorrect path. Make sure to specify a valid app installation package.
  • Invalid app-package file format. App package format must be .tgz, .tar.gz, .spl or .tar file.
Manage Python version400-bad-request
"message": "pythonVersion is invalid".		Determine the correct Python version for your deployment. You can upgrade or downgrade the version using the ACS python-runtime endpoint. Supported Python versions: force_python3, python3, python3.7, python3.9, and python2. See Manage Python versions in Splunk Cloud Platform.
Web Application Firewall (WAF) related errors

No ACS API error message. 403 or 429 status code in the response header.

When a WAF block occurs, the response body is empty. The lack of a response body and a 403/429 status code indicates a likely WAF-related block.

 Possible fixes:
  • For private apps, make sure to specify the complete file path, for example: /Users/<username>/downloads/app.tgz without ../../ in the path.
  • Send the request again on a different machine or network.
  • Check the request rate. Have you exceeded the rate limit of 600 reqs/10 mins? Exceeding the rate limit can cause requests to be blocked for about 5 minutes. Dropped requests return a 429 status code in the response header when running curl with -i flag for detailed info.
Note: For further assistance with ACS errors, contact Splunk Support.