1. Confirm eligibility

To confirm your eligibility for private connectivity, send an HTTP GET request to the following endpoint:

GET /{stack}/adminconfig/v2/private-connectivity/eligibility

For example, to check if TestStack is eligible, send the following request:

curl --location --request GET 'https://admin.splunk.com/TestStack/adminconfig/v2/private-connectivity/eligibility' \
--header 'Authorization: Bearer abcdefgh123456....' 

The request returns a success or failure result based on whether the Splunk Cloud Platform deployment meets the required eligibility criteria.

If your Splunk Cloud Platform deployment is eligible, the request returns:

{
"eligible": true
}

If your Splunk Cloud Platform deployment is not eligible, the request returns:

{
"eligible": false
}

2. Enable private connectivity

Once you confirm that your Splunk Cloud Platform deployment is eligible for private connectivity, send an HTTP POST request to the following endpoint:

POST /{stack}/adminconfig/v2/private-connectivity/endpoints

This request requires you to specify an AWS account ID or Azure subscription ID and a feature that you want to associate with the private connectivity endpoint. For example, to enable private connectivity for both ingest and search for TestStack with your cloud provider-specific ID:

### AWS
curl --location --request POST 'https://admin.splunk.com/TestStack/adminconfig/v2/private-connectivity/endpoints'  \
-H "Content-Type: application/json" --header 'Authorization: Bearer abcdefgh123456...' \
--data-raw '{
"customerAccountIds" : ["112233445566"],
"feature" : ["ingest", "search"]
}'

### Azure
curl --location --request POST 'https://admin.splunk.com/TestStack/adminconfig/v2/private-connectivity/endpoints'  \
-H "Content-Type: application/json" --header 'Authorization: Bearer abcdefgh123456...' \
--data-raw '{
"customerAccountIds" : ["9068c06d-21f8-bcc3-4e68-c569b514b4b6"],
"feature" : ["ingest", "search"]
}'

The API creates a new endpoint and adds the cloud ID(s) to the endpoints. The request returns the cloud ID(s) that were added to the endpoints:

{
"customerAccountIds" : ["112233445566"],
"feature" : ["ingest", "search"]
}

2.a Retrieve endpoint name

To confirm that your private connectivity request was enabled and to retrieve the endpoint ID(s), send an HTTP GET request to the following endpoint:

GET /{stack}/adminconfig/v2/private-connectivity/endpoints

For example, to retrieve the VPC endpoint ID for TestStack for all features:

curl --location --request GET 'https://admin.splunk.com/TestStack/adminconfig/v2/private-connectivity/endpoints?feature=all' \
--header 'Authorization: Bearer abcdefgh123456...' 

When private connectivity for ingest and search have been successfully enabled, the request returns the cloud account ID(s), the name of the private virtual network endpoint, (VPC endpoint in AWS, or Resource ID in Azure S2S forwarder) or Resource ID and Target Sub Resource (Azure HEC and Search), a message confirming enablement, and an available status.

AWS

On AWS, the request returns the AWS account ID(s) and the name of the VPC endpoint. For example:

### AWS
{
    "endpoints":[
        {
        "customerAccountIds": [
            "112233445566",
            "778899101011"
        ],
        "endpoint": "com.amazonaws.vpce.us-east-1.vpce-svc-00f4e0098fce35fb9",
        "feature":"ingest",
        "message": "Private Connectivity for ingest is enabled. Please refer to 
        https://docs.splunk.com/Documentation/SplunkCloud/latest/Security/Privateconnectivity 
        for more information.",
        "status": "available"
        },
        {
        "customerAccountIds": [
            "112233445566",
            "778899101011"
        ],
        "endpoint": "com.amazonaws.vpce.us-east-1.vpce-svc-0d0f607d0a67e50f9",
        "feature":"search",
        "message": "Private Connectivity for search is enabled. Please refer to 
        https://docs.splunk.com/Documentation/SplunkCloud/latest/Security/Privateconnectivity 
        for more information.",
        "privateSearch-DNSRecords":[
            "sh1.pvt.TestStack.splunkcloud.com",
            "shc1.pvt.TestStack.splunkcloud.com",
            "pvt.TestStack.splunkcloud.com"
        ],
        "status": "available"
}]

Azure

On Azure, the request returns the Azure subscription ID(s). The Azure S2S forwarder returns the Resource ID and Azure HEC and Search returns the Resource ID as well as a Target Sub Resource. For example:

### Azure
{
    "endpoints":[
        {
        "customerAccountIds": [
            "7ea26c8a-8ffb-42da-bb7d-7f3992ba3aba",
            "c9c82e13-25a5-4bc4-a4ac-adf8ee343100"
        ],
        "feature":"ingest",
        "message": "Private Connectivity for ingest is enabled. Please refer to 
https://docs.splunk.com/Documentation/SplunkCloud/latest/Security/Privateconnectivity 
for more information.",
        "resourceId":
"/subscriptions/ab2b15d7-1553-4d97-8e6f-df7859dcad17/resourceGroups/TestStack-ingest-lb-rg/providers/Microsoft.Network/privateLinkServices/TestStack-s2s-pl-svc",
        "status": "available"
        },
        {
        "customerAccountIds": [
            "7ea26c8a-8ffb-42da-bb7d-7f3992ba3aba",
            "c9c82e13-25a5-4bc4-a4ac-adf8ee343100"
        ],
        "feature":"ingest",
        "message": "Private Connectivity for ingest is enabled. Please refer to 
https://docs.splunk.com/Documentation/SplunkCloud/latest/Security/Privateconnectivity 
for more information.",
        "resourceId":
"/subscriptions/ab2b15d7-1553-4d97-8e6f-df7859dcad17/resourceGroups/TestStack-hec-appgw-rg/providers/Microsoft.Network/applicationGateways/TestStack-hec-appgw",
        "status": "available",
        "targetSubResource": "TestStack-hec-feip"
        },
        {
        "customerAccountIds": [
            "7ea26c8a-8ffb-42da-bb7d-7f3992ba3aba",
            "c9c82e13-25a5-4bc4-a4ac-adf8ee343100"
       ],
        "feature":"search",
        "message": "Private Connectivity for search is enabled. Please refer to 
https://docs.splunk.com/Documentation/SplunkCloud/latest/Security/Privateconnectivity 
for more information.",
        "privateSearch-DNSRecords": [
            "pvt.TestStack.splunkcloud.com",
            "sh1.pvt.TestStack.splunkcloud.com"
        ],
	"resourceId": 
"/subscriptions/ab2b15d7-1553-4d97-8e6f-df7859dcad17/resourceGroups/TestStack-search-appgw-rg/providers/Microsoft.Network/applicationGateways/TestStack-search-appgw",        
        "status": "available",
        "targetSubResource": "TestStack-search-feip"
	}
]

For search: Note the privateSearch-DNSRecords returned. These search tier DNS record(s) will be used to configure DNS routing within your private virtual network later.

If private connectivity has not been enabled, the request output shows the status of the request and a reason for that status. For example:

Private connectivity for ingest is not enabled:

{
    "endpoints":[
        {
        "feature":"ingest",
        "status": "unavailable",
        "reason": "uninitialized"
        }
    ]
}

Private connectivity for ingest initialization request is still in progress:

{
    "endpoints":[
        {
        "feature":"ingest",
        "status": "unavailable",
        "reason": "initialization is in progress"
        }
    ]
}

Private connectivity for ingest request failed:

{
    "endpoints":[
        {
        "feature":"ingest",
        "status": "unavailable",
        "reason": "initialization failed"
        }
    ]
}

3. (For Data Ingest only) Download universal forwarder credentials for private connectivity

Once you have enabled Splunk Cloud Platform private connectivity, follow the instructions in Deploy the universal forwarder to configure your universal forwarders to connect with your cloud provider's private link technologies. Download the private connectivity package for each universal forwarder that you want to use to send data to private endpoints.

To confirm your forwarder is configured for private connectivity, check the server value in outputs.conf.

• Non-private connectivity: inputs1.<stack-name>.splunkcloud.com, inputs2.<stack-name>.splunkcloud.com, ...
• Private connectivity: inputs1.pvt.<stack-name>.splunkcloud.com, inputs2.pvt.<stack-name>.splunkcloud.com, ...

4. Create private virtual network endpoint and enable private link

Now that you have configured your universal forwarders for your cloud provider's private link technology, follow the instructions from your cloud provider to create an endpoint(s) for the private connectivity features that you enabled.

• Access an AWS service using an interface VPC endpoint - Amazon Virtual Private Cloud

• Quickstart: Create a private endpoint by using the Azure portal

For ingest

AWS

• In PrivateLink Ready partner services, add the VPC endpoint name provided by Splunk in Step 2.a under "feature : Ingest" in the Service Name field.
• Set IP address type to IPv4.
• For security groups, allow port 443 and 9997 as inbound.
• In endpoint service, under the details section, take note of the first DNS record entry with the vpce-xxx.vpce-svc-xxx prefix (in other words, the first entry that does not include the availability zones). You will use this to configure your private DNS zone. For example: com.amazonaws.vpce.us-east-1.vpce-svc-10022f155a5dd8b21

Azure

• In Private Link Center | Private Endpoints, select Connect to an Azure resource by resource ID or alias radial, then add the resource ID (S2S) or resource ID and target sub resource (HEC) provided by Splunk in Step 2.a under "feature : Ingest" in the field(s).
• For security groups, allow port 443 and 9997 as outbound any subnets accessing the endpoint. In Private Link Center | Private Endpoints, in the Private IP column, take note of private IP address that has been assigned to the given endpoint. You will use this to configure your private DNS zone and entries. For example: 10.1.2.3

For search

Set up a new endpoint service to receive search results.

AWS

• In Other endpoint services, add the VPC endpoint name provided by Splunk in Step 2.a under "feature : Search" in the Service Name field.
• Set IP address type to IPv4.
• For security groups, allow port 80, 443 and 8089 as inbound.
• In endpoint service, under the details section, take note of the first DNS record entry with the vpce-xxx.vpce-svc-xxx prefix (in other words, the first entry that does not include the availability zones). You will use this to configure your private DNS zone. For example: com.amazonaws.vpce.us-east-1.vpce-svc-10026f133a3cc1d94

Azure

• In Private Link Center | Private Endpoints, select Connect to an Azure resource by resource ID or alias radial, then add the resource ID and target sub resource (search) provided by Splunk in Step 2.a under "feature : Ingest" in the field(s).
• For security groups, allow 80, port 443 and 8089 as outbound any subnets accessing the endpoint. In Private Link Center | Private Endpoints, in the Private IP column, take note of private IP address that has been assigned to the given endpoint. You will use this to configure your private DNS zone and entries. For example: 10.1.2.3

5. Configure DNS name

To resolve Splunk forwarder or search traffic to the Splunk Cloud Platform from your endpoint services, you must configure appropriate DNS mappings for each cloud provider account that you want to connect to your private link endpoints.

AWS

Follow the instructions in Routing traffic to an Amazon Virtual Private Cloud interface endpoint by using your domain name - Amazon Route 53 with the following caveats:

• Create a private hosted zone with one of the following domain names:
  • Splunk Cloud Platform deployments in standard AWS regions: pvt.<stack-name>.splunkcloud.com
  • Splunk Cloud Platform deployments in GovCloud regions: pvt.<stack_name>.splunkcloudgc.com

For ingest:

AWS

• Create a CNAME record type for ' * ' and point it to the DNS record you copied in the previous section (for example: vpce-xxx.vpce-svc-xxx).

Azure

• Create an A record type for ' * ' and point it to the private IP address associated with the private link endpoint associated with the forwarder (S2S) resource (for example: 10.1.2.3).
• Create an additional CNAME record type for ' http-inputs ' and point it to the private IP address associated with the private link endpoint associated with the HEC resource (for example: 10.2.3.4).

For search:

AWS

• Make a note of all the DNS records associated with your search tier from Step 2a under the "Feature: Search" section.
• Create a CNAME record type for each DNS record obtained in the previous step and point them to the DNS record you copied in the previous section for Search (for example: vpce-xxx.vpce-svc-xxx).

Azure

• Make a note of all the DNS records associated with your search tier from Step 2a under the "Feature: Search" section.
• Create an A record type for each DNS record obtained in the previous step and private IP address associated with private link endpoint associated with the search resource (for example: 10.3.4.5).

Your Splunk Cloud Platform deployment is now configured for private connectivity.

(Optional) Add additional cloud account IDs

If you need to add additional cloud account IDs to the private link endpoint, send an HTTP PATCH request to the following endpoint:

PATCH /{stack}/adminconfig/v2/private-connectivity/endpoints

For example, to add an additional cloud account to the private link endpoint(s) for TestStack for feature ingest:

### AWS
curl --location --request PATCH 'https://admin.splunk.com/{stack}/adminconfig/v2/private-connectivity/endpoints' \
-H "Content-Type: application/json" --header 'Authorization: Bearer abcdefgh123456...' \
--data-raw '{
"customerAccountIds" : ["123456789101"],
"feature" : ["ingest", "search"]
}'

### Azure
curl --location --request PATCH 'https://admin.splunk.com/{stack}/adminconfig/v2/private-connectivity/endpoints' \
-H "Content-Type: application/json" --header 'Authorization: Bearer abcdefgh123456...' \
--data-raw '{
"customerAccountIds" : ["2d585d36-c8c6-ae66-a226-33460a6cf3e5"],
"feature" : ["ingest", "search"]
}'

The request returns the cloud account ID(s) that were added to the endpoints:

### AWS
{
    "feature": [
        "ingest", "search"
    ],
    "patchedCustomerAccountIds": [
        "123456789101"
    ]
}

### Azure
{
    "feature": [
        "ingest", "search"
    ],
    "patchedCustomerAccountIds": [
        "2d585d36-c8c6-ae66-a226-33460a6cf3e5"
    ]
}

(Optional) SAML configuration

Private connectivity--for search or data ingest--creates a private route (using AWS PrivateLink) to search or ingest your data in addition to the publicly-accessible route. Stacks configured for search using private connectivity can use SAML in one of two ways: mixed-mode or PrivateLink-only.

Mixed-mode SAML

Private search-enabled stacks can use SAML in a bimodal mode. Because most SAML IdPs only support a single redirect, this configuration requires that both Splunk and the SAML IdP have been updated to redirect to private search tier DNS entries. Clients will then rely on DNS resolution to resolve either the internet-facing IPs or the VPC endpoint IPs as applicable, respectively. When mixed-mode is enabled, a Splunk-managed private zone will be created that redirects to the regular, non-private, zone and clients accessing the search tier using PrivateLink will rely on the customer-managed private zone resolving to their VPC endpoint for private search.

Internet

DNS resolution flow

1. Client browses <search_tier>.<stack-name>.<domain>.
2. Search tier's SAML configuration sends auth to IdP.
3. Upon successful auth, IdP redirects to <search_tier>.pvt.<stack-name>.<domain>.
4. Client resolves <search_tier>.pvt.<stack-name>.<domain> in Splunk-managed private DNS zone, which are a CNAME(s) resolving to <search_tier>.<stack-name>.<domain>, the Splunk-managed primary DNS zone.
5. Client logs in with appropriate authorization using SAML over the Internet.
6. The client will see the URL change, which is expected, given the configuration. Despite the <search_tier>.pvt.<stack-name>.<domain> in the URL, traffic will traverse the Internet.

Private Link

DNS resolution flow

1. Client browses <search_tier>.<stack-name>.<domain>.
2. Search tier's SAML configuration sends auth to IdP. VPC or AWS Direct Connect connection must allow traffic to reach the SAML IdP.
3. Upon successful auth, IdP redirects to <search_tier>.pvt.<stack-name>.<domain>.
4. Client resolves address <search_tier>.pvt.<stack-name>.<domain> in customer-managed private DNS zone, which ultimately resolves to the VPC endpoint IP(s) from the newly created endpoint.
5. Client logs in with appropriate authorization using SAML over the AWS PrivateLink.

PrivateLink-only SAML

To configure SAML for use in a stack where Private Search is enabled and mixed-mode is set to false, ensure that Splunk and the SAML IdP have been updated to redirect to the private search tier CNAME(s) created in the steps above. DNS must resolve appropriately to the newly created private endpoint for the SAML handshake to complete properly.

Data Ingest

Use the following best practices if you send data to your Splunk Cloud Platform with:

HEC or serverless endpoints

• AWS Lambda function or Azure Function App must be bound to your private virtual network.
• Use the suggested HEC / serverless endpoint URLs:
  • Private endpoint: http-inputs.pvt.<stackname>.splunkcloud.com
  • Public endpoint: http-inputs.<stackname>.splunkcloud.com
• Traffic must be allowed through the security group associated with your serverless endpoint.

AWS Direct Connect, Azure ExpressRoute, or similar technologies

• Set up an additional DNS configuration to send data from on-premise targets via AWS VPC using AWS PrivateLink.
• Configure AWS Route 53 or similar inbound resolver to resolve inputs hostnames for your on-premise forwarder.

Routing traffic from different cloud regions

AWS

The interface VPC endpoint (PrivateLink) must be created in the VPC that is located in the same region as your Splunk Cloud Platform deployment. However, if you want to route traffic through private endpoints from a VPC located in a different region, you can leverage AWS capabilities like inter-region VPC peering. For more details, see AWS documentation on inter-region access to endpoint services.

Azure

The Azure cloud provider makes use of virtual networking, such that private endpoints are not required to be created within the same region as the Splunk Cloud Platform stack. For connectivity from different regions, simply create a new Private Link endpoint attached to a private virtual network in the desired region.