___ __ _______ ________ __ __________

_______ ___ _________ _______ ________ __ __________ _______ _____ ___ _____________ _________ ____ ____ ______

注:

Splunk Observability Cloud は Network Explorer ナビゲータを完全にサポートしていますが、アップストリームの OpenTelemetry eBPF Helm グラフは Splunk 公式サポートの対象外です。機能のアップデート、セキュリティ、またはバグ修正は、いかなる SLA の制約も受けません。

Network Explorerを引き続き使用してSplunk Observability Cloudでデータを確認したい場合は、アップストリームのeBPF Helmチャートをゲートウェイとして実行されているOpenTelemetry Collectorに向けます。

_____________

__ ___ _______ ________ ____ ___________ ___ ____ ____ ___ _________ _____________

____________

___________

___________

_______ ________ __ _________ __ ________________ ____________ __ _____ ______ ___ __________ ___________

_________ ______

  • _____ ______ ________ ____ __ _____ ___ __ _____ ___ ___ __ _____ ______ __________ ___ ________ ________ _______ ________ ___ ______ ___ ___ _________

  • ___ ___ _____ ________ ___ __ ______

  • ______ ________ _____ __ ______

  • ______ ________

  • ______ _____ _

  • ______ ___

__________ _______

_______ ________ ________ __________ ____ __ _______ ___ ____ ____________ ___ ________ __ ___ __________ ______________

____ _______

_______ ________ ________ ____ _______ ___ __ _______

注: _______ ________ __ ___ __________ ____ ___ _________ _________

__ ___ _______ ________ ____ __________ ___ ____ ____ ___ _________ _____________

____________

___________

_________ _______

__ ___________ _________ _______ __ __ _________ ____ _______ _______ _______ __ _______

_____ ____

___ ____ __ __ _____ __ ______ _____________ _____ __ _______ _______ ________ __ _________

_______ ________ __________

___ ____ _____ ___ _______ ________ ________ __ ___ _________ ___________

_________

___________

_________

__ __ ________

___ _______

___ _______ _____ ___ ____ ______ _________ __ ___ __________ ___ _______ ____ __ ______ ______ ____ ______ ______ ___ _______ ____ ________ __ ___ ______ ____________ __ _____________ _________ __ ___ ____ ____ _____

____ _______ ___ _________ __ _____ _ ________ __ ___ ________

___

___ ______ _________

___ ________ ________ ______ ______ ______ _____ ___________ ___ _________ ____ ______ ____ ___ _______

____ _______ ___ _________ ___ ______ _________ __ ____ __ ____ ______

___

___ __________ _________

___ __________ _________ _______ ________ _________ ____ ______ ____ __________ _________

___ __ ___ ____ __ ___ __________ _________ _______ ___ _________ __ _____ ___ ________ __ ___ __________ _________ __ ____ __________ ________

____ __ ___ ____ __ ____ ___ ___ __________ __________ ___ ____________________ __ ______

___ _____ _________

___ _____ _________ _______ ________ _________ ____ ______ ____ __________ _________

___ __ ____ __________ __ ______ ___ __ _________ _______ ____ ___ ___ ____ __ ___ __________ _________ _______ ___ _________ __ _____ _ ________ __ ___ _____ __________

___ __ ___ ____ __ ____ __ ___ _____ __________ ___ ______________________ __ _____

_______ _______ ________

__ _______ ___ ____ _______ ____ __ _______ _________ ___ ____ __ _______ ___ ________ ____ _______ ___ ______ _____________ _________ ____ _____ ___ ___ _____________ _________ ____ ____ ______

_______ ___ _________ ____ _____

___ ___ ______ ____________ __ _____________ _________ __ ____ ____ _______ _________ ___ ____ _______ __ __ ____ __________ _________ ____ ___ ____ ___ _________ _________

  • ____ __ ____ ____ _________ __ ___________ __ ____ ____ ______ ________ __ ___ ________

  • ____ __ ________ ______ __ ___________ _ ________ ________ __ ___ _______ ____ ___ _____ _____ ___ ______ ______

___ ____ ____ ______ ________ ___ ________ ________ ___ _______ __________ __ ___ ____ _____ ___ ___ ______ ____________ __ _____________ __________ __ __ ___ ___ ___ ____ _____ ______ __ _______ ___ ______ ____________ __ _____________ __________ ___ _____ ____ __ _________ _____ ____________ ___________ ___ _______ ___ _________ ___ __________ _____ ____ ___ ________ _____________

___ _________ _____ _____ ________ __________ ___ ____ _____________

_________

___________

_______

_________ ____ __________ _________ _____ _____ __ ________ __ _______ _________

_________

__________ _________ __ _______ _____ ____ _____ ____ _____ ___ _____ ___ ___ _________ __ ___ _______ _________

_________________________

______ _____ __ ____ _________ ____ ___ ___ ________ ____

_______________________________

______ _____ ___ ____ _____________ __ ______ _____ ____ ______ _____ __ ___________ ___ ____ ____________ ___ ______ ___ ______ ____________ ______ ______ _____ ______ _____________ ______

___________

__ _________ _____ ____ __________ ____ __________ ________

_____________

  • __ ___ ___ ______ _______ ________ __ __ ________ ______ ____________ __ _____________ _________ ______________ _____ _____________ __ ___

  • __ ___ ___ __________ _ ___ ________ __ ___ ______ ____________ __ _____________ _________ ___ ____ ____ __ _______ _________ ____ _______ _________ ___ ____ __ _____ __ ____ ___ __________ ___ ______ ____________ __ _____________ _________ __ ____ __________ _______ ____ __ ____ __________ _____

  • __ ___ ___ __________ _ ___ ________ __ ___ ______ ____________ __ _____________ _________ ___ ____ __ _______ _________ ____ ____ _______ ________ ___ ___ __________ _____________ _________ _______ ___ ____ __ _____

_______________________

  • __ ___ ___ ______ _______ ________ __ __ ________ ______ ____________ __ _____________ _________ ______________ _____ _______________________ __ ___

  • __ ___ ___ __________ _ ___ ________ __ ___ ______ ____________ __ _____________ _________ ___ ____ ____ __ _______ _________ ____ _______ _________ ___ ____ __ _____ _____ _______ ________ _______ ___ ___________________________________________

  • __ ___ ___ __________ _ ___ ________ __ ___ ______ ____________ __ _____________ _________ ___ ____ __ _______ _____________ _______ ____ ___ __________ ___ ______ __ ________ __ _______ ________ __________ ___ ____ __ _____

____________________

___ ____ __ _ _____ _______ ________ _______ _______ _____________ __ ________ _______ _________

注: ________ ____ _______ ____ __ ___ ____ ______ ___ _______________ _______ __ ___ ______ _____________ _________ ____ _____ __ ___________ ___ ____________ __ ___ __ _______ ____ ___ _______________ _______ __ ___ ____ ____ ______ ___ _______ ____ _______________ __ ____ ____ ______

_______ ___ ____ ____ _____

_____ ______ ________ ___ ______ ____________ __ ___ _____________ _________ _____ ___ ____ ______ ___ ___ _____________ ____ ____ _____ __ _______ _____ _________

helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update open-telemetry
helm install my-opentelemetry-ebpf -f ./otel-ebpf-values.yaml open-telemetry/opentelemetry-ebpf

____ ____ ____ ___ _____________________ ____ ___ ___ ________________ ______ ___ __ ___ ______ _____________ _________ _______ _______ _____ ___ ___ ___ ___ _______ ____ __ _______ ___ _________ ________

kubectl get svc | grep splunk-otel-collector

___ ___ _____________ _________ ____ ______ ____ ___ ____ ________

___ _____________ _________ ____ ____ _____ ________ ______ _______ __ ___ ___ ______ __ ____ __________ _____ ___ ______ _________ ________ ___ _______ _____________ ______ ____ _____ _____ ____ ______ __ ___ _________

__ ___ ____ __ _______ ___ ________ ________ _________ ___ ___ _________ ________

______
sudo apt-get install --yes linux-headers-$(uname -r)
______ ____________ _____
sudo yum install -y kernel-devel-$(uname -r)

________ _______ _______ ________ ___ __________

__ ____ ________ ___ ________ ___ ______ __________ ___ ___ __________ _________ ___ __________ ________ ____ ___ _____________ _________ ____ ____ ______ ___ _____ _________ _____ ______ ___

  1. ______ ___ ______ ___ ______ _____________ _________ ____ ______

    helm repo add splunk-otel-collector-chart https://signalfx.github.io/splunk-otel-collector-chart
helm repo update

  2. _______ ___ ______ ____________ __ _____________ __________ _______ ___ __________ ____ _____ ___________ _______

    _______ ____ _______ ________ _________
    helm --namespace=<NAMESPACE> install my-splunk-otel-collector \
--set="splunkObservability.realm=<REALM>" \
--set="splunkObservability.accessToken=<ACCESS_TOKEN>" \
--set="clusterName=<CLUSTER_NAME>" \
--set="agent.enabled=false" \
--set="clusterReceiver.enabled=false" \
--set="gateway.replicaCount=1" \
splunk-otel-collector-chart/splunk-otel-collector
    _______ _______ ________ ___ _____ _________
    helm --namespace=<NAMESPACE> install splunk-otel-collector \
--set="splunkObservability.realm=<REALM>" \
--set="splunkObservability.accessToken=<ACCESS_TOKEN>" \
--set="clusterName=<CLUSTER_NAME>" \
--set="splunkObservability.logsEnabled=true" \
--set="splunkObservability.infrastructureMonitoringEventsEnabled=true" \
--set="agent.enabled=true" \
--set="clusterReceiver.enabled=true" \
--set="gateway.replicaCount=1" \
--set="environment=<APM_ENV>" \
--set="gateway.resources.limits.cpu=500m" \
--set="gateway.resources.limits.memory=1Gi" \
splunk-otel-collector-chart/splunk-otel-collector

  3. ______ ___ ______ ___ _____________ _________ ____ ____ ______

    helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update

  4. _______ ___ ______ ____________ __ _____________ __________ _______ ___ __________ ____ _____ ___________ _______

    helm --namespace=<NAMESPACE> install my-opentelemetry-ebpf \
    --set="endpoint.address=<Gateway Service Name>.<Gateway Service Namespace>.svc.cluster.local" \
    open-telemetry/opentelemetry-ebpf

___ __________ ______ ____________ __ _____________ _________ ______________ ___ _______ ___ _________ ___ __________ _____ _____

________ _______ _______ ________ ___ _________

__ ____ ________ ____ ____ __ __ _________ _______ ____ __ ___ ___ __________ _____ _______ _____ ___ _______ _________ __ ________ __ _______ ___ _______ ________ ______ __________ ___ ____ __ _________ ________________ _________ _____ ___ ________ ______ _____ _____ __ _______ _______ _________

  1. ___ ___ _________ ______ __ ______ ___ _______ ___ ______ __ _____ __________ ______ __ _____ ______ __________

    tmp_dir=$(mktemp -d -t EBPF_NET-XXXXX)

cat > "${tmp_dir}/spc_bpf_allow.te" <<END
module spc_bpf_allow 1.0;
require {
    type spc_t;
    class bpf {map_create map_read map_write prog_load prog_run};
}
#============= spc_t ==============

allow spc_t self:bpf { map_create map_read map_write prog_load prog_run };
END
checkmodule -M -m -o "${tmp_dir}/spc_bpf_allow.mod" "${tmp_dir}/spc_bpf_allow.te"
semodule_package -o "${tmp_dir}/spc_bpf_allow.pp" -m "${tmp_dir}/spc_bpf_allow.mod"
semodule -i "${tmp_dir}/spc_bpf_allow.pp"

  2. ___ ___ _________ ________ __ ______ ___ ____ ______

    helm repo add splunk-otel-collector-chart https://signalfx.github.io/splunk-otel-collector-chart

  3. ___ ___ _________ _______ __ ______ ___ ____ ______

    helm repo update

  4. ___ ___ _________ _______ __ _______ ___ ______ ____________ __ _____________ __________ _______ ___ __________ ____ _____ ___________ _______

    helm --namespace=<NAMESPACE> install my-splunk-otel-collector \
--set="splunkObservability.realm=<REALM>" \
--set="splunkObservability.accessToken=<ACCESS_TOKEN>" \
--set="distribution=openshift" \
--set="clusterName=<CLUSTER_NAME>" \
--set="agent.enabled=true" \
--set="clusterReceiver.enabled=true" \
--set="gateway.replicaCount=1" \
splunk-otel-collector-chart/splunk-otel-collector

  5. ______ ___ ______ ___ _____________ _________ ____ ____ ______

    helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update

  6. _______ ___ ______ ____________ __ _____________ __________ _______ ___ __________ ____ _____ ___________ _______

    helm --namespace=<NAMESPACE> install my-opentelemetry-ebpf \
    --set="endpoint.address=<address_of_gateway>" \
    --set="podSecurityPolicy.enabled=false" \
    --set="rbac.create=true" \
    --set="k8sCollector.serviceAccount.create=true" \
    --set="kernelCollector.serviceAccount.create=true" \
    --set="kernelCollector.image.tag=4.18.0-372.51.1.el8_6.x86_64" \
    --set="kernelCollector.image.name=kernel-collector-openshift" \
    open-telemetry/opentelemetry-ebpf

  7. ___ ______ _________ ____ ____ __________ ______ __ _________ ___ ___ _________ _______ __ _________ __________ ______ ___ ___ ______ _________ _____

    oc adm policy add-scc-to-user privileged -z my-opentelemetry-ebpf -n <NAMESPACE>

  8. ___ ___ _________ _______ __ ______ ___ _______ ________ _______ ___________ _____ ___ ____ _________ ________ __ ____ ______ ___ ___ ______ __ ___ __ _ _____________ ____ ___________ _______ ________ ________ ______ __ ___ __________ ____

    oc adm policy add-scc-to-user anyuid -z my-opentelemetry-ebpf -n <NAMESPACE>

______ ___ ________ _________ __ ______ ____________ __ ___ _____________ _________

____ __________ ____ ___ _ ______ ____________ __ ___ _____________ __________ __ ___ _____ ____ __ ______ ____ _________ _________ __ ___ ______ __ __________ _____ ___ _____

___ ___ ______ ___ ______ ____________ __ ___ _____________ _________ ______ ____ _ __ _______ _________ ______ ______ _____________

_____ ___ ___ _______ ________ _______________

resources:
   limits:
      cpu: 4
      memory: 8Gi

___ ___ _________ ______________ __ _________ ____ ________ ______

_____________

________ _____

__ __ ___ ___________ ____ ______ ___ ______

____ _____ _______ _ __

__ __ _____ ____________ ____ ______ ___ ______

____ __ _______ _ __

__ __ _____ ____________ ____ ______ ___ ______

____ __ _______ _ __

_______

__ ___ _________ ________ ___ __ ___ __ _____ ___ ______ __ ___ __ _ ___

______ ___ _____ ____
resources:
  limits:
    cpu: 500m
    memory: 1Gi
____ _________ ______ ____________
helm --namespace=<NAMESPACE> install my-splunk-otel-collector --set="splunkObservability.realm=<REALM>,splunkObservability.accessToken=<ACCESS_TOKEN>,clusterName=<CLUSTER_NAME>,agent.enabled=false,clusterReceiver.enabled=false,gateway.replicaCount=1,gateway.resources.limits.cpu=500m,gateway.resources.limits.memory=1Gi" splunk-otel-collector-chart/splunk-otel-collector

______ ____ _______ ________ ____________

_________ __ ___ ______ __ __________ _____ ___ _____ ____ ________ _____ _____ _____ ___ ___ ____ ___ _________ ___________ __ ____ _____________

______ ___ ________ _________ __ ___ _______

___ _______ __ _ ______ ___ ___ __________ ________ __ ____ _______ ________ _ _____ ______ __ _____ ______ ___ _________ ___ ___ ________ ___ _________ _________ __ ___

___ _______ _________ _________ __ ________ _______ ____ ____ _____ ___________ ____ ___ __ ____ _______ _____ ____ _____ __ _ ________ _______ __________ ___ ______ __ ______ __ ____ _____ _______ ___ ________ __ ___ ________

______ ___ _________ __________ __ ___ _____________ _________ ____ ______ ____ __ ________ __ ________ ___ ______ __ ______ ___ _______ ______ ___ ___ ___ _______ ____ _______

___ _______ _____________ __ _ _____ ___ _______ ______

reducer:
  ingestShards: 1
  matchingShards: 1
  aggregationShards: 1

___ _________ _______ ____ _ ______ ___ _______ ______

reducer:
  ingestShards: 4
  matchingShards: 4
  aggregationShards: 4

__ ________ ___ ___ ___ ______ _____ ___ _______ _____ _______ ____ _ _____ ___ ___ ___ _____ ______ _________

Memory in Mebibytes (Mi) = 4 * Number of nodes in your cluster + 60
Fractional CPU in milliCPU (m) = Number of nodes in your cluster + 30

____ _____ ___ __ ___________ ________ ______ ________ ___ _____ _______ __ _ ______ __ ___ __ _ __ ____ ____ ___ ______ ___ ______ __ ______

_________ _______ _________ _________ __ _______ ________

__ ___ ____ __ _______ _____ __ ____ _______ _________ ________ ___ ___ ______ ___ _____________ _________ ____ ______ ____ _

___ _________ ________ ____ ___ ___ __ ____ ___ __ ____ __ _________ ________

____ __ ___ ________ _________ _______ ______ ___ __ _______

reducer:
  disableMetrics:
    - none

____ ___ ______ ______ __________

reducer:
  disableMetrics:
    - tcp.all
    - udp.all
    - dns.all
    - http.all

____ ___ __ __________ ___ ______

reducer:
  disableMetrics:
    - tcp.bytes
    - tcp.rtt.num_measurements
    - tcp.active
    - tcp.rtt.average
    - tcp.packets
    - tcp.retrans
    - tcp.syn_timeouts
    - tcp.new_sockets
    - tcp.resets

____ ___ __ __________ ___ ______

reducer:
  disableMetrics:
    - udp.bytes
    - udp.packets
    - udp.active
    - udp.drops

____ ___ __ __________ ___ ______

reducer:
  disableMetrics:
    - dns.client.duration.average
    - dns.server.duration.average
    - dns.active_sockets
    - dns.responses
    - dns.timeouts

____ ___ __ __________ ____ ______

reducer:
  disableMetrics:
    - http.client.duration.average
    - http.server.duration.average
    - http.active_sockets
    - http.status_code

____ ___ __ ________ ______

reducer:
  disableMetrics:
    - ebpf_net.bpf_log
    - ebpf_net.otlp_grpc.bytes_sent
    - ebpf_net.otlp_grpc.failed_requests
    - ebpf_net.otlp_grpc.metrics_sent
    - ebpf_net.otlp_grpc.requests_sent
    - ebpf_net.otlp_grpc.successful_requests
    - ebpf_net.otlp_grpc.unknown_response_tags
注: ____ ____ __________ ___ ___ __ ________ _______ _____ ___ _________ __ ________

____ __ ______ ______ __________

reducer:
  enableMetrics:
    - tcp.all
    - udp.all
    - dns.all
    - http.all
    - ebpf_net.all

____ __ __ __________ ___ ______

reducer:
  enableMetrics:
    - tcp.bytes
    - tcp.rtt.num_measurements
    - tcp.active
    - tcp.rtt.average
    - tcp.packets
    - tcp.retrans
    - tcp.syn_timeouts
    - tcp.new_sockets
    - tcp.resets

____ __ __ __________ ___ ______

reducer:
  enableMetrics:
    - udp.bytes
    - udp.packets
    - udp.active
    - udp.drops

____ __ __ __________ ___ ______

reducer:
  enableMetrics:
    - dns.client.duration.average
    - dns.server.duration.average
    - dns.active_sockets
    - dns.responses
    - dns.timeouts

____ __ __ __________ ____ ______

reducer:
  enableMetrics:
    - http.client.duration.average
    - http.server.duration.average
    - http.active_sockets
    - http.status_code

____ __ __ ________ ______

reducer:
  enableMetrics:
    - ebpf_net.span_utilization_fraction
    - ebpf_net.pipeline_metric_bytes_discarded
    - ebpf_net.codetiming_min_ns
    - ebpf_net.entrypoint_info
    - ebpf_net.otlp_grpc.requests_sent
注: ____ ____ ____ ___ _______ ___ ______ ___ __ ________ ________

_______

__ ___ _________ ________ ___ ____ _______ _____ ____ _______ __________ ___ ___ ___ _______ ___ ____________ ___ ___ _______ ___ __________

reducer:
  disableMetrics:
    - http.all
    - tcp.syn_timeouts
    - tcp.new_sockets
    - tcp.resets
    - udp.bytes
    - udp.packets

__ ___ _________ ________ ___ ____ _______ _____ ____ _______ __________ ________ _______ ___ ______ ___

注: ___ ______________ ____ __ _________ ______ ___ _____________ _____
reducer:
  enableMetrics:
    - http.all
    - ebpf_net.codetiming_min_ns
    - ebpf_net.entrypoint_info

_______ ____ _______________ __ ____ ____ _____

________ ____ _______ ____ __ ___ ____ ______ ___ _______________ _______ __ ___ ______ _____________ _________ ____ _____ __ ___________ _______________ ________ ___ _____ __________ ____ ___ _____________ _________ ____ ____ ______ _____ __ _________ __ _______ _________

__ _______ __ ___ _____________ _________ ____ ____ ______ ______ _____ ______

  1. ____ ____ ____ ___ ______ _____________ _________ ____ _____ __ _________ __ ____ __________ _________ _____

    gateway:
  enabled: true

  2. _______ ___ _______________ _______ __ ___ ______ _____________ _________ ____ ______

    networkExplorer:
  enabled: false

  3. ________ ___ ____ __ ___ ______ _____________ _________ _______ ________

    kubectl get svc | grep splunk-otel-collector-gateway

  4. _______ ___ ________ _____________ _________ ____ ____ _____ ________ __ ___ ______ _____________ _________ _______ ________

    helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update open-telemetry
helm install my-opentelemetry-ebpf -f ./otel-ebpf-values.yaml open-telemetry/opentelemetry-ebpf

___ _____________________ ____ ____ ____ ___ ________________ ______ ___ __ ___ ______ _____________ _________ _______ _______ ____ ________ __ ___ _____ _____

endpoint:
  address: <my-splunk-otel-collector-gateway>

_____________ __ ___ ___ ___ ______ ________ __ ___ _______________ ________ ___ ____ __ ____ ____ __ ___ _____________________ _____ ___ ___ _____________ _________ ____ ______ ____ ___ ____ ____________

____ _____

_____ ______ ___ __ _______ _________ ___ ___ _____ __________ _______ _________ _______ ______ ____ ____ ______ ______________ __________ ________ _____ _ __ ____ __ ___ _________ ________

  • ________ _______ ________ ___________ __ ___ ___ _______ ________ ___________ ______ _____ ______

    1. ____ ___ ______ _____________ _____ ____ _____ ______ ______________ __ ___ ____ __________

    2. ______ _______ _________

      _______ ________ _________ _____ __ ___ ______________ _______ _____

    3. ______ ___ ____ ___ ___ _______ ________ _________ ___ ____ __ _____

___ ____ ____________ ___ ___ __________ __ ______ ______________ ___________

___ ____ ___________ __ _______ _________ __ _______ ____ _______ _________ ___ _______ _________ ________