Kubernetes objects receiver
Collects objects from the Kubernetes API server. Supports authentication through service accounts only.
The Kubernetes Objects receiver collects objects from the Kubernetes API server. The supported pipeline is logs
. See Process your data with pipelines and Collect logs and events with the Collector for Kubernetes for more information.
Get started
splunkObservability.logsEnabled
or splunkPlatform.logsEnabled
to true
.To activate the Kubernetes Objects receiver, use this Helm configuration:
clusterReceiver:
k8sobjects:
auth_type: serviceAccount
k8s_leader_elector: k8s_leader_elector
objects:
- name: pods
mode: pull
label_selector: environment in (production),tier in (frontend)
field_selector: status.phase=Running
interval: 15m
- name: events
mode: watch
group: events.k8s.io
namespaces: [default]
You don't need to configure the logs
pipeline.
Main settings
These are the main configuration properties:
auth_type
.serviceAccount
by default. Determines how to authenticate to the Kubernetes API server. Values include none (for no authentication),serviceAccount
(to use the standard service account token provided to the agent pod), orkubeConfig
to use credentials from~/.kube/config
.name
. Name of the resource object to collect.mode
. Defines how the object is collected:pull
mode reads all objects of this type that use the list API at an interval.watch
mode sets up a long connection using the watch API to just get updates.
label_selector
. Select objects by label(s).field_selector
. Select objects by field(s).interval
.60m
(minutes) by default. Inpull
mode, the interval at which the object is pulled.exclude_watch_type
. Inwatch
mode, it allows excluding specific watch types. Valid values areADDED
,MODIFIED
,DELETED
,BOOKMARK
, andERROR
.resource_version
.1
by default. Inwatch
mode, it allows watch resources starting from a specific version. If not specified, the receiver does an initial list to get theresourceVersion
before starting the watch. See Efficient detection of changes in the Kubernetes documentation for details on why this is necessary.namespaces
.all
by default. An array of namespaces to collect events from.group
. Optional. API group name. When a given resource object is present in multiple groups, use this field to specify which group to select. By default, it selects the first group.For example, if the
events
resource is available in both thev1
andevents.k8s.io/v1
APIGroup, it selectsv1
by default.
See more at Settings.
Supported Kubernetes objects
The k8sobjectsreceiver
collects a wide range of standard Kubernetes objects from the API server. Among other, the receiver collects these objects:
pods
events
nodes
jobs
cronjobs
deployments
services
configmaps
secrets
statefulsets
replicasets
daemonsets
namespaces
This receiver supports both pull
and watch
modes, allowing for flexible and real-time monitoring of these objects. Note that custom resources are supported only if their CRDs are available in the cluster.
Configure the resources for the Kubernetes deployment
Follow these sections to set up the various Kubernetes resources required to deploy the Collector with the receiver.
Configure your ConfigMap
Create a ConfigMap with the config for otelcontribcol
, replacing OTLP_ENDPOINT
with a valid value.
apiVersion: v1
kind: ConfigMap
metadata:
name: otelcontribcol
labels:
app: otelcontribcol
data:
config.yaml: |
receivers:
k8sobjects:
objects:
- name: pods
mode: pull
- name: events
mode: watch
exporters:
otlp:
endpoint: <OTLP_ENDPOINT>
tls:
insecure: true
service:
pipelines:
logs:
receivers: [k8sobjects]
exporters: [otlp]
Service account
Create a service account for the Collector to use.
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app: otelcontribcol
name: otelcontribcol
Role-based access control (RBAC)
Use the commands in this section to create a ClusterRole
with the required permissions and a ClusterRoleBinding
to grant the role to the service account created in the previous section.
When using watch
mode you must also specify the list
verb so that the receiver has permission to do its initial list if no resource_version
is supplied, or use a list to recover from 410 Gone scenarios. Learn more in the official Kubernetes documentation at 401 gone responses.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: otelcontribcol
labels:
app: otelcontribcol
rules:
- apiGroups:
- ""
resources:
- events
- pods
verbs:
- get
- list
- watch
- apiGroups:
- "events.k8s.io"
resources:
- events
verbs:
- watch
- list
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: otelcontribcol
labels:
app: otelcontribcol
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: otelcontribcol
subjects:
- kind: ServiceAccount
name: otelcontribcol
namespace: default
Deploy the receiver as one replica
Deploy the Collector with the Kubernetes Objects receiver as one replica, otherwise it produces duplicated data.
apiVersion: apps/v1
kind: Deployment
metadata:
name: otelcontribcol
labels:
app: otelcontribcol
spec:
replicas: 1
selector:
matchLabels:
app: otelcontribcol
template:
metadata:
labels:
app: otelcontribcol
spec:
serviceAccountName: otelcontribcol
containers:
- name: otelcontribcol
image: otelcontribcol:latest # specify image
args: ["--config", "/etc/config/config.yaml"]
volumeMounts:
- name: config
mountPath: /etc/config
imagePullPolicy: IfNotPresent
volumes:
- name: config
configMap:
name: otelcontribcol
Settings
The following table shows the configuration options for the receiver:
included
https://raw.githubusercontent.com/splunk/collector-config-tools/main/cfg-metadata/receiver/k8sobjects.yaml
Troubleshooting
Ensure resources are allocated
If the receiver returns an error similar to the one below, make sure that resource
is added to ClusterRole
.
{"kind": "receiver", "name": "k8sobjects", "pipeline": "logs", "resource": "events.k8s.io/v1, Resource=events", "error": "unknown"}
General troubleshooting
If you are a Splunk Observability Cloud customer and are not able to see your data in Splunk Observability Cloud, you can get help in the following ways.
Available to Splunk Observability Cloud customers
-
Submit a case in the Splunk Support Portal.
-
Contact Splunk Support.
Available to prospective customers and free trial users
-
Ask a question and get answers through community support at Splunk Answers.
-
Join the Splunk #observability user group Slack channel to communicate with customers, partners, and Splunk employees worldwide. To join, see Chat groups.