Best practices for sending data from an Edge Processor to a dataset

Inconsistent event schemas can prevent federated searches from being able to retrieve all of the intended data.

Data processing actions that impact event schemas include the following: adding a field, removing a field, renaming a field, or converting the data type of a field.

For events that match a known source type, process those events using a pipeline that's created from a template instead of a custom-configured pipeline.

See Use templates to create pipelines for Edge Processors for more information.

Templates are designed to maintain standardized schemas for events that are associated with known source types.

Additionally, when you send out data that uses Parquet format, templates can ensure that the schema of the resulting Parquet output is at least partially compatible with federated searches.