> ## Documentation Index > Fetch the complete documentation index at: https://docs.corelayer.com/llms.txt > Use this file to discover all available pages before exploring further. # Anomaly Detection for Data Pipelines > Corelayer detects silent data failures in pipelines and infrastructure, learning a statistical baseline per partition and flagging observations that fall outside it. ## What is Anomaly Detection? Corelayer runs **anomaly rules** against your data. Each rule produces a stream of numeric observations, learns a statistical baseline for every partition in that stream, and raises a finding when a new observation falls outside the expected band. There are two ways to feed observations into a rule: * **Table rules** — Connect a database integration and attach a rule to a table. You describe what to watch in plain English, Corelayer compiles it to a read-only SQL query, and runs that query on a schedule. * **API metrics** — Instrument your own code with the Corelayer SDK and push observations from any pipeline, ETL job, or service. Both paths share the same baseline engine, the same detectors, and the same findings UI. ## How It Works ### 1. Create a rule Go to **Anomaly Rules** in the sidebar and click **Add**. Pick a database table or a codebase, then configure the rule. See [Table Monitoring](/anomalies/table-monitoring) and [API Metrics](/anomalies/sdk-metrics) for the specifics of each. ### 2. Set the cadence and detector Every rule has an **expected cadence** — hourly, daily, weekly, or monthly — which sets how observations are bucketed and, for table rules, how often the query runs. Every rule also has a **detector**: the statistical method that scores each observation. For API metrics you choose it yourself, and `Baseline` is the default. For table rules Corelayer derives it from **Analyze Setup**. ### 3. Build the baseline A rule needs history before it can judge anything. Until a partition has enough observations, it shows as **Collecting** and reports nothing. * **Table rules** can train instantly using **Save & Backfill**, which runs the rule over historical rows. * **API metrics** collect forward from the first observation your SDK sends. Once a partition has enough data points, it flips to **Ready**. ### 4. Detection On each run, Corelayer compares the new observation against that partition's expected band. Anything outside the band is recorded as a **finding**, with a severity scaled to how far past the boundary it landed. ### 5. Review and triage Findings appear on the rule's detail page with a severity (**Critical**, **High**, **Medium**, **Low**, **Info**), the observed value, the expected value, and the band that was breached. Findings are also escalated for automated triage, which can open a Corelayer issue — the finding then links to that issue and shows whether it is still open. Breaches that only marginally clear the boundary are recorded but not escalated, so near-misses do not generate issues. ## Detectors Pick the detector that matches the shape of your metric. | Detector | Use it for | | -------------- | ------------------------------------------------------------------------------------------------------ | | **Baseline** | The default. A robust band around the mean, at `k` standard deviations. Good for most metrics. | | **Asymmetric** | Skewed or one-sided metrics, where a spike up matters but a dip down does not (or vice versa). | | **Seasonal** | Metrics with a repeating daily or weekly cycle. | | **Count** | Low-rate integer counts, where normal values are small and a Gaussian band would be wrong. | | **Conformal** | Distribution-free scoring, when the metric does not look normal and you do not want to assume a shape. | | **CUSUM** | Sustained level shifts, where the metric moves to a new plateau rather than spiking. | Each detector keeps its own rolling window, so switching detectors on an existing rule retrains it from stored history. ## Key Concepts * **Observation** — One numeric value for one partition at one point in time. A rule run produces a batch of them. * **Partition** — A logical slice of the stream, identified by its partition keys. Every partition gets its own independent baseline. Partitioning by `region` and `pipeline` gives you a separate baseline for each region-pipeline pair. * **Baseline** — The statistical state learned from a partition's recent observations, used to compute the expected band. * **Detector** — The method that turns a baseline plus a new observation into a pass or a finding. * **Minimum data points** — How many observations a partition must accumulate before the detector is allowed to flag anything. Leave it blank to use the detector's default. Lower values alert sooner but are noisier. * **Cadence** — How often observations are expected to arrive: hourly, daily, weekly, or monthly. It sets how observations are bucketed and, for table rules, how often the query runs. * **Finding** — A recorded anomaly, with severity, observed value, expected value, and the breached bounds. ## Getting Started Describe an anomaly in plain English and monitor a database table. Push custom metrics from your own pipelines using the Corelayer SDK. Need help? [Contact support](mailto:support@corelayer.com) for assistance with anomaly detection. ## Related topics - [API Metrics: Track Custom Pipeline Metrics](/anomalies/sdk-metrics.md) - [Corelayer Changelog: April 2026 Release](/changelog/april-2026.md) - [Corelayer Changelog: February 2026, v1.2.0](/changelog/february-2026.md)