Skip to main content

Overview

SDK Metrics let you track custom data quality metrics from any codebase — ETL pipelines, batch jobs, microservices, or applications. Corelayer builds statistical baselines for each metric partition and alerts you when values deviate from expected ranges.

Prerequisites

  • A GitHub integration connected to Corelayer (the repository you want to track must be linked)
  • Node.js 18+ (for the SDK)

Onboarding

1. Create an API Key

You need an API key to authenticate SDK calls.
  1. In your Corelayer dashboard, go to Settings > API Keys
  2. Click Create Key
  3. Give the key a name (e.g., Production ETL) and set an expiration (optional)
  4. Copy the key immediately — it will only be shown once

2. Set the Environment Variable

Add the API key as an environment variable in your runtime environment:
export CORELAYER_API_KEY="clr_your_api_key_here"
export CORELAYER_BASE_URL="https://api.corelayer.com"
For production deployments, add both variables to your environment configuration (e.g., .env, Docker secrets, CI/CD variables, or your cloud provider’s secret manager).

3. Install the SDK

npm install @corelayer-ai/sdk

4. Create a Metric in the Dashboard

  1. Go to the Anomalies page in your Corelayer dashboard
  2. Click Add Rule and select a codebase (repository)
  3. On the metrics listing page, click Create Metric
  4. Fill in the configuration:
    • Metric Name — A human-readable label (e.g., Daily Order Volume)
    • Frequency — How often data is reported (hourly, daily, weekly, monthly)
  5. Copy the Metric ID and the SDK Snippet provided on the page
  6. Click Save Metric

5. Instrument Your Code

Add the SDK to your pipeline or application:
import { CorelayerClient } from '@corelayer-ai/sdk';

const client = new CorelayerClient({
  apiKey: process.env.CORELAYER_API_KEY,
  baseUrl: process.env.CORELAYER_BASE_URL,
});

client.trackMetric({
  metricId: "<your-metric-id>",
  partitions: [
    { keys: { "region": "us-east-1", "table": "orders" }, value: 1523 },
  ],
  timestamp: "2025-01-15T00:00:00Z",
});

// Call at process exit to flush remaining metrics
await client.shutdown();
Parameters:
  • metricId — The UUID from step 4. Identifies which metric this data belongs to.
  • partitions — An array of partition entries. Each entry has keys (an object of key-value pairs that define the partition) and value (the numeric count or measurement).
  • timestamp — An ISO 8601 timestamp. Pass an explicit timestamp for batch jobs to ensure correct ordering, especially for reruns.

6. Deploy and Monitor

Once your code is deployed and sending data, Corelayer will:
  1. Collect data to build a baseline for each partition (status shows Collecting)
  2. Switch to active monitoring once the baseline is ready (status shows Ready)
  3. Flag anomalies when values fall outside the expected range

Environment Variables Reference

VariableRequiredDescription
CORELAYER_API_KEYYesAPI key for authenticating SDK calls. Create one at Settings > API Keys.
CORELAYER_BASE_URLYesThe Corelayer API endpoint. Set to https://api.corelayer.com.

Viewing Metrics

Metrics Listing

Navigate to Anomalies and click on a codebase to see all SDK metrics for that repository. Each metric card shows:
  • The metric name (or ID if no name was set)
  • Baseline status (Collecting or Ready)

Metric Detail Page

Click on a metric to see the full detail view:
  • Stat Cards — Total partitions, baselines ready, and anomalies detected in the selected time range
  • Time-Series Chart — Visualize metric values over time for up to 10 partitions. Switch between count, min, max, and avg using the dropdown.
  • Partition Table — Browse all partitions with search and pagination. Each row shows the partition keys, current metric values, baseline statistics, and anomaly count.

Anomaly Findings

When anomalies are detected, they appear in the Anomalies column of the partition table. Click the count badge to expand and see detailed findings:
  • Severity — Critical, High, Medium, Low, or Info
  • Observed vs. Expected — The actual value compared to what the baseline predicted
  • Bounds — The upper and lower thresholds that were breached
  • Timestamp — When the anomaly was detected
Use the time range selector (7d, 30d, 90d) to adjust the window for chart data and findings.

Tips

  • Use descriptive partition keys — Keys like region, pipeline, or table_name make findings easier to diagnose.
  • Set explicit timestamps for batch jobs — This ensures data is bucketed correctly, even during reruns.
  • Call client.shutdown() — Always call shutdown before your process exits to flush any buffered metrics.
  • One metric per logical measurement — Create separate metrics for different measurements (e.g., Order Volume and Payment Processing Time) rather than mixing them.

Managing Metrics

Deleting Metrics

On the metrics listing page, select one or more metrics using the checkboxes and click Delete Selected. This removes the metric configuration and all associated baselines and findings.

Rotating API Keys

If you need to rotate your API key:
  1. Go to Settings > API Keys
  2. Click the rotate icon next to the key
  3. Copy the new key and update your environment variable
  4. The old key is revoked immediately
Need help? Contact support for assistance with SDK metrics.