Encoding plans

The encoding surface manages versioned EncodingPlan documents per tenant. These plans are stored with a numeric version and a status.

What an encoding plan is

An encoding plan tells Esper how to turn a raw incoming request into clean, named fields that the rest of the product can use.

If the incoming request is the raw material, the encoding plan is the translation layer.

When to use this page

Use Encoding when:

• a rule needs a field that does not exist yet
• the raw request shape is messy or inconsistent
• you want to derive a stable identity or action from multiple raw inputs

Version metadata

Each encoding config currently has:

• tenant_id
• version
• status
• config
• created_at

Supported statuses:

• Draft
• Active
• Retired

What the version fields mean

Field	Plain-English meaning	Why you care
tenant_id	The workspace this encoding version belongs to	Keeps data models isolated per tenant
version	The numbered revision of the encoding plan	Lets you reference a specific plan in replay jobs
status	Whether the version is draft, active, or retired	Signals whether it should be the current plan
config	The actual extraction plan	Contains all runtime, field, and derived logic
created_at	When the version was created	Useful for rollout history

Plan structure

An encoding plan contains:

• summary
• runtime
• fields
• derived_fields
• actions

What each section is for

Section	Plain-English meaning
summary	Short explanation of what this version is meant to do
runtime	Global rules for how Esper should treat incoming data
fields	Direct extractions from the request
derived_fields	New fields built from existing extracted values
actions	Canonical action labels such as login_attempt

Runtime policy

The current runtime config supports:

• flatten_objects
• normalize_field_names
• dedup_ttl_seconds
• entity_state_ttl_seconds
• session_state_ttl_seconds
• window_state_ttl_seconds
• session_bucket_minutes
• entity_identity_fields
• default_decision

Supported default decisions:

• Allow
• Observe
• Challenge
• Block

What the important runtime fields mean

Field	Plain-English meaning	Why it matters
flatten_objects	Break nested request objects into flatter field paths	Helps when payloads are deeply nested
normalize_field_names	Make field naming more consistent	Reduces rule-writing mistakes caused by naming drift
dedup_ttl_seconds	How long duplicate events should be treated as duplicates	Helps reduce noisy reprocessing
entity_state_ttl_seconds	How long entity-level state should be kept	Affects long-term memory of a user or identity
session_state_ttl_seconds	How long session-level state should be kept	Affects short-term flow tracking
window_state_ttl_seconds	How long window counters stay relevant	Controls short rolling activity checks
session_bucket_minutes	How Esper groups activity into sessions	Affects session boundaries
entity_identity_fields	Which fields define a unique entity	Critical for stable identity tracking
default_decision	Fallback outcome when no rule changes it	Best starting point is usually Observe

Extracted fields

Each extracted field includes:

• name
• description
• selector
• parser
• normalizers
• required

Supported selector kinds:

• RequestField
• Header
• QueryParam
• Cookie
• RouteParam
• Method
• Path
• SourceId
• ObservedAt

Supported parsers:

• String
• Symbol
• Bool
• U64
• I64
• F64
• IpAddress
• Timestamp
• Duration
• Json

Supported normalizers:

• Trim
• Lowercase
• Uppercase
• CollapseWhitespace

How to think about extracted fields

Each extracted field answers:

• what do I want to name this value?
• where do I read it from?
• how should I parse it?
• should I normalize it before rules use it?

For a new setup, start with only a few high-value fields:

• user identity, such as email
• action type, such as login_attempt
• source or request metadata, such as method or path

Derived fields and actions

Derived fields and actions both use the same expression family. The current UI supports these derived expression kinds:

• FieldRef
• Constant
• FirstNonEmpty
• Concat
• Conditional

Conditionals use these predicate kinds:

• FieldEq
• FieldContains
• FieldExists
• And
• Or
• Not

Plain-English examples

• A derived field might create device_key by combining multiple values.
• An action might map several raw request shapes to one stable label such as password_reset.
• A conditional expression lets you say "if this field looks like X, label it as Y; otherwise use Z."

Example plan

This example matches the app’s default starter plan:

{
  "summary": "Flexible request extraction for login traffic",
  "runtime": {
    "flatten_objects": true,
    "normalize_field_names": true,
    "dedup_ttl_seconds": 60,
    "entity_state_ttl_seconds": 3600,
    "session_state_ttl_seconds": 1800,
    "window_state_ttl_seconds": 300,
    "session_bucket_minutes": 15,
    "entity_identity_fields": ["email", "action"],
    "default_decision": "Observe"
  },
  "fields": [
    {
      "name": "email",
      "description": "Email extracted from the request payload",
      "selector": {
        "kind": "RequestField",
        "path": "body.request.email"
      },
      "parser": "String",
      "normalizers": ["Trim", "Lowercase"],
      "required": true
    }
  ],
  "derived_fields": [
    {
      "name": "device_key",
      "description": "Canonical derived device key",
      "expression": {
        "kind": "Concat",
        "delimiter": ":",
        "parts": [
          { "kind": "FieldRef", "field_name": "email" },
          { "kind": "Constant", "value": "browser" }
        ]
      },
      "parser": "String",
      "normalizers": ["Trim"]
    }
  ],
  "actions": [
    {
      "field_name": "action",
      "description": "Tenant-defined action extracted from the request",
      "expression": {
        "kind": "Constant",
        "value": "login_attempt"
      },
      "parser": "Symbol",
      "normalizers": ["Lowercase"]
    }
  ]
}

Frontend API contract

GET /tenants/{tenant_id}/encoding
POST /tenants/{tenant_id}/encoding/versions
PATCH /tenants/{tenant_id}/encoding/versions/{version}
DELETE /tenants/{tenant_id}/encoding/versions/{version}

Good operating guidance

• Keep your first encoding plan small and easy to explain.
• Prefer stable business names over raw request names.
• Use replay jobs after major encoding changes so you can see how historical traffic would be interpreted.