Detection Engine

Design constraint: in-process, offline, no external model call

Detection runs entirely inside the compiled agent, in memory, with no network call and no external model dependency. This is a deliberate privacy design: sending candidate sensitive text to an external classifier to check whether it's sensitive would defeat the purpose of the control. Detection combines structural pattern matching, statistical entropy analysis, and dictionary/checksum validation. Where the product's interfaces refer to "NLP/NER," that means dictionary and grammar-shape recognizers plus checksum validation plus entropy analysis.

Detector categories

A single pass over a text buffer runs every detector concurrently; each detector reports every occurrence it finds, with byte offsets.

Structural / format-based detectors

Checksum-validated Russian personal-data recognizers

Person names, INN (validated against the official INN checksum algorithm), OGRN, passport numbers, SNILS, driver's license numbers, dates of birth, addresses, and bank card numbers (validated with the Luhn algorithm). A checksum-validated match suppresses a lower-confidence structural match at the same position — a value that already passed a correct INN checksum, for instance, is flagged once by the checksum detector, with the generic high-entropy catch-all suppressed.

Statistical catch-all: entropy analysis

A Shannon-entropy detector catches high-randomness strings that don't match any named format — the intended backstop for arbitrary tokens and keys that don't follow a known vendor's structural convention.

• Pure hexadecimal runs of 32+ characters (optionally 0x-prefixed) are judged against a lower threshold of 3.5–4.0 bits per character — because random hex data structurally concentrates lower in this range at these lengths than mixed-alphabet random data does. Purely decimal runs are explicitly excluded from this relaxation.
  • This calibration is deliberately safe-side: it also flags MD5 (32 hex chars), SHA-1 (40), and SHA-256 (64) digests and Git object IDs for masking alongside genuine key material, so that any value in that length/format range is protected regardless of origin.
• UUIDs are explicitly excluded from the entropy catch-all (a recognizable, non-secret structural format).

Detection quality measurement (calibration)

Every detector activation is aggregated locally by detector, surface, and confidence level — matched values are never retained, only counts and classifications, in keeping with the zero-knowledge principle applied even to internal metrics. An operator can label a specific finding as a false positive, and the control plane builds and Ed25519-signs a calibration bundle from approved labels, exposed via a dedicated endpoint.

Calibration currently adjusts reporting: it changes how a finding is scored/labeled for quality-tracking purposes. Detection thresholds — the detector's underlying trigger condition — stay fixed. The labelled evaluation corpus used in testing covers 25 labelled rows (13 expected-positive, 12 expected-negative) spanning the detector set's major categories.

See also Masking & Streaming Enforcement, which covers how a detected span is redacted in a live stream without breaking output structure.