NER Detection

The NER (Named Entity Recognition) model enables detection of soft PII like person names, organizations, and locations that can’t be captured by regex patterns.

Model Modes

Mode	Description	Size	Use Case
`'disabled'`	No NER, regex only	0	Fast processing, structured PII only
`'quantized'`	Smaller quantized model	~280 MB	Recommended for most use cases
`'standard'`	Full-size model	~1.1 GB	Maximum accuracy
`'custom'`	Your own ONNX model	Varies	Domain-specific models

Basic Setup

import { createAnonymizer } from 'rehydra';

const anonymizer = createAnonymizer({
  ner: { 
    mode: 'quantized',
    onStatus: (status) => console.log(status),
  }
});

await anonymizer.initialize();  // Downloads model on first use

const result = await anonymizer.anonymize('Hello John Smith from Acme Corp!');
// "Hello <PII type="PERSON" id="1"/> from <PII type="ORG" id="1"/>!"

Download Progress

Track model download progress:

const anonymizer = createAnonymizer({
  ner: {
    mode: 'quantized',
    onStatus: (status) => console.log('Status:', status),
    onDownloadProgress: (progress) => {
      console.log(`${progress.file}: ${progress.percent}%`);
    }
  }
});

Output during first initialization:

Status: Downloading model files...
model.onnx: 15%
model.onnx: 30%
model.onnx: 100%
vocab.txt: 100%
Status: Loading NER model...
Status: NER model loaded!

Confidence Thresholds

NER entities have confidence scores (0.0-1.0). Configure minimum thresholds:

const anonymizer = createAnonymizer({
  ner: { 
    mode: 'quantized',
    thresholds: {
      PERSON: 0.8,     // 80% confidence required
      ORG: 0.7,        // 70% for organizations
      LOCATION: 0.6,   // 60% for locations
    }
  }
});

Lower thresholds → more detections (potentially more false positives) Higher thresholds → fewer detections (may miss some entities)

Case Fallback

The NER model is case-sensitive — it works best on properly capitalized text. This means lowercase names like "tom" or "sarah" can be missed. Enable caseFallback to run a second NER pass on title-cased text and merge any new detections:

const anonymizer = createAnonymizer({
  ner: {
    mode: 'quantized',
    caseFallback: true,
  }
});

await anonymizer.initialize();

await anonymizer.anonymize('hey tom, can you ask sarah to call me?');
// "hey <PII type="PERSON" id="1"/>, can you ask <PII type="PERSON" id="2"/> to call me?"

Without caseFallback, neither "tom" nor "sarah" would be detected.

How it works

The primary NER pass runs on the original text
A second pass runs on title-cased text (e.g. "tom" → "Tom")
New detections from the fallback pass that don’t overlap with primary detections are merged in
Fallback detections keep the original lowercase text and character offsets
A confidence penalty is applied to fallback detections to reduce false positives

Confidence penalty

Fallback detections receive a confidence penalty (multiplied by caseFallbackPenalty, default 0.85) since title-casing can introduce false positives. You can tune this:

const anonymizer = createAnonymizer({
  ner: {
    mode: 'quantized',
    caseFallback: true,
    caseFallbackPenalty: 0.7,  // Stricter penalty
  }
});

Enabling caseFallback doubles NER inference time since it runs two passes. Use it when your input text contains informal or uncapitalized names (chat messages, transcripts, etc.).

Auto-Download Control

By default, models are downloaded automatically. To disable:

const anonymizer = createAnonymizer({
  ner: {
    mode: 'quantized',
    autoDownload: false,  // Will throw if model not present
  }
});

Manual Model Management

Pre-download models or manage cache:

import { 
  isModelDownloaded,
  downloadModel,
  clearModelCache,
  listDownloadedModels
} from 'rehydra';

// Check if model exists
const hasModel = await isModelDownloaded('quantized');

// Pre-download with progress
await downloadModel('quantized', (progress) => {
  console.log(`${progress.file}: ${progress.percent}%`);
});

// List downloaded models
const models = await listDownloadedModels();
// ['quantized']

// Clear specific model
await clearModelCache('quantized');

// Clear all models
await clearModelCache();

Inference Server Backend

For batch processing or GPU acceleration, offload NER inference to a remote server:

const anonymizer = createAnonymizer({
  ner: {
    mode: 'quantized',
    backend: 'inference-server',
    inferenceServerUrl: 'http://localhost:8000/predict',
    inferenceServerTimeout: 30000,  // 30 seconds (default)
  }
});

This sends tokenized text to the server for inference instead of running ONNX locally. The server must accept the same input format and return logits in the expected shape.

Custom Models

Use your own ONNX model:

const anonymizer = createAnonymizer({
  ner: {
    mode: 'custom',
    modelPath: './my-model.onnx',
    vocabPath: './vocab.txt',
  }
});

Custom models must follow the same input/output format as the default models. See the model training guide for details.

Cache Locations

Models are cached locally for offline use:

Node.js

Platform	Location
macOS	`~/Library/Caches/rehydra/models/`
Linux	`~/.cache/rehydra/models/`
Windows	`%LOCALAPPDATA%/rehydra/models/`

Browser

In browsers, models are stored using:

Origin Private File System (OPFS) for large model files
IndexedDB for metadata

Data persists across page reloads and browser sessions.

NER-Detected Types

Type	Examples
`PERSON`	John Smith, Maria, Dr. Johnson
`ORG`	Acme Corp, Google, United Nations
`LOCATION`	Berlin, Germany, Central Park
`ADDRESS`	123 Main Street
`DATE_OF_BIRTH`	born on March 15, 1990

Disabling Specific NER Types

Detect only certain entity types:

import { createAnonymizer, PIIType } from 'rehydra';

const anonymizer = createAnonymizer({
  ner: { mode: 'quantized' },
  defaultPolicy: {
    nerEnabledTypes: new Set([
      PIIType.PERSON,  // Only detect names
    ])
  }
});

Performance Tips

Reuse the anonymizer instance

Model loading is expensive. Create once and reuse:

// ✅ Good: create once
const anonymizer = createAnonymizer({ ner: { mode: 'quantized' } });
await anonymizer.initialize();

// Reuse for multiple texts
await anonymizer.anonymize(text1);
await anonymizer.anonymize(text2);

// Dispose when done
await anonymizer.dispose();

Use quantized model for most cases

The quantized model is ~95% as accurate but 4x smaller:

Model	Size	Inference Time
Standard	~1.1 GB	~120ms
Quantized	~280 MB	~100ms

Skip NER for structured-only PII

If you only need emails, phones, IBANs, etc.:

import { anonymizeRegexOnly } from 'rehydra';

const result = await anonymizeRegexOnly(text);
// ~5ms instead of ~150ms

Getting Started

Core Concepts

Guides

Model Modes

Basic Setup

Download Progress

Confidence Thresholds

Case Fallback

How it works

Confidence penalty

Auto-Download Control

Manual Model Management

Inference Server Backend

Custom Models

Cache Locations

Node.js

Browser

NER-Detected Types

Disabling Specific NER Types

Performance Tips

Next Steps

Semantic Enrichment

Custom Recognizers

​Model Modes

​Basic Setup

​Download Progress

​Confidence Thresholds

​Case Fallback

​How it works

​Confidence penalty

​Auto-Download Control

​Manual Model Management

​Inference Server Backend

​Custom Models

​Cache Locations

​Node.js

​Browser

​NER-Detected Types

​Disabling Specific NER Types

​Performance Tips

​Next Steps

Semantic Enrichment

Custom Recognizers

Model Modes

Basic Setup

Download Progress

Confidence Thresholds

Case Fallback

How it works

Confidence penalty

Auto-Download Control

Manual Model Management

Inference Server Backend

Custom Models

Cache Locations

Node.js

Browser

NER-Detected Types

Disabling Specific NER Types

Performance Tips

Next Steps