Skip to main content
Stream-based anonymization for processing text chunk by chunk. Available in Node.js and Bun only.

​
Import

import { createAnonymizerStream, AnonymizerStream, SentenceBuffer } from 'rehydra/streaming';

​
createAnonymizerStream()

Factory function that creates and initializes a streaming anonymizer.

​
Signature

function createAnonymizerStream(config?: StreamConfig): Promise<AnonymizerStream>

​
Parameters

​
config
StreamConfig

​
Returns

Promise<AnonymizerStream> — a Node.js Transform stream.

​
Example

import { createAnonymizerStream } from 'rehydra/streaming';

const stream = await createAnonymizerStream({
  anonymizer: { ner: { mode: 'quantized' } },
  onChunk: (e) => console.log(`Processed: ${e.totalEntities} entities`),
  onFinish: (e) => console.log(`Done: ${e.totalEntities} total`),
});

process.stdin.pipe(stream).pipe(process.stdout);

​
AnonymizerStream

A Node.js Transform stream that anonymizes text passing through it.

​
Methods

MethodReturnsDescription
getPiiMap()EncryptedPIIMap | undefinedGet the cumulative encrypted PII map
statsobjectGet processing statistics

​
Example

const stream = await createAnonymizerStream({ ... });

inputStream.pipe(stream).pipe(outputStream);

stream.on('finish', () => {
  const piiMap = stream.getPiiMap();
  const stats = stream.stats;
  console.log(`Entities: ${stats.totalEntities}, Time: ${stats.totalProcessingTimeMs}ms`);
});

​
SentenceBuffer

Accumulates text and flushes at sentence boundaries with overlap for NER context.

​
Constructor

new SentenceBuffer(anonymizer: Anonymizer, config: ResolvedBufferConfig)

​
Methods

MethodReturnsDescription
append(chunk: string)FlushResult[]Append text, returns flushed results
flush()FlushResult | nullForce flush remaining buffer
getCumulativePiiMap()Map<string, string>Get all detected PII entries
getTotalEntities()numberTotal detected entities

​
Types

​
StreamChunkEvent

interface StreamChunkEvent {
  anonymizedText: string;
  entities: DetectedEntity[];
  totalEntities: number;
  processingTimeMs: number;
}

​
StreamFinishEvent

interface StreamFinishEvent {
  totalEntities: number;
  piiMap?: EncryptedPIIMap;
  totalProcessingTimeMs: number;
}

​
FlushResult

interface FlushResult {
  anonymizedText: string;
  entities: DetectedEntity[];
}