Documentation Index
Fetch the complete documentation index at: https://docs.rehydra.ai/llms.txt
Use this file to discover all available pages before exploring further.
Stream-based anonymization for processing text chunk by chunk. Available in Node.js and Bun only.
Import
import { createAnonymizerStream, AnonymizerStream, SentenceBuffer } from 'rehydra/streaming';
createAnonymizerStream()
Factory function that creates and initializes a streaming anonymizer.
Signature
function createAnonymizerStream(config?: StreamConfig): Promise<AnonymizerStream>
Parameters
Show StreamConfig properties
Anonymizer configuration (same options as createAnonymizer)
policy
Partial<AnonymizationPolicy>
Policy override for the stream
Locale hint (e.g., 'de-DE')
Buffer configuration for sentence detectionShow SentenceBufferConfig properties
Characters of overlap between buffer flushes for NER context
Force flush when buffer reaches this size
Minimum buffer size before allowing flush
Enable low-latency mode (disables NER, reduces buffer sizes)
Custom regex for sentence boundary detection
Session ID for PII map persistence
Encryption key provider (required for session persistence)
Storage provider (required for session persistence)
Interval for periodic PII map saves (milliseconds)
Node.js Transform stream options
onChunk
(event: StreamChunkEvent) => void
Callback after each chunk is anonymized
onFinish
(event: StreamFinishEvent) => void
Callback when the stream ends
Returns
Promise<AnonymizerStream> — a Node.js Transform stream.
Example
import { createAnonymizerStream } from 'rehydra/streaming';
const stream = await createAnonymizerStream({
anonymizer: { ner: { mode: 'quantized' } },
onChunk: (e) => console.log(`Processed: ${e.totalEntities} entities`),
onFinish: (e) => console.log(`Done: ${e.totalEntities} total`),
});
process.stdin.pipe(stream).pipe(process.stdout);
AnonymizerStream
A Node.js Transform stream that anonymizes text passing through it.
Methods
| Method | Returns | Description |
|---|
getPiiMap() | EncryptedPIIMap | undefined | Get the cumulative encrypted PII map |
stats | object | Get processing statistics |
Example
const stream = await createAnonymizerStream({ ... });
inputStream.pipe(stream).pipe(outputStream);
stream.on('finish', () => {
const piiMap = stream.getPiiMap();
const stats = stream.stats;
console.log(`Entities: ${stats.totalEntities}, Time: ${stats.totalProcessingTimeMs}ms`);
});
SentenceBuffer
Accumulates text and flushes at sentence boundaries with overlap for NER context.
Constructor
new SentenceBuffer(anonymizer: Anonymizer, config: ResolvedBufferConfig)
Methods
| Method | Returns | Description |
|---|
append(chunk: string) | FlushResult[] | Append text, returns flushed results |
flush() | FlushResult | null | Force flush remaining buffer |
getCumulativePiiMap() | Map<string, string> | Get all detected PII entries |
getTotalEntities() | number | Total detected entities |
Types
StreamChunkEvent
interface StreamChunkEvent {
anonymizedText: string;
entities: DetectedEntity[];
totalEntities: number;
processingTimeMs: number;
}
StreamFinishEvent
interface StreamFinishEvent {
totalEntities: number;
piiMap?: EncryptedPIIMap;
totalProcessingTimeMs: number;
}
FlushResult
interface FlushResult {
anonymizedText: string;
entities: DetectedEntity[];
}
