Class WordStream`Abstract`

Hierarchy (view full)

WordStream
- BufferedWordStream

Index

Constructors

Accessors

Methods

[asyncIterator] close endInput flush next pushText

Constructors

constructor

new WordStream(): WordStream
Returns WordStream

Accessors

closed

get closed(): boolean
Returns boolean
- Defined in tokenize/tokenizer.ts:101

Methods

[asyncIterator]

[asyncIterator](): WordStream
Returns WordStream
- Defined in tokenize/tokenizer.ts:149

close

close(): void
Close both the input and output of the tokenizer stream

Returns void
- Defined in tokenize/tokenizer.ts:143

endInput

endInput(): void
Mark the input as ended and forbid additional pushes

Returns void
- Defined in tokenize/tokenizer.ts:128

flush

flush(): void
Flush the tokenizer, causing it to process all pending text

Returns void
- Defined in tokenize/tokenizer.ts:117

next

next(): Promise<IteratorResult<TokenData, any>>
Returns Promise<IteratorResult<TokenData, any>>
- Defined in tokenize/tokenizer.ts:138

pushText

pushText(text): void
Push a string of text to the tokenizer
Parameters
- text: string
Returns void
- Defined in tokenize/tokenizer.ts:106