Skip to content

Commit fd490fe

Browse files
committed
Remove append() aborting and add more detail on append
See discussions in #92 (comment) onward.
1 parent 4159013 commit fd490fe

File tree

1 file changed

+9
-12
lines changed

1 file changed

+9
-12
lines changed

README.md

Lines changed: 9 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -324,6 +324,10 @@ analyzeButton.onclick = async (e) => {
324324
};
325325
```
326326

327+
The promise returned by `append()` will reject if the prompt cannot be appended (e.g., too big, invalid modalities for the session, etc.), or will fulfill once the prompt has been validated, processed, and appended to the session.
328+
329+
Note that `append()` can also cause [overflow](#tokenization-context-window-length-limits-and-overflow), in which case it will evict the oldest non-system prompts from the session and fire the `"quotaoverflow"` event.
330+
327331
### Configuration of per-session parameters
328332

329333
In addition to the `systemPrompt` and `initialPrompts` options shown above, the currently-configurable model parameters are [temperature](https://huggingface.co/blog/how-to-generate#sampling) and [top-K](https://huggingface.co/blog/how-to-generate#top-k-sampling). The `params()` API gives the default and maximum values for these parameters.
@@ -429,7 +433,7 @@ The ability to manually destroy a session allows applications to free up memory
429433

430434
### Aborting a specific prompt
431435

432-
Specific calls to `prompt()`, `promptStreaming()`, or `append()` can be aborted by passing an `AbortSignal` to them:
436+
Specific calls to `prompt()` or `promptStreaming()` can be aborted by passing an `AbortSignal` to them:
433437

434438
```js
435439
const controller = new AbortController();
@@ -440,11 +444,11 @@ const result = await session.prompt("Write me a poem", { signal: controller.sign
440444

441445
Note that because sessions are stateful, and prompts can be queued, aborting a specific prompt is slightly complicated:
442446

443-
* If the prompt is still queued behind other prompts in the session, then it will be removed from the queue.
444-
* If the prompt is being currently responded to by the model, then it will be aborted, and the prompt/response pair will be removed from the conversation history.
447+
* If the prompt is still queued behind other prompts in the session, then it will be removed from the queue, and the returned promise will be rejected with an `"AbortError"` `DOMException`.
448+
* If the prompt is being currently responded to by the model, then it will be aborted, the prompt/response pair will be removed from the conversation history, and the returned promise will be rejected with an `"AbortError"` `DOMException`.
445449
* If the prompt has already been fully responded to by the model, then attempting to abort the prompt will do nothing.
446450

447-
Since `append()`ed prompts are not responded to immediately, they can be aborted until a subsequent call to `prompt()` or `promptStreaming()` happens and that response has been finished.
451+
(Note: `append()` calls cannot currently be aborted. We could consider adding that in the future, if people have strong use cases. But see [this discussion](https://github.com/webmachinelearning/prompt-api/issues/92#issuecomment-2811773119) for all the complexities involved in designing such a system.)
448452

449453
### Tokenization, context window length limits, and overflow
450454

@@ -616,10 +620,7 @@ interface LanguageModel : EventTarget {
616620
LanguageModelPrompt input,
617621
optional LanguageModelPromptOptions options = {}
618622
);
619-
Promise<undefined> append(
620-
LanguageModelPrompt input,
621-
optional LanguageModelAppendOptions options = {}
622-
);
623+
Promise<undefined> append(LanguageModelPrompt input);
623624
624625
Promise<double> measureInputUsage(
625626
LanguageModelPrompt input,
@@ -671,10 +672,6 @@ dictionary LanguageModelPromptOptions {
671672
AbortSignal signal;
672673
};
673674
674-
dictionary LanguageModelAppendOptions {
675-
AbortSignal signal;
676-
};
677-
678675
dictionary LanguageModelCloneOptions {
679676
AbortSignal signal;
680677
};

0 commit comments

Comments
 (0)