You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The promise returned by `append()` will reject if the prompt cannot be appended (e.g., too big, invalid modalities for the session, etc.), or will fulfill once the prompt has been validated, processed, and appended to the session.
328
+
329
+
Note that `append()` can also cause [overflow](#tokenization-context-window-length-limits-and-overflow), in which case it will evict the oldest non-system prompts from the session and fire the `"quotaoverflow"` event.
330
+
327
331
### Configuration of per-session parameters
328
332
329
333
In addition to the `systemPrompt` and `initialPrompts` options shown above, the currently-configurable model parameters are [temperature](https://huggingface.co/blog/how-to-generate#sampling) and [top-K](https://huggingface.co/blog/how-to-generate#top-k-sampling). The `params()` API gives the default and maximum values for these parameters.
@@ -429,7 +433,7 @@ The ability to manually destroy a session allows applications to free up memory
429
433
430
434
### Aborting a specific prompt
431
435
432
-
Specific calls to `prompt()`, `promptStreaming()`, or `append()` can be aborted by passing an `AbortSignal` to them:
436
+
Specific calls to `prompt()`or `promptStreaming()` can be aborted by passing an `AbortSignal` to them:
433
437
434
438
```js
435
439
constcontroller=newAbortController();
@@ -440,11 +444,11 @@ const result = await session.prompt("Write me a poem", { signal: controller.sign
440
444
441
445
Note that because sessions are stateful, and prompts can be queued, aborting a specific prompt is slightly complicated:
442
446
443
-
* If the prompt is still queued behind other prompts in the session, then it will be removed from the queue.
444
-
* If the prompt is being currently responded to by the model, then it will be aborted, and the prompt/response pair will be removed from the conversation history.
447
+
* If the prompt is still queued behind other prompts in the session, then it will be removed from the queue, and the returned promise will be rejected with an `"AbortError"``DOMException`.
448
+
* If the prompt is being currently responded to by the model, then it will be aborted, the prompt/response pair will be removed from the conversation history, and the returned promise will be rejected with an `"AbortError"``DOMException`.
445
449
* If the prompt has already been fully responded to by the model, then attempting to abort the prompt will do nothing.
446
450
447
-
Since`append()`ed prompts are not responded to immediately, they can be aborted until a subsequent call to `prompt()` or `promptStreaming()` happens and that response has been finished.
451
+
(Note:`append()` calls cannot currently be aborted. We could consider adding that in the future, if people have strong use cases. But see [this discussion](https://github.com/webmachinelearning/prompt-api/issues/92#issuecomment-2811773119) for all the complexities involved in designing such a system.)
448
452
449
453
### Tokenization, context window length limits, and overflow
0 commit comments