You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The promise returned by `append()` will reject if the prompt cannot be appended (e.g., too big, invalid modalities for the session, etc.), or will fulfill once the prompt has been validated, processed, and appended to the session.
328
+
329
+
Note that `append()` can also cause [overflow](#tokenization-context-window-length-limits-and-overflow), in which case it will evict the oldest non-system prompts from the session and fire the `"quotaoverflow"` event.
330
+
327
331
### Configuration of per-session parameters
328
332
329
333
In addition to the `systemPrompt` and `initialPrompts` options shown above, the currently-configurable model parameters are [temperature](https://huggingface.co/blog/how-to-generate#sampling) and [top-K](https://huggingface.co/blog/how-to-generate#top-k-sampling). The `params()` API gives the default and maximum values for these parameters.
@@ -429,7 +433,7 @@ The ability to manually destroy a session allows applications to free up memory
429
433
430
434
### Aborting a specific prompt
431
435
432
-
Specific calls to `prompt()`, `promptStreaming()`, or `append()` can be aborted by passing an `AbortSignal` to them:
436
+
Specific calls to `prompt()`or `promptStreaming()` can be aborted by passing an `AbortSignal` to them:
433
437
434
438
```js
435
439
constcontroller=newAbortController();
@@ -440,11 +444,20 @@ const result = await session.prompt("Write me a poem", { signal: controller.sign
440
444
441
445
Note that because sessions are stateful, and prompts can be queued, aborting a specific prompt is slightly complicated:
442
446
443
-
* If the prompt is still queued behind other prompts in the session, then it will be removed from the queue.
444
-
* If the prompt is being currently responded to by the model, then it will be aborted, and the prompt/response pair will be removed from the conversation history.
447
+
* If the prompt is still queued behind other prompts in the session, then it will be removed from the queue, and the returned promise will be rejected with an `"AbortError"``DOMException`.
448
+
* If the prompt is being currently responded to by the model, then it will be aborted, the prompt/response pair will be removed from the session, and the returned promise will be rejected with an `"AbortError"``DOMException`.
445
449
* If the prompt has already been fully responded to by the model, then attempting to abort the prompt will do nothing.
446
450
447
-
Since `append()`ed prompts are not responded to immediately, they can be aborted until a subsequent call to `prompt()` or `promptStreaming()` happens and that response has been finished.
451
+
Similarly, the `append()` operation can also be aborted. In this case the behavior is:
452
+
453
+
* If the append is queued behind other appends in the session, then it will be removed from the queue, and the returned promise will be rejected with an `"AbortError"``DOMException`.
454
+
* If the append operation is currently ongoing, then it will be aborted, any part of the prompt that was appended so far will be removed from the session, and the returned promise will be rejected with an `"AbortError"``DOMException`.
455
+
* If the append operation is complete (i.e., the returned promise has resolved), then attempting to abort it will do nothing. This includes all the following states:
456
+
* The append operation is complete, but a prompt generation step has not yet triggered.
457
+
* The append operation is complete, and a prompt generation step is processing.
458
+
* The append operation is complete, and a prompt generation step has used it to produce a result.
459
+
460
+
Finally, note that if either prompting or appending has caused an [overflow](#tokenization-context-window-length-limits-and-overflow), aborting the operation does not re-introduce the overflowed messages into the session.
448
461
449
462
### Tokenization, context window length limits, and overflow
0 commit comments