Skip to content

Conversation

@jeffersoncgo
Copy link

No description provided.

@DannyDaemonic
Copy link
Contributor

DannyDaemonic commented Apr 20, 2023

This adds both [end of text] and [model ready] tags, so maybe another name would be better?

[end of text] will be displayed with --forceendtoken in interactive mode but [model ready] will now be displayed in interactive mode even without --forceendtoken. [model ready] is new, so it should probably only show up when using the --forceendtoken option, although, again, another name might be better.

Also, if you're looking for a "start" token, the initialization token for LLaMA models is the first space - that's my understanding of it anyway. Might want to turn that first space that's always generated into the [start of text] instead of just printing [model ready] which isn't significant.

Another thing to consider is using a switch with multiple options, like --show-signal-tokens=auto,always,never. auto could be the way it's current done, and always and never show or hide the special tokens. With proper control over them, there might even be an argument to be made that they should go to stdout instead of stderr.

I don't know how I feel about this commit as it currently is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants