Serving llama.cpp as an API

[evgenyfedorchenko]

Dear Community,
Is there any possibility to serve llama.cpp on some port not like HTML form, but rather to allow sending JSON request with prompt/history and receiving the response in JSON format? (similar to ChatGPT API, for example).
Of course, i can write it manually for myself, but i wonder if there is any existing solution to not reinvent the wheel.
Appreciate your advises.

[phymbert]

Please have a look to the server

[KissPeter]

404

[CISC]

Refresh. :)