server : (webui) revamp the input area, plus many small UI improvements #13365

ngxson · 2025-05-07T21:02:15Z

Many small UI/UX improvements that I wanted to have, but now finally have time to do.

TODO:

allow upload files
allow uploading non-txt file, for example .c, .py, etc
check if server has multimodal, if not, don't allow image upload
allow renaming conversation
list conversations: group by time
autoscroll: detect size changed instead of checking on each token generated (better performance)

Overall UI:

Detail improvements

(1) Remove background color in assistant message, to make it easier to read

(2) Improve the input UI/UX

(3) Allow uploading files (plus, prepare the code to upload image - enable it once the API support image input via #12898 )

Drap-and-drop file to input area is supported

TODO in a next PR: allow ctrl+v long content, the long content will be converted to file automatically

(4) Use HeroIcons everywhere + more consistent icons

(5) Move conversation options menu to sidebar, add "rename" option

(6) Improve "thought process" display

Note: the default assistant message is removed, this is because more and more models not requiring system message

ngxson · 2025-05-07T21:06:14Z

CC @gary149 if you have some suggestions :)

ZUIcat · 2025-05-08T01:15:08Z

Could you please label the conversation with the model name, like open-web-ui does? For example, displaying the model's name on each message bubble.

ngxson · 2025-05-08T07:21:21Z

Could you please label the conversation with the model name, like open-web-ui does? For example, displaying the model's name on each message bubble.

Yes I thought about this but forgot to note it down, will do it now

tools/server/webui/src/utils/llama-vscode.ts

ngxson · 2025-05-08T09:33:13Z

common/chat.cpp

-        throw std::runtime_error("Failed to parse messages: " + std::string(e.what()) + "; messages = " + messages.dump(2));
+        // @ngxson : disable otherwise it's bloating the API response
+        // printf("%s\n", std::string("; messages = ") + messages.dump(2));
+        throw std::runtime_error("Failed to parse messages: " + std::string(e.what()));


small note @ochafik , we should never reflect user input via the error message, as it usually comes with security risks. In this case, it's not a security risk, but it's a bit inconvenient to display the error in the UI

If you want to print the input for debugging, consider using LOG_DBG

ngxson · 2025-05-08T09:34:54Z

tools/server/webui/src/components/ChatScreen.tsx

+        <b>Server Info</b>
+        <p>
+          <b>Model</b>: {serverProps?.model_path?.split(/(\\|\/)/).pop()}
+          <br />
+          <b>Build</b>: {serverProps?.build_info}
+          <br />
+        </p>


Note to myself: maybe display extra info, like server capabilities (speculative, FIM, multimodal, etc)

ngxson

Leaving some comments to make reviewing easier

ngxson · 2025-05-08T09:43:25Z

tools/server/webui/src/components/useChatExtraContext.tsx

+// WARN: vibe code below
+// This code is a heuristic to determine if a string is likely not binary.
+// It is necessary because input file can have various mime types which we don't have time to investigate.
+// For example, a python file can be text/plain, application/x-python, etc.
+export function isLikelyNotBinary(str: string): boolean {
+  const options = {
+    prefixLength: 1024 * 10, // Check the first 10KB of the string
+    suspiciousCharThresholdRatio: 0.15, // Allow up to 15% suspicious chars
+    maxAbsoluteNullBytes: 2,
+  };


Each file extension like .py, .c, .cpp, .sh, .bat, etc has their own set of mime type which is very difficult to keep track. Therefore, this function was added.

It should also work with unicode text. I tested with:

Quran

Traditional chinese

ngxson · 2025-05-08T09:45:42Z

tools/server/webui/src/utils/types.ts

+  model_path: string;
+  n_ctx: number;
+  has_multimodal: boolean;
+  // TODO: support params


In a next PR, we can use the server's default params to override the settings, this will address #13277

ngxson · 2025-05-08T09:49:20Z

tools/server/webui/src/components/useChatScroll.tsx

+export function useChatScroll(msgListRef: React.RefObject<HTMLDivElement>) {
+  useEffect(() => {
+    if (!msgListRef.current) return;
+
+    const resizeObserver = new ResizeObserver((_) => {
+      scrollToBottomThrottled(true, 10);
+    });
+
+    resizeObserver.observe(msgListRef.current);
+    return () => {
+      resizeObserver.disconnect();
+    };
+  }, [msgListRef]);


This replaces the old logic where scrollToBottom gets triggered every time a new token is received. The new approach relies on ResizeObserver, which is a browser's API to observe if a HTML element changes its size. This should now cover the case where browser window is resized

ngxson · 2025-05-08T09:51:02Z

tools/server/webui/src/utils/misc.ts

+      } else if (extra.type === 'textFile') {
+        contentArr.push({
+          type: 'text',
+          text: `File: ${extra.name}\nContent:\n\n${extra.content}`,
+        });


Here is now the input file will be formatted upon sending to server. The file name will be preserved

slaren · 2025-05-08T11:57:35Z

Looks very nice.

A bit of a nit, but I noticed that there is not enough margin at the bottom of the code boxes and it leads to situations like this:

Caused by a line separator (---) after the code:

    dfs(0, graph);
    std::cout << "\n";

    return 0;
}
```

---

### 🔍 Explanation

- **Graph Representation**: The graph is represented using an **adjacency list** as a `std::vector<std::vector<int>>`.

ngxson · 2025-05-08T12:46:28Z

Thanks for testing, this should be fixed now:

Before:

After:

slaren · 2025-05-08T12:53:31Z

That fixes it for the line separator, but I still think that the code blocks have the wrong margin. For example:

It should have even spacing on the top and bottom (from VSCode markdown preview):

Source:

```cpp
void dfs_recursive(int node, const std::vector<std::vector<int>>& graph, std::vector<bool>& visited) {
    visited[node] = true;
    std::cout << node << " ";
    for (int neighbor : graph[node]) {
        if (!visited[neighbor])
            dfs_recursive(neighbor, graph, visited);
    }
}
```

And in `main`:

```cpp
std::vector<bool> visited(graph.size(), false);
dfs_recursive(0, graph, visited);
std::cout << "\n";
```

ngxson · 2025-05-08T12:56:20Z

Ok I see, I added a margin bottom mb-3 to <pre> block, it should look better now:

slaren · 2025-05-08T12:58:50Z

Looks great now.

* origin/master: (39 commits) server : vision support via libmtmd (ggml-org#12898) sycl : implementation of reordered Q4_0 MMVQ for Intel GPUs (ggml-org#12858) metal : optimize MoE for large batches (ggml-org#13388) CUDA: FA support for Deepseek (Ampere or newer) (ggml-org#13306) llama : do not crash if there is no CPU backend (ggml-org#13395) CUDA: fix crash on large batch size for MoE models (ggml-org#13384) imatrix : Add --parse-special for enabling parsing of special tokens in imatrix calculation (ggml-org#13389) llama-run: add support for downloading models from ModelScope (ggml-org#13370) mtmd : fix batch_view for m-rope (ggml-org#13397) llama : one-off chat template fix for Mistral-Small-2503 (ggml-org#13398) rpc : add rpc_msg_set_tensor_hash_req (ggml-org#13353) vulkan: Allow up to 4096 elements for mul_mat_id row_ids (ggml-org#13326) server : (webui) rename has_multimodal --> modalities (ggml-org#13393) ci : limit write permission to only the release step + fixes (ggml-org#13392) mtmd : Expose helper_decode_image_chunk (ggml-org#13366) server : (webui) fix a very small misalignment (ggml-org#13387) server : (webui) revamp the input area, plus many small UI improvements (ggml-org#13365) convert : support rope_scaling type and rope_type (ggml-org#13349) mtmd : fix the calculation of n_tokens for smolvlm (ggml-org#13381) context : allow cache-less context for embeddings (ggml-org#13108) ...

AeneasZhu · 2025-05-15T14:56:33Z

@ngxson my vertical 3 points icon is missing from the sidebar.

AeneasZhu · 2025-05-15T15:02:02Z

@ngxson my vertical 3 points icon is missing from the sidebar.

I found the reason behind that. After I turned off the fullscreen, the icon showed. Thank you for your help.

ngxson added 3 commits May 7, 2025 19:16

rework the input area

f4af3f3

process selected file

7d59402

change all icons to heroicons

4476129

github-actions bot added examples server labels May 7, 2025

ngxson mentioned this pull request May 7, 2025

(Discussion) Improve usability of llama-server #13367

Open

ngxson added 6 commits May 8, 2025 00:24

fix thought process collapse

c8641ef

move conversation more menu to sidebar

7c87fc4

sun icon --> moon icon

eb0d66d

rm default system message

f994a30

stricter upload file check, only allow image if server has mtmd

9d076e4

build it

d813da8

ngxson added 5 commits May 8, 2025 09:38

add renaming

3ff0710

better autoscroll

2a814a7

build

993b4dd

add conversation group

47e7314

fix scroll

2d2b8de

ngxson marked this pull request as ready for review May 8, 2025 08:48

ngxson requested review from ggerganov and slaren May 8, 2025 08:48

ggerganov reviewed May 8, 2025

View reviewed changes

tools/server/webui/src/utils/llama-vscode.ts Show resolved Hide resolved

ngxson commented May 8, 2025

View reviewed changes

extra context first, then user input in the end

9a24d3f

ggerganov approved these changes May 8, 2025

View reviewed changes

slaren approved these changes May 8, 2025

View reviewed changes

ngxson added 3 commits May 8, 2025 14:44

fix <hr> tag

b0be033

clean up a bit

ae5a807

build

2a3cd10

add mb-3 for <pre>

a64f881

ngxson added 3 commits May 8, 2025 15:01

throttle adjustTextareaHeight to make it less laggy

163bdca

(nits) missing padding in sidebar

3da9380

rm stray console log

e7e28bd

ngxson merged commit 8c83449 into ggml-org:master May 8, 2025
44 checks passed

ngxson mentioned this pull request May 9, 2025

server : add support for file upload to the Web UI #11611

Closed

4 tasks

ZUIcat mentioned this pull request May 10, 2025

Misc. bug: The web UI of llama-server is not displaying correctly. #13428

Open

ngxson mentioned this pull request May 15, 2025

Misc. bug: Web UI's download and delete chat button missing in b5392 #13564

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

server : (webui) revamp the input area, plus many small UI improvements #13365

server : (webui) revamp the input area, plus many small UI improvements #13365

ngxson commented May 7, 2025 •

edited

Loading

ngxson commented May 7, 2025

ZUIcat commented May 8, 2025

ngxson commented May 8, 2025

ngxson May 8, 2025 •

edited

Loading

ngxson May 8, 2025

ngxson left a comment

ngxson May 8, 2025

ngxson May 8, 2025

ngxson May 8, 2025

ngxson May 8, 2025

slaren commented May 8, 2025

ngxson commented May 8, 2025

slaren commented May 8, 2025

ngxson commented May 8, 2025

slaren commented May 8, 2025

AeneasZhu commented May 15, 2025

AeneasZhu commented May 15, 2025

server : (webui) revamp the input area, plus many small UI improvements #13365

server : (webui) revamp the input area, plus many small UI improvements #13365

Conversation

ngxson commented May 7, 2025 • edited Loading

Detail improvements

ngxson commented May 7, 2025

ZUIcat commented May 8, 2025

ngxson commented May 8, 2025

ngxson May 8, 2025 • edited Loading

Choose a reason for hiding this comment

ngxson May 8, 2025

Choose a reason for hiding this comment

ngxson left a comment

Choose a reason for hiding this comment

ngxson May 8, 2025

Choose a reason for hiding this comment

ngxson May 8, 2025

Choose a reason for hiding this comment

ngxson May 8, 2025

Choose a reason for hiding this comment

ngxson May 8, 2025

Choose a reason for hiding this comment

slaren commented May 8, 2025

ngxson commented May 8, 2025

slaren commented May 8, 2025

ngxson commented May 8, 2025

slaren commented May 8, 2025

AeneasZhu commented May 15, 2025

AeneasZhu commented May 15, 2025

ngxson commented May 7, 2025 •

edited

Loading

ngxson May 8, 2025 •

edited

Loading