Tool call errors with Expected 'content' to be a string or an array

[anudit]

Name and Version

version: 5350 (c104023)
built with Apple clang version 17.0.0 (clang-1700.0.13.3) for arm64-apple-darwin24.4.0

Operating systems

Mac

GGML backends

Metal

Hardware

M2 Max

Models

Tested with gemma-3-4b-it-qat and internvl3-8b-instruct

Problem description & steps to reproduce

Tool call errors with Expected 'content' to be a string or an array.

Mainly because llama.cpp server responds with "content": null, in the assistant tool call response, which when sent back to the server with tool call response errors.
If I change the "content": null, to "content": "", it works.

Repro:

import { generateText } from '@xsai/generate-text'
import { tool } from '@xsai/tool'
import * as v from 'valibot'

async function main() {

  const weather = await tool({
    description: 'Get the weather in a location',
    execute: ({ location }) => JSON.stringify({
      location,
      temperature: 42,
    }),
    name: 'weather',
    parameters: v.object({
      location: v.pipe(
        v.string(),
        v.description('The location to get the weather for'),
      ),
    }),
  })

  const { text } = await generateText({
    apiKey: "xx",
    baseURL: 'http://localhost:9001/v1/',
    maxSteps: 10,
    messages: [
      {
        content: 'You are a helpful assistant.',
        role: 'system',
      },
      {
        content: 'What is the weather in San Francisco?',
        role: 'user',
      },
    ],
    model: 'xx',
    tools: [weather],
  })

  console.log({text})
}

main()

Final Req (this errors),

{
  "messages": [
    {
      "content": "You are a helpful assistant.",
      "role": "system"
    },
    {
      "content": "What is the weather in San Francisco?",
      "role": "user"
    },
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        {
          "type": "function",
          "function": {
            "name": "weather",
            "arguments": "{\"location\":\"San Francisco\"}"
          },
          "id": "IbcHFjCPI7tAwLeMoOlBhn9d5ZEqWY6W"
        }
      ]
    },
    {
      "content": "{\"location\":\"San Francisco\",\"temperature\":42}",
      "role": "tool",
      "tool_call_id": "IbcHFjCPI7tAwLeMoOlBhn9d5ZEqWY6W"
    }
  ],
  "model": "xx",
  "tools": [
    {
      "function": {
        "description": "Get the weather in a location",
        "name": "weather",
        "parameters": {
          "$schema": "http://json-schema.org/draft-07/schema#",
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The location to get the weather for"
            }
          },
          "required": [
            "location"
          ],
          "additionalProperties": false
        },
        "strict": true
      },
      "type": "function"
    }
  ],
  "stream": false
}

[CISC]

The offending check is here:

llama.cpp/tools/server/utils.hpp

Lines 647 to 653 in 33eff40

	if (content.is_string()) {
	continue;
	}
	if (!content.is_array()) {
	throw std::runtime_error("Expected 'content' to be a string or an array");
	}

Since null content is valid it should be enough to add content.is_null() to the first check, but that might cause things to break elsewhere, can you test/add a PR?

[anudit]

@CISC Made a PR, works for me locally in our projects and tests.