Picture description API

[dalton5]

Hi,

I try to use picture description api but t's not working.

I tried a lot of different things to call the api in javascript with picture_description_api. Nothing works. Can you confirm it works well or show an example?

If I use the parameter like the usage help document is showing no parameters are taken in account.

const parameters = {
    to_formats: ['json'],
    table_mode: 'fast',
    do_ocr: true,
    return_as_file: false,
    picture_description_api: {
      url: "https://openrouter.ai/api/v1",
      headers: {
        Authorization: "Bearer sk-or-v1-xxxx",
        "Content-Type": "application/json"
      },
      params: {
        model: "google/gemma-3-4b-it"
      },
      timeout: 20,
      prompt: "describe the image with data details, numbers and values"
    }
  };
   formData.append('parameters', JSON.stringify(parameters));

And with a call with

formData.append("from_formats", "pdf");
formData.append("to_formats", "json");
formData.append("table_mode", "fast");
formData.append("do_ocr", "true");
formData.append("return_as_file", "false");


formData.append("picture_description_api", JSON.stringify({
  url: "https://openrouter.ai/api/v1",
  headers: {
    Authorization: "Bearer sk-or-v1-xxxx",
    "Content-Type": "application/json"
  },
  params: {
    model: "google/gemma-3-4b-it"
  },
  timeout: 20,
  prompt: "describe the image with data details, numbers and values"
}));

It gives an error: 422 unprocessable entity

What is the good syntax?

If I do not use picture_description_api it works well.

Thanks,

[AlexisGitHu]

I think this is not a syntax error rather how HTTP petitions work.

If one file or more is being sent in the request, then the Content-Type is multipart/form-data which is incompatible with BODY parameter as the content type for those would be "application/json".

As it makes sense to keep the dictionary format for the picture_description_api parameter, the solution would be to change the Pydantic Schema to a simple str and then encode back to json once it has reached the API.

Sending the str with the actual code won't just work because the definition is incorrect due to schema enforcement when this parameter is sent.

[ReddaHawk]

I am encountering the same problem while utilizing Python and curl command. What is the solution?

[vaclcer]

Right. I really tried to look into the docs, but did not figure out how to use picture description feature via docling-serve REST API. Could you please improve the docs to accomodate for that? Do I need to pre-download some VLLM? What parameters in payload do I need to setup?