Structured output with XML Schema

[tomayac]

JSON isn't very streaming-friendly, although there're some attempts at making streaming work with JSON. In contrast, XML is very streaming-friendly and structured (as hell), so maybe it'd make sense to allow structured output with an XML Schema?

[domenic]

Can you explain what makes XML streaming-friendly? It seems to suffer from the same problem as JSON: any unclosed tag makes the whole document unparseable.

[tomayac]

Let's say you have the model generate a list of people with names and birth dates following an XML Schema. Here's an example document to illustrate the idea:

<people>
  <person>
    <name>Jane</name>
    <age born="1990-02-24">25</age>
  </person>
  <person>
    <name>Helen</name>
    <age born="1985-01-01">30</age>
  </person>
  <person>
    <name>Liza</name>
    <age born="1980-01-01">30</age>
  </person>
</people>

You could then work with a streaming parser like saxophone or sax and whenever you see a <person>…</person> tag, already work with this intermediate result.

[domenic]

Sure, but you could also use a streaming JSON parser, if you're not planning to wait for valid XML / valid JSON. I remain confused what's different about XML and JSON in this regard.

[tomayac]

There's no difference functionality-wise between JSON and XML from the streaming perspective. It's simply an alternative, more traditional approach with well-established tooling, both around streaming parsers and schema support. Since #91 raised potential RegExp support, there may be developer demand for XML, maybe for enterprise use cases. With JSON support, you could of course try to convert to XML on the client, as long as you don't rely on features of XML Schema that JSON Schema doesn't support.

[domenic]

more traditional approach with well-established tooling, both around streaming parsers and schema support.

I don't find that to be the case. Do you have any sources for these statements?

there may be developer demand for XML, maybe for enterprise use cases.

Could you point to some of that developer demand?

[tomayac]

Do you have any sources for these statements?

Not sure what you're asking for. SAX (not on https:) is around since the early 2000s, as is XML Schema, both with significant enterprise adoption [citation needed?].

Could you point to some of that developer demand?

I haven't heard of concrete developer demand for the Prompt API, but there's evidence of this being a thing for cloud AI for both prompts and LLM output, for example, Mastering Structured Output in LLMs 3: LangChain and XML or Using XML Schema in AI System Prompts.

[domenic]

Not sure what you're asking for.

I'm asking for a source that streaming XML is more prevalent than streaming JSON. Just from the fact that JSON is so much more prevalent than XML, I'd suspect that's not the case.

[tomayac]

I'm asking for a source that streaming XML is more prevalent than streaming JSON. Just from the fact that JSON is so much more prevalent than XML, I'd suspect that's not the case.

Oh, I see. I wasn't saying that one is more prevalent than the other, simply that the XML landscape is more traditional with well-established tooling. I guess all I'm saying is that we spec LanguageModelPromptOptions in such a way that, should developer demand arise for XML Schema support in the Prompt API, we can easily add object responseXMLSchema to it, which, given the current design, is very much the case, so I guess it's fine to just close this Issue for now.

[tomayac]

Reopening as per a discussion (Google-only link, sorry) with @clarkduvall to focus on possibly being able to extend responseConstraint with a type.

[domenic]

This isn't really a good way to track issues. Please file a new issue with public information, focused on this new proposal.