Fixing JSON errors in OpenAI functions using more OpenAI Functions

Yes, you read that right.

Sept. 21, 2023

OpenAI functions are a really powerful add-on to its ChatGPT-style models. Basically, along with your normal chat message, you give OpenAI a function spec, and OpenAI—instead of returning a just a message—will return JSON that adheres to that function spec.

Usually.

Except, because it's an LLM, and it's still generating its result probabilistically, sometimes it...doesn't. Sure—it tries to return JSON, but sometimes it will return something like this:

{
  "answer": "The word "dassie" is another name for a rock hyrax, a small rodent common in Southern Africa.",
  "sources": ["https://en.wikipedia.org/wiki/Rock_hyrax"]
}

An example OpenAI function response with invalid JSON. In this case, there is an escaping problem with the quotes around the word "dassie" in our "answer" parameter.

And while at first glance this looks like vaild JSON, it is not, because the quotation marks around "dassie" are not escaped. This causes the JSON parser to choke as soon as it encounters the text after the first quote.

So how do we deal with this? As a human, it's obvious that we need to escape the quotation marks, but how do we get a computer to do it? This answer can quickly devolve into complex regular expressions and hard-coding our expectations about the ways in which the answer needs to be fixed. And, as we all know, parsing things with regular expressions ca̶̶̶n le̸͖̾ad̴̩̄ tȍ̴̰̑̓ ę̸̣͓̀́r̶̯̻͕̒͛ͅŗ̸̬͙̻̪̔̑̒͋ǫ̶̣̫̱͋͂̀͘͝r̵̦͘s̸̺̖̜̰̲̋.

Hmm...so we have a general problem that is relatively easy for humans to understand, but hard to explain explicitly to a computer. You know what tool is good at solving problems like that? LLMs! And a wacky solution turns out to be that we can fix the mistakes made by OpenAI functions using... you guessed it... OpenAI functions!

Here's a function declaration you can use to tell it to clean invalid JSON.

clean_json_fn = {
    "name": "clean_json",
    "description": "Cleans JSON.",
    "parameters": {
        "type": "object",
        "properties": {
            "clean_json": {
                "type": "string",
                "description": "The cleaned JSON.",
            },
        },
        "required": ["clean_json"],
    },
}

You can then pass this function, along with the invalid JSON, to OpenAI like this:

bad_json = """
{
  "answer": "The word "dassie" is another name for a rock hyrax, a small rodent common in Southern Africa.",
  "sources": ["https://en.wikipedia.org/wiki/Rock_hyrax"]
}
"""
messages = [
    {
        "role": "system",
        "content": "You are a helpful assistant.",
    },
    {
        "role": "user",
        "content": f"The bad JSON: {bad_json}",
    },
]
openai_response = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=messages,
    functions=[clean_json_fn],
    function_call={"name": clean_json_fn["name"]},
)

And it works!

You just have to take care to load the returned JSON twice (once for the original function arguments, and then a second time to load it from the newly cleaned value).

# the arguments come back as text, so those get converted to JSON
arguments_raw = openai_response.choices[0].message.function_call.arguments
arguments = json.loads(arguments_raw)
# Then the cleaned json is also text, which needs to be loaded
cleaned_json_raw = arguments["clean_json"]
cleaned_json = json.loads(cleaned_json_raw)
print(cleaned_json["answer"])

This finally correctly outputs the correct value:

The word "dassie" is another name for a rock hyrax, a small rodent common in Southern Africa.

AI is crazy!

Here is the complete source code of a function you can use to clean JSON with OpenAI.

Interested in learning about building with LLMs?