AI · 4 min read

How to Fix OpenAI Structured Output Malformed JSON Errors

openai structured output json schema langchain pydantic

The malformed JSON you're seeing from gpt-5-mini structured output almost always comes from one of three causes: using response_format: { type: "json_object" } instead of the stricter json_schema mode, having unsupported schema features like optional fields or additionalProperties defaulting to true, or using the legacy Chat Completions endpoint when the Responses API handles structured output more reliably. The fix is to use response_format: { type: "json_schema" } with a fully explicit schema where every field is required, additionalProperties is false, and no unsupported keywords appear.

Request Structured Output with json_schema Mode, Not json_object

Use json_schema mode, not json_object mode. The json_object mode guarantees only valid JSON syntax, not conformance to any schema. That's why you get unexpected keys, missing fields, or wrong types.

from openai import OpenAI

client = OpenAI()

response = client.chat.completions.create(
    model="gpt-4o-mini",
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "extraction",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "summary": {"type": "string"},
                    "sentiment": {"enum": ["positive", "negative", "neutral"]},
                    "confidence": {"type": "number"}
                },
                "required": ["summary", "sentiment", "confidence"],
                "additionalProperties": False
            }
        }
    },
    messages=[{"role": "user", "content": "Analyze: 'The product is great but shipping was slow'"}]
)

Why Strict Mode and additionalProperties Matter

When strict is true, OpenAI uses constrained decoding. The model literally can't produce tokens that violate your schema. Without it, you're relying on the model's best effort, which is where malformed output creeps in. The API enforces the additionalProperties: false requirement when strict mode is on; omitting it triggers a 400 error. Every property in your schema must also appear in the required array. If you need optional fields, mark them as required but allow null through a union type.

# Handling "optional" fields in strict mode.
# Use a union type with null instead of omitting from required.
schema = {
    "type": "object",
    "properties": {
        "name": {"type": "string"},
        "nickname": {
            "type": ["string", "null"]
        }
    },
    "required": ["name", "nickname"],
    "additionalProperties": False
}

Using the OpenAI Responses API for Structured Output

OpenAI's newer Responses API has first-class structured output support through text.format. If you're starting a new project, prefer this over Chat Completions. The structured output behavior is identical under the hood, but the API surface is cleaner and supports built-in tool use alongside structured output without conflicts.

from openai import OpenAI

client = OpenAI()

response = client.responses.create(
    model="gpt-4o-mini",
    input="Analyze: 'The product is great but shipping was slow'",
    text={
        "format": {
            "type": "json_schema",
            "name": "extraction",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "summary": {"type": "string"},
                    "sentiment": {"enum": ["positive", "negative", "neutral"]},
                    "confidence": {"type": "number"}
                },
                "required": ["summary", "sentiment", "confidence"],
                "additionalProperties": False
            }
        }
    }
)

LangChain with_structured_output Common Pitfalls

LangChain's with_structured_output works well but has a subtle default that causes problems: it sends schemas without strict: true unless you pass method="json_schema" and strict=True explicitly. Without these, LangChain might fall back to function-calling-based extraction, which doesn't use constrained decoding and produces the malformed JSON you're debugging.

There's another catch when you pass a Pydantic model. LangChain generates the JSON Schema for you, but older Pydantic v1 models can produce schemas with definitions instead of $defs, which the API rejects.

from langchain_openai import ChatOpenAI
from pydantic import BaseModel

class Extraction(BaseModel):
    summary: str
    sentiment: str
    confidence: float

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

# Pass strict=True and method="json_schema" explicitly.
structured_llm = llm.with_structured_output(
    Extraction,
    method="json_schema",
    strict=True
)

result = structured_llm.invoke(
    "Analyze: 'The product is great but shipping was slow'"
)
print(result.summary, result.sentiment, result.confidence)

Handle Refusals and Truncated Structured Output

Even with strict mode, you can still get incomplete JSON in one scenario: when the response hits max_tokens and gets truncated. The finish_reason is "length" instead of "stop", and the partial JSON won't parse. Always check finish_reason before parsing.

There's a second failure mode. If the model refuses a request for safety reasons, the message.refusal field is populated and message.content is null. Calling json.loads(None) is another common source of crashes.

import json

choice = response.choices[0]

# Check for refusal before parsing.
if choice.message.refusal:
    raise ValueError(f"Model refused: {choice.message.refusal}")

# Check for truncation before parsing.
if choice.finish_reason != "stop":
    raise ValueError(f"Incomplete response: finish_reason={choice.finish_reason}")

parsed = json.loads(choice.message.content)
print(parsed)

JSON Schema Restrictions That Silently Break Things

OpenAI's strict mode supports a subset of JSON Schema. Unsupported features either throw a 400 error or get ignored silently, producing output that doesn't match your expectations.

Constraints like minLength, maxLength, pattern, minimum, and maximum are all ignored in strict mode. The model might still follow them as soft guidance, but constrained decoding won't enforce them. Recursive schemas work but only up to a limited depth; deeply nested recursive types can hit the API's schema complexity limit. oneOf and anyOf work only for discriminated unions, and overlapping schemas in anyOf cause a validation error.

If you need regex-level validation, do it client-side after parsing. Constrained decoding guarantees structural correctness, not semantic correctness.

from pydantic import BaseModel, field_validator

class Extraction(BaseModel):
    summary: str
    sentiment: str
    confidence: float

    # Enforce constraints client-side after parsing.
    @field_validator("confidence")
    @classmethod
    def validate_confidence(cls, v: float) -> float:
        if not 0.0 <= v <= 1.0:
            raise ValueError(f"confidence must be 0-1, got {v}")
        return v

    @field_validator("sentiment")
    @classmethod
    def validate_sentiment(cls, v: str) -> str:
        allowed = {"positive", "negative", "neutral"}
        if v not in allowed:
            raise ValueError(f"sentiment must be one of {allowed}")
        return v

Which Structured Output Approach to Use

The Pydantic + SDK approach (the client.beta.chat.completions.parse method) auto-generates the schema and auto-parses the response into a typed object. This is the least error-prone path. The raw json_schema approach gives you fine-grained control over the schema and works well in non-Python environments. LangChain's with_structured_output fits best when you're already in a LangChain pipeline and need model-agnostic code, but you should always pass strict=True and method="json_schema" explicitly.

In all three cases, always validate finish_reason, check for refusals, and add client-side validation for constraints that JSON Schema strict mode doesn't enforce.

← Back to all articles