Fix with_structured_output Malformed JSON in LangChain OpenAI
The most common cause of malformed JSON from with_structured_output is using method="json_mode" instead of the default method="function_calling". Function calling delegates JSON generation to OpenAI's constrained decoding, which guarantees valid JSON that matches your schema. JSON mode only guarantees syntactically valid JSON with no schema enforcement, so the model frequently omits fields, invents extra keys, or returns nested structures that don't match your Pydantic model. Switch to function calling (the default) and ensure that your Pydantic model has explicit field descriptions.
from langchain_openai import ChatOpenAI
from pydantic import BaseModel, Field
class MovieReview(BaseModel):
title: str = Field(description="The movie title exactly as given.")
sentiment: str = Field(description="One of: positive, negative, neutral.")
score: int = Field(description="Rating from 1 to 10.")
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
# Use the default method (function_calling), not json_mode.
structured_llm = llm.with_structured_output(MovieReview)
result = structured_llm.invoke("Review: Inception was mind-blowing, 9/10.")
print(result)Why json_mode Breaks and function_calling Does Not
When you pass method="json_mode", LangChain sets response_format={"type": "json_object"} on the API request. OpenAI guarantees that this returns parseable JSON, but it doesn't enforce any particular schema. The model sees your Pydantic schema only through a system prompt that LangChain injects, and newer models sometimes ignore or reinterpret those instructions. The result is valid JSON that fails Pydantic validation: missing required fields, wrong types, or extra whitespace in string values.
Function calling, by contrast, passes your schema as a tool definition. OpenAI's API uses constrained decoding to force the output to conform to the JSON Schema derived from your Pydantic model. Fields, types, and enum values are enforced at the token generation level. This is why function calling is the default and recommended method.
# BAD: json_mode does not enforce your schema.
bad_llm = llm.with_structured_output(MovieReview, method="json_mode")
# GOOD: function_calling (default) enforces the schema.
good_llm = llm.with_structured_output(MovieReview)Common Pitfall: Missing Field Descriptions
Even with function calling, vague or missing field descriptions cause the model to guess. OpenAI's tool-calling implementation sends your JSON Schema to the model, and the description field in that schema is the primary way the model learns what each field should contain. If you use bare type annotations without Field(description=...), the model fills fields based on the field name alone. This leads to unexpected values, especially for ambiguous names like type, label, or status.
from typing import Literal
from pydantic import BaseModel, Field
class ExtractedEntity(BaseModel):
"""A named entity extracted from the input text."""
name: str = Field(description="The entity name as it appears in the text.")
# Use Literal to constrain values the model can return.
entity_type: Literal["person", "organization", "location"] = Field(
description="The category of the named entity."
)
confidence: float = Field(
description="Confidence score between 0.0 and 1.0.",
ge=0.0,
le=1.0,
)Handling Validation Errors Gracefully
In production, you should never trust that structured output will always parse correctly. Network issues, model updates, and edge-case inputs can all cause failures. LangChain provides with_fallbacks, and you can also wrap the call with include_raw=True to get the raw API response alongside the parsed object. This is invaluable for debugging.
structured_llm = llm.with_structured_output(
MovieReview,
# Return raw response alongside parsed output for debugging.
include_raw=True,
)
response = structured_llm.invoke("Review: Terrible movie, 2/10.")
# Access the parsed Pydantic object.
parsed = response["parsed"]
# Access the raw AIMessage if parsing failed.
raw_message = response["raw"]
# Check for parsing errors.
parsing_error = response["parsing_error"]
if parsing_error:
print(f"Parse failed: {parsing_error}")
print(f"Raw output: {raw_message.content}")Using strict=True for Guaranteed Schema Compliance
OpenAI's API supports a strict parameter on function definitions that enables full JSON Schema validation at the API level. LangChain exposes this through the strict argument on with_structured_output. When enabled, the API rejects any response that doesn't conform exactly to your schema before returning it to you. The tradeoff is that strict mode doesn't support all JSON Schema features: it requires additionalProperties: false on all objects and doesn't allow certain constructs like oneOf. For most Pydantic models, this works out of the box.
# Enable strict mode for API-level schema enforcement.
strict_llm = llm.with_structured_output(MovieReview, strict=True)
# This will never return malformed JSON relative to MovieReview.
result = strict_llm.invoke("Review: Decent film, solid 7.")
print(result.title, result.sentiment, result.score)When You're Stuck With json_mode
Some workflows require json_mode because you need freeform JSON that doesn't map to a fixed tool schema, or you're using a model that doesn't support function calling. In that case, add explicit retry logic with Pydantic validation. Parse the raw output yourself and retry on failure. This approach is strictly worse than function calling for fixed schemas, but it's the best option when you're constrained.
from langchain_core.output_parsers import PydanticOutputParser
from langchain_core.prompts import ChatPromptTemplate
parser = PydanticOutputParser(pydantic_object=MovieReview)
prompt = ChatPromptTemplate.from_messages([
("system", "Return JSON matching this schema:\n{format_instructions}"),
("human", "{input}"),
])
# Inject format instructions into the prompt.
chain = prompt.partial(
format_instructions=parser.get_format_instructions()
) | llm.bind(response_format={"type": "json_object"}) | parser
result = chain.invoke({"input": "Review: Amazing, 10/10."})
print(result)Debugging Checklist
If you're still getting malformed output after switching to function calling with strict=True, work through these issues in order.
First, check your langchain-openai version by running pip install --upgrade langchain-openai. Versions before 0.1.19 had bugs in schema serialization that produced invalid tool definitions.
Second, make sure that every field in your Pydantic model has a type annotation. Bare assignments like name = "default" without a type hint are silently dropped from the generated schema.
Third, avoid deeply nested Optional unions. OpenAI's strict mode requires all object properties to appear in required, so Optional[str] fields must have an explicit default of None.
Finally, check that you aren't exceeding the model's output token limit. If the response is truncated mid-JSON, parsing always fails. Set max_tokens high enough to accommodate your expected output size.