How json_repair Works
json_repair is a lightweight Python library designed to correct malformed JSON strings generated by large language models, APIs, logs, or user input. It operates by first attempting to parse the input using Python’s built-in json.loads() function. If parsing fails, the library applies a series of repairs to address common syntax errors, such as missing quotes, trailing commas, unbalanced brackets, or stray prose. This two-step process ensures that valid JSON is preserved while broken JSON is repaired with minimal structural changes.
The library supports schema-guided repairs in beta, allowing users to enforce JSON Schema or Pydantic v2 models to fill missing values, coerce data types (e.g., converting strings to integers), and drop disallowed properties. This feature is particularly useful for pipelines requiring strict validation while accommodating imperfect input.
Users can bypass this initial check with skipjsonloads=True if they are certain the input is invalid, though this may alter valid JSON unintentionally. The library also includes a strict mode (strict=True) that raises errors on unrecoverable structural issues, such as duplicate keys or missing separators, providing clearer feedback for debugging.
Non-Latin characters are handled with the ensureascii=False parameter to preserve original encoding, while performance optimizations like returnobjects=True skip unnecessary serialization steps. The library intentionally avoids third-party JSON parsers to maintain predictable behavior and reduce overhead on valid inputs.
Use Cases
json_repair is primarily used in environments where JSON output is unreliable, such as pipelines processing LLM-generated responses, webhook payloads, or log data. Common scenarios include:
- LLM Output Handling: Repairing JSON strings generated by language models that may include syntax errors, hallucinated text, or incorrect boolean/null values. The library removes extraneous prose and restores structural integrity without losing meaningful data.
- API and Webhook Resilience: Tolerating malformed JSON in real-time systems where strict validation would otherwise cause failures. For example, it can correct trailing commas in arrays or incomplete key-value pairs before further processing.
- Log and User Input Parsing: Cleaning up JSON-like strings extracted from logs or user-submitted data, where errors may arise from manual entry or system quirks. The schema-guided repair feature ensures compliance with expected data structures.
- Schema-Validated Pipelines: Integrating with JSON Schema or Pydantic models to enforce type safety and defaults. This is useful in data extraction workflows where raw input must conform to predefined contracts before downstream processing.
The library’s drop-in compatibility with json.loads() and json.load() makes it easy to integrate into existing codebases, while its CLI and online demo provide immediate validation for manual testing.
Syntax Error Correction
- Missing or mismatched quotes around strings and keys.
- Trailing commas in arrays or objects (e.g., [1, 2, 3,]).
- Unbalanced or misplaced brackets, braces, or parentheses.
- Unescaped characters or improperly formatted values (e.g., true instead of true).
- Case-insensitive handling of JSON booleans (True, TRUE, true) and null values (null, NULL, None).
The library also strips non-JSON characters, such as comments or stray text, while preserving the underlying data structure.
Auto-Completion and Default Values
For incomplete JSON fields, json_repair automatically fills missing values with reasonable defaults, such as empty strings ("") or null. This ensures that partially formed objects or arrays remain parseable without losing context.
Schema-Guided Repairs (Beta)
When a JSON Schema or Pydantic model is provided, the library enforces structural and type constraints during repair.
- Filling missing required fields with default values specified in the schema.
- Coercing scalar values to match expected types (e.g., converting the string "1" to the integer 1 for an integer field).
- Dropping properties or array items that violate schema rules.
- Supporting case-insensitive boolean and null representations during type conversion.
This feature is useful for ensuring repaired JSON adheres to API contracts or internal data standards.
Performance and Configuration
- This minimizes overhead for valid JSON.
- Skip Validation: skipjsonloads=True bypasses the initial check for inputs known to be invalid, improving speed but risking unintended modifications to valid JSON.
- Strict Mode: strict=True raises errors on unrecoverable issues (e.g., duplicate keys) instead of attempting repairs, useful for validation-heavy workflows.
- Non-Latin Character Support: ensureascii=False preserves original characters in languages like Chinese, Japanese, or Korean.
Integration and Compatibility
- jsonrepair.loads(): Replaces json.loads() with repair logic.
- CLI Tool: Processes JSON from files or stdin with optional strict mode or schema validation.
Transparency and Debugging
The online demo and CLI include repair logs that detail every correction applied, allowing users to audit changes before using the output in production. This is particularly valuable for AI workflows where model hallucinations or syntax errors may introduce subtle bugs.
Recent Improvements
- Preserving positive exponents in numeric values (e.g., 1e10).
- Maintaining leading quotes when valid delimiters close strings.
- Handling trailing garbage in otherwise valid JSON (e.g., {"key": "value"} extra text).
These changes reflect ongoing efforts to handle real-world JSON quirks encountered in LLM outputs and log data.