Validate JSON Schema which has fixed keys and user defined keys in Python [closed]

Question

Closed. This question needs details or clarity. It is not currently accepting answers.

Want to improve this question? As written, this question is lacking some of the information it needs to be answered. If the author adds details in comments, consider editing them into the question. Once there's sufficient detail to answer, vote to reopen the question.

Closed 5 months ago.

Improve this question

I'm trying to validate a JSON file that is provided by a user. The JSON will contain certain fixed keys, but also contain some user-defined keys too. I want to validate that this JSON object contains these fixed keys, in a certain format, and the user-defined keys are in a certain format too (as these keys will always have values in a defined format).

I came across this post Validate JSON data using python, but the documentation for jsonschema.validate doesn't really show anything to do with user-defined keys, and also how to define if a key should have a list of dicts, or a dict which its key-values must be of a list of dicts.

Here's a sample schema:

{
    "a": "some value",
    "b": "some value",
    "c": {
        "custom_a": [{...}],
        "custom_b": [{...}]
    },
    "d": [{...}]
}

I have tried doing the following:

import json
from jsonschema import validate

my_json = json.loads(<JSON String following above pattern>) 

schema = {
    "a" : {"type": "string"},
    "b" : {"type": "string"},
    "c" : {[{}]},
    "d": [{}]
}

validate(instance=my_json, schema=schema) #raises TypeError on "c" and "d" in schema spec

I have also tried the following schema spec, but I get stuck on how to handle the custom keys, and also nested lists within dicts, etc.

schema = {
    "a" : {"type": "string"},
    "b" : {"type": "string"},
    "c" : {
        "Unsure what to define here": {"type": "list"} #but this is a list of dicts
    },
    "d": {"type": "list"} #but this is a list of dicts
}

hc_dev · Accepted Answer · 2025-05-15 16:31:53Z

There are several Python libraries available for validating JSON data, especially when it comes to complex schemas with fixed and user-defined keys. Here are some commonly used libraries, each with unique strengths and options for managing dynamic structures.

The most common are:

Using jsonschema

from jsonschema import validate, ValidationError

# Define JSON Schema
schema = {
    "type": "object",
    "properties": {
        "a": {"type": "string"},
        "b": {"type": "string"},
        "c": {
            "type": "object",
            "patternProperties": {
                "^custom_": {  # Any key in "c" must start with "custom_"
                    "type": "array",
                    "items": {"type": "object"}
                }
            },
            "additionalProperties": False
        },
        "d": {
            "type": "array",
            "items": {"type": "object"}
        }
    },
    "required": ["a", "b", "c", "d"],
    "additionalProperties": False
}

# Sample JSON data
data = {
    "a": "some value",
    "b": "another value",
    "c": {
        "custom_a": [{"key1": "value1"}, {"key2": "value2"}],
        "custom_b": [{"key3": "value3"}]
    },
    "d": [{"key4": "value4"}, {"key5": "value5"}]
}

# Validate the JSON data
try:
    validate(instance=data, schema=schema)
    print("Validation successful!")
except ValidationError as e:
    print("Validation failed:", e.message)

Using marshmallow

from marshmallow import Schema, fields, validate, ValidationError

class CustomEntrySchema(Schema):
    # This allows any string keys and values in each dictionary
    class Meta:
        unknown = 'include'

class MainSchema(Schema):
    a = fields.String(required=True)
    b = fields.String(required=True)
    c = fields.Dict(
        keys=fields.String(validate=validate.Regexp(r'^custom_')),
        values=fields.List(fields.Nested(CustomEntrySchema)),
        required=True
    )
    d = fields.List(fields.Nested(CustomEntrySchema), required=True)

# Sample JSON data
data = {
    "a": "some value",
    "b": "another value",
    "c": {
        "custom_a": [{"key1": "value1"}, {"key2": "value2"}],
        "custom_b": [{"key3": "value3"}]
    },
    "d": [{"key4": "value4"}, {"key5": "value5"}]
}

# Validate the JSON data
schema = MainSchema()
try:
    schema.load(data)
    print("Validation successful!")
except ValidationError as e:
    print("Validation failed:", e.messages)

Using pydantic

from pydantic import BaseModel, Field, ValidationError, RootModel, model_validator
from typing import List, Dict
import re

class CustomEntryModel(RootModel[Dict[str, str]]):
    """This allows arbitrary key-value pairs in each entry of 'c' and 'd'."""

class MainModel(BaseModel):
    a: str
    b: str
    c: Dict[str, List[CustomEntryModel]]  # We'll validate keys in 'c' manually
    d: List[CustomEntryModel]

    @model_validator(mode="before")
    def validate_custom_keys(cls, values):
        # Check that all keys in 'c' start with "custom_"
        c_data = values.get("c", {})
        for key in c_data:
            if not re.match(r'^custom_', key):
                raise ValueError(f"Key '{key}' in 'c' must start with 'custom_'")
        return values

# Sample JSON data
data = {
    "a": "some value",
    "b": "another value",
    "c": {
        "custom_a": [{"key1": "value1"}, {"key2": "value2"}],
        "custom_b": [{"key3": "value3"}]
    },
    "d": [{"key4": "value4"}, {"key5": "value5"}]
}

# Validate the JSON data
try:
    model = MainModel(**data)
    print("Validation successful!")
except ValidationError as e:
    print("Validation failed:", e)

Output

When I ran all of them at once, the validation succeeded equally for each:

Validation successful!
Validation successful!
Validation successful!

Excellent overview of JSON-schema and model validation libraries. The code examples serve as illustrative comparison: the model definition differs most among them, ordered from simple to complex they increase also in use-cases (starting with asked JSON-schema, over lightweight ORM, to fully-integrative data validation).

Jeremy Fiel · Accepted Answer · 2024-11-11 15:41:42Z

Define the known properties as usual and the unknown properties as additionalProperties with a schema defined

{
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
        "a": {"type": "string"},
        "b": {"type": "string"},
        "c": {
            "additionalProperties": {
                "type": "string"
            }
        },
        "d": {
            "type": "array",
            "items": {
                "type": "object"
            }
        }
    }
}

This will allow an instance such as

{
    "a": "some_value",
    "b": "some_value",
    "c": {
        "custom_keyword": "some_value"
    },
    "d": [
        {
            "custom_keyword": 1
        }
    ]
}

Collectives™ on Stack Overflow

Validate JSON Schema which has fixed keys and user defined keys in Python [closed]

2 Answers 2

Using jsonschema

Using marshmallow

Using pydantic

Output

1 Comment

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Using jsonschema

Using marshmallow

Using pydantic

Output

1 Comment

Comments

Linked

Related