Insert Multiple JSON files to MongoDB using python

Question

The JSON files are as follows a.json,b.json.....z.json (26 json files)

The json format of each of the file looks as:

{
    "a cappella": {
        "word": "a cappella",
        "wordset_id": "5feb6f679a",
        "meanings": [
            {
                "id": "492099d426",
                "def": "without musical accompaniment",
                "example": "they performed a cappella",
                "speech_part": "adverb"
            },
            {
                "id": "0bf8d49e2e",
                "def": "sung without instrumental accompaniment",
                "example": "they sang an a cappella Mass",
                "speech_part": "adjective"
            }
        ]
    },
    "A.D.": {
        "word": "A.D.",
        "wordset_id": "b7e9d406a0",
        "meanings": [
            {
                "id": "a7482f3e30",
                "def": "in the Christian era",
                "speech_part": "adverb",
                "synonyms": [
                    "AD"
                ]
            }
        ]
    },.........
}

How could I store these in MongoDB such that if queried with word the results shows meanings,synonyms(if available)?

I have never used Mongo on how to approach, but the same was done with SO suggestions for a single json file in mysql as:

**cursor has db connection

with open('a.json') as f:
    d = json.load(f)

for word in d:
    word_obj = d[word]
    wordset_id = word_obj['wordset_id']

    sql = "INSERT INTO Word (word, wordset_id) VALUES (%s, %s)"
    values = (word, wordset_id)
    cursor.execute(sql, values)
    conn.commit()

similarly to store meanings and synonyms as different tables,

But as suggetsed I guess this would become better if MongoDB is used

Sorry, but "I have never used Mongo" just begs for a tutorial on that topic. Run through that first. Then, try to replicate these things in Python. Stack Overflow can't teach you things from scratch and it doesn't want to either. — Ulrich Eckhardt
– Ulrich Eckhardt, Commented Dec 27, 2019 at 10:23
makes sense, I can connect to mongo , but looking for an approach on inserting all json files at once , thanks for suggestion — KcH
– KcH, Commented Dec 27, 2019 at 10:26
Yeah I have tried with sql , same I want to replicate with mongo — KcH
– KcH, Commented Dec 27, 2019 at 10:50

Ivan Vinogradov · Accepted Answer · 2019-12-27 11:51:14Z

1

If you want to insert data from multiple .json files, do it in a loop:

file_names = ['a.json', 'b.json', ...]

for file_name in file_names:
    with open(file_name) as f:
        file_data = json.load(f)  # load data from JSON to dict
        for k, v in file_data.items():  # iterate over key-value pairs
            collection.insert_one(v)  # your collection object here

edited Dec 27, 2019 at 11:51

answered Dec 27, 2019 at 11:30

Ivan Vinogradov

4,5236 gold badges34 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

10 Comments

KcH Over a year ago

when I am just trying with a.json single file as test, it has thrown an error : bson.errors.InvalidDocument: key 'A.D.' must not contain '.' (this data is in above question)

Ivan Vinogradov Over a year ago

In mongo, keys cannot contain . character. If your files contain . only at top level, it's okay since they are not inserted in collection (in my example answer). Otherwise you should change key names.

Ivan Vinogradov Over a year ago

No, because . is used in their query language like key1.key2

Ivan Vinogradov Over a year ago

Well sometimes people just replace . with _. If you need to "restore" your keys to their "original" format in future and avoid collisions with real "_" character, you can use your own custom char sequence to encode .. Like double underscores or something.

Ivan Vinogradov Over a year ago

What you wrote above sounds exactly like table partitioning. I haven't used this feature in Mongo, but you can google for that. I have no details on your application features, so partitioning may not be the most efficient trick you can use.

|

Collectives™ on Stack Overflow

Insert Multiple JSON files to MongoDB using python

1 Answer 1

10 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

10 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related