parse decoded json array into csv

Question

I have a file that contains millions of arrays of this size:

{
  "leagues" : [{
      "tier" : "SILVER",
      "entries" : [{
          "playerOrTeamId" : "359",
          "playerOrTeamName" : "TryHard",
          "division" : "II",
          "leaguePoints" : "63",
          "wins" : "65"
        }],
      "id" : "359"
    }],
  "summonerId" : "359",
  "region" : "euw",
  "updatedAt" : "1412122432"
},

That's, for example, the smallest array.,There are some arrays that have additional linked arrays that contain extra information in relation to the primary array. Example:

{
  "summonerId" : "477",
  "region" : "euw",
  "leagues" : [{
      "tier" : "GOLD",
      "entries" : [{
          "playerOrTeamId" : "477",
          "playerOrTeamName" : "Alucard662545",
          "division" : "V",
          "leaguePoints" : "9",
          "wins" : "128"
        }]
    }, {
      "tier" : "SILVER",
      "entries" : [{
          "playerOrTeamId" : "TEAM-8d6a3640-2da8-11e2-99dc-782bcb4ce61a",
          "playerOrTeamName" : "CAPCOMP BE",
          "division" : "V",
          "leaguePoints" : "0",
          "wins" : "24"
        }]
    }, {
      "tier" : "BRONZE",
      "entries" : [{
          "playerOrTeamId" : "TEAM-8d6a3640-2da8-11e2-99dc-782bcb4ce61a",
          "playerOrTeamName" : "CAPCOMP BE",
          "division" : "I",
          "leaguePoints" : "55",
          "wins" : "8"
        }]
    }],
  "updatedAt" : "1410786559"
},

I have been literally pulling hair out of my head, spend 2 day and nights to figure it out. I have MongoDB where this information is stored in, when I export it I can only get decoded JSON arrays. I need this stuff to be fully CSV formatted. How name can I CSV format a million arrays like these?

other than column separators, csv has no "format". there's no provision for nested records. there's just columns. And since you've provided NO details about how you want your csv to look like, we can't help you - even if someone did feel like writing the code for you. We're here to HELP, not do your job for you. — Marc B
– Marc B, Commented Oct 8, 2014 at 21:05
A bit harsh Mark B, but @floppy floppy to your question, I think you may have to consider creating a csv string using java script and loop through your current js object to extract then add to your csv string. Has that crossed your mind? I would start with a smaller data set to test, obviously. — Mr. Concolato
– Mr. Concolato, Commented Oct 8, 2014 at 21:10
I did not find Marc's matter-of-fact and true statement to be harsh @Mr.Concolato. — Jay Blanchard
– Jay Blanchard, Commented Oct 8, 2014 at 21:12
I understand marc b, could you please point me where i clearly stated that i wanted the community to fix this? Im for sure that i was asking how this could be achieved. By helping people could provide new ways of thinking that i couldve been missing, or a piece of code that did the trick for a similar situation. You are not helping me with your comment, but thanks for sharing your opinion. Have a nice day sir. — floppy
– floppy, Commented Oct 9, 2014 at 13:13

hd1 · Accepted Answer · 2014-10-08 21:27:46Z

1

You have two options:

mongoexport is a utility that produces a JSON or CSV export of data stored in a MongoDB instance. Usage example:

mongoexport --db users --collection contacts --csv --fieldFile fields.txt --out /opt/backups/contacts.csv

which takes the fields specified in the \r-terminated fields.txt file, one per line, from the collection contacts and puts them into /opt/backups/contacts.csv.

Any other way to read in JSON to a language and write csv. An example in python follows:

from pymongo import  MongoClient
import csv
client = MongoClient()
db = client['test-database']
collection = db.test_collection
writer = csv.writer('/opt/backups/contacts.csv')
writer.writerow([k for k in collection])
writer writerows([[v for v in c] for c in collection])

... and does the same Hope that helps.

answered Oct 8, 2014 at 21:27

hd1

34.9k5 gold badges83 silver badges95 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

parse decoded json array into csv

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related