0

I have been trying to write to a CSV file from a JSON file using NorthConcepts datapipeline library in the following way, however I have problems. The data in the JSON file is either repeated or the fields are Seperated and written in different rows. For instance from the given code below I could have a combination of the following rows : Row 1 : CreatedAt, ID, Row 2 : CreatedAT, ID, IDString, Row 3 : CreatedAt, IDString likewise. I have about 30 fields and about 4 rows in the CSV file writing data for the same record but with split fields filled in. Would anyone know what the problem is? The JSON file I am reading from doesnt have repetitive data.

I read data the following way : 

DataReader reader = new JsonReader(new File(FILE))
.addField("CreatedAt", "//array/object/created_at")
.addField("ID", "//array/object/id")
.addField("IDString", "//array/object/id_str")
.addField("Text", "//array/object/actualTweet")

Writing data the following way : 

DataWriter writer = new  CSVWriter(new File("../Twix/files/x.csv" ));
JobTemplate.DEFAULT.transfer(reader, writer);

The library used : http://northconcepts.com/data-pipeline/ javadocs : http://northconcepts.com/data-pipeline/javadocs/

Example CSV : 

  Name  ProfileImageURL Location    CreatedAt   FollowRequestSent   IsTranslator    DefaultProfile  FavouritesCount ContributorsEnabled
  The Guardian      London                      
  The Guardian      London                      
  The Guardian      London  Thu Nov 05 23:49:19 +0000 2009              130 
  The Guardian      London  Thu Nov 05 23:49:19 +0000 2009              130 
  The Guardian      London  Thu Nov 05 23:49:19 +0000 2009              130 
    http://pbs.twimg.com/profile_images/2814613165/f3c9e3989acac29769ce01b920f526bb_normal.png          FALSE   FALSE   FALSE       FALSE

Here the URL given above is for the second column.

Improve this question
3
  • This doesn't really help, but I think I would convert the Json to a list of objects. One object representing a row of data from the ultimate csv file. Then have a CSV writer that adds the column headings and then writes each object as required. The blank object fields could become empty strings in the CSV file. Commented Jul 4, 2014 at 10:19
  • What are you passing into JsonReader.addRecordBreak()? Commented Jul 4, 2014 at 12:12
  • This is what i pass into addRecordBreak : //array/object. Although there are nested objects within this JSON i don't need them so not using them. Do you think this could be a problem? Commented Jul 4, 2014 at 12:13
Related questions
5
Converting an CSV file to a JSON object in Java
4
Converting csv row to JSON object in Java
0
How to write multiple records to JSON in file on Java
Load 7 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.