Reading multiple classes from single csv file using CsvHelper

Question

I've been using Josh Close' CsvHelper a bit recently to parse CSV files, I quite like the fluent api for class mapping.

I'm trying to map a csv file which contains multiple record types, the file structure is

C,Comment,Timestamp
I,Class1,Header1,Header2
D,Class1,Data1,Data2
D,Class1,Data1,Data2
...
I,Class2,Header1,Header2,Header3
D,Class2,Data1,Data2,Data3
D,Class2,Data1,Data2,Data3
...
C,Checksum

Is this something which can be handled by CsvHelper? I've writen a custom parser which basically works but all it really does is filter out the Header and Data fields for a specific class - I'd really like to be able to do something like

csv.Configuration.RegisterClassMap<Class1>();
csv.Configuration.RegisterClassMap<Class2>();

var data1 = csv.GetRecords<Class1>().ToList();             
var data2 = csv.GetRecords<Class2>().ToList();

And read the file in one pass? Is this possible or am I using the wrong parser?

Regards Dave

I've been parsing text files like this for 40 years. If I can't parse the file nobody can. I can do this lots of ways with a single pass. You have to use a method that doesn't use a header row since you have multiple sections with different headers and each section may contain a different number of columns. You need to use a GroupBy that sorts the data on the first two columns (Type [C,I,D] and Class). — jdweng
– jdweng, Commented Dec 3, 2015 at 3:53
That doesn't really answer the question though, can CsvHelper parse this file? — David Waterworth
– David Waterworth, Commented Dec 3, 2015 at 5:01
You may not need help on this anymore, but how do you know which record should map to which class? For example, are all rows distinct lengths, is there a discriminator column, etc? — drneel
– drneel, Commented Feb 16, 2016 at 14:22
Hi @drneel, yes there's a discriminator column. The 2nd column is the class type. The first is the row type (I is headers, D is data) — David Waterworth
– David Waterworth, Commented Feb 17, 2016 at 20:53

drneel · Accepted Answer · 2016-02-18 13:52:14Z

10

There is a way to do this; you just have to do it manually.

You manually read the csv file row by row
Inspect the first column for the discriminator that will indicate that you need to map to a Class object.
Inspect the second column for the class to map to.

Map the entire row to that given class.

public static void ReadMultiClassCsv()
{
    var class1Data = new List<Class1>();
    var class2Data = new List<Class2>();

    using (StreamReader reader = File.OpenText(@"C:\filename.csv"))
    using (var csvReader = new CsvReader(reader))
    {
        //1.  You manually read the csv file row by row
        while (csvReader.Read())
        {
            var discriminator = csvReader.GetField<string>(0);

            //2.  Inspect the first column for the discriminator that will indicate that you need to map to a Class object.
            if (discriminator == "D")
            {
                var classType = csvReader.GetField<string>(1);

                //3.  Inspect the second column for the class to map to.
                switch (classType)
                {
                    //4.  Map the entire row to that given class.
                    case "Class1":
                        class1Data.Add(csvReader.GetRecord<Class1>());
                        break;
                    case "Class2":
                        class2Data.Add(csvReader.GetRecord<Class2>());
                        break;
                    default:
                        break;
                }
            }
        }
    }
}

answered Feb 18, 2016 at 13:52

drneel

2,9055 gold badges32 silver badges48 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

stripathi Over a year ago

Thanks for the answer. I am guessing the class maps need to be index based. Is it correct or can I actually have a class map with name mapping?

stripathi Over a year ago

I posted above question to github.com/JoshClose/CsvHelper/issues/441. Got a response from Josh with code sample to accomplish this. This can be done in version 3 though.

Collectives™ on Stack Overflow

Reading multiple classes from single csv file using CsvHelper

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related