Validating data from a file in .NET

Question

In my program (a console application that does edit and imputation of data) I give the user the ability to provide a data dictionary in a number of different ways: tab-delimited text files, Excel workbooks or in a database. The dictionary consists of several (12-15) files/sheets/tables. I am trying to come up with a nice way to load the data from the various sources into the database.

My solution thus far has been to use a repository to isolate the various data sources and have these repositories spit out DTOs that I map onto my domain model. I use the Builder pattern to control the entire sequence of events.

Basically the sequence of events for each file/sheet/table is:

Get the DTOs from the repository
Validate the information in the DTOs
Then
- If the data is good, map the domain entity
- else keep a running list of errors.

My question is this: I am trying to figure out where the best place is to validate the information in the DTOs? One possible solution was to add an interface on the DTOs like so

public interface IValidate
{
    void Validate();
    bool HasErrors { get; }
    IEnumerable<string> GetErrorMessages();
}

is this too heavy for a DTO? Should the validation happen somewhere else? Sorry if this is a little subjective.

I think "Data Transfer Object" from Martin Fowler's PoEAA ;) — amantur
– amantur, Commented Sep 5, 2009 at 12:43
Yes it is here: martinfowler.com/eaaCatalog/dataTransferObject.html — amantur
– amantur, Commented Sep 5, 2009 at 12:44
@TheVillageIdiot: sounds like an FNFAS (Fancy Name For A Structure). — MusiGenesis
– MusiGenesis, Commented Sep 5, 2009 at 13:42
@MusiGenesis: DTOs serve a very specific purpose, separating the domain from the persistence layer. How an object looks in a domain might look very different from how it might look in the database for instance. FNFAS doesn't really apply here either, like I say, I'm trying to have DTOs be able to validate themselves for instance, though there are other things you could do with them depending on your domain — Jeffrey Cameron
– Jeffrey Cameron, Commented Sep 5, 2009 at 21:41
p.s. I'm merely looking for experience of someone who has been in this situation themselves. Think of it as validating configuration information stored in an external source (like an XML config file) if that helps at all. — Jeffrey Cameron
– Jeffrey Cameron, Commented Sep 5, 2009 at 21:43

Daniel Segan · Accepted Answer · 2009-09-05 13:58:01Z

1

I can't answer your question definitively because, as you said, the question does seem to be subjective and any answer would ultimately be an opinion. It sounds like you are really struggling with a design decision based on the "academic" definition of a DTO as opposed to some kind of pragmatic requirement. We've all been there.

When faced with similar situations I generally tend to carry out the implementation in the most simple and straight forward way as I can avoiding things like complex tightly coupled relationships and excessive amounts of design. This way, once I am able to get everything running I can refractor from there will less impact.

Overall, it sounds like you are constructing some kind of ETL system. I don't know what platform you are dealing with but if you are using SQL Server Have you looked at SQL Server Integration Services? It has a lot of constructs to handle things like Office documents, XML, and flat files as data sources.

Anyway, good luck with your struggle.

answered Sep 5, 2009 at 13:58

Daniel Segan

1842 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Peter Mortensen Over a year ago

ETL = Extract, transform, and load ? en.wikipedia.org/wiki/Extract,_transform,_load

Jeffrey Cameron Over a year ago

Thanks for the answer Daniel, but I'm merely trying to get infomation (a data dictionary in this case) into my program so that I can use it. The program is a batch Edit and Imputation system and doesn't have anything to do with ETL at all.

Collectives™ on Stack Overflow

Validating data from a file in .NET

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related