I have a huge amount of data in flat-file format that I would like to validate with Visual Basic 2010. Basically the data is a text file with 300 comma separated fields and 500,000 records.

Any tips, sample code or examples? Are any .NET classes specifically geared for data validation?

As I have different validation rules for each field (some fields are dates, some are currency, etc.) I would like to store the rules away from the code (e.g. in an SQL database table). Has anyone seen this done before?

Many thanks

6 Years
Discussion Span
Last Post by agrothe

If your going to store the rules in SQL Server, why not store the data in SQL server?

If you really can't move the data, you can use an XML file to store the validation rules and then compare the data in the text file.

So your XML file would specify that column 1 is a DateTime.
Open the data file and do a TryParse to a DateTime. Most if not all the data types expose a TryParse method which use an output parameter and return boolean results.

Using a Repository type pattern you can abstract that away and make the code more usable too.

This topic has been dead for over six months. Start a new discussion instead.
Have something to contribute to this discussion? Please be thoughtful, detailed and courteous, and be sure to adhere to our posting rules.