Member Avatar for JohnDove

I have a huge amount of data in flat-file format that I would like to validate with Visual Basic 2010. Basically the data is a text file with 300 comma separated fields and 500,000 records.

Any tips, sample code or examples? Are any .NET classes specifically geared for data validation?

As I have different validation rules for each field (some fields are dates, some are currency, etc.) I would like to store the rules away from the code (e.g. in an SQL database table). Has anyone seen this done before?

Many thanks

  • 2 Contributors
  • 1 Reply
  • 167 Views
  • 1 Day Discussion Span
  • Latest Post Latest Post by agrothe
Member Avatar for agrothe

If your going to store the rules in SQL Server, why not store the data in SQL server?

If you really can't move the data, you can use an XML file to store the validation rules and then compare the data in the text file.

So your XML file would specify that column 1 is a DateTime.
Open the data file and do a TryParse to a DateTime. Most if not all the data types expose a TryParse method which use an output parameter and return boolean results.

Using a Repository type pattern you can abstract that away and make the code more usable too.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.