Reading in certain columns from CSV files into array C++

[johnnyrocket]

I've placed a CSV file into an array with C++ with each row being a single array element, but the file contains many, many columns that I don't need. I only need columns 1,2,3,4,6,7 out of about 50. Is is best to read the full value into the array and then edit it or only read in those values that I need? Either way, I'm stuck and don't know how to proceed either way. I'm new at this and I've been unable to find resources about this online. Any help would be appreciated.

[WolfPack]

I've placed a CSV file into an array with C++ with each row being a single array element, but the file contains many, many columns that I don't need. I only need columns 1,2,3,4,6,7 out of about 50. Is is best to read the full value into the array and then edit it or only read in those values that I need?

This depends on what you are going to do with the data read. If you are going to edit the data in those columns and write it back to a file, along with the rest of the columns, you will have to read and store all the data. If you only want the values, and they are discarded after use, you can create an array of only 6 elements with multiple rows, and store the columns you need to that array.

As for how to proceed, paste the current code where you are reading the csv values into the array, and maybe we can work something out.

[johnnyrocket]

Remember I'm new at this, so I'm sure my code is horrible. This is in Visual Studio 2005. The Console::WriteLine and the cout of the lineCount are only for my purposes to make sure the program was actually working. They will be removed later.

#include "stdafx.h"
#include <iostream>
#include <string>
using namespace std;
using namespace System;
using namesapce System::IO;
using namespace System::Collections;
 
int _tmain()
{
StreamReader^ reader;
int lineCount = 0;
reader = gcnew StreamReader(" <a href="http://www.daniweb.com/techtalkforums/">\\\\<server>\\<file>.csv</a> ");
String^ data;
 
ArrayList^ theArray = gcnew ArrayList(lineCount);
 
while(0<reader->Peek())
{
data = reader->ReadLine();
theArray->Add(data);
Console::WriteLine(theArray[lineCount]);
lineCount++;
}
 
cout << lineCount << endl;

[iamthwee]

Remember I'm new at this, so I'm sure my code is horrible. This is in Visual Studio 2005. The Console::WriteLine and the cout of the lineCount are only for my purposes to make sure the program was actually working. They will be removed later.

#include "stdafx.h"
#include <iostream>
#include <string>
using namespace std;
using namespace System;
using namesapce System::IO;
using namespace System::Collections;
 
int _tmain()
{
StreamReader^ reader;
int lineCount = 0;
reader = gcnew StreamReader(" <a href="http://www.daniweb.com/techtalkforums/">\\\\<server>\\<file>.csv</a> ");
String^ data;
 
ArrayList^ theArray = gcnew ArrayList(lineCount);
 
while(0<reader->Peek())
{
data = reader->ReadLine();
theArray->Add(data);
Console::WriteLine(theArray[lineCount]);
lineCount++;
}
 
cout << lineCount << endl;

That ain't c++ sorrie.

[WaltP]

I've placed a CSV file into an array with C++ with each row being a single array element, but the file contains many, many columns that I don't need. I only need columns 1,2,3,4,6,7 out of about 50. Is is best to read the full value into the array and then edit it or only read in those values that I need?

This depends on what you are going to do with the data read. If you are going to edit the data in those columns and write it back to a file, along with the rest of the columns, you will have to read and store all the data. If you only want the values, and they are discarded after use, you can create an array of only 6 elements with multiple rows, and store the columns you need to that array.

Remember I'm new at this, so I'm sure my code is horrible. This is in Visual Studio 2005. The Console::WriteLine and the cout of the lineCount are only for my purposes to make sure the program was actually working. They will be removed later.

This response doesn't seem to answer any ofWolfPack's questions. We still don't know enough to answer your question. And yes, your format is horrible. After every { indent 4 spaces. Before every } unindent. Then you program can be followed.

[johnnyrocket]

This response doesn't seem to answer any of WolfPack's questions. We still don't know enough to answer your question. And yes, your format is horrible. After every { indent 4 spaces. Before every } unindent. Then you program can be followed.

I need to read in the data, sort it, add a few columns from another CSV file and output it to a file. So what I need is to read in a CSV file of about 500 rows and 50 columns, keeping all the rows, but only keeping columns 1,2,3,4,6,7 and appending those elements with data from another CSV file to go into an output report. So I'll end up with about 500 rows and 10 columns. Again, I'm new at this and struggling at it. Thanks.

[Salem]

Well this being C++, I would go with something like

class csvRow {
  public:
    parseLine(std::string);
    std::string operator[](int) const;
  private:
    std::vector< std::string > row;
};

Where parseLine takes a line and splits out all the comma separated values, and stores them in the internal vector.
The overload of the [] array operator allows you to pick out individual fields.

So you could say for example something like this

csvRow foo;
foo.parseLine("1,2,3,\"hello\",5,6");
cout << foo[0] << foo[2] << endl;

The whole file would just be a std::vector of csvRow's

[johnnyrocket]

I'm not familiar with vectors. So I need a csvRow for each element in the array? I'm having trouble creating those. What else do I need to add to the code?

[Salem]

Ok, start with the basics - can you split this line at each comma?
"1,2,3,\"hello, world\",5,6"

Start real simple, just print each field as you find it, say Found 1
Found 2
Found 3
Found hello, world
Found 5
Found 6
Test it with a variety of lines to make sure that bit of the code works.

This is the essence of the problem, without which all the packaging will do you no good at all. My csvRow thing just wraps it up into a useful class where you can treat each field of the CSV as a separate index of an array.

> I'm not familiar with vectors
Consider it an opportunity to learn about them then - read a book, find a tutorial or two.

[WaltP]

Since you keep explaining you are new (thinking we'll forget I guess), I'll assume you know practically nothing about C++ but know just enough to understand arrays and the string type. Define a two dimensional array of strings with as many rows and colums as you need
Open the data file
Start a loop to read each line into another stringread a line
look for a comma
move the previous substring into your string array into col 1
look for the next comma
move the previous substring into your string array into col 2
keep going until you have all the columns saved
loop back

Close the file

[johnnyrocket]

Since you keep explaining you are new (thinking we'll forget I guess), I'll assume you know practically nothing about C++ but know just enough to understand arrays and the string type.

• Define a two dimensional array of strings with as many rows and colums as you need
• Open the data file
• Start a loop to read each line into another string
  • read a line
  • look for a comma
  • move the previous substring into your string array into col 1
  • look for the next comma
  • move the previous substring into your string array into col 2
  • keep going until you have all the columns saved
  • loop back

• Close the file

Ok. So it looks like I was approaching the problem incorrectly. If I create the 2-dimensional array, I think I can remove entire columns using a loop with ArrayList->RemoveAt. For example, to remove column 5: theArray->RemoveAt[i][5] and iterate through the end of the rows. I'll give it a shot. Thanks.

[WaltP]

Ok. So it looks like I was approaching the problem incorrectly. If I create the 2-dimensional array, I think I can remove entire columns using a loop with ArrayList->RemoveAt. For example, to remove column 5: theArray->RemoveAt[i][5] and iterate through the end of the rows. I'll give it a shot. Thanks.

Not what I said. Please read againcarefully to see the easy solution. You don't start with 50 columns. You start with the total you need at the end of processing (like 10).

[iamthwee]

Ok. So it looks like I was approaching the problem incorrectly. If I create the 2-dimensional array, I think I can remove entire columns using a loop with ArrayList->RemoveAt. For example, to remove column 5: theArray->RemoveAt[i][5] and iterate through the end of the rows. I'll give it a shot. Thanks.

Nope, there ain't no ArrayList->RemoveAt in c++.