how do i parse a custom delimited file

•

What is DaniWeb IT Discussion Community?

You're currently browsing the C++ section within the Software Development category of DaniWeb, a massive community of 375,168 software developers, web developers, Internet marketers, and tech gurus who are all enthusiastic about making contacts, networking, and learning from each other. In fact, there are 2,264 IT professionals currently interacting right now! Registration is free, only takes a minute and lets you enjoy all of the interactive features of the site.

Join our community!

Please support our C++ advertiser:

Thread Tools Display Modes

Views: 2313 | Replies: 7

•

Join Date: Apr 2007

Posts: 10

Reputation: Lance Wassing is an unknown quantity at this point

Rep Power: 2

Solved Threads: 0

Lance Wassing Lance Wassing is offline

Offline

Newbie Poster

how do i parse a custom delimited file

Apr 2nd, 2007

Hello Everyone,

My question is pretty basic, i hope. Basically, what i have is an extremely large text file, which acts as a flat file database... in essence.

Text file formatting looks like this:
CLIENT CODE | CLIENT NAME| CLIENT ADDRESS 1 ...

This pattern goes on for about 23 fields, including one field per line, and there is about 1100 records in the "flat file database at the moment". My thinking was to use the strtok function, and extract the fields into aarays, for the full 1100 records. so that wrap some functions around it like get next record... etc.

what i need to know is, does anyone know of a fast, and efficient way to extract this data, from the delimited text file, and basically place the data into my gui line by line, as requested by the button click events. If it helps, i am using BCB 6 Professional.

Thankyou for any help
Lance.

•

Join Date: Apr 2007

Posts: 10

Reputation: Lance Wassing is an unknown quantity at this point

Rep Power: 2

Solved Threads: 0

Lance Wassing Lance Wassing is offline

Offline

Newbie Poster

Re: how do i parse a custom delimited file

Apr 2nd, 2007

Sorry... i meant "including one memo field per line" not "including one field per line"

•

Join Date: Dec 2005

Posts: 3,052

Reputation: Salem has much to be proud of

Rep Power: 20

Solved Threads: 348

Salem

Offline

void main'ers are DOOMed

Re: how do i parse a custom delimited file

Apr 2nd, 2007

I'd suggest you start with the simple obvious thing of using strtok (in C), or using getline() with a delimiter (C++).

File I/O is going to have a lot more impact on how long this is going to take than what you might do in finding and splitting a string based on a delimiter.

If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.

•

Join Date: Dec 2006

Location: india

Posts: 1,036

Reputation: vijayan121 is a glorious beacon of light

Rep Power: 9

Solved Threads: 156

vijayan121 vijayan121 is offline

Offline

Veteran Poster

Re: how do i parse a custom delimited file

Apr 2nd, 2007

u could memory map the file (unix mmap/mmap64 or windows CreateFileMapping/MapViewOfFile) to reduce the i/o overhead (or use a large file buffer with C/C++ file i/o).

•

Join Date: Apr 2007

Posts: 10

Reputation: Lance Wassing is an unknown quantity at this point

Rep Power: 2

Solved Threads: 0

Lance Wassing Lance Wassing is offline

Offline

Newbie Poster

Re: how do i parse a custom delimited file

Apr 2nd, 2007

Well guys,

I have tried the obvious.

I think I may be doing something a little wrong, or the strtok(str, delims) function is not going to suffice.

The test file i made looks like this
BLUE | GREEN
RED | ORANGE
PURPLE | PINK
YELLOW | BLACK

and the code looks like this:

 Help with Code Tags 
 c Syntax (Toggle Plain Text) 
        //Open the file, and copy the file into the buffer
        FILE * f;
        char * buffer;
        long size;
        f = fopen("test.txt","rt+");
        if( f == NULL) exit(1);
 
        fseek(f,0,SEEK_END);
        size = ftell(f);
        rewind(f);
 
        buffer = (char*) malloc(size);
        if(buffer == NULL) exit(2);
 
        fread(buffer, 1, size, f);
        fclose(f);
 
        //Now we're going to parse the file
        char * temp;
        String * Array = new String[5];
        temp = strtok(buffer,"|");
        RichEdit1->Lines->Add(temp);
        //Open the file, and copy the file into the buffer
        FILE * f;
        char * buffer;
        long size;
        f = fopen("test.txt","rt+");
        if( f == NULL) exit(1);

        fseek(f,0,SEEK_END);
        size = ftell(f);
        rewind(f);

        buffer = (char*) malloc(size);
        if(buffer == NULL) exit(2);

        fread(buffer, 1, size, f);
        fclose(f);

        //Now we're going to parse the file
        char * temp;
        String * Array = new String[5];
        temp = strtok(buffer,"|");
        RichEdit1->Lines->Add(temp);

unfortunately, the output in the RichEdit1 Box, is BLUE. Does anyone know why? or how i can fix this. If so please... PLEASE... respond. Im a bit stumped here.

Thanks,
Lance

Last edited by WaltP : Apr 2nd, 2007 at 9:02 pm. Reason: Fixed CODE tags -- doesn't anyone know what the PREVIEW button is for???

•

Join Date: Dec 2005

Posts: 3,052

Reputation: Salem has much to be proud of

Rep Power: 20

Solved Threads: 348

Salem

Offline

void main'ers are DOOMed

Re: how do i parse a custom delimited file

Apr 2nd, 2007

http://www.daniweb.com/techtalkforum...cement8-3.html

Well for starters, fread() doesn't append a \0, so the thing you read isn't a proper string.

If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.

•

Join Date: Apr 2007

Posts: 10

Reputation: Lance Wassing is an unknown quantity at this point

Rep Power: 2

Solved Threads: 0

Lance Wassing Lance Wassing is offline

Offline

Newbie Poster

Re: how do i parse a custom delimited file

Apr 3rd, 2007

Well everyone, i have figured out a way of doing this sort of.

What i have done is created strings, and used the substring function within a huge for loop. This method works better than strtok, mainly because i can't figureout strtok... not quite, but the problem is that, the file must be hard coded intot he executable, but i want to read the file to a string array, and then use the substring function to parse.

I guess my question, and im sorry to ask such a novice question as i am a novice in c, how do i load my file, preferably with the fopen function of stdio.h and stdlib.h, int String arrays?

bassically i would like

Help with Code Tags

(Toggle Plain Text)

String Lines = new String[i]
//Where i is the number of lines in the file
//load the file into the String array so that it //looks like this:
Lines[0] = "Lance , Wassing , 11 Wonderway";
/*
and so on then i can just loop through the array doing the Lines[i].SubString(int, int); function */

if you have something that will work for this purpose, or know i how i can do this, please let me know. Your help is MUCH appreciated!

Thankyou,
Lance