how do i parse a custom delimited file

[Lance Wassing]

Hello Everyone,

My question is pretty basic, i hope. Basically, what i have is an extremely large text file, which acts as a flat file database... in essence.

Text file formatting looks like this:
CLIENT CODE | CLIENT NAME| CLIENT ADDRESS 1 ...

This pattern goes on for about 23 fields, including one field per line, and there is about 1100 records in the "flat file database at the moment". My thinking was to use the strtok function, and extract the fields into aarays, for the full 1100 records. so that wrap some functions around it like get next record... etc.

what i need to know is, does anyone know of a fast, and efficient way to extract this data, from the delimited text file, and basically place the data into my gui line by line, as requested by the button click events. If it helps, i am using BCB 6 Professional.

Thankyou for any help
Lance.

[Lance Wassing]

Sorry... i meant "including one memo field per line" not "including one field per line"

[Salem]

I'd suggest you start with the simple obvious thing of using strtok (in C), or using getline() with a delimiter (C++).

File I/O is going to have a lot more impact on how long this is going to take than what you might do in finding and splitting a string based on a delimiter.

[vijayan121]

u could memory map the file (unix mmap/mmap64 or windows CreateFileMapping/MapViewOfFile) to reduce the i/o overhead (or use a large file buffer with C/C++ file i/o).

[Lance Wassing]

Well guys,

I have tried the obvious.

I think I may be doing something a little wrong, or the strtok(str, delims) function is not going to suffice.

The test file i made looks like this
BLUE | GREEN
RED | ORANGE
PURPLE | PINK
YELLOW | BLACK

and the code looks like this:

//Open the file, and copy the file into the buffer
        FILE * f;
        char * buffer;
        long size;
        f = fopen("test.txt","rt+");
        if( f == NULL) exit(1);

        fseek(f,0,SEEK_END);
        size = ftell(f);
        rewind(f);

        buffer = (char*) malloc(size);
        if(buffer == NULL) exit(2);

        fread(buffer, 1, size, f);
        fclose(f);

        //Now we're going to parse the file
        char * temp;
        String * Array = new String[5];
        temp = strtok(buffer,"|");
        RichEdit1->Lines->Add(temp);

unfortunately, the output in the RichEdit1 Box, is BLUE. Does anyone know why? or how i can fix this. If so please... PLEASE... respond. Im a bit stumped here.

Thanks,
Lance:?:

[Salem]

http://www.daniweb.com/techtalkforums/announcement8-3.html

Well for starters, fread() doesn't append a \0, so the thing you read isn't a proper string.

[Lance Wassing]

Well everyone, i have figured out a way of doing this sort of.

What i have done is created strings, and used the substring function within a huge for loop. This method works better than strtok, mainly because i can't figureout strtok... not quite, but the problem is that, the file must be hard coded intot he executable, but i want to read the file to a string array, and then use the substring function to parse.

I guess my question, and im sorry to ask such a novice question as i am a novice in c, how do i load my file, preferably with the fopen function of stdio.h and stdlib.h, int String arrays?

bassically i would like

String Lines = new String[i]
//Where i is the number of lines in the file
//load the file into the String array so that it //looks like this:
Lines[0] = "Lance , Wassing , 11 Wonderway";
/*
and so on then i can just loop through the array doing the Lines[i].SubString(int, int); function */

if you have something that will work for this purpose, or know i how i can do this, please let me know. Your help is MUCH appreciated!

Thankyou,
Lance

[WaltP]

fopen(), fgets(), fclose()