Problems using fscanf with CSV files

[JohnMcPherson]

I am trying read CSV files in a C program on an AIX system. I am having problems getting fscanf to recognize the comma delimeter. I have read previous posts on this issue, and used the techniques dicussed in those threads, but I am still having the issue. Specifically, this problem was in a thread from a year ago, and the solution was to remove the comma using %c on integer/decimal fields, and adding a mask for string fields, i.e.,
%[^ \t\n\r\v\f,]%c. I have tried that, but I still am not getting the fscanf to work. It reads all of the line into the first variable, a string variable. I don't know if the problem is that I am running on AIX, and the solution I found is for Linux, or if the problem is that my input CSV file does not have any whitespace, all of the fields are separated only by commas. Anyone have any ideas on what the issue is?

[deceptikon]

Please post a small program that exhibits the problem and a sample of your CSV file. scanf() format strings can be tricky, and scanf() is usually unsuitable for true CSV, but if it's just a naive comma separated format without embedded commas, you can get away with using scanf().

[JohnMcPherson]

Sample CSV file:

HDR,1,ACC,ES,427,FRB427ESACC20120731020000,31/07/2012,2:00:00,[e-mail]

Note that this is a test program to test the fscanf concept, to use in the actual program I am working on. Some items in the test program are commented out as I was trying different ideas and fixes as I worked.

Here is the program:

/*************** INCLUDES ****************/

#include    <stdio.h>
#include    <stdlib.h>
#include    <string.h>

/************** DEFINES *****************/

#ifdef  NULLTERM
    #undef NULLTERM
#endif

#define NULLTERM(A) A##.arr[A##.len] = '\0'

#define TRUE  0
#define FALSE 1

/************** GLOBALS ****************/

char *fileHeaderField[5];
long recordCount;
char *fileType[6];
char *countryCode[4];
char *partnerCode[5];
char *fileName[32];
char *createDate[12];
char *createTime[10];
char *operatorID[52];
char *dummyVal1[2];
char *dummyVal2[2];
char *dummyVal3[2];
char *dummyVal4[2];
char *dummyVal5[2];
char *dummyVal6[2];
/* char fDelm[2]; */

/**************************************/

int main(int argc, char *argv[])
{
    char log_msg[80];
    FILE *fp;

 sprintf(log_msg, " There are [%i] parameters. \n", argc);
    fprintf( stdout, "%s\n", log_msg);

    /* Validate input parameter count. */

    if (argc != 2)
    {
        strcpy(log_msg, "Invalid argument count.\n");
        fprintf(stderr, "%s\n", log_msg );
        strcpy(log_msg, "           1) address of CSV file to be read.\n");
        fprintf(stderr, "%s\n", log_msg );
        strcpy(log_msg, "           2) delimeter between fields in the record.\n");
        fprintf(stderr, "%s\n", log_msg );

        return(FALSE);
    }
    /* strcpy(fDelm, argv[2]); */
    if ( (fp = fopen(argv[1], "r")) == NULL)
    {
        strcpy(log_msg,"failed to open the input file");
        fprintf(stderr, "%s\n", log_msg);
        return(FALSE);
    }

    fscanf( fp, "%s%*c %ld%*c %si%*c %s%*c %s%*c %s%*c %s%*c %s%*c %s%*c %s%*c %s%*c %s%*c %s%*c %s%*c %s",
                fileHeaderField, &recordCount, fileType, countryCode, partnerCode,
                fileName, createDate, createTime, operatorID, dummyVal1, dummyVal2, dummyVal3, dummyVal4, dummyVal5, dummyVal6);

    if ( strcmp(fileHeaderField, "HDR") != 0)
    {
         sprintf(log_msg,"First record of file read is not a header record, value read is %s.", fileHeaderField);
         fprintf(stderr, "%s\n", log_msg);
         return(FALSE);
    }
    else
    {
         sprintf(log_msg,"Successfully read the HDR field of the record, with value %s.", fileHeaderField);
         fprintf(stdout, "%s\n", log_msg);
    }
    if ( recordCount == NULL || recordCount == 0)
    {
        sprintf(log_msg,"Record count of file read is not a valid value, value read is %ld.", recordCount);
        fprintf(stderr, "%s\n", log_msg);
 return(FALSE);
    }
    else
    {
        sprintf(log_msg,"Successfully read the record count field of file with value of %ld.", recordCount);
        fprintf(stdout, "%s\n", log_msg);
    }
    if ( strcmp(fileType, "ACC") != 0)
    {
        sprintf(log_msg,"File read is not an ACC file, value read is %s.", fileType);
        fprintf(stderr, "%s\n", log_msg);
        return(FALSE);
    }
    else
    {
        sprintf(log_msg,"Successfully read the ACC field of the record, with value %s.", fileType);
        fprintf(stdout, "%s\n", log_msg);
    }

    fclose(fp);
    return(TRUE);

}

[deceptikon]

You're using the %s specifier, which is whitespace delimited. It won't stop reading when it sees a comma. Try this:

#include <stdio.h>

int main(void)
{
    char buf[BUFSIZ];

    while (scanf("%[^,\n]%*c", buf) == 1)
        printf("'%s'\n", buf);

    return 0;
}

[JohnMcPherson]

Thank you, that worked just fine. I do now get compile warnings when I use strcmp against the parsed value( now not defined as a string), but the code works, and I can live with the warnings.

[JohnMcPherson]

Never mind about the warnings, those are now fixed too. Thanks again for your help!