Trouble reading data input from vi editor

[JohnMcPherson]

Hello, this is a very strange problem. I had first put this discussion in the Linux and Unix thread, since I believe it is actually a problem with how the data is entered via but the vi editor, but I closed those out and will put it in the C thread, since it is being read with C code. Below is the code that is reading the data:

  strcpy(passw,"P7");
  fscanf(fip, "%3s", ckod); /* card # 164 */
  fprintf(ff16,"\n\n%12s%12s\n", "card# 164 :", "Password ");
  printf("passwd is %s\n", ckod);
  if(strcmp(passw,ckod)!=0)
  {
    printf("\n%64s\n%80s\n", "E R R O R !", "Password is incorrect. Please resubmit.");
    exit(164);
  }

  /***** Cards 165 - 174 *****/

  fgets(str, 82, fip);
  fprintf(ff16, "\nLIST OF SOLVENCY SURCHARGE RATES\n");
  fgets(str, 82, fip);
  if (sscanf(&str[1], "%d", &baseYearSurcharge) < 1)
  {
    fprintf(ff16, "Failed to correctly read the screen input for base year\n");
  }
  else
  {
    fprintf(ff16, "\nIS SURCHARGE APPLIED TO THE BASE YEAR ( YES = 1, NO = 0 ) = %d\n", baseYearSurcharge);
  }
  fgets(str, 82, fip);
  if (sscanf(&str[1], "%lf  %lf", &solvpctchkinitial, &solvpctchkmaximum) < 2)
  {
    fprintf(ff16, "Failed to correctly read the screen input for floor and ceiling\n");
  }
  else
  {
    fprintf(ff16, "\nINITIAL PERCENTAGE = %lf, MAXIMUM PERCENTAGE = %lf\n", solvpctchkinitial, solvpctchkmaximum);
    surchargeCheckPercent = solvpctchkinitial;
  }
  firstLpCtr = (knri + 1)/9.0;
  if ( firstLpCtr == 0 )
  {
     firstLpCtr = 1;
     readCnt = knri + 1;
  }
  else
  {
     readCnt = 9;
  }
  secondLpCtr = fmod((knri + 1), 9.0);
  for(i=1; i <= firstLpCtr; i++)
  {
    fgets( str, 82, fip);
    for ( kk = 1; kk <= readCnt; kk++ )
    {
      if (sscanf(&str[fieldPos], "%lf", &solvsurchgrt[jj-1]) < 1)
      {
        fprintf(ff16, "Failed to correctly read the screen input for solvency surcharges\n");
      }
      else
      {
        fprintf(ff16, "\n SOLVENCY SURCHARGE RATE = %lf\n", solvsurchgrt[jj-1]);
        fprintf(ff16, "The value of jj is %d\n", jj);
      }
      jj++;
      fieldPos += 7;
    }
    fieldPos = 1;
  }
  if ( secondLpCtr > 0 )
  {
    fgets(str, 82, fip);
    for ( kk = 1; kk <= secondLpCtr; kk++)
    {
      if (sscanf(&str[fieldPos], "%lf",&solvsurchgrt[jj-1]) < 1)
      {
        fprintf(ff16, "Failed to correctly read the screen input for solvency surcharges\n");
      }
      else
      {
        fprintf(ff16, "\n SOLVENCY SURCHARGE RATE = %lf\n", solvsurchgrt[jj-1]);
        fprintf(ff16, "The value of jj is %d\n", jj);
      }
      jj++;
      fieldPos += 7;
    }
  }
  fprintf(ff16, "\n");

This code is reading from a data file that has much more data, including the knri value this code is using to set the loop counters. I have verified that knri is 24 in this file, knri + 1 will be 25. I have also verified that knri can never be 0, so there is no need to handle the 0 case in the code. The maximum knri + 1 can be is 81. Below is the data in my file this code is reading:

P7
1
0.005 0.007
3.150 3.050 2.900 2.850 2.700 2.600 2.550 2.450 2.300
2.150 2.050 1.900 1.800 1.650 1.500 1.400 1.250 1.150
1.050 1.000 0.900 0.850 0.750 0.700 0.650 0.000 0.000
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000

Reading this data with the above code gives the following result:

card# 164 : Password

LIST OF SOLVENCY SURCHARGE RATES

IS SURCHARGE APPLIED TO THE BASE YEAR ( YES = 1, NO = 0 ) = 1

INITIAL PERCENTAGE = 0.005000, MAXIMUM PERCENTAGE = 0.007000

SOLVENCY SURCHARGE RATE = 3.150000
The value of jj is 1

SOLVENCY SURCHARGE RATE = 3.050000
The value of jj is 2

SOLVENCY SURCHARGE RATE = 2.900000
The value of jj is 3

SOLVENCY SURCHARGE RATE = 2.850000
The value of jj is 4

SOLVENCY SURCHARGE RATE = 0.000000
The value of jj is 5

SOLVENCY SURCHARGE RATE = 0.000000
The value of jj is 6

SOLVENCY SURCHARGE RATE = 600.000000
The value of jj is 7

SOLVENCY SURCHARGE RATE = 0.550000
The value of jj is 8

SOLVENCY SURCHARGE RATE = 2.450000
The value of jj is 9

SOLVENCY SURCHARGE RATE = 3.150000
The value of jj is 10

SOLVENCY SURCHARGE RATE = 3.050000
The value of jj is 11

SOLVENCY SURCHARGE RATE = 2.900000
The value of jj is 12

SOLVENCY SURCHARGE RATE = 2.850000
The value of jj is 13

SOLVENCY SURCHARGE RATE = 0.000000
The value of jj is 14

SOLVENCY SURCHARGE RATE = 0.000000
The value of jj is 15

SOLVENCY SURCHARGE RATE = 600.000000
The value of jj is 16

SOLVENCY SURCHARGE RATE = 0.550000
The value of jj is 17

SOLVENCY SURCHARGE RATE = 2.450000
The value of jj is 18

SOLVENCY SURCHARGE RATE = 3.150000
The value of jj is 19

SOLVENCY SURCHARGE RATE = 3.050000
The value of jj is 20

SOLVENCY SURCHARGE RATE = 2.900000
The value of jj is 21

SOLVENCY SURCHARGE RATE = 2.850000
The value of jj is 22

SOLVENCY SURCHARGE RATE = 0.000000
The value of jj is 23

SOLVENCY SURCHARGE RATE = 0.000000
The value of jj is 24

SOLVENCY SURCHARGE RATE = 600.000000
The value of jj is 25

The first line of output indiecates that the password was sucessfully read. Note that the next two lines are read and processed without a problem, and then it starts to go haywire. Moving the sscanf pointer by seven should put it at the start of the next field, and does for the first four fields in the line. Why it gets lost in the remaining five fields, I do not understand. Also, fgets starts reading the same line over and over, and I don't know why that is happening. I still feel that there is something wrong with how the data is entered in the vi editor. Note that it has has been verified that the lines are space filled between the fields, the lines are 80 columns long( space filled ), and there is a line feed at the end of each line. Any help with this is greatly appreciated.

[rubberman]

I think you posted this question previously. What responses did you get?

[Nutster]

I see that you included the data validation I recommended last time. Good.

The editor you used to create the file does not appear to be the problem because your data file looks reasonable for the situation.

Why are you reading the string starting at the second character with sscanf(&str[1],? Typically, I would read from the first character (index 0) unless there is a specific reason to skip the first character, using sscanf(str,.

In line 25, use only 1 space between the fields, or even none. The %lf will skip leading spaces.

What are the definitions, use and expected values of knri, firstLpCtr and secondLpCtr?

[JohnMcPherson]

 Hello, Rubberman and Nutster, this is the same question I had posted on the Linux and Unix discussion thread( accidentally twice, sorry about that). I was getting mostly C responses back, so I decided to close it out on that thread and reopen it here.
 The values being read in from this file are tax rates, and the variable knri is the number of rates, minus 1, to read in. That is why you see knri + 1 as the value to set the loop counters. For this particular file, knri is 24, meaning 25 tax rates, but that is a value that can be changed by the user. Therefore, I have to handle different loop counter values, so I decided to put nine fields per line, and have nine lines, to handle the maximum of 81 possible tax rates. The fields that are not used if there are less than 81 rates are set to zero. That is why I have firstLpCtr and secondLpCtr. I set firstLpCtr to the the number of lines that have tax rates in all nine lines, and secondLpCtr to the remainder, so the fields in the last line with rates, less than nine, is processed. If the number of rates is divisible by nine ( say 27), secondLpCtr will be zero.
 In the sscanf lines, I am using using &str[subscript] instead of just str to control precisely where the file position pointer is placed. I put the spaces in the data fore readability, and carefully set the number of spaces so I could code to that spacing. That is wht you see str[1], since I put a space before the first data field. Each field is five characters followed by two spaces, so I add seven in each pass through the loop, so the pointer is at the start of the next field. If I just use str, it also does not run right. It again gives me the first two lines correctly, but in the nine field lines, it just reads the first field over and over.
 Thanks again for looking at this and helping me with this problem. 

[Nutster]

What type are firstLpCtr and secondLpCtr? It looks like they could be integer, but you are using operations with them that return (and assign) double.

If you specify two spaces in the [f/s]scanf format, but only provide 1 space, then the second value will not be read. All the numeric patterns will skip over any leading whitespace to read the number.

I think you are making this more complicated than it needs to be. You always need to read 9 lines, so read 9 lines. In each line there are 9 values, so parse 9 times by calling sscanf 9 times for each line. The fields after knri are not being used, but are still taking up space, so stop parsing after you have reached knri, but keep reading from the file.

/* Read tax rates from the file. */
/* Always read 9 lines, with 9 fields each, but only the first (knri) are valid. */
for (iLine = 0; iLine < 9; ++iLine)
{
    /* Read the next line into str, an array at least 81 characters in size. */
    fgets(str, 82, fip);
    iOffset = 0;
    for (iField = 0; iField < 9; ++iField)
    {
        if (iLine * 9 + iField > knri)
            /* If we are past the last significant rate, stop processing, but keep reading. */
            /* 
            break;
        else if (sscanf(str + iOffset, "%lf%n", &solvsurchgrt[iLine * 9 + iField], &iNewOffset) < 1)
        {
            /* Alert the user that we have bad input in line (iLine + 1) */
            fprintf(ff16, "Unable to read a valid number at position %d of line %s.\n", iOffset+1, str);
            /* stop reading this line and try the next. */
            break;
        }
        else
        {
            /* Add on the length of the last field so that we start reading the next number after that */
            iOffset += iNewOffset;
        }
    }
}

First things first: I came up with this off the top of my head. It is not tested. It is likely that there are still bugs in it, even though I have gone through it a few times to remove bugs and add comments. Use with caution to help clean up your code.

Each of the variables I introduced that begins with 'i' is an int. %n in scanf gets the number of characters processed, so iNewOffset will get the length of the text that was read to produce the double. In other words, it will point to the beginning of the next field. Assigning to an %n variable does not count in the return value of sscanf().