Memory Problem After a certain limit

Question

Dream2code 12 Junior Poster

14 Years Ago

Hi All,

My requirement is like this.It a part of a software devlopment team as i am devloping a tool its a part of it.
Its bit long but my explanation is clear to get it.

1)we need to read a data file using a description file.
Usage: programname <description_file> <data_file>
2)Then need to dump the datafile on the screen according to the description file.

Detailed descritpin is given below:

Thanks,
DP

Description file format:
========================

begin
columname datatype("delimeter");
columname datatype("delimeter");
.
.
.
.
end;

EXAMPLE:
--------

begin
name string("|");
age string("|");
sex string("\n");
end;

Datafile Sample:
===============
Deepak|23|M
puja|21|f
|XY|

OUTPUT WILL BE:
================

Record_1:

name:"Deepak"
age:"23"
sex:"M"

Record_2:

name:"puja:
age:"21"
sex:"F"

Record_3:

name:""
age:"XY"
sex:""

I think the picture is clear now what i am doing.

now i am telling about the important functions and structures i am using.

A) The below structure is used to describe the Description file.

typedef struct dml_s
{
char **col_headers;
char **column_delim;
int *col_datatype;
int *col_len;
int col_cnt;
int record_length;
int file_type; //file type can be fixed,delimeted or max len with delimeters
}dml;

concentrate on 3 columns :
===========================

char **col_headers; it will store the column headers from the description file.
char **column_delim; it will store the column delimeters from the description file.
int col_cnt; it will store the column count from the description file.

B) dml* extract_dml_info(char* description_fname,dml *record);

this function will read the description file and store the necessary information in the variables.

C) void read_delimeted(char *data_filename,char **delmimeter,char **header,int col_cnt);

now this fuction will take the data file name along with the data retured by the above function.
and displays data on the screen as per the output format.

++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++PROBLEM++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++

1)When we pass hard coded values to the fucntion

void read_delimeted(char *data_filename,char **delmeter,char **header,int col_cnt);

it works fine for any number of columns and any size of data file

e.g
char *delmimeter[]={"name","age","sex");
char *delimeter[]={"|","|","\n"};

2)When we pass values returned from extract_dml_info(char* description_fname,dml *record);

to the fucntion read_delimeted(char *data_filename,char **delmeter,char **header,int col_cnt);

it works upto 10 columns.

when the column name exceeds 10

it shows "segmentation fault"

ANALYSIS:
==========
A)
if error in

read_delimeted(char *data_filename,char **delmeter,char **header,int col_cnt);

then it should not have worked for hard coded values.

B)if error in extract_dml_info(char* description_fname,dml *record);

then it should not print datas extrated beyond 10 columns but it prints data for any number of
columns.

but as a whole

program fails if the column number from descrition file exceeds 10 columns.

c

8 Contributors
46 Replies
429 Views
2 Days Discussion Span
Latest Post 14 Years Ago Latest Post by Dave Sinkula

All 46 Replies

jephthah 1,888 Posting Maven

14 Years Ago

use code tags correctly. and make sure your code is indented. otherwise, it's a ridiculous mess, and no one will want to look at it.

int main(void)
{
    // hello, i am properly formatted C code.
    // aren't i nice?

    if (your_code == properly_formatted)
    {
        youWillGetResponse = TRUE;
    }
    else 
    {
        youWillGetResponse = NULL;
    }

    return toBeginning(andTryAgain);
}

Edited 11 Years Ago by happygeek because: fixed formatting

mvmalderen commented: Heh, nice code :P +15

Hiroshe 499 Posting Whiz in Training

14 Years Ago

First off, use astyle this one and only time to format your code. I compiled your code from post 5 (didn't know which code to use bacouse you reposted it instead of editing your original) and got 100's of errors and warnings. You need a tutorial on debugging. If your code somehow managed to work for you, but it wont work for more than 10 records, I'd guess that your eather using some undifined behaver or you are going out of the range of the memory you allocated. That or you typed the input files wrong.
EDIT: You are calling string functions that don't exist You forgot to include <string.h>. It seems there is something wrong on line 527. Also it looks like your looping without checking if fgets != NULL. I bet you fgetting a line that dosent exist.
Good luck with your program.

Reply to this topic

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.

Dream2code 12 Junior Poster · Answer 1 · 2009-07-13T22:20:44+00:00

Stand alone code is pasted below:
i have compiled it on gcc compiler on 64bit mode.

e.g gcc -o output prog.c -m64

code below
===========

#include<stdio.h>
#include<math.h>
#include<stdlib.h>

#define FIXED 1
#define DELIMETED 2
#define MIXED 3

int errno;

typedef struct dml_s
{
char **col_headers;
char **column_delim;
int *col_datatype;
int *col_len;
int col_cnt;
int record_length;
int file_type;          //file type can be fixed,delimeted or max len with delimeters
}dml;


void display_intro();
dml* extract_dml_info(char*,dml*);
int str_wc(char *);
int str_semicl(char *);
int pow_user(int,int);
void display_dml_info(struct dml_s *record);
void read_fixed(char *filename,int *size,char **header,int col_cnt,int record_len);
void read_delimeted(char *filename,char **delmimeter,char **header,int col_cnt);
char * ustrtok(char *base_string,char *next_str,char *delim);

int main(int argc,char *argv[])
{

if(argc!=3)
{
display_intro();
return;
}


dml *record;
record=(dml*)malloc(sizeof(dml));
record=extract_dml_info(argv[1],record);
if(record==NULL) exit(1);

display_dml_info(record);


if(record->file_type==DELIMETED)
{

read_delimeted(argv[2],record->column_delim,record->col_headers,record->col_cnt);
}


/*
else if(record->file_type=FIXED)
{
read_fixed(argv[2],record->col_len,record->col_headers,record->col_cnt,record->record_length);
}

else if(record->file_type=MIXED)
{
printf("\n\nCONSTRUCTION ON PROGRESS FOR DELIMTED FILE WITH LENGTH\n");
return;
}
*/

return 0;
}





/********************THIS FUNCTION EXTRACTS INFORMATION FROM THE
*********************DML FILE AND RETURNS A STRUCTRUE***********/

dml* extract_dml_info(char *dmlfile,dml *record)
{
char buffer[100];       //IT BUFFERS ONE LINE AT A TIME FORM THE FILE
char **line;            //ALL THE LINES FROM THE FILE STORED IN LINE ARRAY
char **col_desc;        //STORES THE COLUMN DESCRIPTIONS
char *temp,*token;
FILE *fp;
int i,j;

fp=fopen(dmlfile,"r");
if(fp==NULL)
{
printf("Can't read from file: \"%s\"\n",dmlfile);
perror("Error");
exit(1);
}

record=(dml*)malloc(20*sizeof(dml));
record->col_headers=(char**)malloc(200*sizeof(char*));
record->column_delim=(char**)malloc(200*sizeof(char*));
col_desc=(char**)malloc(200*sizeof(char*));
line=(char**)malloc(200*sizeof(char*));

fgets(buffer,100,fp);
i=0;
while(!feof(fp))
{
if(buffer[strlen(buffer)-1]=='\n')
buffer[strlen(buffer)-1]='\0';
line[i]=(char*)malloc(strlen(buffer)+1);
strcpy(line[i],buffer);
i++;
fgets(buffer,100,fp);
}

record->col_cnt=i-2;                    //Getting the column count from the dml file



/****************DML VALIDATION***************************************
The following validation taken place
1)DML starts with a 'begin' flag and end with 'end;'
2)Every statement ends with a semicolon except the begin
3)no semicolon should encounter inside
4)each line should consist of 2 words (a)column name (b)column desc
*********************************************************************/

if(strcmp(line[0],"begin")!=0)
{
printf("Dml Error: at line 1\n");
}

int k=0;
int w_count,semicl_flag;
for(j=1;j<i;j++)
{
k=strlen(line[j]);


if(line[j][k-1]!=';')
{
printf("Dml error: Missing ; at line %d\n",j+1);
return NULL;
}
if((strchr(line[j],';'))!=NULL && line[j][k-1]!=';' )
{
printf("Dml error: at line %d\n",j+1);
return NULL;
}


w_count=str_wc(line[j]);
if( w_count != 2 && w_count != 3 && j!=i-1 )
{
printf("Dml Error xxx at line %d\n",j);
return NULL;
}

semicl_flag=str_semicl(line[j]);
if(semicl_flag==-1)
{
printf("Dml Error yyy at line %d\n",j);
return NULL;
}
else if(semicl_flag==-2)
{
printf("Dml Error zzz at line %d\n",j);
return NULL;
}
}

if(strcmp(line[i-1],"end;")!=0)
{
printf("Dml Error: at line %d\n",i);
return NULL;
}

/****************END DML VALIDATION***********************/


for(j=0;j<record->col_cnt;j++)
{
token=(char*)malloc(strlen(line[j])+1);
token=strtok(line[j+1]," ");
if(token==NULL)
{
token="\0";
return NULL;
}
record->col_headers[j]=(char*)malloc(strlen(token)+1);
strcpy(record->col_headers[j],token);
token=strtok(NULL,"\n");
col_desc[j]=(char*)malloc(strlen(token)+1);
if(token!=NULL)
strcpy(col_desc[j],token);
else
{
printf("Incorrect DML at line %d for %s\n",j+1,record->col_headers[j]);
return NULL;
}

}



/****************DETERMINE THE DATATYPE OF THE COLUMNS*********/
char *del;
record->col_datatype=(int*)malloc(2*i*sizeof(int));
int flag;
for(j=0;j<record->col_cnt;j++)
{
del=strstr(col_desc[j],"string");
if(del==NULL)
{
record->col_datatype[j]=1;
del='\0';
}
else
record->col_datatype[j]=0;
}
/*****************************************************************/


int count_fixed=0;
int count_del=0;
char *temp_del_fst;
char *temp_del_lst;
int tempv;
record->column_delim=(char**)malloc(10*sizeof(char*));
for(j=0;j<record->col_cnt;j++)
{
temp_del_fst=strchr(col_desc[j],'"');
temp_del_lst=strrchr(col_desc[j],'"');
if(temp_del_fst==NULL && temp_del_lst==NULL)
{
count_fixed++;
record->column_delim[j]=(char*)malloc(2);
record->column_delim[j]="\0";
}
else if(temp_del_fst!=NULL && temp_del_lst!=NULL && temp_del_lst!=temp_del_fst)
{
count_del++;
tempv=temp_del_lst-temp_del_fst;
record->column_delim[j]=(char*)malloc(temp_del_lst-temp_del_fst);
strncpy(record->column_delim[j],temp_del_fst+1,temp_del_lst-temp_del_fst-1);
}
else if(temp_del_fst!=NULL && temp_del_lst!=NULL && temp_del_lst==temp_del_fst)
{
printf("DML Error :at line %d ,inproper description\n",j+1);
record->column_delim[j]="\0";
return  NULL;
}
}



//printf("count fixed:%d and count del :%d \n",count_fixed,count_del);
/***********/

record->record_length=0;
char *c_len;
if(count_fixed==record->col_cnt && count_del==0)
{
 record->file_type=FIXED;
 record->col_len=(int *)malloc(record->col_cnt*sizeof(int));
 for(j=0;j<record->col_cnt;j++)
 {
temp_del_fst=strchr(col_desc[j],'(');
temp_del_lst=strrchr(col_desc[j],')');
  if(temp_del_fst==NULL && temp_del_lst==NULL)
    {
     printf("Dml Error..( ) missing in column description\n");
    }
    tempv=temp_del_lst-temp_del_fst;

        c_len=(char*)malloc(10*sizeof(char));
    strncpy(c_len,temp_del_fst+1,temp_del_lst-temp_del_fst-1);

        int len=strlen(c_len);

        record->col_len[j]=atoi(c_len);
        record->record_length=record->record_length+record->col_len[j];
       // printf("length:%d\tclen:%s\tlen:%d\t j=%d\n",record->col_len[j],c_len,tempv,j);

  }

        //printf("record length is :%d\n",record->record_length);

}


/**********/
else if(count_fixed==0 && count_del==record->col_cnt)
{
record->file_type=DELIMETED;

  if(strcmp(record->column_delim[record->col_cnt-1],"\\n")!=0)
  {
    printf("DML error,in the last field '\\n' missing\n");
   return NULL;
  }
}
else
{
record->file_type=-1;
printf("DML error,Unable to determine the file type\n");
return NULL;
}

return record;
}


/******A function to get the word count of a string*******/

int str_wc(char *str)
{
int limit = strlen(str);
    int i, change = 1, words = 0;
    for(i = 0; i < limit; ++i)
    {
        if(!isspace(str[i]))
        {
            if(change)
            {
                ++words;
                change = 0;
            }
        }
        else
        {
            change = 1;
        }
            }

  return words;
}

/*************************************************************/

/*********A fucntion to check semicolon occurs in between the line***/

int str_semicl(char *str)
{
int limit = strlen(str);
int i,flag=0;

    for(i = 0; i < limit; ++i)
    {
        if(i < limit-1 && str[i]==';' )
        {
                 flag=-1;
        }

        else if( str[limit-1]==';' && isspace(str[limit-2]) )
        {
                flag=-2;
        }
        else if( str[limit-1]==';' && !(isspace(str[limit-2])))
        {
                flag=0;
        }

        else
                flag=0;

   }

  return flag;
}

/************************************************************/

/******CALCULATION POWER****************/
int pow_user(base,exp)
{
int i;
int pow=1;
for(i=0;i<exp;i++)
{
pow=pow*base;
}
return pow;
}
/****************************************/


/******IT'S A FUNCTION SIMILAR TO STRTOK BUT CAN HANDLE NULL VALUES******/
/******THIS FUNCTION SPILTS A STRING INTO TWO STRINGS W.R.T A TOKEN******/

char * ustrtok(char *base_string,char *next_str,char *delim)
{
char *str2;
str2=(char*)malloc(100);
if(base_string==NULL && strlen(base_string)<1)
{
next_str=NULL;
return NULL;
}
int len=strlen(base_string);
char *temp;
temp=(char*)malloc(100);

temp=strstr(base_string,delim);
if(temp==NULL)
{
strcpy(next_str,base_string);
next_str[strlen(next_str)]='\0';
return NULL;
}
else
{
int sub_len=temp-base_string;
str2=(char*)malloc(100);
strncpy(str2,base_string,sub_len);
str2[strlen(str2)]='\0';
strcpy(next_str,str2);
return (temp+strlen(delim));
}
}



/******************THIS FUNCTION DISPLAYS THE DML DESCRIPTION****************/


void display_dml_info(dml *record)
{

int i,j;
if(record==NULL)return;
printf("=========================\n");
printf("NO OF COLUMNS:%d\n",record->col_cnt);
printf("RECORD LENGTH:%d\n",record->record_length);
printf("FILE TYPE :\"%d\"\n",record->file_type);
printf("=========================\n");

for(j=0;j<record->col_cnt;j++)
{
printf("===COLUMN %d===\n\n",j+1);
printf("HEADER:\"%s\"\n",record->col_headers[j]);
printf("DATATYPE:\"%d\"\n",record->col_datatype[j] );
if(record->file_type==DELIMETED)printf("DELIMETER:\"%s\"\n",record->column_delim[j]);
if(record->file_type==FIXED )printf("LENGTH:%d\n",record->col_len[j]);
}
printf("=========================\n\n");
}

/***************************************************************************************/



/*****************************Begin read_fixed**************************************

THIS FUNCTION READS A FIXED LENGTH FILE AS PER THE SPECIFED DML STEPS ARE AS FOLLOWS:

INPUT PARAMETER:
A) File Name
B) Column Headers
C) Length of all the Columns
D) No Of columns
E) Lenggh of Each Record

OUTPUT:

Reads the file as per the input parametes and Display it on the Screen

*********************************************************************************/


void read_fixed(char *filename,int *size,char **header,int col_cnt,int record_len)
{
  FILE *file_read;
  long lSize;
  char * buffer;
  size_t result;
  fpos_t position;


  file_read=fopen(filename,"rb");
  if(file_read==NULL)
  {
   printf("Can't read from file \"%s\"\n",filename);
   perror("Error");
   exit(1);
  }


fseek(file_read,0,SEEK_END);
lSize=ftell(file_read);
rewind(file_read);

int j=0;
int i=0;
float record_count=lSize/record_len;
int multiply=record_count*record_len;

buffer=(char*)malloc(4);
result=fread(buffer,1,size[0],file_read);
buffer[size[0]]='\0';
rewind(file_read);
while(!feof(file_read) || buffer==NULL )
{
if(j>=col_cnt)j=0;
if(j==0 && i<=record_count)
{
i++;
}
if(j==0 && i<=record_count)
{
printf("Record %d\n",i);
printf("=============\n");
}

buffer=(char*)malloc(10);
result=fread(buffer,1,size[j],file_read);
buffer[size[j]]='\0';
if(i<=record_count)
printf("%s:\"%s\"\n",header[j],buffer);
free(buffer);
j++;
}

/*setting up the file pointer to get the leftover bytes*/
fgetpos (file_read, &position);
position=(fpos_t)multiply;
fsetpos (file_read, &position);
/******************************************************/
if(lSize!=multiply)
{

char *left_over;
left_over=(char*)malloc(record_len+1);
result=fread(left_over,1,record_len,file_read);
left_over[record_len]='\0';
printf("\n\n\nLeft Over Bytes:\"%s\"\n",left_over);
free(left_over);

printf("\nError:Incomplete record at end of file.\n");
printf("Either (1) The metadata does not accurately describe the data,\n");
printf("    or (2) The data has somehow been corrupted.\n");
}
}

/*********************************END read_fixed ****************************/



void read_delimeted(char *filename,char **column_delim,char **col_headers,int col_cnt)
{
FILE *data_file;
char buffer[20000];
char *ext_line=NULL;
char *temp=NULL;
char **fields=NULL;
int i,j;

printf("filename:%s\n",filename);
printf("columncount:%d\n",col_cnt);
for(i=0;i<col_cnt;i++)
{
printf("del:%s\n",column_delim[i]);
printf("header:%s\n",col_headers[i]);
}






data_file=fopen(filename,"r");
if(data_file==NULL)
{
printf("can't open file\n");
return;
}

printf("TESTING BREAK1\n");

j=1;
fgets(buffer,2000,data_file);
while(!feof(data_file))
{
i=0;

if(buffer[strlen(buffer)-1]=='\n')
buffer[strlen(buffer)-1]='\0';
if(strlen(buffer)==0)
goto nextline;
ext_line=(char*)malloc(strlen(buffer)+1);
if(ext_line==NULL)
{
printf("NUll Record Found\n");
}
strcpy(ext_line,buffer);
printf("\nRecord %d:\n",j);
printf("=========\n");

fields=(char**)malloc(col_cnt*sizeof(char*));
char *next_str;
temp=(char*)malloc(strlen(buffer)+1);
next_str=(char*)malloc(strlen(buffer)+1);
next_str=ustrtok(ext_line,temp,column_delim[0]);
while(i<col_cnt)
{
if(next_str==NULL)
next_str="\0";
if(ext_line==NULL)
ext_line="\0";
if(temp==NULL)
temp="\0";


fields[i]=(char*)malloc(strlen(temp));
strcpy(fields[i],temp);

fprintf(stdout,"%s:\"%s\"\n",col_headers[i],fields[i]);
ext_line=next_str;
free(temp);
temp=(char*)malloc(100*sizeof(char));
if(i==col_cnt-1)
{
column_delim[i+1]=column_delim[i];
}
next_str=ustrtok(ext_line,temp,column_delim[i+1]);


if(i==col_cnt-2)
{
if(ext_line==NULL)
ext_line="\0";
strcpy(temp,ext_line);
}

if(i==col_cnt-1 && strlen(ext_line)>0)
{
printf("\n\nError:Incomplete field at end of record.\n");
printf("Leftover bytes :\"%s\"\n",ext_line);
printf("Either (1) The metadata does not accurately describe the data,\n");
printf("    or (2) The data has somehow been corrupted.\n");
}
i++;
}
j++;
nextline:
fgets(buffer,2000,data_file);
free(fields);
}
fclose(data_file);
}



void display_intro()
{
 printf("\n mdump: Version 1.0\n\n");
 printf(" Copyright (c) 2009, Deepak Panigrahy.  All rights reserved.\n\n");
 printf(" Purpose: This program is a simple utility that display\n");
 printf("          the source file as per the Record format.\n\n");

 printf(" Usage:   mdump <Dml FileName> <Data FileName>\n\n");

 printf("          Use -h option for additional help.\n");
}

Salem 5,138 Posting Sage · Answer 2 · 2009-07-13T22:25:51+00:00

http://www.daniweb.com/forums/announcement118-3.html
http://www.daniweb.com/forums/announcement118-3.html
http://www.daniweb.com/forums/announcement118-3.html
http://www.daniweb.com/forums/announcement118-3.html
http://www.daniweb.com/forums/announcement118-3.html
You already missed at least FIVE places where the board asks (nay, implores) you to use code tags, but you missed them all.

Try editing your posts again, and this time see if you can make them more readable.
Check out some other posts on the forum for readability, and make sure yours is just as good.

Dream2code 12 Junior Poster · Answer 3 · 2009-07-13T22:27:28+00:00

for more detail:
================

instread of strtok i have used my own advanced function ustrtok.

it works like this
==================

char * ustrtok(char *base_string,char *next_str,char *delim);

return=ustrtok(char *base_string,char *next_str,char *delim);

basesting= a|b|c|d
next_str=NULL
delim=|

after one call
===============
next_str=a
return=b|c|d

then we need to assing

base_string=return for further calls.

note:
1)No issue due to this function its only a help to understand the code.
2)i have tried my besr from my side.please look into this.
3)i have provided the sample data u can run the code very easily to see the output.
4)i suggest run the code with different type of descrition files and data files
change the columns numbers and all then u will observe the failure or else it works fine.

Dream2code 12 Junior Poster · Answer 4 · 2009-07-13T22:30:15+00:00

sory friend i appolize for this.okie i am posting my code again.

#include<stdio.h>
#include<math.h>
#include<stdlib.h>

#define FIXED 1
#define DELIMETED 2
#define MIXED 3

int errno;

typedef struct dml_s
{
char **col_headers;
char **column_delim;
int *col_datatype;
int *col_len;
int col_cnt;
int record_length;
int file_type;          //file type can be fixed,delimeted or max len with delimeters
}dml;


void display_intro();
dml* extract_dml_info(char*,dml*);
int str_wc(char *);
int str_semicl(char *);
int pow_user(int,int);
void display_dml_info(struct dml_s *record);
void read_fixed(char *filename,int *size,char **header,int col_cnt,int record_len);
void read_delimeted(char *filename,char **delmimeter,char **header,int col_cnt);
char * ustrtok(char *base_string,char *next_str,char *delim);

int main(int argc,char *argv[])
{

if(argc!=3)
{
display_intro();
return;
}


dml *record;
record=(dml*)malloc(sizeof(dml));
record=extract_dml_info(argv[1],record);
if(record==NULL) exit(1);

display_dml_info(record);


if(record->file_type==DELIMETED)
{

read_delimeted(argv[2],record->column_delim,record->col_headers,record->col_cnt);
}


/*
else if(record->file_type=FIXED)
{
read_fixed(argv[2],record->col_len,record->col_headers,record->col_cnt,record->record_length);
}

else if(record->file_type=MIXED)
{
printf("\n\nCONSTRUCTION ON PROGRESS FOR DELIMTED FILE WITH LENGTH\n");
return;
}
*/

return 0;
}





/********************THIS FUNCTION EXTRACTS INFORMATION FROM THE
*********************DML FILE AND RETURNS A STRUCTRUE***********/

dml* extract_dml_info(char *dmlfile,dml *record)
{
char buffer[100];       //IT BUFFERS ONE LINE AT A TIME FORM THE FILE
char **line;            //ALL THE LINES FROM THE FILE STORED IN LINE ARRAY
char **col_desc;        //STORES THE COLUMN DESCRIPTIONS
char *temp,*token;
FILE *fp;
int i,j;

fp=fopen(dmlfile,"r");
if(fp==NULL)
{
printf("Can't read from file: \"%s\"\n",dmlfile);
perror("Error");
exit(1);
}

record=(dml*)malloc(20*sizeof(dml));
record->col_headers=(char**)malloc(200*sizeof(char*));
record->column_delim=(char**)malloc(200*sizeof(char*));
col_desc=(char**)malloc(200*sizeof(char*));
line=(char**)malloc(200*sizeof(char*));

fgets(buffer,100,fp);
i=0;
while(!feof(fp))
{
if(buffer[strlen(buffer)-1]=='\n')
buffer[strlen(buffer)-1]='\0';
line[i]=(char*)malloc(strlen(buffer)+1);
strcpy(line[i],buffer);
i++;
fgets(buffer,100,fp);
}

record->col_cnt=i-2;                    //Getting the column count from the dml file



/****************DML VALIDATION***************************************
The following validation taken place
1)DML starts with a 'begin' flag and end with 'end;'
2)Every statement ends with a semicolon except the begin
3)no semicolon should encounter inside
4)each line should consist of 2 words (a)column name (b)column desc
*********************************************************************/

if(strcmp(line[0],"begin")!=0)
{
printf("Dml Error: at line 1\n");
}

int k=0;
int w_count,semicl_flag;
for(j=1;j<i;j++)
{
k=strlen(line[j]);


if(line[j][k-1]!=';')
{
printf("Dml error: Missing ; at line %d\n",j+1);
return NULL;
}
if((strchr(line[j],';'))!=NULL && line[j][k-1]!=';' )
{
printf("Dml error: at line %d\n",j+1);
return NULL;
}


w_count=str_wc(line[j]);
if( w_count != 2 && w_count != 3 && j!=i-1 )
{
printf("Dml Error xxx at line %d\n",j);
return NULL;
}

semicl_flag=str_semicl(line[j]);
if(semicl_flag==-1)
{
printf("Dml Error yyy at line %d\n",j);
return NULL;
}
else if(semicl_flag==-2)
{
printf("Dml Error zzz at line %d\n",j);
return NULL;
}
}

if(strcmp(line[i-1],"end;")!=0)
{
printf("Dml Error: at line %d\n",i);
return NULL;
}

/****************END DML VALIDATION***********************/


for(j=0;j<record->col_cnt;j++)
{
token=(char*)malloc(strlen(line[j])+1);
token=strtok(line[j+1]," ");
if(token==NULL)
{
token="\0";
return NULL;
}
record->col_headers[j]=(char*)malloc(strlen(token)+1);
strcpy(record->col_headers[j],token);
token=strtok(NULL,"\n");
col_desc[j]=(char*)malloc(strlen(token)+1);
if(token!=NULL)
strcpy(col_desc[j],token);
else
{
printf("Incorrect DML at line %d for %s\n",j+1,record->col_headers[j]);
return NULL;
}

}



/****************DETERMINE THE DATATYPE OF THE COLUMNS*********/
char *del;
record->col_datatype=(int*)malloc(2*i*sizeof(int));
int flag;
for(j=0;j<record->col_cnt;j++)
{
del=strstr(col_desc[j],"string");
if(del==NULL)
{
record->col_datatype[j]=1;
del='\0';
}
else
record->col_datatype[j]=0;
}
/*****************************************************************/


int count_fixed=0;
int count_del=0;
char *temp_del_fst;
char *temp_del_lst;
int tempv;
record->column_delim=(char**)malloc(10*sizeof(char*));
for(j=0;j<record->col_cnt;j++)
{
temp_del_fst=strchr(col_desc[j],'"');
temp_del_lst=strrchr(col_desc[j],'"');
if(temp_del_fst==NULL && temp_del_lst==NULL)
{
count_fixed++;
record->column_delim[j]=(char*)malloc(2);
record->column_delim[j]="\0";
}
else if(temp_del_fst!=NULL && temp_del_lst!=NULL && temp_del_lst!=temp_del_fst)
{
count_del++;
tempv=temp_del_lst-temp_del_fst;
record->column_delim[j]=(char*)malloc(temp_del_lst-temp_del_fst);
strncpy(record->column_delim[j],temp_del_fst+1,temp_del_lst-temp_del_fst-1);
}
else if(temp_del_fst!=NULL && temp_del_lst!=NULL && temp_del_lst==temp_del_fst)
{
printf("DML Error :at line %d ,inproper description\n",j+1);
record->column_delim[j]="\0";
return  NULL;
}
}



//printf("count fixed:%d and count del :%d \n",count_fixed,count_del);
/***********/

record->record_length=0;
char *c_len;
if(count_fixed==record->col_cnt && count_del==0)
{
 record->file_type=FIXED;
 record->col_len=(int *)malloc(record->col_cnt*sizeof(int));
 for(j=0;j<record->col_cnt;j++)
 {
temp_del_fst=strchr(col_desc[j],'(');
temp_del_lst=strrchr(col_desc[j],')');
  if(temp_del_fst==NULL && temp_del_lst==NULL)
    {
     printf("Dml Error..( ) missing in column description\n");
    }
    tempv=temp_del_lst-temp_del_fst;

        c_len=(char*)malloc(10*sizeof(char));
    strncpy(c_len,temp_del_fst+1,temp_del_lst-temp_del_fst-1);

        int len=strlen(c_len);

        record->col_len[j]=atoi(c_len);
        record->record_length=record->record_length+record->col_len[j];
       // printf("length:%d\tclen:%s\tlen:%d\t j=%d\n",record->col_len[j],c_len,tempv,j);

  }

        //printf("record length is :%d\n",record->record_length);

}


/**********/
else if(count_fixed==0 && count_del==record->col_cnt)
{
record->file_type=DELIMETED;

  if(strcmp(record->column_delim[record->col_cnt-1],"\\n")!=0)
  {
    printf("DML error,in the last field '\\n' missing\n");
   return NULL;
  }
}
else
{
record->file_type=-1;
printf("DML error,Unable to determine the file type\n");
return NULL;
}

return record;
}


/******A function to get the word count of a string*******/

int str_wc(char *str)
{
int limit = strlen(str);
    int i, change = 1, words = 0;
    for(i = 0; i < limit; ++i)
    {
        if(!isspace(str[i]))
        {
            if(change)
            {
                ++words;
                change = 0;
            }
        }
        else
        {
            change = 1;
        }
            }

  return words;
}

/*************************************************************/

/*********A fucntion to check semicolon occurs in between the line***/

int str_semicl(char *str)
{
int limit = strlen(str);
int i,flag=0;

    for(i = 0; i < limit; ++i)
    {
        if(i < limit-1 && str[i]==';' )
        {
                 flag=-1;
        }

        else if( str[limit-1]==';' && isspace(str[limit-2]) )
        {
                flag=-2;
        }
        else if( str[limit-1]==';' && !(isspace(str[limit-2])))
        {
                flag=0;
        }

        else
                flag=0;

   }

  return flag;
}

/************************************************************/

/******CALCULATION POWER****************/
int pow_user(base,exp)
{
int i;
int pow=1;
for(i=0;i<exp;i++)
{
pow=pow*base;
}
return pow;
}
/****************************************/


/******IT'S A FUNCTION SIMILAR TO STRTOK BUT CAN HANDLE NULL VALUES******/
/******THIS FUNCTION SPILTS A STRING INTO TWO STRINGS W.R.T A TOKEN******/

char * ustrtok(char *base_string,char *next_str,char *delim)
{
char *str2;
str2=(char*)malloc(100);
if(base_string==NULL && strlen(base_string)<1)
{
next_str=NULL;
return NULL;
}
int len=strlen(base_string);
char *temp;
temp=(char*)malloc(100);

temp=strstr(base_string,delim);
if(temp==NULL)
{
strcpy(next_str,base_string);
next_str[strlen(next_str)]='\0';
return NULL;
}
else
{
int sub_len=temp-base_string;
str2=(char*)malloc(100);
strncpy(str2,base_string,sub_len);
str2[strlen(str2)]='\0';
strcpy(next_str,str2);
return (temp+strlen(delim));
}
}



/******************THIS FUNCTION DISPLAYS THE DML DESCRIPTION****************/


void display_dml_info(dml *record)
{

int i,j;
if(record==NULL)return;
printf("=========================\n");
printf("NO OF COLUMNS:%d\n",record->col_cnt);
printf("RECORD LENGTH:%d\n",record->record_length);
printf("FILE TYPE :\"%d\"\n",record->file_type);
printf("=========================\n");

for(j=0;j<record->col_cnt;j++)
{
printf("===COLUMN %d===\n\n",j+1);
printf("HEADER:\"%s\"\n",record->col_headers[j]);
printf("DATATYPE:\"%d\"\n",record->col_datatype[j] );
if(record->file_type==DELIMETED)printf("DELIMETER:\"%s\"\n",record->column_delim[j]);
if(record->file_type==FIXED )printf("LENGTH:%d\n",record->col_len[j]);
}
printf("=========================\n\n");
}

/***************************************************************************************/



/*****************************Begin read_fixed**************************************

THIS FUNCTION READS A FIXED LENGTH FILE AS PER THE SPECIFED DML STEPS ARE AS FOLLOWS:

INPUT PARAMETER:
A) File Name
B) Column Headers
C) Length of all the Columns
D) No Of columns
E) Lenggh of Each Record

OUTPUT:

Reads the file as per the input parametes and Display it on the Screen

*********************************************************************************/


void read_fixed(char *filename,int *size,char **header,int col_cnt,int record_len)
{
  FILE *file_read;
  long lSize;
  char * buffer;
  size_t result;
  fpos_t position;


  file_read=fopen(filename,"rb");
  if(file_read==NULL)
  {
   printf("Can't read from file \"%s\"\n",filename);
   perror("Error");
   exit(1);
  }


fseek(file_read,0,SEEK_END);
lSize=ftell(file_read);
rewind(file_read);

int j=0;
int i=0;
float record_count=lSize/record_len;
int multiply=record_count*record_len;

buffer=(char*)malloc(4);
result=fread(buffer,1,size[0],file_read);
buffer[size[0]]='\0';
rewind(file_read);
while(!feof(file_read) || buffer==NULL )
{
if(j>=col_cnt)j=0;
if(j==0 && i<=record_count)
{
i++;
}
if(j==0 && i<=record_count)
{
printf("Record %d\n",i);
printf("=============\n");
}

buffer=(char*)malloc(10);
result=fread(buffer,1,size[j],file_read);
buffer[size[j]]='\0';
if(i<=record_count)
printf("%s:\"%s\"\n",header[j],buffer);
free(buffer);
j++;
}

/*setting up the file pointer to get the leftover bytes*/
fgetpos (file_read, &position);
position=(fpos_t)multiply;
fsetpos (file_read, &position);
/******************************************************/
if(lSize!=multiply)
{

char *left_over;
left_over=(char*)malloc(record_len+1);
result=fread(left_over,1,record_len,file_read);
left_over[record_len]='\0';
printf("\n\n\nLeft Over Bytes:\"%s\"\n",left_over);
free(left_over);

printf("\nError:Incomplete record at end of file.\n");
printf("Either (1) The metadata does not accurately describe the data,\n");
printf("    or (2) The data has somehow been corrupted.\n");
}
}

/*********************************END read_fixed ****************************/



void read_delimeted(char *filename,char **column_delim,char **col_headers,int col_cnt)
{
FILE *data_file;
char buffer[20000];
char *ext_line=NULL;
char *temp=NULL;
char **fields=NULL;
int i,j;

printf("filename:%s\n",filename);
printf("columncount:%d\n",col_cnt);
for(i=0;i<col_cnt;i++)
{
printf("del:%s\n",column_delim[i]);
printf("header:%s\n",col_headers[i]);
}






data_file=fopen(filename,"r");
if(data_file==NULL)
{
printf("can't open file\n");
return;
}

printf("TESTING BREAK1\n");

j=1;
fgets(buffer,2000,data_file);
while(!feof(data_file))
{
i=0;

if(buffer[strlen(buffer)-1]=='\n')
buffer[strlen(buffer)-1]='\0';
if(strlen(buffer)==0)
goto nextline;
ext_line=(char*)malloc(strlen(buffer)+1);
if(ext_line==NULL)
{
printf("NUll Record Found\n");
}
strcpy(ext_line,buffer);
printf("\nRecord %d:\n",j);
printf("=========\n");

fields=(char**)malloc(col_cnt*sizeof(char*));
char *next_str;
temp=(char*)malloc(strlen(buffer)+1);
next_str=(char*)malloc(strlen(buffer)+1);
next_str=ustrtok(ext_line,temp,column_delim[0]);
while(i<col_cnt)
{
if(next_str==NULL)
next_str="\0";
if(ext_line==NULL)
ext_line="\0";
if(temp==NULL)
temp="\0";


fields[i]=(char*)malloc(strlen(temp));
strcpy(fields[i],temp);

fprintf(stdout,"%s:\"%s\"\n",col_headers[i],fields[i]);
ext_line=next_str;
free(temp);
temp=(char*)malloc(100*sizeof(char));
if(i==col_cnt-1)
{
column_delim[i+1]=column_delim[i];
}
next_str=ustrtok(ext_line,temp,column_delim[i+1]);


if(i==col_cnt-2)
{
if(ext_line==NULL)
ext_line="\0";
strcpy(temp,ext_line);
}

if(i==col_cnt-1 && strlen(ext_line)>0)
{
printf("\n\nError:Incomplete field at end of record.\n");
printf("Leftover bytes :\"%s\"\n",ext_line);
printf("Either (1) The metadata does not accurately describe the data,\n");
printf("    or (2) The data has somehow been corrupted.\n");
}
i++;
}
j++;
nextline:
fgets(buffer,2000,data_file);
free(fields);
}
fclose(data_file);
}



void display_intro()
{
 printf("\n mdump: Version 1.0\n\n");
 printf(" Copyright (c) 2009, Deepak Panigrahy.  All rights reserved.\n\n");
 printf(" Purpose: This program is a simple utility that display\n");
 printf("          the source file as per the Record format.\n\n");

 printf(" Usage:   mdump <Dml FileName> <Data FileName>\n\n");

 printf("          Use -h option for additional help.\n");
}

Salem 5,138 Posting Sage · Answer 5 · 2009-07-13T22:49:25+00:00

Salem 5,138 Posting Sage

14 Years Ago

Shame about the indentation - good luck.

Dream2code 12 Junior Poster · Answer 6 · 2009-07-13T23:46:57+00:00

i have written the code in notepad and compiled in UNIX environment with gcc compiler so couldn't emphasized on the indentation.

sorry again salem.

i am eagerly waiting for a solution.

salem, it will be appreciable if u go to the code.

Hiroshe 499 Posting Whiz in Training · Answer 7 · 2009-07-14T03:56:20+00:00

Try making a smaller example of the problems... that program looks a over complicated. If what you have work's, exept for a few errors, don't worry about changing it. Here's how I would go around doing it:
First read the description file to make a sscanf string, and make two arrays, one to hold the names of the variables, and another to hold the variables. Since the datatype can vary, maybe make the second array an array of pointers.
Next read the data file, record by record and as you index along print out the variables with there names.
Good luck.

Ancient Dragon 5,243 Achieved Level 70 Team Colleague Featured Poster · Answer 8 · 2009-07-14T06:58:16+00:00

i have written the code in notepad and compiled in UNIX environment with gcc compiler .

The most horrible programming environment ever. Why code it on Windows and compile it on *nix? You have a perfectly good IDE on *nix called vi, and I know for a fact there are even better ones. On Windows you have several free IDEs that will do auto-intent -- VC++ 2008 Express and Code::Bocks are just two of them. Learn to use IDEs because they will make your life a lot simpler so that you can become a more efficient coder.

And if you still insist on using Notepad, I know it has a tab key -- learn to use it.

Dream2code 12 Junior Poster · Answer 9 · 2009-07-14T11:58:03+00:00

Try making a smaller example of the problems... that program looks a over complicated. If what you have work's, exept for a few errors, don't worry about changing it. Here's how I would go around doing it:
First read the description file to make a sscanf string, and make two arrays, one to hold the names of the variables, and another to hold the variables. Since the datatype can vary, maybe make the second array an array of pointers.
Next read the data file, record by record and as you index along print out the variables with there names.
Good luck.

Here the problem is not so big.Complie and run the code once.it will work fine.

and its about two functions dont go to the code now just run it and
face the problem then u will definately get sumthing.

Thanks in advance.

Dream2code 12 Junior Poster · Answer 10 · 2009-07-14T12:01:38+00:00

Hello Ancient Dragon,

we are operating on UNIX environment.It will be used in UNIX only.
anyways next time onwards will take care about indetation it a word.

can you please go though my problem:

1st compile and run the code as i said in #1. then while running for different files u will face the problem after the column count exceeds 10.

but why only 10???

but both the functions a running fine independently.

check it one please.

tuse 22 Junior Poster · Answer 11 · 2009-07-14T17:55:09+00:00

tuse 22 Junior Poster

14 Years Ago

did you try debugging using gdb?

Ancient Dragon 5,243 Achieved Level 70 Team Colleague Featured Poster · Answer 12 · 2009-07-14T17:58:30+00:00

You need to post the input data file (zip it up first!) because the program you posted produces these errors on the three example lines you posted in your post #1.

Dml Error: at line 1
Dml error: Missing ; at line 2
Press any key to continue . . .

it works upto 10 columns.
when the column name exceeds 10
it shows "segmentation fault"

>>record->column_delim=(char**)malloc(10*sizeof(char*));

That line looks awfully suspicious of the reason for your error. It is only allocating room for 10 pointers

Dream2code 12 Junior Poster · Answer 13 · 2009-07-14T19:41:09+00:00

You need to post the input data file (zip it up first!) because the program you posted produces these errors on the three example lines you posted in your post #1.
>>record->column_delim=(char**)malloc(10*sizeof(char*));
That line looks awfully suspicious of the reason for your error. It is only allocating room for 10 pointers

i have given the dml syntax.put in mind that

format like
begin
column_1 string("|");
column_2 string("|");
....

column_n string("\n");
end;

at the End of line 2 in the dml a space is there.

"EVERY STATEMENT SHOULD END WITH A SEMICOLON EXCEPT THE BEGIN AND ENSURE THAT NO SPACE SHOULD BE THERE AFTER THE SEMICOLON"

Thats not the program error..its the error of symantic checker of my program.

record->column_delim=(char**)malloc(10*sizeof(char*));

if its the problem then we should not be able to print the informations from the description file.But we can print it.

and i changed this 10 to 1 ..5 ..20..50 ...many times.

M waiting for a miracle.lets see.its actually a strange problem.

Thanks
DP

Dream2code 12 Junior Poster · Answer 14 · 2009-07-14T19:45:30+00:00

did you try debugging using gdb?

i dont know actually how to use gdb.

please u try once from your side to debug and find the solution.

And please teach to about the usage of gdb.

Thanks,
DP

Ancient Dragon 5,243 Achieved Level 70 Team Colleague Featured Poster · Answer 15 · 2009-07-14T19:45:50+00:00

I'm only going to say this once more -- post the file, dummy it up with fake names if you want, but we need your file to work with. The example lines you posted in post #1 don't work

Deepak|23|M
puja|21|f
|XY|

Ancient Dragon 5,243 Achieved Level 70 Team Colleague Featured Poster · Answer 16 · 2009-07-14T19:49:11+00:00

i dont know actually how to use gdb.
please u try once from your side to debug and find the solution.
And please teach to about the usage of gdb.
Thanks,
DP

Well why don't you learn how to use gdg? Start with google.

>>please u try once from your side to debug and find the solution.
DaniWeb is not your a debugger.

Dream2code 12 Junior Poster · Answer 17 · 2009-07-14T19:55:25+00:00

description file:
=================

data file:
============

x|y|x|a|b|c|d|d|d|a|a|d
x|y|x|a|b|c|d|d|d|a|a|d
x|y|x|a|b|c|d|d|d|a|a|d
x|y|x|a|b|c|d|d|d|a|a|d

anyways u can create any kind of file as per the description file syntax and after that
create any data file as per the description file.

In my post#1 every thing described properly.
In fact it a realtime utility to be used in the server.

Dream2code 12 Junior Poster · Answer 18 · 2009-07-14T19:57:21+00:00

Well why don't you learn how to use gdg? Start with google.
>>please u try once from your side to debug and find the solution.
DaniWeb is not your a debugger.

i am ready to learn and take the tips all time.will learn gdb soon.
anyways now i am concentraing on this problem.

Thanks for your advice.

Ancient Dragon 5,243 Achieved Level 70 Team Colleague Featured Poster · Answer 19 · 2009-07-14T19:59:18+00:00

>>In my post#1 every thing described properly.
Then why didn't your program like it?

I don't want to create the file myself, I want to use the one YOU created.

Dream2code 12 Junior Poster · Answer 20 · 2009-07-14T20:12:11+00:00

>>In my post#1 every thing described properly.
Then why didn't your program like it?
I don't want to create the file myself, I want to use the one YOU created.

description file_1:
=================

data file_1:
============

x|y|x|a|b|c|d|d|d|a|a|d
x|y|x|a|b|c|d|d|d|a|a|d
x|y|x|a|b|c|d|d|d|a|a|d
x|y|x|a|b|c|d|d|d|a|a|d

output log _1:
===========

sdetlab01:/export/home/dpani/cprograms/mdump> strbk delim.dml delim.dat
=========================
NO OF COLUMNS:10
RECORD LENGTH:0
FILE TYPE :"2"
=========================
===COLUMN 1===

HEADER:"name"
DATATYPE:"0"
DELIMETER:"|"
===COLUMN 2===

HEADER:"age"
DATATYPE:"0"
DELIMETER:"|"
===COLUMN 3===

HEADER:"a"
DATATYPE:"0"
DELIMETER:"|"
===COLUMN 4===

HEADER:"a"
DATATYPE:"0"
DELIMETER:"|"
===COLUMN 5===

HEADER:"a"
DATATYPE:"0"
DELIMETER:"|"
===COLUMN 6===

HEADER:"a"
DATATYPE:"0"
DELIMETER:"|"
===COLUMN 7===

HEADER:"a"
DATATYPE:"0"
DELIMETER:"|"
===COLUMN 8===

HEADER:"a"
DATATYPE:"0"
DELIMETER:"|"
===COLUMN 9===

HEADER:"a"
DATATYPE:"0"
DELIMETER:"|"
===COLUMN 10===

HEADER:"sex"
DATATYPE:"0"
DELIMETER:"\n"
=========================
TESTING BREAK1

Record 1:
=========
name:"x"
age:"y"
a:"x"
a:"a"
a:"b"
a:"c"
a:"d"
a:"d"
a:"d"
sex:"a|a|d"

Record 2:
=========
name:"x"
age:"y"
a:"x"
a:"a"
a:"b"
a:"c"
a:"d"
a:"d"
a:"d"
sex:"a|a|d"

Record 3:
=========
name:"x"
age:"y"
a:"x"
a:"a"
a:"b"
a:"c"
a:"d"
a:"d"
a:"d"
sex:"a|a|d"

Record 4:
=========
name:"x"
age:"y"
a:"x"
a:"a"
a:"b"
a:"c"
a:"d"
a:"d"
a:"d"
sex:"a|a|d"
sdetlab01:/export/home/dpani/cprograms/mdump>

Dream2code 12 Junior Poster · Answer 21 · 2009-07-14T20:14:06+00:00

description file_2:(here column count exceeds 10) but still it can print description info on screen.
=================

data file_2:
============

x|y|x|a|b|c|d|d|d|a|a|d
x|y|x|a|b|c|d|d|d|a|a|d
x|y|x|a|b|c|d|d|d|a|a|d
x|y|x|a|b|c|d|d|d|a|a|d

sdetlab01:/export/home/dpani/cprograms/mdump> strbk delim.dml delim.dat
=========================
NO OF COLUMNS:12
RECORD LENGTH:0
FILE TYPE :"2"
=========================
===COLUMN 1===

HEADER:"name"
DATATYPE:"0"
DELIMETER:"|"
===COLUMN 2===

HEADER:"age"
DATATYPE:"0"
DELIMETER:"|"
===COLUMN 3===

HEADER:"a"
DATATYPE:"0"
DELIMETER:"|"
===COLUMN 4===

HEADER:"a"
DATATYPE:"0"
DELIMETER:"|"
===COLUMN 5===

HEADER:"a"
DATATYPE:"0"
DELIMETER:"|"
===COLUMN 6===

HEADER:"a"
DATATYPE:"0"
DELIMETER:"|"
===COLUMN 7===

HEADER:"a"
DATATYPE:"0"
DELIMETER:"|"
===COLUMN 8===

HEADER:"a"
DATATYPE:"0"
DELIMETER:"|"
===COLUMN 9===

HEADER:"a"
DATATYPE:"0"
DELIMETER:"|"
===COLUMN 10===

HEADER:"a"
DATATYPE:"0"
DELIMETER:"|"
===COLUMN 11===

HEADER:"a"
DATATYPE:"0"
DELIMETER:"|"
===COLUMN 12===

HEADER:"sex"
DATATYPE:"0"
DELIMETER:"\n"
=========================

TESTING BREAK1
Segmentation Fault (core dumped)
sdetlab01:/export/home/dpani/cprograms/mdump>

Ancient Dragon 5,243 Achieved Level 70 Team Colleague Featured Poster · Answer 22 · 2009-07-14T20:57:11+00:00

Please attach the file -- not post it. To do that hit the "Go Advanced" butten, scroll down until you see "Manage Attachments" button. That will let you browse your computer file system for the file to upload. Attempting to recreate the file(s) from what you posted can cause a lot of errors.

I admire your perseverance, I gave up caring about this thread a long time ago.

Dream2code 12 Junior Poster · Answer 23 · 2009-07-14T21:36:11+00:00

First off, use astyle this one and only time to format your code. I compiled your code from post 5 (didn't know which code to use bacouse you reposted it instead of editing your original) and got 100's of errors and warnings. You need a tutorial on debugging. If your code somehow managed to work for you, but it wont work for more than 10 records, I'd guess that your eather using some undifined behaver or you are going out of the range of the memory you allocated. That or you typed the input files wrong.
EDIT: You are calling string functions that don't exist You forgot to include <string.h>. It seems there is something wrong on line 527. Also it looks like your looping without checking if fgets != NULL. I bet you fgetting a line that dosent exist.
Good luck with your program.

Hello Hiroshe,

1)The program is running fine.
2)Each modules are running fine individually.
3)string.h is not needed in case we compile it on gcc. and please compile it on gcc/cc in unix environment.
it will not work for vc++ because variable declaration is done inside the code also.
4)look at the prototypes defined.to know the work of the modules.you need to understand what this code is doing before going into the technical things.
5)use the executable file for different set of data i have given in my examples.
6)No error in line 527 because requirement is sumthing differnt.
and think logically if its a error so
it will not work for hardcoded values but its working

i request you to go to the #1 to get the details again.thanks.
In the end if u will be able to fix it then its highly appriciable.

Regards,
DP

Dream2code 12 Junior Poster · Answer 24 · 2009-07-14T21:44:46+00:00

Dream2code 12 Junior Poster

14 Years Ago

Hi,

i have attached files for testing.

Thanks,
DP

data.txt (0.1 KB)

x|y|x|a|b|c|d|d|d|a|a|d
x|y|x|a|b|c|d|d|d|a|a|d
x|y|x|a|b|c|d|d|d|a|a|d
x|y|x|a|b|c|d|d|d|a|a|d

desc.txt (0.21 KB)

begin
col_1 string("|");
col_2 string("|");
col_3 string("|");
col_4 string("|");
col_5 string("|");
col_6 string("|");
col_7 string("|");
col_8 string("|");
col_9 string("|");
col_12 string("\n");
end;

Hiroshe 499 Posting Whiz in Training · Answer 25 · 2009-07-14T23:57:00+00:00

Hello Hiroshe,
1)The program is running fine.
2)Each modules are running fine individually.
3)string.h is not needed in case we compile it on gcc. and please compile it on gcc/cc in unix environment.
it will not work for vc++ because variable declaration is done inside the code also.
4)look at the prototypes defined.to know the work of the modules.you need to understand what this code is doing before going into the technical things.
5)use the executable file for different set of data i have given in my examples.
6)No error in line 527 because requirement is sumthing differnt.
and think logically if its a error so
it will not work for hardcoded values but its working
i request you to go to the #1 to get the details again.thanks.
In the end if u will be able to fix it then its highly appriciable.
Regards,
DP

You using string functions such as: "strncpy". That is defined in string.h in any ISO/ANSI C99 standard compiler, which includes gcc. I'm not sure what the operating system has to do with this, as long as the code is standard. The line 527 thing is comming directly from gcc. I am using Ubuntu Netbook Remix 9.04 with gcc 4.3 and here is the output with string.h:

ubuntu@ubuntu-laptop:~/Documents/C$ cc test.c -o test
test.c: In function ‘read_fixed’:
test.c:527: error: conversion to non-scalar type requested
ubuntu@ubuntu-laptop:~/Documents/C$

I'll include an output without string.h as a file attachment(its too long to include in the message). What version of gcc are you using? Are you sure you uploaded the right code (happend to me once:$ )?

P.S. yes I also tried it with the -m64 tag, but it shouldent matter anyway.

Dream2code 12 Junior Poster · Answer 26 · 2009-07-15T01:05:38+00:00

I have edited the code and removed that function creating will look into that later.now u can compile this piece of code it works.

#include<stdio.h>
#include<math.h>
#include<stdlib.h>
#include<string.h>

#define FIXED 1
#define DELIMETED 2
#define MIXED 3


typedef struct dml_s
{
char **col_headers;
char **column_delim;
int *col_datatype;
int *col_len;
int col_cnt;
int record_length;
int file_type;          //file type can be fixed,delimeted or max len with delimeters
}dml;


void display_intro();
dml* extract_dml_info(char*,dml*);
int str_wc(char *);
int str_semicl(char *);
int pow_user(int,int);
void display_dml_info(struct dml_s *record);
void read_delimeted(char *filename,char **delmimeter,char **header,int col_cnt);
char * ustrtok(char *base_string,char *next_str,char *delim);

int main(int argc,char *argv[])
{

if(argc!=3)
{
display_intro();
return;
}


dml *record;
record=(dml*)malloc(sizeof(dml));
record=extract_dml_info(argv[1],record);
if(record==NULL) exit(1);

display_dml_info(record);


if(record->file_type==DELIMETED)
{
read_delimeted(argv[2],record->column_delim,record->col_headers,record->col_cnt);
}



return 0;
}


/********************THIS FUNCTION EXTRACTS INFORMATION FROM THE
*********************DML FILE AND RETURNS A STRUCTRUE***********/

dml* extract_dml_info(char *dmlfile,dml *record)
{
char buffer[100];       //IT BUFFERS ONE LINE AT A TIME FORM THE FILE
char **line;            //ALL THE LINES FROM THE FILE STORED IN LINE ARRAY
char **col_desc;        //STORES THE COLUMN DESCRIPTIONS
char *temp,*token;
FILE *fp;
int i,j;

fp=fopen(dmlfile,"r");
if(fp==NULL)
{
printf("Can't read from file: \"%s\"\n",dmlfile);
perror("Error");
exit(1);
}

record=(dml*)malloc(20*sizeof(dml));
record->col_headers=(char**)malloc(200*sizeof(char*));
record->column_delim=(char**)malloc(200*sizeof(char*));
col_desc=(char**)malloc(200*sizeof(char*));
line=(char**)malloc(200*sizeof(char*));

fgets(buffer,100,fp);
i=0;
while(!feof(fp))
{
if(buffer[strlen(buffer)-1]=='\n')
buffer[strlen(buffer)-1]='\0';
line[i]=(char*)malloc(strlen(buffer)+1);
strcpy(line[i],buffer);
i++;
fgets(buffer,100,fp);
}

record->col_cnt=i-2;                    //Getting the column count from the dml file



/****************DML VALIDATION***************************************
The following validation taken place
1)DML starts with a 'begin' flag and end with 'end;'
2)Every statement ends with a semicolon except the begin
3)no semicolon should encounter inside
4)each line should consist of 2 words (a)column name (b)column desc
*********************************************************************/

if(strcmp(line[0],"begin")!=0)
{
printf("Dml Error: at line 1\n");
}

int k=0;
int w_count,semicl_flag;
for(j=1;j<i;j++)
{
k=strlen(line[j]);


if(line[j][k-1]!=';')
{
printf("Dml error: Missing ; at line %d\n",j+1);
return NULL;
}
if((strchr(line[j],';'))!=NULL && line[j][k-1]!=';' )
{
printf("Dml error: at line %d\n",j+1);
return NULL;
}


w_count=str_wc(line[j]);
if( w_count != 2 && w_count != 3 && j!=i-1 )
{
printf("Dml Error xxx at line %d\n",j);
return NULL;
}

semicl_flag=str_semicl(line[j]);
if(semicl_flag==-1)
{
printf("Dml Error yyy at line %d\n",j);
return NULL;
}
else if(semicl_flag==-2)
{
printf("Dml Error zzz at line %d\n",j);
return NULL;
}
}

if(strcmp(line[i-1],"end;")!=0)
{
printf("Dml Error: at line %d\n",i);
return NULL;
}

/****************END DML VALIDATION***********************/


for(j=0;j<record->col_cnt;j++)
{
token=(char*)malloc(strlen(line[j])+1);
token=strtok(line[j+1]," ");
if(token==NULL)
{
token="\0";
return NULL;
}
record->col_headers[j]=(char*)malloc(strlen(token)+1);
strcpy(record->col_headers[j],token);
token=strtok(NULL,"\n");
col_desc[j]=(char*)malloc(strlen(token)+1);
if(token!=NULL)
strcpy(col_desc[j],token);
else
{
printf("Incorrect DML at line %d for %s\n",j+1,record->col_headers[j]);
return NULL;
}

}



/****************DETERMINE THE DATATYPE OF THE COLUMNS*********/
char *del;
record->col_datatype=(int*)malloc(2*i*sizeof(int));
int flag;
for(j=0;j<record->col_cnt;j++)
{
del=strstr(col_desc[j],"string");
if(del==NULL)
{
record->col_datatype[j]=1;
del='\0';
}
else
record->col_datatype[j]=0;
}
/*****************************************************************/


int count_fixed=0;
int count_del=0;
char *temp_del_fst;
char *temp_del_lst;
int tempv;
record->column_delim=(char**)malloc(10*sizeof(char*));
for(j=0;j<record->col_cnt;j++)
{
temp_del_fst=strchr(col_desc[j],'"');
temp_del_lst=strrchr(col_desc[j],'"');
if(temp_del_fst==NULL && temp_del_lst==NULL)
{
count_fixed++;
record->column_delim[j]=(char*)malloc(2);
record->column_delim[j]="\0";
}
else if(temp_del_fst!=NULL && temp_del_lst!=NULL && temp_del_lst!=temp_del_fst)
{
count_del++;
tempv=temp_del_lst-temp_del_fst;
record->column_delim[j]=(char*)malloc(temp_del_lst-temp_del_fst);
strncpy(record->column_delim[j],temp_del_fst+1,temp_del_lst-temp_del_fst-1);
}
else if(temp_del_fst!=NULL && temp_del_lst!=NULL && temp_del_lst==temp_del_fst)
{
printf("DML Error :at line %d ,inproper description\n",j+1);
record->column_delim[j]="\0";
return  NULL;
}
}




/***********/

record->record_length=0;
char *c_len;
if(count_fixed==record->col_cnt && count_del==0)
{
 record->file_type=FIXED;
 record->col_len=(int *)malloc(record->col_cnt*sizeof(int));
 for(j=0;j<record->col_cnt;j++)
 {
temp_del_fst=strchr(col_desc[j],'(');
temp_del_lst=strrchr(col_desc[j],')');
  if(temp_del_fst==NULL && temp_del_lst==NULL)
    {
     printf("Dml Error..( ) missing in column description\n");
    }
    tempv=temp_del_lst-temp_del_fst;

        c_len=(char*)malloc(10*sizeof(char));
    strncpy(c_len,temp_del_fst+1,temp_del_lst-temp_del_fst-1);

        int len=strlen(c_len);

        record->col_len[j]=atoi(c_len);
        record->record_length=record->record_length+record->col_len[j];
       

  }

       

}


/**********/
else if(count_fixed==0 && count_del==record->col_cnt)
{
record->file_type=DELIMETED;

  if(strcmp(record->column_delim[record->col_cnt-1],"\\n")!=0)
  {
    printf("DML error,in the last field '\\n' missing\n");
   return NULL;
  }
}
else
{
record->file_type=-1;
printf("DML error,Unable to determine the file type\n");
return NULL;
}

return record;
}


/******A function to get the word count of a string*******/

int str_wc(char *str)
{
int limit = strlen(str);
    int i, change = 1, words = 0;
    for(i = 0; i < limit; ++i)
    {
        if(!isspace(str[i]))
        {
            if(change)
            {
                ++words;
                change = 0;
            }
        }
        else
        {
            change = 1;
        }
            }

  return words;
}

/*************************************************************/

/*********A fucntion to check semicolon occurs in between the line***/

int str_semicl(char *str)
{
int limit = strlen(str);
int i,flag=0;

    for(i = 0; i < limit; ++i)
    {
        if(i < limit-1 && str[i]==';' )
        {
                 flag=-1;
        }

        else if( str[limit-1]==';' && isspace(str[limit-2]) )
        {
                flag=-2;
        }
        else if( str[limit-1]==';' && !(isspace(str[limit-2])))
        {
                flag=0;
        }

        else
                flag=0;

   }

  return flag;
}

/************************************************************/

/******CALCULATION POWER****************/
int pow_user(base,exp)
{
int i;
int pow=1;
for(i=0;i<exp;i++)
{
pow=pow*base;
}
return pow;
}
/****************************************/


/******IT'S A FUNCTION SIMILAR TO STRTOK BUT CAN HANDLE NULL VALUES******/
/******THIS FUNCTION SPILTS A STRING INTO TWO STRINGS W.R.T A TOKEN******/

char * ustrtok(char *base_string,char *next_str,char *delim)
{
char *str2;
str2=(char*)malloc(100);
if(base_string==NULL && strlen(base_string)<1)
{
next_str=NULL;
return NULL;
}
int len=strlen(base_string);
char *temp;
temp=(char*)malloc(100);

temp=strstr(base_string,delim);
if(temp==NULL)
{
strcpy(next_str,base_string);
next_str[strlen(next_str)]='\0';
return NULL;
}
else
{
int sub_len=temp-base_string;
str2=(char*)malloc(100);
strncpy(str2,base_string,sub_len);
str2[strlen(str2)]='\0';
strcpy(next_str,str2);
return (temp+strlen(delim));
}
}



/******************THIS FUNCTION DISPLAYS THE DML DESCRIPTION****************/


void display_dml_info(dml *record)
{

int i,j;
if(record==NULL)return;
printf("=========================\n");
printf("NO OF COLUMNS:%d\n",record->col_cnt);
printf("RECORD LENGTH:%d\n",record->record_length);
printf("FILE TYPE :\"%d\"\n",record->file_type);
printf("=========================\n");

for(j=0;j<record->col_cnt;j++)
{
printf("===COLUMN %d===\n\n",j+1);
printf("HEADER:\"%s\"\n",record->col_headers[j]);
printf("DATATYPE:\"%d\"\n",record->col_datatype[j] );
if(record->file_type==DELIMETED)printf("DELIMETER:\"%s\"\n",record->column_delim[j]);
if(record->file_type==FIXED )printf("LENGTH:%d\n",record->col_len[j]);
}
printf("=========================\n\n");
}

/***************************************************************************************/


void read_delimeted(char *filename,char **column_delim,char **col_headers,int col_cnt)
{
FILE *data_file;
char buffer[20000];
char *ext_line=NULL;
char *temp=NULL;
char **fields=NULL;
int i,j;

data_file=fopen(filename,"r");
if(data_file==NULL)
{
printf("can't open file\n");
return;
}

printf("TESTING BREAK1\n");

j=1;
fgets(buffer,2000,data_file);
while(!feof(data_file))
{
i=0;

if(buffer[strlen(buffer)-1]=='\n')
buffer[strlen(buffer)-1]='\0';
if(strlen(buffer)==0)
goto nextline;
ext_line=(char*)malloc(strlen(buffer)+1);
if(ext_line==NULL)
{
printf("NUll Record Found\n");
}
strcpy(ext_line,buffer);
printf("\nRecord %d:\n",j);
printf("=========\n");

fields=(char**)malloc(col_cnt*sizeof(char*));
char *next_str;
temp=(char*)malloc(strlen(buffer)+1);
next_str=(char*)malloc(strlen(buffer)+1);
next_str=ustrtok(ext_line,temp,column_delim[0]);
while(i<col_cnt)
{
if(next_str==NULL)
next_str="\0";
if(ext_line==NULL)
ext_line="\0";
if(temp==NULL)
temp="\0";


fields[i]=(char*)malloc(strlen(temp));
strcpy(fields[i],temp);

fprintf(stdout,"%s:\"%s\"\n",col_headers[i],fields[i]);
ext_line=next_str;
free(temp);
temp=(char*)malloc(100*sizeof(char));
if(i==col_cnt-1)
{
column_delim[i+1]=column_delim[i];
}
next_str=ustrtok(ext_line,temp,column_delim[i+1]);


if(i==col_cnt-2)
{
if(ext_line==NULL)
ext_line="\0";
strcpy(temp,ext_line);
}

if(i==col_cnt-1 && strlen(ext_line)>0)
{
printf("\n\nError:Incomplete field at end of record.\n");
printf("Leftover bytes :\"%s\"\n",ext_line);
printf("Either (1) The metadata does not accurately describe the data,\n");
printf("    or (2) The data has somehow been corrupted.\n");
}
i++;
}
j++;
nextline:
fgets(buffer,2000,data_file);
free(fields);
}
fclose(data_file);
}



void display_intro()
{
 printf("\n mdump: Version 1.0\n\n");
 printf(" Copyright (c) 2009, Deepak Panigrahy.  All rights reserved.\n\n");
 printf(" Purpose: This program is a simple utility that display\n");
 printf("          the source file as per the Record format.\n\n");

 printf(" Usage:   mdump <Dml FileName> <Data FileName>\n\n");

 printf("          Use -h option for additional help.\n");
}

rcbhat 0 Newbie Poster · Answer 27 · 2009-07-15T01:12:42+00:00

Its a linker error caused due to a shared library

Memory Problem After a certain limit

Recommended Answers Collapse Answers

All 46 Replies

Recommended Answers