| | |
Memory Problem After a certain limit
Thread Solved |
Hi All,
My requirement is like this.It a part of a software devlopment team as i am devloping a tool its a part of it.
Its bit long but my explanation is clear to get it.
1)we need to read a data file using a description file.
Usage: programname <description_file> <data_file>
2)Then need to dump the datafile on the screen according to the description file.
Detailed descritpin is given below:
Thanks,
DP
Description file format:
========================
begin
columname datatype("delimeter");
columname datatype("delimeter");
.
.
.
.
end;
EXAMPLE:
--------
begin
name string("|");
age string("|");
sex string("\n");
end;
Datafile Sample:
===============
Deepak|23|M
puja|21|f
|XY|
OUTPUT WILL BE:
================
Record_1:
name:"Deepak"
age:"23"
sex:"M"
Record_2:
name:"puja:
age:"21"
sex:"F"
Record_3:
name:""
age:"XY"
sex:""
I think the picture is clear now what i am doing.
now i am telling about the important functions and structures i am using.
A) The below structure is used to describe the Description file.
typedef struct dml_s
{
char **col_headers;
char **column_delim;
int *col_datatype;
int *col_len;
int col_cnt;
int record_length;
int file_type; //file type can be fixed,delimeted or max len with delimeters
}dml;
concentrate on 3 columns :
===========================
char **col_headers; it will store the column headers from the description file.
char **column_delim; it will store the column delimeters from the description file.
int col_cnt; it will store the column count from the description file.
B) dml* extract_dml_info(char* description_fname,dml *record);
this function will read the description file and store the necessary information in the variables.
C) void read_delimeted(char *data_filename,char **delmimeter,char **header,int col_cnt);
now this fuction will take the data file name along with the data retured by the above function.
and displays data on the screen as per the output format.
++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++PROBLEM++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++
1)When we pass hard coded values to the fucntion
void read_delimeted(char *data_filename,char **delmeter,char **header,int col_cnt);
it works fine for any number of columns and any size of data file
e.g
char *delmimeter[]={"name","age","sex");
char *delimeter[]={"|","|","\n"};
2)When we pass values returned from extract_dml_info(char* description_fname,dml *record);
to the fucntion read_delimeted(char *data_filename,char **delmeter,char **header,int col_cnt);
it works upto 10 columns.
when the column name exceeds 10
it shows "segmentation fault"
ANALYSIS:
==========
A)
if error in
read_delimeted(char *data_filename,char **delmeter,char **header,int col_cnt);
then it should not have worked for hard coded values.
B)if error in extract_dml_info(char* description_fname,dml *record);
then it should not print datas extrated beyond 10 columns but it prints data for any number of
columns.
but as a whole
program fails if the column number from descrition file exceeds 10 columns.
My requirement is like this.It a part of a software devlopment team as i am devloping a tool its a part of it.
Its bit long but my explanation is clear to get it.
1)we need to read a data file using a description file.
Usage: programname <description_file> <data_file>
2)Then need to dump the datafile on the screen according to the description file.
Detailed descritpin is given below:
Thanks,
DP
Description file format:
========================
begin
columname datatype("delimeter");
columname datatype("delimeter");
.
.
.
.
end;
EXAMPLE:
--------
begin
name string("|");
age string("|");
sex string("\n");
end;
Datafile Sample:
===============
Deepak|23|M
puja|21|f
|XY|
OUTPUT WILL BE:
================
Record_1:
name:"Deepak"
age:"23"
sex:"M"
Record_2:
name:"puja:
age:"21"
sex:"F"
Record_3:
name:""
age:"XY"
sex:""
I think the picture is clear now what i am doing.
now i am telling about the important functions and structures i am using.
A) The below structure is used to describe the Description file.
typedef struct dml_s
{
char **col_headers;
char **column_delim;
int *col_datatype;
int *col_len;
int col_cnt;
int record_length;
int file_type; //file type can be fixed,delimeted or max len with delimeters
}dml;
concentrate on 3 columns :
===========================
char **col_headers; it will store the column headers from the description file.
char **column_delim; it will store the column delimeters from the description file.
int col_cnt; it will store the column count from the description file.
B) dml* extract_dml_info(char* description_fname,dml *record);
this function will read the description file and store the necessary information in the variables.
C) void read_delimeted(char *data_filename,char **delmimeter,char **header,int col_cnt);
now this fuction will take the data file name along with the data retured by the above function.
and displays data on the screen as per the output format.
++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++PROBLEM++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++
1)When we pass hard coded values to the fucntion
void read_delimeted(char *data_filename,char **delmeter,char **header,int col_cnt);
it works fine for any number of columns and any size of data file
e.g
char *delmimeter[]={"name","age","sex");
char *delimeter[]={"|","|","\n"};
2)When we pass values returned from extract_dml_info(char* description_fname,dml *record);
to the fucntion read_delimeted(char *data_filename,char **delmeter,char **header,int col_cnt);
it works upto 10 columns.
when the column name exceeds 10
it shows "segmentation fault"
ANALYSIS:
==========
A)
if error in
read_delimeted(char *data_filename,char **delmeter,char **header,int col_cnt);
then it should not have worked for hard coded values.
B)if error in extract_dml_info(char* description_fname,dml *record);
then it should not print datas extrated beyond 10 columns but it prints data for any number of
columns.
but as a whole
program fails if the column number from descrition file exceeds 10 columns.
Stand alone code is pasted below:
i have compiled it on gcc compiler on 64bit mode.
e.g gcc -o output prog.c -m64
code below
===========
i have compiled it on gcc compiler on 64bit mode.
e.g gcc -o output prog.c -m64
code below
===========
c Syntax (Toggle Plain Text)
#include<stdio.h> #include<math.h> #include<stdlib.h> #define FIXED 1 #define DELIMETED 2 #define MIXED 3 int errno; typedef struct dml_s { char **col_headers; char **column_delim; int *col_datatype; int *col_len; int col_cnt; int record_length; int file_type; //file type can be fixed,delimeted or max len with delimeters }dml; void display_intro(); dml* extract_dml_info(char*,dml*); int str_wc(char *); int str_semicl(char *); int pow_user(int,int); void display_dml_info(struct dml_s *record); void read_fixed(char *filename,int *size,char **header,int col_cnt,int record_len); void read_delimeted(char *filename,char **delmimeter,char **header,int col_cnt); char * ustrtok(char *base_string,char *next_str,char *delim); int main(int argc,char *argv[]) { if(argc!=3) { display_intro(); return; } dml *record; record=(dml*)malloc(sizeof(dml)); record=extract_dml_info(argv[1],record); if(record==NULL) exit(1); display_dml_info(record); if(record->file_type==DELIMETED) { read_delimeted(argv[2],record->column_delim,record->col_headers,record->col_cnt); } /* else if(record->file_type=FIXED) { read_fixed(argv[2],record->col_len,record->col_headers,record->col_cnt,record->record_length); } else if(record->file_type=MIXED) { printf("\n\nCONSTRUCTION ON PROGRESS FOR DELIMTED FILE WITH LENGTH\n"); return; } */ return 0; } /********************THIS FUNCTION EXTRACTS INFORMATION FROM THE *********************DML FILE AND RETURNS A STRUCTRUE***********/ dml* extract_dml_info(char *dmlfile,dml *record) { char buffer[100]; //IT BUFFERS ONE LINE AT A TIME FORM THE FILE char **line; //ALL THE LINES FROM THE FILE STORED IN LINE ARRAY char **col_desc; //STORES THE COLUMN DESCRIPTIONS char *temp,*token; FILE *fp; int i,j; fp=fopen(dmlfile,"r"); if(fp==NULL) { printf("Can't read from file: \"%s\"\n",dmlfile); perror("Error"); exit(1); } record=(dml*)malloc(20*sizeof(dml)); record->col_headers=(char**)malloc(200*sizeof(char*)); record->column_delim=(char**)malloc(200*sizeof(char*)); col_desc=(char**)malloc(200*sizeof(char*)); line=(char**)malloc(200*sizeof(char*)); fgets(buffer,100,fp); i=0; while(!feof(fp)) { if(buffer[strlen(buffer)-1]=='\n') buffer[strlen(buffer)-1]='\0'; line[i]=(char*)malloc(strlen(buffer)+1); strcpy(line[i],buffer); i++; fgets(buffer,100,fp); } record->col_cnt=i-2; //Getting the column count from the dml file /****************DML VALIDATION*************************************** The following validation taken place 1)DML starts with a 'begin' flag and end with 'end;' 2)Every statement ends with a semicolon except the begin 3)no semicolon should encounter inside 4)each line should consist of 2 words (a)column name (b)column desc *********************************************************************/ if(strcmp(line[0],"begin")!=0) { printf("Dml Error: at line 1\n"); } int k=0; int w_count,semicl_flag; for(j=1;j<i;j++) { k=strlen(line[j]); if(line[j][k-1]!=';') { printf("Dml error: Missing ; at line %d\n",j+1); return NULL; } if((strchr(line[j],';'))!=NULL && line[j][k-1]!=';' ) { printf("Dml error: at line %d\n",j+1); return NULL; } w_count=str_wc(line[j]); if( w_count != 2 && w_count != 3 && j!=i-1 ) { printf("Dml Error xxx at line %d\n",j); return NULL; } semicl_flag=str_semicl(line[j]); if(semicl_flag==-1) { printf("Dml Error yyy at line %d\n",j); return NULL; } else if(semicl_flag==-2) { printf("Dml Error zzz at line %d\n",j); return NULL; } } if(strcmp(line[i-1],"end;")!=0) { printf("Dml Error: at line %d\n",i); return NULL; } /****************END DML VALIDATION***********************/ for(j=0;j<record->col_cnt;j++) { token=(char*)malloc(strlen(line[j])+1); token=strtok(line[j+1]," "); if(token==NULL) { token="\0"; return NULL; } record->col_headers[j]=(char*)malloc(strlen(token)+1); strcpy(record->col_headers[j],token); token=strtok(NULL,"\n"); col_desc[j]=(char*)malloc(strlen(token)+1); if(token!=NULL) strcpy(col_desc[j],token); else { printf("Incorrect DML at line %d for %s\n",j+1,record->col_headers[j]); return NULL; } } /****************DETERMINE THE DATATYPE OF THE COLUMNS*********/ char *del; record->col_datatype=(int*)malloc(2*i*sizeof(int)); int flag; for(j=0;j<record->col_cnt;j++) { del=strstr(col_desc[j],"string"); if(del==NULL) { record->col_datatype[j]=1; del='\0'; } else record->col_datatype[j]=0; } /*****************************************************************/ int count_fixed=0; int count_del=0; char *temp_del_fst; char *temp_del_lst; int tempv; record->column_delim=(char**)malloc(10*sizeof(char*)); for(j=0;j<record->col_cnt;j++) { temp_del_fst=strchr(col_desc[j],'"'); temp_del_lst=strrchr(col_desc[j],'"'); if(temp_del_fst==NULL && temp_del_lst==NULL) { count_fixed++; record->column_delim[j]=(char*)malloc(2); record->column_delim[j]="\0"; } else if(temp_del_fst!=NULL && temp_del_lst!=NULL && temp_del_lst!=temp_del_fst) { count_del++; tempv=temp_del_lst-temp_del_fst; record->column_delim[j]=(char*)malloc(temp_del_lst-temp_del_fst); strncpy(record->column_delim[j],temp_del_fst+1,temp_del_lst-temp_del_fst-1); } else if(temp_del_fst!=NULL && temp_del_lst!=NULL && temp_del_lst==temp_del_fst) { printf("DML Error :at line %d ,inproper description\n",j+1); record->column_delim[j]="\0"; return NULL; } } //printf("count fixed:%d and count del :%d \n",count_fixed,count_del); /***********/ record->record_length=0; char *c_len; if(count_fixed==record->col_cnt && count_del==0) { record->file_type=FIXED; record->col_len=(int *)malloc(record->col_cnt*sizeof(int)); for(j=0;j<record->col_cnt;j++) { temp_del_fst=strchr(col_desc[j],'('); temp_del_lst=strrchr(col_desc[j],')'); if(temp_del_fst==NULL && temp_del_lst==NULL) { printf("Dml Error..( ) missing in column description\n"); } tempv=temp_del_lst-temp_del_fst; c_len=(char*)malloc(10*sizeof(char)); strncpy(c_len,temp_del_fst+1,temp_del_lst-temp_del_fst-1); int len=strlen(c_len); record->col_len[j]=atoi(c_len); record->record_length=record->record_length+record->col_len[j]; // printf("length:%d\tclen:%s\tlen:%d\t j=%d\n",record->col_len[j],c_len,tempv,j); } //printf("record length is :%d\n",record->record_length); } /**********/ else if(count_fixed==0 && count_del==record->col_cnt) { record->file_type=DELIMETED; if(strcmp(record->column_delim[record->col_cnt-1],"\\n")!=0) { printf("DML error,in the last field '\\n' missing\n"); return NULL; } } else { record->file_type=-1; printf("DML error,Unable to determine the file type\n"); return NULL; } return record; } /******A function to get the word count of a string*******/ int str_wc(char *str) { int limit = strlen(str); int i, change = 1, words = 0; for(i = 0; i < limit; ++i) { if(!isspace(str[i])) { if(change) { ++words; change = 0; } } else { change = 1; } } return words; } /*************************************************************/ /*********A fucntion to check semicolon occurs in between the line***/ int str_semicl(char *str) { int limit = strlen(str); int i,flag=0; for(i = 0; i < limit; ++i) { if(i < limit-1 && str[i]==';' ) { flag=-1; } else if( str[limit-1]==';' && isspace(str[limit-2]) ) { flag=-2; } else if( str[limit-1]==';' && !(isspace(str[limit-2]))) { flag=0; } else flag=0; } return flag; } /************************************************************/ /******CALCULATION POWER****************/ int pow_user(base,exp) { int i; int pow=1; for(i=0;i<exp;i++) { pow=pow*base; } return pow; } /****************************************/ /******IT'S A FUNCTION SIMILAR TO STRTOK BUT CAN HANDLE NULL VALUES******/ /******THIS FUNCTION SPILTS A STRING INTO TWO STRINGS W.R.T A TOKEN******/ char * ustrtok(char *base_string,char *next_str,char *delim) { char *str2; str2=(char*)malloc(100); if(base_string==NULL && strlen(base_string)<1) { next_str=NULL; return NULL; } int len=strlen(base_string); char *temp; temp=(char*)malloc(100); temp=strstr(base_string,delim); if(temp==NULL) { strcpy(next_str,base_string); next_str[strlen(next_str)]='\0'; return NULL; } else { int sub_len=temp-base_string; str2=(char*)malloc(100); strncpy(str2,base_string,sub_len); str2[strlen(str2)]='\0'; strcpy(next_str,str2); return (temp+strlen(delim)); } } /******************THIS FUNCTION DISPLAYS THE DML DESCRIPTION****************/ void display_dml_info(dml *record) { int i,j; if(record==NULL)return; printf("=========================\n"); printf("NO OF COLUMNS:%d\n",record->col_cnt); printf("RECORD LENGTH:%d\n",record->record_length); printf("FILE TYPE :\"%d\"\n",record->file_type); printf("=========================\n"); for(j=0;j<record->col_cnt;j++) { printf("===COLUMN %d===\n\n",j+1); printf("HEADER:\"%s\"\n",record->col_headers[j]); printf("DATATYPE:\"%d\"\n",record->col_datatype[j] ); if(record->file_type==DELIMETED)printf("DELIMETER:\"%s\"\n",record->column_delim[j]); if(record->file_type==FIXED )printf("LENGTH:%d\n",record->col_len[j]); } printf("=========================\n\n"); } /***************************************************************************************/ /*****************************Begin read_fixed************************************** THIS FUNCTION READS A FIXED LENGTH FILE AS PER THE SPECIFED DML STEPS ARE AS FOLLOWS: INPUT PARAMETER: A) File Name B) Column Headers C) Length of all the Columns D) No Of columns E) Lenggh of Each Record OUTPUT: Reads the file as per the input parametes and Display it on the Screen *********************************************************************************/ void read_fixed(char *filename,int *size,char **header,int col_cnt,int record_len) { FILE *file_read; long lSize; char * buffer; size_t result; fpos_t position; file_read=fopen(filename,"rb"); if(file_read==NULL) { printf("Can't read from file \"%s\"\n",filename); perror("Error"); exit(1); } fseek(file_read,0,SEEK_END); lSize=ftell(file_read); rewind(file_read); int j=0; int i=0; float record_count=lSize/record_len; int multiply=record_count*record_len; buffer=(char*)malloc(4); result=fread(buffer,1,size[0],file_read); buffer[size[0]]='\0'; rewind(file_read); while(!feof(file_read) || buffer==NULL ) { if(j>=col_cnt)j=0; if(j==0 && i<=record_count) { i++; } if(j==0 && i<=record_count) { printf("Record %d\n",i); printf("=============\n"); } buffer=(char*)malloc(10); result=fread(buffer,1,size[j],file_read); buffer[size[j]]='\0'; if(i<=record_count) printf("%s:\"%s\"\n",header[j],buffer); free(buffer); j++; } /*setting up the file pointer to get the leftover bytes*/ fgetpos (file_read, &position); position=(fpos_t)multiply; fsetpos (file_read, &position); /******************************************************/ if(lSize!=multiply) { char *left_over; left_over=(char*)malloc(record_len+1); result=fread(left_over,1,record_len,file_read); left_over[record_len]='\0'; printf("\n\n\nLeft Over Bytes:\"%s\"\n",left_over); free(left_over); printf("\nError:Incomplete record at end of file.\n"); printf("Either (1) The metadata does not accurately describe the data,\n"); printf(" or (2) The data has somehow been corrupted.\n"); } } /*********************************END read_fixed ****************************/ void read_delimeted(char *filename,char **column_delim,char **col_headers,int col_cnt) { FILE *data_file; char buffer[20000]; char *ext_line=NULL; char *temp=NULL; char **fields=NULL; int i,j; printf("filename:%s\n",filename); printf("columncount:%d\n",col_cnt); for(i=0;i<col_cnt;i++) { printf("del:%s\n",column_delim[i]); printf("header:%s\n",col_headers[i]); } data_file=fopen(filename,"r"); if(data_file==NULL) { printf("can't open file\n"); return; } printf("TESTING BREAK1\n"); j=1; fgets(buffer,2000,data_file); while(!feof(data_file)) { i=0; if(buffer[strlen(buffer)-1]=='\n') buffer[strlen(buffer)-1]='\0'; if(strlen(buffer)==0) goto nextline; ext_line=(char*)malloc(strlen(buffer)+1); if(ext_line==NULL) { printf("NUll Record Found\n"); } strcpy(ext_line,buffer); printf("\nRecord %d:\n",j); printf("=========\n"); fields=(char**)malloc(col_cnt*sizeof(char*)); char *next_str; temp=(char*)malloc(strlen(buffer)+1); next_str=(char*)malloc(strlen(buffer)+1); next_str=ustrtok(ext_line,temp,column_delim[0]); while(i<col_cnt) { if(next_str==NULL) next_str="\0"; if(ext_line==NULL) ext_line="\0"; if(temp==NULL) temp="\0"; fields[i]=(char*)malloc(strlen(temp)); strcpy(fields[i],temp); fprintf(stdout,"%s:\"%s\"\n",col_headers[i],fields[i]); ext_line=next_str; free(temp); temp=(char*)malloc(100*sizeof(char)); if(i==col_cnt-1) { column_delim[i+1]=column_delim[i]; } next_str=ustrtok(ext_line,temp,column_delim[i+1]); if(i==col_cnt-2) { if(ext_line==NULL) ext_line="\0"; strcpy(temp,ext_line); } if(i==col_cnt-1 && strlen(ext_line)>0) { printf("\n\nError:Incomplete field at end of record.\n"); printf("Leftover bytes :\"%s\"\n",ext_line); printf("Either (1) The metadata does not accurately describe the data,\n"); printf(" or (2) The data has somehow been corrupted.\n"); } i++; } j++; nextline: fgets(buffer,2000,data_file); free(fields); } fclose(data_file); } void display_intro() { printf("\n mdump: Version 1.0\n\n"); printf(" Copyright (c) 2009, Deepak Panigrahy. All rights reserved.\n\n"); printf(" Purpose: This program is a simple utility that display\n"); printf(" the source file as per the Record format.\n\n"); printf(" Usage: mdump <Dml FileName> <Data FileName>\n\n"); printf(" Use -h option for additional help.\n"); }
Last edited by Ancient Dragon; Jul 13th, 2009 at 9:52 pm. Reason: add code tags
http://www.daniweb.com/forums/announcement118-3.html
http://www.daniweb.com/forums/announcement118-3.html
http://www.daniweb.com/forums/announcement118-3.html
http://www.daniweb.com/forums/announcement118-3.html
http://www.daniweb.com/forums/announcement118-3.html
You already missed at least FIVE places where the board asks (nay, implores) you to use code tags, but you missed them all.
Try editing your posts again, and this time see if you can make them more readable.
Check out some other posts on the forum for readability, and make sure yours is just as good.
http://www.daniweb.com/forums/announcement118-3.html
http://www.daniweb.com/forums/announcement118-3.html
http://www.daniweb.com/forums/announcement118-3.html
http://www.daniweb.com/forums/announcement118-3.html
You already missed at least FIVE places where the board asks (nay, implores) you to use code tags, but you missed them all.
Try editing your posts again, and this time see if you can make them more readable.
Check out some other posts on the forum for readability, and make sure yours is just as good.
If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
--
If your code lacks code tags, you will be IGNORED
--
If your code lacks code tags, you will be IGNORED
for more detail:
================
instread of strtok i have used my own advanced function ustrtok.
it works like this
==================
char * ustrtok(char *base_string,char *next_str,char *delim);
return=ustrtok(char *base_string,char *next_str,char *delim);
basesting= a|b|c|d
next_str=NULL
delim=|
after one call
===============
next_str=a
return=b|c|d
then we need to assing
base_string=return for further calls.
note:
1)No issue due to this function its only a help to understand the code.
2)i have tried my besr from my side.please look into this.
3)i have provided the sample data u can run the code very easily to see the output.
4)i suggest run the code with different type of descrition files and data files
change the columns numbers and all then u will observe the failure or else it works fine.
================
instread of strtok i have used my own advanced function ustrtok.
it works like this
==================
char * ustrtok(char *base_string,char *next_str,char *delim);
return=ustrtok(char *base_string,char *next_str,char *delim);
basesting= a|b|c|d
next_str=NULL
delim=|
after one call
===============
next_str=a
return=b|c|d
then we need to assing
base_string=return for further calls.
note:
1)No issue due to this function its only a help to understand the code.
2)i have tried my besr from my side.please look into this.
3)i have provided the sample data u can run the code very easily to see the output.
4)i suggest run the code with different type of descrition files and data files
change the columns numbers and all then u will observe the failure or else it works fine.
sory friend i appolize for this.okie i am posting my code again.
C Syntax (Toggle Plain Text)
#include<stdio.h> #include<math.h> #include<stdlib.h> #define FIXED 1 #define DELIMETED 2 #define MIXED 3 int errno; typedef struct dml_s { char **col_headers; char **column_delim; int *col_datatype; int *col_len; int col_cnt; int record_length; int file_type; //file type can be fixed,delimeted or max len with delimeters }dml; void display_intro(); dml* extract_dml_info(char*,dml*); int str_wc(char *); int str_semicl(char *); int pow_user(int,int); void display_dml_info(struct dml_s *record); void read_fixed(char *filename,int *size,char **header,int col_cnt,int record_len); void read_delimeted(char *filename,char **delmimeter,char **header,int col_cnt); char * ustrtok(char *base_string,char *next_str,char *delim); int main(int argc,char *argv[]) { if(argc!=3) { display_intro(); return; } dml *record; record=(dml*)malloc(sizeof(dml)); record=extract_dml_info(argv[1],record); if(record==NULL) exit(1); display_dml_info(record); if(record->file_type==DELIMETED) { read_delimeted(argv[2],record->column_delim,record->col_headers,record->col_cnt); } /* else if(record->file_type=FIXED) { read_fixed(argv[2],record->col_len,record->col_headers,record->col_cnt,record->record_length); } else if(record->file_type=MIXED) { printf("\n\nCONSTRUCTION ON PROGRESS FOR DELIMTED FILE WITH LENGTH\n"); return; } */ return 0; } /********************THIS FUNCTION EXTRACTS INFORMATION FROM THE *********************DML FILE AND RETURNS A STRUCTRUE***********/ dml* extract_dml_info(char *dmlfile,dml *record) { char buffer[100]; //IT BUFFERS ONE LINE AT A TIME FORM THE FILE char **line; //ALL THE LINES FROM THE FILE STORED IN LINE ARRAY char **col_desc; //STORES THE COLUMN DESCRIPTIONS char *temp,*token; FILE *fp; int i,j; fp=fopen(dmlfile,"r"); if(fp==NULL) { printf("Can't read from file: \"%s\"\n",dmlfile); perror("Error"); exit(1); } record=(dml*)malloc(20*sizeof(dml)); record->col_headers=(char**)malloc(200*sizeof(char*)); record->column_delim=(char**)malloc(200*sizeof(char*)); col_desc=(char**)malloc(200*sizeof(char*)); line=(char**)malloc(200*sizeof(char*)); fgets(buffer,100,fp); i=0; while(!feof(fp)) { if(buffer[strlen(buffer)-1]=='\n') buffer[strlen(buffer)-1]='\0'; line[i]=(char*)malloc(strlen(buffer)+1); strcpy(line[i],buffer); i++; fgets(buffer,100,fp); } record->col_cnt=i-2; //Getting the column count from the dml file /****************DML VALIDATION*************************************** The following validation taken place 1)DML starts with a 'begin' flag and end with 'end;' 2)Every statement ends with a semicolon except the begin 3)no semicolon should encounter inside 4)each line should consist of 2 words (a)column name (b)column desc *********************************************************************/ if(strcmp(line[0],"begin")!=0) { printf("Dml Error: at line 1\n"); } int k=0; int w_count,semicl_flag; for(j=1;j<i;j++) { k=strlen(line[j]); if(line[j][k-1]!=';') { printf("Dml error: Missing ; at line %d\n",j+1); return NULL; } if((strchr(line[j],';'))!=NULL && line[j][k-1]!=';' ) { printf("Dml error: at line %d\n",j+1); return NULL; } w_count=str_wc(line[j]); if( w_count != 2 && w_count != 3 && j!=i-1 ) { printf("Dml Error xxx at line %d\n",j); return NULL; } semicl_flag=str_semicl(line[j]); if(semicl_flag==-1) { printf("Dml Error yyy at line %d\n",j); return NULL; } else if(semicl_flag==-2) { printf("Dml Error zzz at line %d\n",j); return NULL; } } if(strcmp(line[i-1],"end;")!=0) { printf("Dml Error: at line %d\n",i); return NULL; } /****************END DML VALIDATION***********************/ for(j=0;j<record->col_cnt;j++) { token=(char*)malloc(strlen(line[j])+1); token=strtok(line[j+1]," "); if(token==NULL) { token="\0"; return NULL; } record->col_headers[j]=(char*)malloc(strlen(token)+1); strcpy(record->col_headers[j],token); token=strtok(NULL,"\n"); col_desc[j]=(char*)malloc(strlen(token)+1); if(token!=NULL) strcpy(col_desc[j],token); else { printf("Incorrect DML at line %d for %s\n",j+1,record->col_headers[j]); return NULL; } } /****************DETERMINE THE DATATYPE OF THE COLUMNS*********/ char *del; record->col_datatype=(int*)malloc(2*i*sizeof(int)); int flag; for(j=0;j<record->col_cnt;j++) { del=strstr(col_desc[j],"string"); if(del==NULL) { record->col_datatype[j]=1; del='\0'; } else record->col_datatype[j]=0; } /*****************************************************************/ int count_fixed=0; int count_del=0; char *temp_del_fst; char *temp_del_lst; int tempv; record->column_delim=(char**)malloc(10*sizeof(char*)); for(j=0;j<record->col_cnt;j++) { temp_del_fst=strchr(col_desc[j],'"'); temp_del_lst=strrchr(col_desc[j],'"'); if(temp_del_fst==NULL && temp_del_lst==NULL) { count_fixed++; record->column_delim[j]=(char*)malloc(2); record->column_delim[j]="\0"; } else if(temp_del_fst!=NULL && temp_del_lst!=NULL && temp_del_lst!=temp_del_fst) { count_del++; tempv=temp_del_lst-temp_del_fst; record->column_delim[j]=(char*)malloc(temp_del_lst-temp_del_fst); strncpy(record->column_delim[j],temp_del_fst+1,temp_del_lst-temp_del_fst-1); } else if(temp_del_fst!=NULL && temp_del_lst!=NULL && temp_del_lst==temp_del_fst) { printf("DML Error :at line %d ,inproper description\n",j+1); record->column_delim[j]="\0"; return NULL; } } //printf("count fixed:%d and count del :%d \n",count_fixed,count_del); /***********/ record->record_length=0; char *c_len; if(count_fixed==record->col_cnt && count_del==0) { record->file_type=FIXED; record->col_len=(int *)malloc(record->col_cnt*sizeof(int)); for(j=0;j<record->col_cnt;j++) { temp_del_fst=strchr(col_desc[j],'('); temp_del_lst=strrchr(col_desc[j],')'); if(temp_del_fst==NULL && temp_del_lst==NULL) { printf("Dml Error..( ) missing in column description\n"); } tempv=temp_del_lst-temp_del_fst; c_len=(char*)malloc(10*sizeof(char)); strncpy(c_len,temp_del_fst+1,temp_del_lst-temp_del_fst-1); int len=strlen(c_len); record->col_len[j]=atoi(c_len); record->record_length=record->record_length+record->col_len[j]; // printf("length:%d\tclen:%s\tlen:%d\t j=%d\n",record->col_len[j],c_len,tempv,j); } //printf("record length is :%d\n",record->record_length); } /**********/ else if(count_fixed==0 && count_del==record->col_cnt) { record->file_type=DELIMETED; if(strcmp(record->column_delim[record->col_cnt-1],"\\n")!=0) { printf("DML error,in the last field '\\n' missing\n"); return NULL; } } else { record->file_type=-1; printf("DML error,Unable to determine the file type\n"); return NULL; } return record; } /******A function to get the word count of a string*******/ int str_wc(char *str) { int limit = strlen(str); int i, change = 1, words = 0; for(i = 0; i < limit; ++i) { if(!isspace(str[i])) { if(change) { ++words; change = 0; } } else { change = 1; } } return words; } /*************************************************************/ /*********A fucntion to check semicolon occurs in between the line***/ int str_semicl(char *str) { int limit = strlen(str); int i,flag=0; for(i = 0; i < limit; ++i) { if(i < limit-1 && str[i]==';' ) { flag=-1; } else if( str[limit-1]==';' && isspace(str[limit-2]) ) { flag=-2; } else if( str[limit-1]==';' && !(isspace(str[limit-2]))) { flag=0; } else flag=0; } return flag; } /************************************************************/ /******CALCULATION POWER****************/ int pow_user(base,exp) { int i; int pow=1; for(i=0;i<exp;i++) { pow=pow*base; } return pow; } /****************************************/ /******IT'S A FUNCTION SIMILAR TO STRTOK BUT CAN HANDLE NULL VALUES******/ /******THIS FUNCTION SPILTS A STRING INTO TWO STRINGS W.R.T A TOKEN******/ char * ustrtok(char *base_string,char *next_str,char *delim) { char *str2; str2=(char*)malloc(100); if(base_string==NULL && strlen(base_string)<1) { next_str=NULL; return NULL; } int len=strlen(base_string); char *temp; temp=(char*)malloc(100); temp=strstr(base_string,delim); if(temp==NULL) { strcpy(next_str,base_string); next_str[strlen(next_str)]='\0'; return NULL; } else { int sub_len=temp-base_string; str2=(char*)malloc(100); strncpy(str2,base_string,sub_len); str2[strlen(str2)]='\0'; strcpy(next_str,str2); return (temp+strlen(delim)); } } /******************THIS FUNCTION DISPLAYS THE DML DESCRIPTION****************/ void display_dml_info(dml *record) { int i,j; if(record==NULL)return; printf("=========================\n"); printf("NO OF COLUMNS:%d\n",record->col_cnt); printf("RECORD LENGTH:%d\n",record->record_length); printf("FILE TYPE :\"%d\"\n",record->file_type); printf("=========================\n"); for(j=0;j<record->col_cnt;j++) { printf("===COLUMN %d===\n\n",j+1); printf("HEADER:\"%s\"\n",record->col_headers[j]); printf("DATATYPE:\"%d\"\n",record->col_datatype[j] ); if(record->file_type==DELIMETED)printf("DELIMETER:\"%s\"\n",record->column_delim[j]); if(record->file_type==FIXED )printf("LENGTH:%d\n",record->col_len[j]); } printf("=========================\n\n"); } /***************************************************************************************/ /*****************************Begin read_fixed************************************** THIS FUNCTION READS A FIXED LENGTH FILE AS PER THE SPECIFED DML STEPS ARE AS FOLLOWS: INPUT PARAMETER: A) File Name B) Column Headers C) Length of all the Columns D) No Of columns E) Lenggh of Each Record OUTPUT: Reads the file as per the input parametes and Display it on the Screen *********************************************************************************/ void read_fixed(char *filename,int *size,char **header,int col_cnt,int record_len) { FILE *file_read; long lSize; char * buffer; size_t result; fpos_t position; file_read=fopen(filename,"rb"); if(file_read==NULL) { printf("Can't read from file \"%s\"\n",filename); perror("Error"); exit(1); } fseek(file_read,0,SEEK_END); lSize=ftell(file_read); rewind(file_read); int j=0; int i=0; float record_count=lSize/record_len; int multiply=record_count*record_len; buffer=(char*)malloc(4); result=fread(buffer,1,size[0],file_read); buffer[size[0]]='\0'; rewind(file_read); while(!feof(file_read) || buffer==NULL ) { if(j>=col_cnt)j=0; if(j==0 && i<=record_count) { i++; } if(j==0 && i<=record_count) { printf("Record %d\n",i); printf("=============\n"); } buffer=(char*)malloc(10); result=fread(buffer,1,size[j],file_read); buffer[size[j]]='\0'; if(i<=record_count) printf("%s:\"%s\"\n",header[j],buffer); free(buffer); j++; } /*setting up the file pointer to get the leftover bytes*/ fgetpos (file_read, &position); position=(fpos_t)multiply; fsetpos (file_read, &position); /******************************************************/ if(lSize!=multiply) { char *left_over; left_over=(char*)malloc(record_len+1); result=fread(left_over,1,record_len,file_read); left_over[record_len]='\0'; printf("\n\n\nLeft Over Bytes:\"%s\"\n",left_over); free(left_over); printf("\nError:Incomplete record at end of file.\n"); printf("Either (1) The metadata does not accurately describe the data,\n"); printf(" or (2) The data has somehow been corrupted.\n"); } } /*********************************END read_fixed ****************************/ void read_delimeted(char *filename,char **column_delim,char **col_headers,int col_cnt) { FILE *data_file; char buffer[20000]; char *ext_line=NULL; char *temp=NULL; char **fields=NULL; int i,j; printf("filename:%s\n",filename); printf("columncount:%d\n",col_cnt); for(i=0;i<col_cnt;i++) { printf("del:%s\n",column_delim[i]); printf("header:%s\n",col_headers[i]); } data_file=fopen(filename,"r"); if(data_file==NULL) { printf("can't open file\n"); return; } printf("TESTING BREAK1\n"); j=1; fgets(buffer,2000,data_file); while(!feof(data_file)) { i=0; if(buffer[strlen(buffer)-1]=='\n') buffer[strlen(buffer)-1]='\0'; if(strlen(buffer)==0) goto nextline; ext_line=(char*)malloc(strlen(buffer)+1); if(ext_line==NULL) { printf("NUll Record Found\n"); } strcpy(ext_line,buffer); printf("\nRecord %d:\n",j); printf("=========\n"); fields=(char**)malloc(col_cnt*sizeof(char*)); char *next_str; temp=(char*)malloc(strlen(buffer)+1); next_str=(char*)malloc(strlen(buffer)+1); next_str=ustrtok(ext_line,temp,column_delim[0]); while(i<col_cnt) { if(next_str==NULL) next_str="\0"; if(ext_line==NULL) ext_line="\0"; if(temp==NULL) temp="\0"; fields[i]=(char*)malloc(strlen(temp)); strcpy(fields[i],temp); fprintf(stdout,"%s:\"%s\"\n",col_headers[i],fields[i]); ext_line=next_str; free(temp); temp=(char*)malloc(100*sizeof(char)); if(i==col_cnt-1) { column_delim[i+1]=column_delim[i]; } next_str=ustrtok(ext_line,temp,column_delim[i+1]); if(i==col_cnt-2) { if(ext_line==NULL) ext_line="\0"; strcpy(temp,ext_line); } if(i==col_cnt-1 && strlen(ext_line)>0) { printf("\n\nError:Incomplete field at end of record.\n"); printf("Leftover bytes :\"%s\"\n",ext_line); printf("Either (1) The metadata does not accurately describe the data,\n"); printf(" or (2) The data has somehow been corrupted.\n"); } i++; } j++; nextline: fgets(buffer,2000,data_file); free(fields); } fclose(data_file); } void display_intro() { printf("\n mdump: Version 1.0\n\n"); printf(" Copyright (c) 2009, Deepak Panigrahy. All rights reserved.\n\n"); printf(" Purpose: This program is a simple utility that display\n"); printf(" the source file as per the Record format.\n\n"); printf(" Usage: mdump <Dml FileName> <Data FileName>\n\n"); printf(" Use -h option for additional help.\n"); }
use code tags correctly. and make sure your code is indented. otherwise, it's a ridiculous mess, and no one will want to look at it.
[code=c] [/code]
[code=c]
c Syntax (Toggle Plain Text)
int main(void) { // hello, i am properly formatted C code. // aren't i nice? if (your_code == properly_formatted) { youWillGetResponse = TRUE; } else { youWillGetResponse = NULL; } return toBeginning(andTryAgain); }
Last edited by jephthah; Jul 13th, 2009 at 6:10 pm.
Try making a smaller example of the problems... that program looks a over complicated. If what you have work's, exept for a few errors, don't worry about changing it. Here's how I would go around doing it:
First read the description file to make a sscanf string, and make two arrays, one to hold the names of the variables, and another to hold the variables. Since the datatype can vary, maybe make the second array an array of pointers.
Next read the data file, record by record and as you index along print out the variables with there names.
Good luck.
First read the description file to make a sscanf string, and make two arrays, one to hold the names of the variables, and another to hold the variables. Since the datatype can vary, maybe make the second array an array of pointers.
Next read the data file, record by record and as you index along print out the variables with there names.
Good luck.
Last edited by Hiroshe; Jul 13th, 2009 at 6:56 pm.
"Sometimes, when I lie in bed at night and look up at the stars, I think to myself, "Man! I really need to fix that roof."-Jack Handy
•
•
•
•
i have written the code in notepad and compiled in UNIX environment with gcc compiler .
And if you still insist on using Notepad, I know it has a tab key -- learn to use it.
Last edited by Ancient Dragon; Jul 13th, 2009 at 9:59 pm.
![]() |
Similar Threads
- Physical memory problem (Windows Vista and Windows 7)
- Memory Problem? (C++)
- Dell Inspiron 8200 Memory Problem (Troubleshooting Dead Machines)
- ASUS mainboard memory problem (Motherboards, CPUs and RAM)
- Out Of Memory Problem (C++)
- Weird memory problem in Windows XP (Windows NT / 2000 / XP)
- Problem w/ LIMIT in CASE statement (MySQL)
- Memory problem? (Motherboards, CPUs and RAM)
- memory problem (Motherboards, CPUs and RAM)
Other Threads in the C Forum
- Previous Thread: Finding the number of variables equal to a certain value
- Next Thread: falling letters
Views: 1754 | Replies: 46
| Thread Tools | Search this Thread |
Tag cloud for C
api array arrays bash binary binarysearch bind c++ calculator char coke command conversion convert copypdffile createcopyoffile csyntax data database decimal dev-c++ directory directorystructure dude dynamic error exec fgetc fgets file fork frequency function functions givemetehcodez grade graphics help|help|help|help highest homework i/o infiniteloop initialization input int integer kernel lazy line linked linked-list linkedlist linux list loop malloc matrix measuring memory mysql no-code no-effort no-good operator output path pointer pointers problem process processing program programming read recursion recursive recv repetition reversal reverse scanf segmentationfault send socketprograming spoonfeeding stack string strings strtok structures student system telephone test turbo-c undefined unix user variable windows






