•
•
•
•
What is DaniWeb IT Discussion Community?
You're currently browsing the C section within the Software Development category of DaniWeb, a massive community of 456,428 software developers, web developers, Internet marketers, and tech gurus who are all enthusiastic about making contacts, networking, and learning from each other. In fact, there are 2,614 IT professionals currently interacting right now! Registration is free, only takes a minute and lets you enjoy all of the interactive features of the site.
Please support our C advertiser: Programming Forums
Views: 32872 | Replies: 32
 |
•
•
Join Date: Jul 2004
Posts: 16
Reputation: 
Rep Power: 5
Solved Threads: 0
Newbie Poster
I have a bcp output file from sybase that i need to parse using 'c' and write to a new file.
A sample line from bcp o/p file is below
9890000501:74667:0:6::2:0000:0:6:0:5:0:0:0:0:0:0:0:::9890000501:1:1:0::::::::0:0:3::::::::::\202^B:\202^B^D:0:0
:1:0:0:0:0:0:0:0:0:0::::0:0:0:::::::::0::0:0:0:0:
The file has ":" has the delimitor.
I need to create a new file with a single line string as something like this
CUST:NAME=9890000501,HT=74667&0&6,LT=PT-0&PQ-5;
There will be thousands of subscribers.I also will need to get data from different bcp output files to get the data to be filled in this string.
The 'c' program needs to be fast.
I'm a novice with 'c'.
The only way i know is read char by char and check for ":" and then store the value into a char array.
I would have as many variables as there are in the string!!
What would be the fast way to do this?
Is there any function that would in one shot give me the values separated by a delimitor of my choice?
Which is the best way to build up the final string?
Can i create the fixed parts of the string and then just fill up values that need to be taken from the bcp output file??
please HELP!!!
A sample line from bcp o/p file is below
9890000501:74667:0:6::2:0000:0:6:0:5:0:0:0:0:0:0:0:::9890000501:1:1:0::::::::0:0:3::::::::::\202^B:\202^B^D:0:0
:1:0:0:0:0:0:0:0:0:0::::0:0:0:::::::::0::0:0:0:0:
The file has ":" has the delimitor.
I need to create a new file with a single line string as something like this
CUST:NAME=9890000501,HT=74667&0&6,LT=PT-0&PQ-5;
There will be thousands of subscribers.I also will need to get data from different bcp output files to get the data to be filled in this string.
The 'c' program needs to be fast.
I'm a novice with 'c'.
The only way i know is read char by char and check for ":" and then store the value into a char array.
I would have as many variables as there are in the string!!
What would be the fast way to do this?
Is there any function that would in one shot give me the values separated by a delimitor of my choice?
Which is the best way to build up the final string?
Can i create the fixed parts of the string and then just fill up values that need to be taken from the bcp output file??
please HELP!!!
•
•
Join Date: Jun 2004
Location: Marin, CA, USA
Posts: 434
Reputation:
Rep Power: 5
Solved Threads: 10
Unprevaricator
Check out 'strtok()'; it will parse a string, stopping on one or more tokens; in this case your ':'.
The strings you parse could be referenced by an array of string pointers:
now all parsed strings are in the foundThings[] array.
The strings you parse could be referenced by an array of string pointers:
const char* foundThings[30]; // or however big
while (going)
{
foundThings[n] = strtok( inputString, ":" );
if (foundThings[n] == NULL)
going = false;
else
n++;
} •
•
Join Date: Jul 2004
Posts: 16
Reputation:
Rep Power: 5
Solved Threads: 0
Newbie Poster
WOW! thats neat.
Thank you!!
A couple of things!!
1) I read the man page for strtok.It says that it returns a (char *) pointer back.
Suppose i have a fixed line in which i want to fill up values from foundthing array.
for example ,
CUST:NAME=<value from array>,TP=<next value from array>&<next value from array>;
This values would be of variable length.
In perl i would write it in a single line with the '.' operator.
Is there a similar way here or i should store the fixed strings in a char array and do 'strcat' with each value.
The ouput file would have thousands of lines as above.
Example,
char S1[100]="CUST:NAME=";
strcat (S1,foundThing[0]);
strcat(S1,",TP=");
strcat(S1,foundThing[1]);
and so on.
Is there a faster way to do this work since efficiency is very important(reason to choose 'c' over perl)
Thank you!!
A couple of things!!
1) I read the man page for strtok.It says that it returns a (char *) pointer back.
Suppose i have a fixed line in which i want to fill up values from foundthing array.
for example ,
CUST:NAME=<value from array>,TP=<next value from array>&<next value from array>;
This values would be of variable length.
In perl i would write it in a single line with the '.' operator.
Is there a similar way here or i should store the fixed strings in a char array and do 'strcat' with each value.
The ouput file would have thousands of lines as above.
Example,
char S1[100]="CUST:NAME=";
strcat (S1,foundThing[0]);
strcat(S1,",TP=");
strcat(S1,foundThing[1]);
and so on.
Is there a faster way to do this work since efficiency is very important(reason to choose 'c' over perl)
•
•
Join Date: Apr 2004
Posts: 3,755
Reputation: 
Rep Power: 17
Solved Threads: 147
You may want to try fgets/sscanf. Maybe something like this. Then, if this is too slow, I'd look into other faster techniques.
#include <stdio.h>
int main(void)
{
const char filename[] = "file.txt";
FILE *file = fopen(filename, "r");
if ( file )
{
char line [ BUFSIZ ];
while ( fgets(line, sizeof line, file) )
{
char substr[32], *ptr = line;
int n;
fputs(line, stdout);
while ( *ptr )
{
if ( sscanf(ptr, "%31[^:]%n", substr, &n) == 1 )
{
ptr += n;
puts(substr);
}
else
{
puts("---empty field---");
}
++ptr;
}
}
}
else
{
perror(filename);
}
return 0;
} •
•
Join Date: Jun 2004
Location: Marin, CA, USA
Posts: 434
Reputation:
Rep Power: 5
Solved Threads: 10
Unprevaricator
Well, generally the fastest way to do it is to loop through the string in a while loop, not relying on standard string routines. Some string library functions may be implemented in assembler on some platforms, generally printf/scanf/strtok and the like aren't.
the non-standard-library parser could be something like this:
And then, if it were me, I'd have an array of the static strings to build up the final string:
If that's not fast enough, you could store the token lengths when you are parsing them and use memcpy() rather than strcpy() in the final loop.
the non-standard-library parser could be something like this:
// parse the source line into tokens[], returns the number of tokens found.
// note that some tokens may be empty.
int ParseLine( char* theSourceLine, const char* tokens[], int maxTokens )
{
currentToken = 0;
tokens[currentToken] = theSourceLine;
while (*theSourceLine)
{
if (*theSourceLine == ':')
{
*theSourceLine = 0; // null terminate this string
currentToken++;
if (currentToken >= maxTokens) return currentToken; // reached the limit; maybe return -1?
tokens[currentToken] = theSourceLine; // next token starts here
}
theSourceLine++;
}
return currentToken;
}static const char* finalStringConstants[MAX_FINAL_CONSTANTS] =
{
"CUST:NAME=",
"HT=",
"LT=",
<etc>
};
char outputLine[500]; // make it big enough
outputLine[0] = 0; // null terminate it to start
for (i = 0; i < MAX_FINAL_CONSTANTS; i++)
{
strcat(outputLine, finalStringConstants[i]);
strcat(outputLine, tokens[i] );
}If that's not fast enough, you could store the token lengths when you are parsing them and use memcpy() rather than strcpy() in the final loop.
•
•
Join Date: Apr 2004
Posts: 3,755
Reputation:
Rep Power: 17
Solved Threads: 147
My highlighing in red: http://en.wikipedia.org/wiki/Unix_philosophy
•
•
•
•
Rob Pike, a leading expert on applying the C programming language, offers the following "rules" in Notes on Programming in C as programming maxims (but they can be easily viewed as points of a Unix philosophy):Pike's rules 1 and 2 restate Donald Knuth's famous maxim, "Premature optimization is the root of all evil." Ken Thompson rephrased Pike's rule 4 as "When in doubt, use brute force." Rule 5 was previously stated by Fred Brooks in The Mythical Man-Month.
- Rule 1. You can't tell where a program is going to spend its time. Bottlenecks occur in surprising places, so don't try to second guess and put in a speed hack until you've proven that's where the bottleneck is.
- Rule 2. Measure. Don't tune for speed until you've measured, and even then don't unless one part of the code overwhelms the rest.
- Rule 3. Fancy algorithms are slow when n is small, and n is usually small. Fancy algorithms have big constants. Until you know that n is frequently going to be big, don't get fancy. (Even if n does get big, use Rule 2 first.)
- Rule 4. Fancy algorithms are buggier than simple ones, and they're much harder to implement. Use simple algorithms as well as simple data structures.
- Rule 5. Data dominates. If you've chosen the right data structures and organized things well, the algorithms will almost always be self-evident. Data structures, not algorithms, are central to programming.
- Rule 6. There is no Rule 6.
•
•
Join Date: Jul 2004
Posts: 16
Reputation:
Rep Power: 5
Solved Threads: 0
Newbie Poster
Thanks for the solutions!!
Can we use sprintf() to format the string at the end instead of strcat()?
I can use multiple "%s" to pick from the static string and the parsed output strings
and write to a buffer in one shot.
This buffer can be written to the file.
Is this the best way to go??
Can we use sprintf() to format the string at the end instead of strcat()?
I can use multiple "%s" to pick from the static string and the parsed output strings
and write to a buffer in one shot.
This buffer can be written to the file.
Is this the best way to go??
•
•
Join Date: Apr 2004
Posts: 3,755
Reputation:
Rep Power: 17
Solved Threads: 147
Do you something mean like this? Sure -- but the fields in each record would need to be all filled or all empty in the same places. And the format string can get a bit unwieldy -- unless you are only looking for the first several fields out of it.
#include <stdio.h>
int main(void)
{
const char filename[] = "file.txt";
FILE *file = fopen(filename, "r");
if ( file )
{
char line [ BUFSIZ ];
while ( fgets(line, sizeof line, file) )
{
char name[16], ht[8], output[128];
int a,b,c,d;
if ( sscanf(line, "%15[^:]:%7[^:]:%d:%d::%*d:%*d:%*d:%*d:%d:%d",
name, ht, &a, &b, &c, &d) == 6 )
{
snprintf(output, sizeof output,
"CUST:NAME=%s,HT=%s,&%d&%d,LT=PT-%d&PQ-%d",
name, ht, a, b, c, d);
puts(output);
}
}
}
else
{
perror(filename);
}
return 0;
}
/* my output
CUST:NAME=9890000501,HT=74667,&0&6,LT=PT-0&PQ-5
*/ •
•
Join Date: Jul 2004
Posts: 16
Reputation:
Rep Power: 5
Solved Threads: 0
Newbie Poster
I guess sscanf wont work since i have fields with variable length delimited by ":"
%31[^:]
will it expect minimum 31 chars..i dont understand it
Can i use fprintf() to write straight to the file??
Since it is variable length fields i will have to clear the buffer for each record and memset would be expensive!!
I believe it wont write "\0" after each string ..right?
Using fprintf(), can i use "\n" for the succeding fields to be written to the next line?
I want the lines in my file to be separated by a carriage return and a line feed.
Is it the same as "\n"?????
%31[^:]
will it expect minimum 31 chars..i dont understand it
Can i use fprintf() to write straight to the file??
Since it is variable length fields i will have to clear the buffer for each record and memset would be expensive!!
I believe it wont write "\0" after each string ..right?
Using fprintf(), can i use "\n" for the succeding fields to be written to the next line?
I want the lines in my file to be separated by a carriage return and a line feed.
Is it the same as "\n"?????
•
•
Join Date: Apr 2004
Posts: 3,755
Reputation:
Rep Power: 17
Solved Threads: 147
•
•
•
•
Originally Posted by reuben12
I guess sscanf wont work since i have fields with variable length delimited by ":"
%31[^:]
will it expect minimum 31 chars..i dont understand it
•
•
•
•
Originally Posted by reuben12
Can i use fprintf() to write straight to the file??
•
•
•
•
Originally Posted by reuben12
Since it is variable length fields i will have to clear the buffer for each record and memset would be expensive!!
•
•
•
•
Originally Posted by reuben12
I believe it wont write "\0" after each string ..right?
•
•
•
•
Originally Posted by reuben12
Using fprintf(), can i use "\n" for the succeding fields to be written to the next line?
•
•
•
•
Originally Posted by reuben12
I want the lines in my file to be separated by a carriage return and a line feed.
Is it the same as "\n"?????
|
•
•
•
•
•
•
•
•
DaniWeb C Marketplace
•
•
•
•
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
Linear Mode
Similar Threads
Other Threads in the C Forum
- Previous Thread: error coding switch
- Next Thread: recursive algorithm
