| | |
A function which will transfer the data from a file to STL( vector or map) object.
Please support our C++ advertiser: Intel Parallel Studio Home
![]() |
•
•
Join Date: Sep 2006
Posts: 9
Reputation:
Solved Threads: 0
A function which will transfer the data from a file to STL( vector or map) object.
1
#1 Sep 14th, 2006
I'm creating a code which will map two distict strings.
I used two ways for that:
1) created a MAP and added raw data to it from file.
It is not that efficient.
2) created an object holding mapped values:
eg.
class c
{
string str1,str2;
public:
/// opearations
};
and Added these objects to VECTOR.
I have to search it sequentially As there are around 1.5 million enrties or objects
.
Can enyone show me the way to improve my program.
Or some other approach towards the problem?:rolleyes:
Does anybody having a function which takes the input as STL object reference and file path And load the data to STL object.
Also include exception handellings.
I used two ways for that:
1) created a MAP and added raw data to it from file.
It is not that efficient.
2) created an object holding mapped values:
eg.
class c
{
string str1,str2;
public:
/// opearations
};
and Added these objects to VECTOR.
I have to search it sequentially As there are around 1.5 million enrties or objects
.
Can enyone show me the way to improve my program.
Or some other approach towards the problem?:rolleyes:
Does anybody having a function which takes the input as STL object reference and file path And load the data to STL object.
Also include exception handellings.
Re: A function which will transfer the data from a file to STL( vector or map) object.
0
#2 Sep 14th, 2006
one suggestion: load them into a vector, sort them, then use a binary search algorithm to search for given string. But I suspect that is something like what a <map> would do for you.
Another suggestion: If the file that contains the 1.5 million records rarely changes, don't read them into memory. Write another small program that will be run only when the primary file changes. It will create an index file that contains fixed-length records with key string and integer offset into primary file where the record starts, sort the key file. Then in the main program you are writing use binary search to search the keys in the file. Since the index file will already be sorted when your program starts, you could just read the whole thing into a vectory and then do in-memory binary search. C++ std::string is a nice class, but will kill a program that uses millions of them. When reading the structure below don't convert the strings to std::string, leave them as c-style strings because they will load into vector a lot quicker.
Another suggestion: If the file that contains the 1.5 million records rarely changes, don't read them into memory. Write another small program that will be run only when the primary file changes. It will create an index file that contains fixed-length records with key string and integer offset into primary file where the record starts, sort the key file. Then in the main program you are writing use binary search to search the keys in the file. Since the index file will already be sorted when your program starts, you could just read the whole thing into a vectory and then do in-memory binary search. C++ std::string is a nice class, but will kill a program that uses millions of them. When reading the structure below don't convert the strings to std::string, leave them as c-style strings because they will load into vector a lot quicker.
C++ Syntax (Toggle Plain Text)
struct keys { char keystr[255]; unsigned long offset; }
Last edited by Ancient Dragon; Sep 14th, 2006 at 10:21 am.
Re: A function which will transfer the data from a file to STL( vector or map) object.
1
#3 Sep 14th, 2006
The best data structure to keep track of millions of record (i mean there is no such best but still...) B Trees.
B-Trees are specifically designed for managing indexes on secondary storage such as hard disks, compact discs, and so on, providing efficient insert, delete, and search operations. Like binary search trees, B-Trees contain nodes. Unlike binary search trees, however, the nodes of a
B-Tree contain not one, but multiple, keys, up to some defined maximum—usually determined by the size of a disk block. The keys in a node are stored in sorted order, with an associated child node holding keys that sort lower than it—every nonleaf node containing k keys must have k+1 children.
So if poss you can try to implement a class for B-Trees since they exhibit excellent performance where numerous records are concerned. Most Databases like MySQL also use some kind or variation of B+ Trees.
An implementation for ur reference can be found here:
http://www.cse.mrt.ac.lk/lecnotes/cs...nt5/index.html
Hope it helped, bye.[IMG]file:///C:/DOCUME%7E1/sos/LOCALS%7E1/Temp/moz-screenshot-1.jpg[/IMG]
[IMG]file:///C:/DOCUME%7E1/sos/LOCALS%7E1/Temp/moz-screenshot.jpg[/IMG]
B-Trees are specifically designed for managing indexes on secondary storage such as hard disks, compact discs, and so on, providing efficient insert, delete, and search operations. Like binary search trees, B-Trees contain nodes. Unlike binary search trees, however, the nodes of a
B-Tree contain not one, but multiple, keys, up to some defined maximum—usually determined by the size of a disk block. The keys in a node are stored in sorted order, with an associated child node holding keys that sort lower than it—every nonleaf node containing k keys must have k+1 children.
So if poss you can try to implement a class for B-Trees since they exhibit excellent performance where numerous records are concerned. Most Databases like MySQL also use some kind or variation of B+ Trees.
An implementation for ur reference can be found here:
http://www.cse.mrt.ac.lk/lecnotes/cs...nt5/index.html
Hope it helped, bye.[IMG]file:///C:/DOCUME%7E1/sos/LOCALS%7E1/Temp/moz-screenshot-1.jpg[/IMG]
[IMG]file:///C:/DOCUME%7E1/sos/LOCALS%7E1/Temp/moz-screenshot.jpg[/IMG]
I don't accept change; I don't deserve to live.
•
•
Join Date: Sep 2006
Posts: 9
Reputation:
Solved Threads: 0
Re: A function which will transfer the data from a file to STL( vector or map) object.
1
#4 Sep 15th, 2006
I also thought about using the B-tree only. But problem is with the string match.
The key, which is used for searching is string
the format is AB12333 or DB4839.
I'm not getting, what should be the maximimum degree of each node.
I 'm supposed to implement the the wild match also.
ie for A* output may be AB3847, AG48783, A62674.....
and AB?473 will yield AB1367, AB2677,
The key, which is used for searching is string
the format is AB12333 or DB4839.
I'm not getting, what should be the maximimum degree of each node.
I 'm supposed to implement the the wild match also.
ie for A* output may be AB3847, AG48783, A62674.....
and AB?473 will yield AB1367, AB2677,
Re: A function which will transfer the data from a file to STL( vector or map) object.
0
#5 Sep 15th, 2006
Then maybe you are looking for something along the lines of "Ternary Trees" which are used in games like Scrabble and solving crosswords which also require wildcard matching. Maybe you would want to look here:
http://www.cs.princeton.edu/~rs/strings/
Hope it helped, bye.
http://www.cs.princeton.edu/~rs/strings/
Hope it helped, bye.
I don't accept change; I don't deserve to live.
![]() |
Similar Threads
- STL vector - deleting the last element (C++)
- help:stl vector of user defined objects (C)
- conversion of text file into data base file (Visual Basic 4 / 5 / 6)
- empty or delete data index file in temp internet files (Windows NT / 2000 / XP)
Other Threads in the C++ Forum
- Previous Thread: Basic doubt
- Next Thread: (was) STL <map> question
| Thread Tools | Search this Thread |
api array based beginner bitmap c++ c/c++ calculator char class classes code coding compile compiler console conversion count database delete deploy desktop developer directshow dll download dynamic email encryption error file forms fstream function functions game givemetehcodez google graph gui homeworkhelp homeworkhelper iamthwee ifstream input int integer java lib linkedlist linker linux list loop looping loops map math matrix memory multiple news node number output parameter pointer problem program programming project python random read recursion recursive return sorting string strings struct temperature template templates test text text-file tree unix url variable vector video visualstudio win32 windows winsock word wordfrequency wxwidgets






