An OpenSource Database

[Lance Wassing]

Hello Everyone,

I have started creating an open source database architecture, with a waa, and much more. There is many features that can blow SQL, and access out of the water, including but not limited to:
1) true multi-dimensionalism,
2) array fields
3)File embedding, of 30 or more formats..
4) compression, encryption, and file lock
5)13 Field Types
6) Maximum 123,456 Fields
7) 8,000,438 Records
8)25.gig file limit
9)Compatible with c/c++, python, Perl, PHP, CGI, VB, and qBasic
10)WAA

If anyone is interested in helping, please let me know

Thanks,
Lance Wassing

[Infarction]

1) Will your multi-dimensionalism come even close to SQL in terms of speed? How do you intend to implement it?
2) How will the performance of your arrays be? Why is this a better design?
3) Why do you need file formats in a DBMS?
4) Compression and encryption will only slow down a DBMS. Locking happens on any DBMS worth using.
5) T-SQL has at least 28 types that I counted. MySQL has at least 16.
6) That's a remarkably small number.
7) Also seems very limiting.
8) I don't know how big databases get, but that also sounds like a potentially limiting factor.
9) Most other DBMSs also have APIs in several languages
10) What is WAA?
11) Have you put any planning into a memory management system yet? How familiar are you with dealing with data that doesn't fit in RAM?

Not to be shooting you down too much, but you really haven't provided nearly enough information to make this project sounds usable, much less feasible.

[Lance Wassing]

Basically,

what i have put in the post is a bare minimum of what i have, the encryption is a 1gig encryption, which takes about 2 seconds to encrypt and decrypt. As far as my compression algorithms go, were running about a 2mb compression algorithm for now because like you said, that will slow down the system, and a better algortihm is yet to be found. Also, the memmory mangement technique im using is far from lmitting, and proves to be worth while, as ive tried a sample database, with multiple fields, and 500 records, which comes to about 18meg. The file opens in seconds, and doesnt cause a flood of system resource failures, as it is unique to the point where it bypasses the memmory completely and communicates directily with the processor, like on a MAC.

Further more, a WAA is a Web Application Adapter, allowing you to create cgi and php applications for example the can directly communicate with a c/c++ executable. To do this a proper web server must be host by the remote machine that will act aas a data store, or file server.

There is an API, but it is being developed as I go. A good reference would be the API for xBase 64 dbms. Further more, this dbms, is more of a db engine and front-end there of, rather than an editor for multiple formats, which is where the file formats come in.

As well, the largest database i personally have ever worked with is about 587 meg give or take, and the limits i have used give les of a restraint on those small boundaries.

Also, unlike SQL, the dbms, does not require any custom scripting, it will like to a cgi or C++ application using a library, and you develop from there, as it's built in multi-threading, does not require the user to code there own threads in there application.

If you require any more infor plpease let me know... constructive critism is always welcome.

oh... and no other database format to my knowledge allows for multimedia, or files in general to be stored within a field, which is the purpose for designing this structure. If i am wrong PLEASE let me know.

Thankyou,
Lance Wassing

[~s.o.s~]

Building something that famous requires a lot of planning and expertise by your side. How many years of developement experience do you have under your belt ? How many people have you by your side ? How far have you reached in this endeavour of yours? Building a database might require many things which might not be apparent to you right now.

[~s.o.s~]

oh... and no other database format to my knowledge allows for multimedia, or files in general to be stored within a field, which is the purpose for designing this structure. If i am wrong PLEASE let me know.

Ever heard of the BLOB datatype....

[vijayan121]

oh... and no other database format to my knowledge allows for multimedia, or files in general to be stored within a field, which is the purpose for designing this structure. If i am wrong PLEASE let me know.

you mean no other open source database.

this is what oracle claims about interMedia.
'With interMedia, you can type columns of relational tables as containing media. For example, you can take an existing human resources table that contains name, address, and other traditional data, and add an image column to contain a picture of an employee. Or you could take a product table and add a video column to show products in use. There is not limit on the number of media columns that can be added to a table. Through this mechanism, media can be easily associated with traditional, relational data.'

is this just hype by oracle?

[Infarction]

Basically,

what i have put in the post is a bare minimum of what i have, the encryption is a 1gig encryption, which takes about 2 seconds to encrypt and decrypt. As far as my compression algorithms go, were running about a 2mb compression algorithm for now because like you said, that will slow down the system, and a better algortihm is yet to be found. Also, the memmory mangement technique im using is far from lmitting, and proves to be worth while, as ive tried a sample database, with multiple fields, and 500 records, which comes to about 18meg. The file opens in seconds, and doesnt cause a flood of system resource failures, as it is unique to the point where it bypasses the memmory completely and communicates directily with the processor, like on a MAC.

1gig encryption? 2mb compression? What do those mean? Are you using a 8589934592 bit encryption key? This isn't making sense. Also, the 2 seconds to encrypt-decrypt: are those for each database query? If so, that is irrationally long, and would never survive in a production environment.

There is an API, but it is being developed as I go. A good reference would be the API for xBase 64 dbms. Further more, this dbms, is more of a db engine and front-end there of, rather than an editor for multiple formats, which is where the file formats come in.

I'd recommend you take a few weeks to design before you code. Developing an API as you write the system will almost always yield a very ugly, disorganized code base and API.

Also, unlike SQL, the dbms, does not require any custom scripting, it will like to a cgi or C++ application using a library, and you develop from there, as it's built in multi-threading, does not require the user to code there own threads in there application.

Actually, the structure of SQL is such that the specific implementation of operations can be determined by the DBMS optimizer rather than by the coder. For example, if you have a query:

SELECT * FROM Users u INNER JOIN Items i ON i.owner = u.id

Will the programmer (writing the client app) know which type of join operation will be most effective? Likely not, as it depends on the data set being worked with. SQL optimizers can quickly estimate the number of expected results, then determine which algorithm for the join is likely to be most efficient.

oh... and no other database format to my knowledge allows for multimedia, or files in general to be stored within a field, which is the purpose for designing this structure. If i am wrong PLEASE let me know.

If I were designing a database, I'd have the DB hold links to the multimedia. A few reasons for this: 1) saving a link is compatible with DBs that only deal in text, 2) it's more portable for a system design, i.e. if the text is part of a link which may have other parts in client-side storage or elsewhere in the DB, and 3) it will always take longer when you have to work with more data; by keeping only text in the database, the DBMS can execute faster and the client app can request any multimedia files returned from a DB query.

If you're doing this simply as an excersize, I think it's a great idea, but I'd say you should do a lot more research and planning. Unfortunately, research and planning are two of the most important parts of a project of any real size. Anyways, good luck :)