XML or SQLite?

[JOSheaIV]

Hello Daniwebbers,

Recently I have begun a new project that will go out to the internet and fetch information off IMDb about a source of media. Now currently when I do this I use an XML file to store what I have downloaded and parsed. For each piece of media a new XML is created. Each file is stored under the ID code used for the media on IMDb. There is also a master index file that contains the ID number and Title for the media.

Now XML is working great and everything, but I have started to wonder if it's the best choice. As time goes on more and more of these files will be created, to the point where I could have 100s, and while they are only say around 6 KBs, the number of them will be huge. Also, I want to allow for easy editing. The main files themselves might not require a lot of editing all the time but there may be times they do (plus the master index file will).

So my question is, should I stick to using XML or move over to SQLite? The SQLite would allow me one file for everything, and not to mention would be a lot easier for me to edit (I still can't find really clear guides on how to edit an XML file). However, I wonder what hte performance toll will be for using a SQLite file. As the file grows will it become slower?

What do you guys think? Which would you use and why? Is there positives and negatives I left out for both? I want to be able to access the data stored in these files rather quickly.

Thanks

[Ketsuekiame]

It depends how you use the data. If you want to do index and searching, the database is going to be far better at it. This comes at the expense that your data is (marginally) harder to read and edit.

If your objects are going to be loaded and cached, XML is going to be better as it's easier to read and edit. This comes at the expense of poorer performance as you will need to parse the XML. Also, if you're going to cache it all, I hope the target system has enough RAM :)

Personally, I wouldn't use XML for anything other than basic configuration and data serialisation for transmission (on small objects of a couple of Mb max, otherwise custom Binary serialisation)

[JOSheaIV]

Personally I am finding the Database to be easier to edit. Been trying to find a good guide to edit XML, and well not finding a lot (seems like you have to load in the whole document). Now granted I don't expect to edit alot, but when I do, I know how to do it in SQL already (which at this point means easier).

Also, each file is less then 10 KBs so loading into memory won't be to hard, however, I feel I could alter the program to fetch from a Database the necessary info a head of time.

But you kind of confirmed what I was thinking, that SQLite would be a lot easier to work with (and personally I am start to think accessing the data period would be simple).

Anyone else have reasons for or against?