Program State - save as?

[Slavi]

In the past couple of days, I've been practicing with JavaFX and I made myself an app, which I am planning to use in the future. It is supposed to keep track of web pages and the passwords I use for them, usernames as well of course. The program itself works great now, however, when I have to keep the state/last used(or already written in) items, I am facing a problem. Mainly, databases are used for such purpose, arent they? However, right now I am having java objects as my data, not database items or what they are called. How should I save my information? Currently, I am saving to an XML file, and then reading from the file, but I have no idea whether that is a good practice to do so

This is how the app looks like( Only missing CSS to make it fancy)

[Hiroshe]

So your essentially making a clone of KeePass (in practice I'd probably stick to that; they take several steps to prevent side channel-attacks, and already use strong encryption).

Saving passwords in plaintext is never good practice.

You can use SQLite to work with the database. It will be less work for you and much faster.

[Slavi]

Hey Hiroshe,
Its more for personal knowledge that use. I 'll take a look at SQLite,
Do I have to convert my data type(java objects) to match what the database expects or it already accepts the same data type?

[Taywin]

If you want to deal with Java only (without DBM), simply encrypt the data before you write it out to xml. Using DBM is more convenient because you do not need to worry much about maintaining the file (and it is easier to read to & write from DB.

Usually, data type in DB is not neccessary to be the same as data in Java object, but it would be easier and better to have similar data type. However, some datatype could be difference between SQL and Java (i.e. integer in SQLite3 is 2^(63-1) but is 2^(31-1) in Java7), so you must know the limit of each data type in order to implement a workaround and such.

[Slavi]

How about encryption with DBM? Also, should I try to implement the encryption myself or use an existing library? I guess it depends on the purpose doesn't it? If I just want to deal with ecryption and practice on it, then I should definately go for it but if the app is/has to actually be used then no doubt go for a standard encryption to make sure no glitches or vulnerabilities would occur?

Also, what encryption is tipically used for this type of data in the real life?

@Taywin, thanks for pointing out limitations

[Taywin]

Normally, you would need extension to encrypt data for free DBM (not sure about Oracle itself because it is an enterprise version which should includes everything). If you are using SQLite3, you can look at this forum.

[JamesCherrill]

Before you start installing some random datatbase engine, remember that Oracle's JavaDB is already on your system. It's installed as a part of the standard JDK. It works very well indeed with Java (obviously!) and NetBeans knows all about it. You can start with it as a singe-user simple embedded database engine, and move it to a multi-user server configuration later if you need. There are loads of beginner tutorials on the web, including ones specifically for NetBeans.
Unless you already have some other database engine, I can't imagine why you would not use JavaDB

[Slavi]

So, would AES before saving into database be a good choice?

[JamesCherrill]

AES is a good algorithm, especially with a long key.
You wouldn't normally store a password, no matter how it's encrypted. You store a secure one-way hash of the password (salted) which cannot be decoded. To check the user's password you process it with the same salt and hash mechanism, and check if that matches the stored value. Here's a decent tutorial:
http://www.codeproject.com/Articles/704865/Salted-Password-Hashing-Doing-it-Right

[Slavi]

Nono, I don't want to hash it because I want to read it in plain text so I think encrypting algorithm is what I need, such as AES. Actually, what I am thinking now is about RC4, as far as I know it is easy to implement, might be a good personal thing to do and try to implement and use it, still not sure when to encrypt, what comes to my mind is encrypt what user enters and save into database? Then for decryption use the same key(simetric encryption) and get the details in plain text again

[JamesCherrill]

I was just referring to passwords. The other data will be OK with AES. You should never store a password, no matter how it's encrypted.

[Slavi]

So, if I want to keep track of my usernames and passwords on different pages, best way to do so would be pen and paper?

[JamesCherrill]

See the tutorial I linked you to.

[Slavi]

Hey James,
I've already read it as I am doing Masters in computer security. I've read couple similiar as well but my point here was that I want to implement these stuff with main goal java practicing. If I hash the passwords I won't be able to go backwards and show them back to me as I want to keep them in such a way that at any time I would like to check what was my password for a specific web page I would do it.

I do understand that passwords in databases should not be kept in plain text or simple hashing function such as MD5(from the point of view that with brute force or dictionary it is possible to find out what the password is), with different salt for each password, thats how people should do it so even if sqli(for example) is successful the acquired passwords won't be possible ( well to some extend practically) to be guessed.

So yeah, what I want is to encrypt information to store it in(Still i have no idea where thats why I am doing it in an XML file, and it is in plain text but empty of course still) and then when I select an item from the web page column, it would decrypt the data about it and present it to me

[JamesCherrill]

OK. If it's a learning exercise rather than a live system, then of course you can try stuff out rather than automatically implementing "best practice". AES would be the obvious choice.
If the data is all encrypted then its hard to see what database functionality could be usefully applied, other than just store & retrive... in which case you may as well just XMLEncode and encrypt the resulting text file(s).

[Slavi]

Kk, gonna take a look at XMLEncode, thanks :)

[JamesCherrill]

Sorry - correct class name is XMLEncoder
Just make sure your classes follow Beans standards (no-args constructor, getters/setters etc), put them in standard Collections (ArrayList etc) and you just need a single call to encode the top-level object of your app and it will automatically encode the entire tree below that. Use a ByteArrayOutputStream for the output so you can encode that before writing to a file.
XMLDecoder then re-creates your entire app tree from the decoded XML in one call.

[Slavi]

Hey James, I am sorry that i am asking in here but don't want to spam the forum with new thread for a question that came up now concerning XMLEncoder. I started looking around to find out more about it and came accross this article. What I see there is that XMLEncoder is writing information in plain text to an XML file, is that true?

It is probably my fault that i haven't mentioned it yet, i am using XStream to write to an xml file, I already got it, i can create new/modify existing/ delete items in the xml file. The problem that i am facing was that my data was in plain text inside the xml file. What I would like to do is to have in encrypted so that the only way to view the data is inside the application and in the xml file everything inside would be shown as "unrecognizable" text. Did I understand the idea behind XMLEncoder wrong, or could you point me to a different direction?

[Hiroshe]

Do I have to convert my data type(java objects) to match what the database expects or it already accepts the same data type?

Well, kind of. When you run an SQL query, it's a string. So you'll need to format your data into a string. Then when you read the data, generally the interface you're using to access the database will convert it for you. I would imagine that JavaDB might be more convienient.

Also, should I try to implement the encryption myself or use an existing library?

Since this is a learning excersize, it doesn't matter. If you want to learn about implementing it yourself (and some interesting optimizations you can do), I encourage you do to so. In practice, you always use a well supported and updated inplmentation of the encryption. You'll never implement your own cryptography youself in practice, unless your on a team that's expirenced with writing secure software (preventing side-channel attacks, etc...).

Also, what encryption is tipically used for this type of data in the real life?

Depends. Generally most people stick with AES. But other well tested algorithms work as well. Some others off the top of my head are Twofish, Serpant, Camellia and Chacha20.

Their are some other bits of cryptography you should mind as well. Such as a KDF used for key streching. As well as the mode of operation it's running it (always use a good IV and mode of operation). Validation and padding are also important (like Poly1305-1).

Also, you should mind what your encrypting, and why your encrypting it. If you encrypt the entire file, then it is indistinguishable from random data. That can be an advantage if you want to deny the existance of the file, or if you want to embed it.

So, would AES before saving into database be a good choice?

It depends. If you do that then people will be able to recognize the file. If you don't mind that then it's fine. You might also want to encrypt the username along with the password though (as well as other fields).

Before you start installing some random datatbase engine, remember that Oracle's JavaDB is already on your system.

This is a good suggestion. Here is a good compairison of the two http://www.sqlite.org/cvstrac/wiki?p=SqliteVersusDerby . I'd suggest using a database over using XML for this kind of thing (faster, easier to program, good expirience esp if you haven't dealt with databases). Text formats like XML, CVS, etc. are generally nice for sharing information between things. Databases are generally better for use within an application.

You wouldn't normally store a password, no matter how it's encrypted.

This is definatly correct - a raw password should always be stored using a strong KDF (not just a hash!), wheather it's in a database or not. No one but the user should ever know the password - not even the administrator. The exception is a users personal password manager. In this case, a lot of care should be put into the password manager. It's a great exersize, but I woulsn't suggest using it in practice without rigorus testing and review by lots of people.

Actually, what I am thinking now is about RC4

I wouldn't really recommend RC4, (you can if it's just for fun). I would expect to see RC4 publicly broken to a practical extent. I'd suggest Chacha20. It's A LOT more secure, and still simple to implement. Furthermore, it's designed in a way such that naive implementations should still show resiliance to timing attacks (don't depend on that though). Also, since it's a stream cipher, you won't need to worry about modes of operation. You still need to use a nonce though (with stream ciphers, you can never encrypt data with the same bitstream, so you'll need to introduce something that works like an IV). It can be verified with Poly1305-1 (otherwise you would ne unsure if the data was currupter - or worse - modified). You still need to choose a good KDF or key strenching mechanism though to derive a key for the cipher.

I do understand that passwords in databases should not be kept in plain text or simple hashing function such as MD5(from the point of view that with brute force or dictionary it is possible to find out what the password is), with different salt for each password, thats how people should do it so even if sqli(for example) is successful the acquired passwords won't be possible ( well to some extend practically) to be guessed.

I wouldn't recommend using a simple hash+salt. You're going to need to use a strong KDF and a salt. Hashes arn't ment for passwords - they're ment for verifying larger bodies of information.

However, since this is a password manager, you need to encrypt the passwords (and the key you use for the cipher should be derived from the password using some kind of KDF).

[JamesCherrill]

I just mentioned XMLEncoder as a simple way to encode your complete program state as XML. It's an alternative to XStream but it's a standard part of the Java API. Each has it's own plus and minus points - it's your choice.
With XMLEncoder you specify the output stream where the XML is to be written. If you use a ByteArrayOutputStream then the XML is written to a byte array in memory. You can then encode that with AES or whatever and write that to a file safely. I belive XStream will write your XML to a String, so you can encode that before writing it to a file.

[Slavi]

Much appreciated guys, thank you