How to secure connections over the internet?

[zachattack05]

I'm sorry if this is the wrong place to put this, but since I normally code in C#, and my potential solution would involve using C# I figured this would be the best place to start.

I've been given a task at work to allow our customers the ability to transmit confidential record information from their office to our in-house, or possibly a new cloud-based, server.

I have been rather adamint that to transmit said data, a SSL certificate must be aquired by us through a third party. But I'm wondering if that really is true.

The more I researched SSL certificates the more I've come to realize that all they really are is one company vouching for another. The encryption (even on an expired certificate or on a self-signed certificate) works and the encryption is just as secure as one that isn't. Sure, the user is presented with nasty icons and red screens showing that "hey! this may not be safe!" But if the user doesn't visit a "https" prefixed website and only visits a "http" website, what would be wrong with encrypting data client-side, submitting it to our server, decrypting it server side and vice-versa?

Encryption is encryption right?

Or what if a WinForms app was created that did the same as above? Encrypt data, submit it to our servers and the servers decrypt it.

I just can't justify paying thousands of dollars a year to have Verisign, or whoever, issue us a certificate when 99% (I'm willing to bet) of the users on the internet don't even bother checking the validity of the certificate.

I obviously want to make sure everything IS secure, and I'm not downplaying the role of SSL certificates or keeping things secure, but I just fail to see the logic behind aquiring one, if the same type of encryption can be achieved in-house with better control and, if you ask me, better security.

Any thoughts or opinions?

[JorgeM]

a SSL certificate must be aquired by us through a third party. But I'm wondering if that really is true.

Wondering what exactly? can you clarify...

The more I researched SSL certificates the more I've come to realize that all they really are is one company vouching for another.

The certificate is only good if the parties using the certificate trust the issuer. A classic example.... when someone shows you their driver's license, you trust the data that is on it, because you trust the issuer.

The encryption (even on an expired certificate or on a self-signed certificate) works and the encryption is just as secure

Yes of course it is still secure. Your browser will generaly warn you for one of three conditions... 1) your computer doesnt trust the issuer of the cert, 2) the cert is expired, or 3) the host your computer is talking to doesnt match the name on the cert. AGain, with a driver's license...say it expires, that doesnt make the information on the card invalid. its just expired and you may consider asking for another piece of identification that you trust.

I just can't justify paying thousands of dollars a year to have Verisign, or whoever, issue us a certificate when 99% (I'm willing to bet) of the users on the internet don't even bother checking the validity of the certificate.

You are not required to use a provider like Verisign. Your organization can set up its own Cert root and issuing servers. The problem is that no one on the internet trusts your root. With verisign, for example, windows computers already have the public certs installed on the system so it doesnt require any other action for the user. If you have your own certificate infrastructure, you have to distribute hte public certs to your customers so they can import the certs in their systems OR as you indicated, deal with the browser warnings...

achieved in-house with better control and, if you ask me, better security.

I doubt that the in-house solution would have better controls. Say you setup your own infrastructure and you have 100,000 customers. Now your private keys are comprimised... what are you going to tell your customers? Verisign and other cert companies have invested quite a bit into security and have alot of experience in that field.

In your case, i think you simply need to make a decision on whether or not its worth the risk vs cost. its a business decision.

As far as which cert is more secure, assuming both certs are created equally....it doesnt matter if it comes from verisign or your private cert infrastructure. a cert is a cert.

[zachattack05]

Wondering what exactly? can you clarify...

I'm wondering if a Certificate is even necessary.

I do understand your examples of a drivers license. I understand the purpose of a certificate and the logistics of how they work, but I'm just wondering if I have an application that runs in a browser or a WinForms app, and the app itself encrypts the data on the client side, then sends it to our server where it is decrypted and used is there any real advantage in having a certificate?

I mean, I guess my application can't really use a certificate if it is a WinForms app (I don't think) but if a web app at say http://www.somerandomwebsite.com/myapp was doing the same thing as my WinForm app would do and it is obviously not hosted using a certificate, what would be the major drawback? The data is encrypted either way.

Granted there is always the chance of a MITM attack, and I'm sure it could happen and has happened, but the same could be said for a site hosted with a certificate. Hack into the site, change the code and now your encrypted, SSL certificate is encrypting data and sending it somewhere else instead of where it's supposed to go. I don't see how a certificate can deter that.

For example, if I have a login page at https://www.somerandomwebsite.com/login and it's secured with a certificate and when a user submits the form it posts the data to https://www.somerandomwebsite.com/checklogin before sending them to the users area, but someone hacks into the server, and changes the "checklogin" script to not only verify the credentials but save a copy to the server's hard drive so it can be retrieved later...the user doesn't know any better and no warnings or errors are thrown up.

I would say having my own encryption methods and keys that don't rely on a third party is a lot safer and easier to control. But I'm really not sure.

I'm not so sure that even going so far as setting up a certificate authority is the best route. If I just encrypt the data and rotate keys on a regular basis (and update the app to reflect that change) I fail to see how getting a certificate from a third party is all that advantageous. Is it?

[hericles]

My take is that, for knowledgeable web users, the https and the lack of warnings is a sign of trust. Your self encrypton method, while it would be valid if done right, is invisible to the user. they wouldn't know what was happening in the background and so your website LOOKS unsecured.

[JorgeM]

If you are using considering a web application, your only option is to use HTTPS with certificates to encrypt your traffic over the web, unless all you are interested in encrypting is file uploads. For file uploads, your users would have to first encrypt the file using whatever encrypting process you have and when the user uses the file upload control in the browser, the traffic will be transferred via HTTP (unsecured) to your target server. The only thing in that stream that would be encrypted would be the file in the payload. Someone that intercepts this traffic would have your data and could begin their process to decrypt the file using a variety of methods. With a browser, the only way to encrypt the traffic is to use HTTPS with certificates.

If you are considering a winform, and I have little experience with winform dev, yes i would assume that you can create your own encryption/decryption process since you have contorl over the client app and target server app. However, how are you going to manage the keys used to encrypt the data? Both the client and server would have to have this information ahead of time prior to the transfer of data or you would ahve to use one set of pre-shared keys for all of your clients and server. Not very secure if the pre-shared keys are exposed.

So to clarify... using certificates in a web app environment.. the certificates arent actually used to encrypt the data, they are used to quickly and very effectivty transfer the pre-shared key that is generated for the session between the client and server. Every set of client/server session will have their own keys generated. Once these key(s) are negotiated and transferred from client/server, the client can now encrypt the traffic using the pre-shared session key. Once the session is over, the keys are not reused. When the next visitor access the web server, the process starts again with the certicates being used to create a secure channel, the client and server negotiate the keys and the process continues.

Im not a PKI expert by no means though...

[Hiroshe]

First, lets look at how (simple) symetric encryption works.

1. If Alice wants to send a message to Bob.
2. Alice tells Bob the password beforehand.
3. When Alice sends the message, she encrypts the message with the password. encrypt(plaintext, password) -> ciphertext.
4. Then Bob uses the passeord to get the message back. decrypt(cipthertext, password) -> plaintext.

The problem with this is that Alice needs to tell bob the password in advance.

Now, let's look at how (simple) assymetric encryption works:

1. Alice and Bob both create secret and public keys.
2. If Alice wants to send a message to Bob, Alice looks up his public key, and encrypts a messasge with it. encrypt(plaintext, bob_public) -> cipthertext.
3. Bob reads the message using his private key. decrypt(cipthetext, bob_secret) -> plaintext.

The problem with this is that Alice must be use that the public key IS Bobs. Otherwise someone can easily intercept the message, and send their own key as Bob's private key. This is called a MiTM attack (as refferenced earlier).

Now lets look at how a simplified tls session works:

1. Alice perminantly creates a public and private key. Alice keeps the private key secret.
2. When Bob wants to converse with Alice, Bob asks for the public key.
3. For all Bob knows, the public key could be fake. He needs to verify that it's real by asking a predefined list of CA's.
4. After it was verified to be real, Bob chooses a shared-secret key for this session, and sends it to Alice uisng her public key.
5. Now both parties use thr shared-secret for regular symetric encryption.

Back to your question. If you ask the user to ignore the security warning (which isn't easy on all web browsers), then you open yourself for a MiTM attack. This attack is fairly easy to do. The attacker just intercepts and sends his own certificate at the server. Yes, the data is still encrypted. But the encryption is between the attacker and the user. You might at well not enrypt anything if your not going to use a CA.

In short: If your using the users web browser, use a certificate with a CA. There is no other practical way around this.

Now, if your considering writting your own client, you do have a few more secure options.
1. Your first option is to use tls and a CA in the cleint, as well as to send the client.
2. Your second option is to use tls without a CA, and transfer the client to the user over the internet over a secure connection. (ie, include the certificate, but make sure that you host the client on a service that provides tls file transfers, like dropbox perhaps).
3. Your third option is to use tls without a CA, and transfer the client with a CD (or other physical medium).
4. Another option would be not to use tls or a CA at all. You still need to give the cleint to the user securely to the user (cd, or through a secure hosting service). And then your client asks the user for their password, and a KDF of that password is used for symmetric encryption.
5. You might also be able to devise a system using PGP and email instead.

[Hiroshe]

Granted there is always the chance of a MITM attack, and I'm sure it could happen and has happened, but the same could be said for a site hosted with a certificate. Hack into the site, change the code and now your encrypted, SSL certificate is encrypting data and sending it somewhere else instead of where it's supposed to go. I don't see how a certificate can deter that

The problem with that logic is that it's relatively easy to perform a MiTM attack without ever touching the servers. If you implement your own encryption on the web page itself, it is incredably easy to MiTM, send the user a fake web page, and get the fake web page to send the password back to the user. This can all be done within the users own network.

Whereas if you use a CA, then you need to actually attack the servers themselves which is a very difficult task. By using a CA, you are oliminating the possibility of simply intercepting and giving the user a fake page. Now the attacket needs to target your sever, which will be much more difficult.

[zachattack05]

I'm still not sure I see the benefit.

Hiroshe: If your example is correct and my end users have private and public keys and our servers have the same, I fail to see (especially if we assign those keys to our clients) how a CA is useful at all. If our application is distributed to our clients and we include our server's public key with the software and the end user's private key is generated on their machine based on data we have (as in, we also know their private key ahead of time) why would we need to bother asking a CA to verify it if we already have it? We can just check it internally couldn't we? If our servers check our client's private key and it doesn't match, then it's fake. If the client's check our server's public key and it doesn't match, then it's fake.

Rotating keys wouldn't be too difficult if you ask me either. Our servers can have say, 5 active keys at one time. Each time a client connects, if the client meets certain criteria (say it rotates after every 50 logins or something) then our servers keep track of it and say "hey, this is login #50! use key #2 now instead of key #1, but encrypt key #2 with key #1 and send it to the client first, confirm the client has the key by having the client respond with a confirmation that can only be decoded with key #2. If it works, great, move on, if not, try it again or end the session." that way if Joe Blow hacker steals traffic data and finds the key, by then hopefully the key is different now and if they break into our systems and steal any encrypted data, they can't do much with it if the keys rotate fast enough.

I don't know, am I missing something? Is the idea I just laid out so terrible?

[Hiroshe]

There are 2 trivial attacks that you need to consider:

1. If the client is sent over http, then the attacker will try to intercept and replace it with his own. Say your program asked the user for a number, ecrypted it, and sent it to your servers. An attacker would say use ARP Cache Poisoning to set up the MITM attack, wait for the server to send the client, and the attacker would send his own version instead. The user wont no the difference if the fake looks exactly the same. The fake version emails the attacker the information instead of sending it to your server.

2. If the client doesn't verify a tls certificate that is sent from the server, then the attacker can again use something like ARP-CP to make himself "look" like the server, and then send his own certificate, and carry on to steal the information.

If your considering sending the client (be it a webpage or an application) over http, don't. It's trivial to attack using method 1.

If your considering sending the client over https with a certificate not verified by a CA, don't. It's trivial to attack using method 2.

The only way you can securely do this is without a CA is to give the client to the user using a physical medium, or to give the client to the user using another hosts verified https. Additionally, this client must already contain the certificate to be used, or the client must use symmetric encryption. (otherwise it's vunerable to method 2).

If you want to go with the symmetric encryption, use a strong KDF, strong encryption as well as good integrity checking. Never ever store plaintext passwords anywhere. Not on the server, not on the users computer, and you should probably avoid saving it in RAM for too long (erase it securely from RAM once the KDF has been calculated).

If you are going to do anything more advanced then a webpage with verified https, I would strongly reccomend you consult someone who knows cryptography and security well.

[zachattack05]

Hiroshe,

What is the difference? If I use a CA and the hacker intercepts the data, couldn't they also just immitate the CA?

I guess I'm missing something here right? I mean, the encryption level is the same regardless of if a CA issues a certificate or I generate the same encryption, right?

Why would there be any difference? Couldn't a hacker still intercept the transaction and send their own data?

[zachattack05]

Oh, to clarify too...the connection would NOT be over a http or https. It would be through a protocol that we would create in-house (if we use WinForms that is)

[Hiroshe]

The CA's also use certificates to verify themselves. The difference is that the users computer already contains a list of pre-defined CA certificates. These come with the operating system (a secure physical medium). The only way for an attacker to impersonate a CA is for them to have physical access and the ability to modify the users computer/the installation medium.

And since you transfer your client after the connection has been verified by a CA, the attacker cannot impersonate the server, and cannot change the client in transit.

Thus, you know that the client on the users computer is unmodified.

The easiest way to explain it is: "An attacker cannot perform a MiTM attack when tls is used. The user knows he's talking to the server, the server knows he's talking to the user, and all of the information is encrypted".

You only 2 options is to use tls, (the certificates are already on the users computer), or to use a physical medium. It is impossible to create a secure connection only using the internet without using tls.

I strongly advise not diviating from standard cryptographic protocols. A huge number of people oversee that these protocols are secure. If you roll your own, you'll make mistakes. And by the sounds of it, your not trained for creating cryptographic protocols. I dout you have the man power (programmers and mathematicians) to roll your own protocol.

Read my last post. Those are all of your secure options.

[zachattack05]

I don't see how a certificate installed in a users browser affects a WinForms application? The data would not be transmitted through HTTP. Can a SSL certificate even be used with WinForms?

[Ketsuekiame]

They don't get installed into a browser, they're installed in the computers certificate store. (either by user or by machine)

Certificate/SSL Certificate no difference. Certificates are produced to a well defined standard and they include a key of a certain size along with descriptor information. That is a certificate. You could make one yourself with the right knowledge of the protocol :)

Encrypting a connection still involves using a key to encrypt data and relies on the secrecy of said key for it to be secure (otherwise known as the private key)

There is something that exists called the Diffie-Hellman Elliptic Curve. This is a good place to look for connection based, non-certificate based symmetrical encryption.

ECDH relies on generating a private key and sending a public key to the client. The client starts the ECDH process by using the public key to generate their own private key. The server responds with its own public key and the client generates the same private key. These private keys can be used to encrypt over the wire without ever being sent to one another.

Example;

// Client Side

public byte[] GeneratePrivateKey()
{
    using(var keyGenerator = new ECDiffieHellmanCng())
    {
        keyGenerator.KeyDerivationFunction = ECDiffieHellmanKeyDerivationFunction.Hash;
        keyGenerator.HashAlgorithm = CngAlgorithm.Sha256;

        byte[] clientPublicKey = keyGenerator.PublicKey.ToByteArray();
        byte[] serverPublicKey = myService.ExchangeKey(clientPublicKey);

        byte[] privateKey = keyGenerator.DeriveKeyMaterial(serverPublicKey);

        return privateKey;
    }
}

// Server Side
public byte[] ExchangeKey(byte[] clientPublicKey, out byte[] privateKey)
{
    using(var keyGenerator = new ECDiffieHellmanCng())
    {
        keyGenerator.KeyDerivationFunction = ECDiffieHellmanKeyDerivationFunction.Hash;
        keyGenerator.HashAlgorithm = CngAlgorithm.Sha256;

        byte[] serverPublicKey = keyGenerator.PublicKey.ToByteArray();

        privateKey = keyGenerator.DeriveKeyMaterial(clientPublicKey);

        return serverPublicKey;
    }
}

[Hiroshe]

I don't see how a certificate installed in a users browser affects a WinForms application?

How is the winforms client downloaded? If the answer is "through a web browser using http", then the client could have been easily compromised. Even if your winforms client has it's own protocol, it's irrelevant if the client is sent over http.

There is something that exists called the Diffie-Hellman Elliptic Curve. This is a good place to look for connection based, non-certificate based symmetrical encryption.

DHEC alone wont authenticate the server. For example, after establishing a key, you still need to make sure you established it with the right person! Though it is often used in conjunction with signing (which involves certificates unfortunatly).

--

In the field of cryptography, there is no known way to establish a secure connection with a server without having some previous information on the server. Attempts to do so will be vunerable to a MITM attack. It doesn't matter how you look at it.

It must be bootstrapped with something already on the users computer (ie, pre-existing CA certificates -> CA -> getting servers certificate -> establishing secure connection), or be boostrapped with a physical medium.

Since TLS/SSL is the only 'pre-existing' information on most computers, you're stuck with (a) using TLS/SSL (even if it's just to download the client) or (b) use a physical medium to transfer your client.

There are no other possibilities. You must have pre-existing information that is known to be correct to securly connect to a server.

--

So, why not upload your client to some hosting service with https like dropbox? It's free, secure and means less work for you.

If your using .net, then you can use something like System.Security.Cryptography.Aes along with System.Security.Cryptography.Rfc2898DeriveBytes.

Encrypting data:

1. Send the server PBKDF2('username', iterations=100000, salt=0). The server responds by sending you your salt.

2. Then you use PBKDF2('password', iterations=100000, salt=salt) to calculate the key.

3. Encrypt the message with the key. In System.Security.Cryptography.Aes, the IV, mode and padding are automatically handled with secure defaults.

Decrypting data:

Use the derived key to decrypt the message.

Note: The server never stores the users password. The only thing that should be stored is the username, PBKDF2 of the username and PBKDF2+salt of the passwrod and the salt.

With this you know that the client is not compromised (unless the users computer is already compromised). The client is able to start a secure connection with the server because you and the server have a shared secret. On order for a MITM attack to be successfull, you would need to already know the shared secret.

[Ketsuekiame]

DHEC alone wont authenticate the server. For example, after establishing a key, you still need to make sure you established it with the right person! Though it is often used in conjunction with signing (which involves certificates unfortunatly).

True, but this isn't the requirement as far as I read it. The requirement being "I want to encrypt some data on the connection". If you want any kind of server validation (which is generally why TLS is used) then some information must have been exchanged beforehand.

Attempts to do so will be vunerable to a MITM attack. It doesn't matter how you look at it.

Agreed, but at least with ECDH they have to have intercepted the start of the exchange to make it easy. It's not perfect, but if you don't want/can't have pre-shared information it's the best place to start.

Additionally, we are using the WAN to establish a connection, but the WWW isn't really in use. If we're verifying that the server is correct, the server should also verify that the client is correct.

Finally, it's important to note that certs aren't the be-all and end-all of security. Any cryptographically strong pre-shared key would suffice as a mechanism of client/server validation. Certs just happen to contain this information. The downside is that you need to pay for them and that might not be necessary, depending on your requirement.

For example; if you had a crypto pre-shared key of 512 bytes, you could use the shared secret to encrypt the public ECDH keys in a symmetric encryption function. Or; as Hiroshe pointed out, use it to derive a second key.

Note: Do not use PBKDF2 to encrypt every message to and from your system. It's good for protecting things over a long term (such as hashing passwords for the DB) but the processing power required is large and does not scale linearly with increased/decreased iterations. So attempt to keep it to a single exchange per session if possible, if you decide to use it.

[Hiroshe]

but if you don't want/can't have pre-shared information it's the best place to start.

I wouldn't transfer important client information with a connection vunerable to a simple MITM attack. If the information isn't as important, and you simply don't care, go for it. Just make sure that people know how it's vunerable.

The OP originally stated The encryption (even on an expired certificate or on a self-signed certificate) works and the encryption is just as secure as one that isn't. The thing I'm trying to make clear is that that statement is not true, and that proper TLS/SSL is much safer then one that doesn't use any TLS/SSL or a physical meduim. If the OP is ok with the fact that it is less secure, then by all means.

Any cryptographically strong pre-shared key would suffice as a mechanism of client/server validation.

Definatly. In fact, I like the idea of using a physical medium with a symmetric cipher better then using assymetric cryptography. But it's more inconvienient depending on what you're using it for.

Do not use PBKDF2 to encrypt every message to and from your system.

True. The server should only need to calculate the KDF once per password, not once per session. My laptop doesn't have any problem calculating 17 million SHA512 iterations in a second as a guideline for how much work it takes to compute them.

[zachattack05]

I tend to agree Ketsuekiame, certificates generated by CA's contain cryptographic information (keys) and other information that doesn't really change the cryptography of the data. If I can generate keys myself without a CA (basically act as my own "CA" if you will) then I really don't see how a certificate is any better.

Granted, a CA has more experience than I do in Cryptography I'm sure, but even if I purchased a certificate, I would still have to know how to use it correctly in order to keep things secure so what difference does it make where the key comes from or if it is stored on our server or some CA server somewhere?

I think I'm just going to try things on my own. Granted, it might be a bad idea, but I'll make sure I read up on it first. If anyone here would like to offer their help, I'd love to have advice. Maybe I'll start a new discussion with my plan/idea and see how it fairs?

Thanks for the help! I really do appreciate it!

[Hiroshe]

If I can generate keys myself without a CA (basically act as my own "CA" if you will) then I really don't see how a certificate is any better.

A certificate from a CA is better because the CA is registered with the users computer. You're not registered with the users computer.

You WILL be vunerable to a MITM attack, while a CA will not be.

If you're OK with being vunerable to a (relatively) simple attack, go for it. But you need to be aware that it is vunerable.

What scares me is your insisting that not using a CA is just as secure as using a CA! That is simply not true dude! If your using your own, you need to be aware that it is simply not as secure as a physical medium or TLS/SSL. Be aware that a practical attack exists.

I would still have to know how to use it correctly in order to keep things secure so what difference does it make where the key comes from or if it is stored on our server or some CA server somewhere?

That's like saying "if it's vunerable to this attack, even if I fix it, there will still be some other vunerability, so whats the point?"

[zachattack05]

Hiroshe;

If I create my own CA, a end user can install my Root Certificate onto their computer can't they?

If so, how is this any different?

[Hiroshe]

How are you planning to send the CA cert to them? If your sending it over http, then again - it can be intercepted. Even worse - now that the attacker has control over a CA that the users computer, all connections that use user makes may be compromised (ie MiTM, then send a certificate that authenticates with the attackers CA). Even if it wasn't intercepted, what if your CA's private key was stolen (are you sure you're capable of protecting it)? Again, all of your clients would be vunerable to attack - even if they're using another service. At this point it's safer to use an unsigned certificate (though thats also easily defeated, but atleast it's just your program that's compromised).

I would never recommend to a user that she installs an untrusted CA. I would never touch a service that required it's own CA. The only time that might be usefull is within an organization.

What makes "rolling your own CA" and using a default CA is that the default CA's are either installed with a physical medium, or they're installed with a system update which itself is protected by tls. Also, they are trusted. They have a lot of experts constantly making sure everything is safe - and it's not an easy job.

If you don't want to pay for a certificate - fine. There are good and easy alternatives that are free. Like using a hosting service to securely send a client that uses it's own block cipher. Or use GnuPG (which is downloaded through tls, and connects to keyservers using tls) to send the client. Or use a CD + block cipher.

Why would you want to go out of your way to avoid the safer, easier solutions?

[RichardGalaviz]

Having a quick Internet association that is "constantly on" when you need to surf the Web is extraordinary for you, yet its likewise incredible for programmers from around the globe who clear through a great many irregular IP locations searching for machines that they can misuse. What's more what they can do is truly very frightening. Without any noticeable sign or cautioning, programmers can penetrate your framework to acquire individual data about you or to utilize your machine to camouflage themselves when they assault different machines.