943,942 Members | Top Members by Rank

Ad:
  • PHP Discussion Thread
  • Unsolved
  • Views: 12330
  • PHP RSS
You are currently viewing page 2 of this multi-page discussion thread; Jump to the first page
Nov 5th, 2009
0
Re: Password encoding/decoding
I think she recommended high memory usage so that the effort needed to try to recreate/hack any of the passwords would be excessive and not worth it. But if you use up that much processing/memory, wouldn't you make it untenable to be used within a login/registering system? For even a reasonable amount of requests even.
Reputation Points: 13
Solved Threads: 19
Junior Poster
jomanlk is offline Offline
103 posts
since Oct 2009
Nov 5th, 2009
0
Re: Password encoding/decoding
Click to Expand / Collapse  Quote originally posted by Atli ...
@digital-ether
I agree with you 100%, although 100.000 iterations seem a bit excessive to me. (But that's just me :-P)
However, I got to ask why you specifically mention high memory usage?
You can save the number of iterations used on the hash. That way you can start with just a few, say 1000, then add more if needed. Computation speed is always increasing so to keep the password safe in a few years time, you may need more iterations.

100.000 iterations of SHA1 for instance only takes 300ms on a Core2 Duo 2.83Ghz.
On an production (under heavy use) 8 core Xeon 2.27 Ghz it took 500 ms.

So it looks like 100.000 is too high. (I just threw it in there). Maybe around 10.000 iterations. (around 50 ms to do).

Usually, the logins are not a bottleneck in your application.

See: http://chargen.matasano.com/chargen/...w-about-s.html
under "adaptive hashing"

I can't find any implementations of this in PHP. Though what is described in that article (BCrypt) uses Blowfish which PHP 5.3 has a native implementation. http://php.net/manual/en/function.crypt.php
So you could do something equivalent.

Click to Expand / Collapse  Quote originally posted by jomanlk ...
I think she recommended high memory usage so that the effort needed to try to recreate/hack any of the passwords would be excessive and not worth it. But if you use up that much processing/memory, wouldn't you make it untenable to be used within a login/registering system? For even a reasonable amount of requests even.
Yes, exactly. You can tune the memory usage also. This is to make distributed attacks less effective since there is a limited amount of memory, even for 1000 parallel machines.
Moderator
Reputation Points: 457
Solved Threads: 101
Nearly a Posting Virtuoso
digital-ether is offline Offline
1,250 posts
since Sep 2005
Nov 6th, 2009
1
Re: Password encoding/decoding
Thought I'd write an example:

note: for PHP4 just remove the "public static" in front of each function declaration.

The class:
php Syntax (Toggle Plain Text)
  1. /**
  2.  * Generate cryptographic Hashes for passwords
  3.  *
  4.  * Features:
  5.  * Harderned against precomputation attacks like rainbow tables (using salt)
  6.  * Harderned against brute force and dictionary attacks (using key stretching, large memory usage and an optional secret key)
  7.  *
  8.  * http://en.wikipedia.org/wiki/Password_cracking
  9.  *
  10.  * Note: for PHP4 and lower, just remove the "public static" before function declaration
  11.  *
  12.  * @author gabe@fijiwebdesign.com
  13.  * @link http://www.fijiwebdesign.com/
  14.  * @version $Id$
  15.  */
  16. class Password_Hash
  17. {
  18.  
  19. /**
  20. * Generate the Hash
  21. * @return String
  22. * @param $password String
  23. * @param $salt String[optional]
  24. * @param $iterations Int[optional]
  25. * @param $secret String[optional]
  26. */
  27. public static function generate($password, $salt = null, $iterations = 10000, $hash_function = 'sha1', $secret = '')
  28. {
  29. $salt or $salt = self::generateToken();
  30. $hashes = array();
  31. $hash = $password;
  32. // stores a sequence of 10000 unique hashes in memory
  33. $i = $iterations;
  34. while(--$i)
  35. {
  36. $hash = $hashes[] = $hash_function($hash.$salt.$secret);
  37. }
  38. // hash the 10000 unique hashes into a single hash
  39. $hash = $hash_function(implode('', $hashes).$salt.$secret);
  40. return implode(':', array($hash, $iterations, $hash_function, $salt));
  41. }
  42.  
  43. /**
  44. * Verify a password meets a hash
  45. * @return Bool
  46. * @param $password String
  47. * @param $hash String
  48. * @param $secret String[optional]
  49. */
  50. public static function verify($password, $hash, $secret = '')
  51. {
  52. list($_hash, $iterations, $hash_function, $salt) = explode(':', $hash);
  53. return ($hash == self::generate($password, $salt, $iterations, $hash_function, $secret));
  54. }
  55.  
  56. /**
  57. * Generate a random hex based token
  58. * @return String
  59. * @param $length Int[optional]
  60. */
  61. public static function generateToken($length = 40)
  62. {
  63. $token = array();
  64. for( $i = 0; $i < $length; ++$i )
  65. {
  66. $token[] = dechex( mt_rand(0, 15) );
  67. }
  68. return implode('', $token);
  69. }
  70.  
  71. }

Example using SHA1 with default of 10000 iterations.

php Syntax (Toggle Plain Text)
  1. // generating the hash
  2. $password = 'test';
  3. $hash = Password_Hash::generate($password);
  4.  
  5. // verifying a password
  6. $result = Password_Hash::verify($password, $hash);
  7.  
  8. // dump results
  9. var_dump($hash, $result);

Example using whirlpool as the hash function, a 128 length salt as well as a secret.

php Syntax (Toggle Plain Text)
  1. // define our custom hash function
  2. function whirlpool($str) {
  3. return hash('whirlpool', $str);
  4. }
  5.  
  6. $password = 'test';
  7. $salt = password_Hash::generateToken(128);
  8. $secret = password_Hash::generateToken(128);
  9. $iterations = 10000;
  10.  
  11. // generate hash
  12. $hash = Password_Hash::generate($password, $salt, $iterations, 'whirlpool', $secret);
  13.  
  14. // verify
  15. $result = Password_Hash::verify($password, $hash, $secret);
  16.  
  17. // dump results
  18. var_dump($result);
Last edited by digital-ether; Nov 6th, 2009 at 1:46 pm.
Moderator
Reputation Points: 457
Solved Threads: 101
Nearly a Posting Virtuoso
digital-ether is offline Offline
1,250 posts
since Sep 2005
Nov 6th, 2009
0
Re: Password encoding/decoding
Thanks for that.
I see what you mean now, about the memory.

Interesting article to. I had never really considered using bcrypt before, although after doing some testing, and considering your memory point, I'm inclined to keep using the hashing algorithms.

Your code looks pretty awesome to. Very flexible :-)
Looks like it could easily starve you for memory if you aren't careful. (Which is what you were aiming for, I assume.)

I'm trying out a system where I store details on the hash inside the database, everything from which algorithm to use to how much RAM it should be using. I use a similar method you used, to make sure the RAM suffers during the process.

Been working fine so far:
php Syntax (Toggle Plain Text)
  1. /**
  2.  * Handles member security.
  3.  *
  4.  * Assumes a DB table named `member` structured like so:
  5.  * CREATE TABLE `member` (
  6.  * `id` int Unsigned NOT NULL Auto_Increment Primary Key,
  7.  * `name` VarChar(64) Not Null Unique,
  8.  * `pw_salt` char(32) NOT NULL,
  9.  * `pw_memory` int(10) unsigned DEFAULT 10485760,
  10.  * `pw_algo` varchar(32) DEFAULT 'sha512',
  11.  * `password` char(128) NOT NULL
  12.  * ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
  13.  *
  14.  * Static Methods:
  15.  * generate_hash - Generates a has based on the input and the user's pw_ details.
  16.  * validate_password - Validates the given password against the member password.
  17.  *
  18.  */
  19. class MemberSecurity
  20. {
  21. /**
  22.   * Generates a hash based on the user-specific salt and the passed data.
  23.   * @param string $input The keyword (password) to hash.
  24.   * @param uint $userID The ID of the user, who's salt will be used.
  25.   * @return string The hash. Variable length, based on the algorithm used.
  26.   * @throws Exception
  27.   */
  28. public static function generate_hash($input, $userID)
  29. {
  30. try
  31. {
  32. // Fetch and initialize data.
  33. $sql = "SELECT `pw_salt`, `pw_memory`, `pw_algo` FROM `member` WHERE `id` = {$userID}";
  34. $resultSet = Database::Get()->query($sql);
  35. $resultRow = $resultSet->fetch_assoc();
  36.  
  37. $salt = $resultRow['pw_salt'];
  38. $algo = $resultRow['pw_algo'];
  39. $memorySize = $resultRow['pw_memory'];
  40. $hash = $input . $salt;
  41.  
  42. // In case of a normal hash algorithm.
  43. if(in_array($algo, hash_algos()))
  44. {
  45. // Set up a memory buffer and fill it with hashes created using
  46. // the input and the salt, then hash the whole thing as the output.
  47. // Should eat up loads of memory and CPU time.
  48. $buffer = "";
  49. while(strlen($buffer) < $memorySize)
  50. {
  51. $buffer .= hash($algo, $input . $salt);
  52. }
  53. $hash = hash($algo, $buffer);
  54. $buffer = null;
  55. }
  56.  
  57. // Fall back to bcrypt, if possible.
  58. // No memory limit. (The memory field will be used as the key length)
  59. // This eats up much more CPU time than memory.
  60. else if($algo == 'bcrypt' && CRYPT_BLOWFISH == 1)
  61. {
  62. $keyLength = ($memorySize < 10 ? "0".$memorySize : $memorySize);
  63. $hash = substr(crypt($input, '$2a$'. $keyLength .'$'. $salt . '$'), 28);
  64. }
  65.  
  66. // Algorithm not available. Bail out!
  67. else
  68. {
  69. throw new Exception("The hashing algorithm '$algo' is not available.", 501);
  70. }
  71.  
  72. return $hash;
  73. }
  74. catch(DatabaseException $ex)
  75. {
  76. throw new Exception("Failed to fetch user data.", 304, $ex);
  77. }
  78. }
  79.  
  80. /**
  81.   * Validates the password against the one stored in the user database.
  82.   * @param string $password The password to validate.
  83.   * @param string $userName The name of the member.
  84.   * @return bool
  85.   * @throws Exception
  86.   */
  87. public static function validate_password($password, $userName)
  88. {
  89. try
  90. {
  91. $sql = "SELECT `id`, `password` FROM `member` WHERE `name` = '{$userName}'";
  92. $resultSet = Database::Get()->query($sql);
  93. if($resultSet && $resultSet->num_rows == 1)
  94. {
  95. $resultRow = $resultSet->fetch_assoc();
  96. $generatedHash = self::generate_hash($password, $resultRow['id']);
  97.  
  98. return ($resultRow['password'] == $generatedHash);
  99. }
  100. else
  101. {
  102. throw new DatabaseException("Failed to fetch user data.", 500, $sql, Database::Get()->error);
  103. return false;
  104. }
  105.  
  106. }
  107. catch(DatabaseException $ex)
  108. {
  109. throw new Exception("Failed to fetch user data.", 500, $ex);
  110. }
  111. }
  112. }
I use my own database wrapper classes in there, but I trust you can read around them ;-)

Using the following, validating a single password uses more than 10Mb memory and takes just under a second.
(Using Core Duo 3.16GHz CPU and 1333Mhz DDR3 RAM)
text Syntax (Toggle Plain Text)
  1. mysql> SELECT * FROM member\G
  2. *************************** 1. row ***************************
  3. id: 1
  4. name: User 1
  5. pw_salt: 0e20d1dc3c6e60225597be
  6. pw_memory: 10485760
  7. pw_algo: sha512
  8. password: 09254dc706eb1acb5420adcbc97d67a0992fb3530e8c844c4c3a930614d447f371f2f0a2af369dd1c73452819d3aac30bdeea7f889b11f4cf038435c26d5036e

Looks promising... I think :-)
Reputation Points: 93
Solved Threads: 70
Posting Pro
Atli is offline Offline
526 posts
since May 2007
Nov 6th, 2009
1
Re: Password encoding/decoding
Looks great. I modified the class I wrote a bit after looking at your code.

Just a few suggestions:

You could optimize the string concatenation.

php Syntax (Toggle Plain Text)
  1. $buffer .= hash($algo, $input . $salt);

A lot of the execution time is the concatenation of the $buffer string.

If you use an array however, it becomes negligible. I believe the reason is the whole string has to be copied/destroyed in memory each time you append to it. Arrays are mutable so appending would not create a copy.

You could calculate the number of iterations you'll need by dividing the memory usage (in bytes) by the length of the hash. That way you also don't have to re-check the length of the string.
Moderator
Reputation Points: 457
Solved Threads: 101
Nearly a Posting Virtuoso
digital-ether is offline Offline
1,250 posts
since Sep 2005
Nov 7th, 2009
0
Re: Password encoding/decoding
Thanks.

Quote ...
You could optimize the string concatenation.
I did use an array at first, adding each hash as an element and them imploding it before creating the final hash. (Like you do in your code.)

However, after testing that I found that this method uses double the amount of memory the string concatenation method uses.
Makes sense when you think about it. Both the array and the string will have to exist in memory at the same time when the implode function is called.

After testing this a bit more, I've decided to just use half the array size. The array takes up half the memory, and the string the implode creates uses the rest. Seems to be most efficient.
php Syntax (Toggle Plain Text)
  1. $buffer = array();
  2. $hash = hash($algo, $input . $salt);
  3. for($y = floor($memorySize / strlen($hash) / 2); $y > 0; --$y)
  4. {
  5. $buffer[] = hash($algo, $y . $hash . $salt);
  6. }
  7. $hash = hash($algo, implode($buffer));
This takes only half the time my previous code did :-)

This also fixed another problem in my previous code.
The loop that created the memory buffer in my previous code just hashed and added the same hash over and over. It could have been optimized to practically nothing by hashing it once outside the loop and adding that each loop, rather than recreating the hash each time.
By adding the loop index to the string that gets hashed, I now get a unique hash added to the buffer each loop.

Quote ...
You could calculate the number of iterations you'll need by dividing the memory usage (in bytes) by the length of the hash. That way you also don't have to re-check the length of the string.
Good point, thanks.
Last edited by Atli; Nov 7th, 2009 at 6:07 am. Reason: Fixed. Sorry XD
Reputation Points: 93
Solved Threads: 70
Posting Pro
Atli is offline Offline
526 posts
since May 2007
Nov 7th, 2009
0
Re: Password encoding/decoding
Click to Expand / Collapse  Quote originally posted by Atli ...
Thanks.


I did use an array at first, adding each hash as an element and them imploding it before creating the final hash. (Like you do in your code.)

However, after testing that I found that this method uses double the amount of memory the string concatenation method uses.
Makes sense when you think about it. Both the array and the string will have to exist in memory at the same time when the implode function is called.

After testing this a bit more, I've decided to just use half the array size. The array takes up half the memory, and the string the implode creates uses the rest. Seems to be most efficient.
php Syntax (Toggle Plain Text)
  1. $buffer = array();
  2. $hash = hash($algo, $input . $salt);
  3. for($y = floor($memorySize / strlen($hash) / 2); $y > 0; --$y)
  4. {
  5. $buffer[] = hash($algo, $y . $hash . $salt);
  6. }
  7. $hash = hash($algo, implode($buffer));
This takes only half the time my previous code did :-)

This also fixed another problem in my previous code.
The loop that created the memory buffer in my previous code just hashed and added the same hash over and over. It could have been optimized to practically nothing by hashing it once outside the loop and adding that each loop, rather than recreating the hash each time.
By adding the loop index to the string that gets hashed, I now get a unique hash added to the buffer each loop.


Good point, thanks.
After thinking about this again, I realized I made an assumption that is very incorrect. I assumed hash functions require the whole input in order to produce the hash.

But it appears the hash is generated by reading the input in sequence. That way it does not require storing the whole input in memory.
A bit of searching online seemed to prove it: http://www.partow.net/programming/ha...gMethodologies

And a test case:

php Syntax (Toggle Plain Text)
  1. // generate a hash for 2MB file
  2. $hash = sha1_file('hash.txt');
  3. echo 'memory peak use: '.floor(memory_get_peak_usage(true)/1000).' Kb';

The result was only 524 Kb memory allocated for the PHP process even though the file was 2MB which shows that the file was read sequentially.

For example with PHP, sha1_file() supports the stream wrappers. So you could just write each hash generated to a stream/pipe that sha1_file() would read as the input. A unidirectional stream is not kept in memory as a whole, only each packet is read into memory and then dropped.

So I guess the memory consumption bit has to be rewritten some other way.
Moderator
Reputation Points: 457
Solved Threads: 101
Nearly a Posting Virtuoso
digital-ether is offline Offline
1,250 posts
since Sep 2005
Nov 7th, 2009
0
Re: Password encoding/decoding
But neither of our codes need the hash functions themselves to use a lot of memory. They both use arrays and strings we created to fill the memory.
I mean, using my last code, setting the $memorySize variable to 10Mb, the peak usage of the script goes to around 10.5Mb.

Or am I not getting your point?

P.S.
floor(memory_get_peak_usage(true)/1000)
1Kb = 1024b ;-)
Reputation Points: 93
Solved Threads: 70
Posting Pro
Atli is offline Offline
526 posts
since May 2007
Nov 8th, 2009
0
Re: Password encoding/decoding
Click to Expand / Collapse  Quote originally posted by Atli ...
But neither of our codes need the hash functions themselves to use a lot of memory. They both use arrays and strings we created to fill the memory.
I mean, using my last code, setting the $memorySize variable to 10Mb, the peak usage of the script goes to around 10.5Mb.

Or am I not getting your point?

P.S.
floor(memory_get_peak_usage(true)/1000)
1Kb = 1024b ;-)
The whole array or string does not need to be kept in memory, in order to generate the hash. The hash can be generated by writing the input in 1 byte at a time.

Example:

php Syntax (Toggle Plain Text)
  1. // sha1_stream.php
  2. echo sha1_file('php://stdin');

Saved as file sha1_stream.php the above would read the input from STDIN and write the hash to STDOUT.

Then I used proc_open() to write the input to it bit by bit. So it never stays in memory.

PHP Syntax (Toggle Plain Text)
  1. $salt = 'e5cbe45c71a0b805278f2b5f94eb108dba532af3';
  2. $hash = 'test';
  3.  
  4. $descriptorspec = array(
  5. 0 => array("pipe", "r"), // stdin is a pipe that the child will read from
  6. 1 => array("pipe", "w"), // stdout is a pipe that the child will write to
  7. 2 => array("file", "error-output.txt", "a") // stderr is a file to write to
  8. );
  9.  
  10. $cwd = './';
  11. $env = array();
  12. $options = array('bypass_shell');
  13. $php_path = 'c:\xampp\php\php.exe'; // change to php path
  14. $file_path = dirname(__FILE__).'/sha1_stream.php';
  15. $process = proc_open("$php_path $file_path", $descriptorspec, $pipes, $cwd, $env, $options);
  16. if (is_resource($process)) {
  17. $i = 1000;
  18. while(--$i)
  19. {
  20. // instead of saving the hashes in memory, we write each hash to the other PHP process which computes the overall hash for the whole string.
  21. fwrite($pipes[0], sha1($hash.$salt));
  22. }
  23. fclose($pipes[0]);
  24.  
  25. $hash = stream_get_contents($pipes[1]);
  26.  
  27. fclose($pipes[1]);
  28. $return_value = proc_close($process);
  29. } else {
  30. throw new Exception('Could not create process. see: error-output.txt');
  31. }
  32. echo $hash;
  33. echo '<br />';


With 100.000 iterations the full process still takes only about 1.12secs and memory usage of both processes was around 700Kb.
Moderator
Reputation Points: 457
Solved Threads: 101
Nearly a Posting Virtuoso
digital-ether is offline Offline
1,250 posts
since Sep 2005

This thread is more than three months old

No one has posted to this discussion for at least three months. Please let old threads die and do not reply to them unless you feel you have something new and valuable to contribute that absolutely must be added to make the discussion complete. Otherwise, please start a new thread in this forum instead.
Message:
Previous Thread in PHP Forum Timeline: To restrict users to choose date from date picker only
Next Thread in PHP Forum Timeline: Messaging System





About Us | Contact Us | Advertise | Acceptable Use Policy
Forum Index | Build Custom RSS Feed


Follow us on Twitter


© 2011 DaniWeb® LLC