943,648 Members | Top Members by Rank

Ad:
  • PHP Discussion Thread
  • Marked Solved
  • Views: 2673
  • PHP RSS
Sep 17th, 2009
0

Best compression

Expand Post »
Hi and I am looking for a compression mechonism that can compress text (just letters and numbers) into binary or all weird symbols. Does anybody know of any good php/mysql codes as I can't get any to work.
Similar Threads
Sponsor
Featured Poster
Reputation Points: 410
Solved Threads: 258
Occupation: Genius
cwarn23 is offline Offline
3,004 posts
since Sep 2007
Sep 18th, 2009
0

Re: Best compression

I have tried running the following mysql queries through the phpmyadmin command console (mysql_query() code) but I seem to be unable to retrieve the value. Here is my code.
INSERT INTO `table` SET `column`=COMPRESS("abcd")
SELECT * FROM `table` WHERE UNCOMPRESS(`column`)="abcd"
Also tried
INSERT INTO `table` SET `column`=COMPRESS("abcd")
SELECT * FROM `table` WHERE `column`=COMPRESS("abcd")
Please advice on how I should use the select query.
Last edited by cwarn23; Sep 18th, 2009 at 2:24 am.
Sponsor
Featured Poster
Reputation Points: 410
Solved Threads: 258
Occupation: Genius
cwarn23 is offline Offline
3,004 posts
since Sep 2007
Sep 18th, 2009
0

Re: Best compression

I think I may have found something with the bzip2/bz2 library because the gz library didn't compress very well. I will try storing the bzcompress() result in a binary column. Well let you know how my result goes.
Sponsor
Featured Poster
Reputation Points: 410
Solved Threads: 258
Occupation: Genius
cwarn23 is offline Offline
3,004 posts
since Sep 2007
Sep 18th, 2009
0

Re: Best compression

Click to Expand / Collapse  Quote originally posted by cwarn23 ...
I have tried running the following mysql queries through the phpmyadmin command console (mysql_query() code) but I seem to be unable to retrieve the value. Here is my code.
INSERT INTO `table` SET `column`=COMPRESS("abcd")
SELECT * FROM `table` WHERE UNCOMPRESS(`column`)="abcd"
Also tried
INSERT INTO `table` SET `column`=COMPRESS("abcd")
SELECT * FROM `table` WHERE `column`=COMPRESS("abcd")
Please advice on how I should use the select query.
I tried this and it worked:

PHP Syntax (Toggle Plain Text)
  1. SELECT * FROM `table` WHERE `compressed` = compress('test')

The only thing I can assume is the column containing the compressed data has to be a blob or varbinary type.
Moderator
Reputation Points: 457
Solved Threads: 101
Nearly a Posting Virtuoso
digital-ether is offline Offline
1,250 posts
since Sep 2005
Sep 18th, 2009
0

Re: Best compression

I just tried varbinary like before and again varbinary didn't work however I tried blob which I have never heard of before and it worked! Also the bz library works but won't decompress. So using the method you have described how can I decompress the result?
Sponsor
Featured Poster
Reputation Points: 410
Solved Threads: 258
Occupation: Genius
cwarn23 is offline Offline
3,004 posts
since Sep 2007
Sep 18th, 2009
1

Re: Best compression

Click to Expand / Collapse  Quote originally posted by cwarn23 ...
I just tried varbinary like before and again varbinary didn't work however I tried blob which I have never heard of before and it worked! Also the bz library works but won't decompress. So using the method you have described how can I decompress the result?
Which bz library? Do you mean zlib? Is this in mysql or PHP?

You can just decompress the result in the SQL query:

PHP Syntax (Toggle Plain Text)
  1. SELECT uncompress(test) as test FROM `table` WHERE `compressed` = compress('test')

This should give you the column test in decompressed form.

Note:

I wouldn't use:

PHP Syntax (Toggle Plain Text)
  1. SELECT * FROM `table` WHERE UNCOMPRESS(`column`)="abcd"

The reason is that you'll run uncompress() on each row in the database table.
You will also be unable to use any index made on that column.

If you use:
PHP Syntax (Toggle Plain Text)
  1. `column`= COMPRESS("abcd")
Then you only have to compress the input "abcd" once. You will also be able to use indexes made on the `column` column since you're comparing directly.
Moderator
Reputation Points: 457
Solved Threads: 101
Nearly a Posting Virtuoso
digital-ether is offline Offline
1,250 posts
since Sep 2005
Sep 18th, 2009
0

Re: Best compression

Quote ...
SELECT uncompress(test) as test FROM `table` WHERE `compressed` = compress('test')
Wow, if that works that will be brillient. I will try it out.

And the confusion you may have had before about the bz library also known as bzip2 or bz2 depend on what version (I found out the hard way) that was another option I was exploring untill I found out it couldn't compress crc32 hashes. So as I said, if that above code works, that will be the solution to all my endless reverse hashing storage problems.
Sponsor
Featured Poster
Reputation Points: 410
Solved Threads: 258
Occupation: Genius
cwarn23 is offline Offline
3,004 posts
since Sep 2007
Sep 18th, 2009
0

Re: Best compression

Moderator
Reputation Points: 457
Solved Threads: 101
Nearly a Posting Virtuoso
digital-ether is offline Offline
1,250 posts
since Sep 2005
Sep 18th, 2009
0

Re: Best compression

It works!
Even though it requires twice as much space on the mysql cache it works perfectly. Thanks for that very usefull code that I could not find anywhere else.
Sponsor
Featured Poster
Reputation Points: 410
Solved Threads: 258
Occupation: Genius
cwarn23 is offline Offline
3,004 posts
since Sep 2007
Sep 18th, 2009
0

Re: Best compression

Click to Expand / Collapse  Quote originally posted by cwarn23 ...
It works!
Even though it requires twice as much space on the mysql cache it works perfectly. Thanks for that very usefull code that I could not find anywhere else.
I wouldn't worry about the cache. But if you want, you can uncompress the result on the PHP side.

Assuming you're using the default compression, zlib, then you need to cut off the first 4 bytes before you uncompress the data.

The first 4 bytes is reserved for the length of the uncompressed string.

see: http://dev.mysql.com/doc/refman/5.0/...ction_compress

Here if a full example using PDO:

php Syntax (Toggle Plain Text)
  1. <?php
  2. /* Connect to an ODBC database using driver invocation */
  3. $dsn = 'mysql:dbname=test;host=localhost';
  4. $user = 'root';
  5. $password = '';
  6.  
  7. try {
  8. $dbh = new PDO($dsn, $user, $password);
  9. } catch (PDOException $e) {
  10. echo 'Connection failed: ' . $e->getMessage();
  11. }
  12.  
  13. $sql = 'SELECT binary(compressed) AS compressed FROM `compressed` WHERE compress(:test) = compressed ';
  14. $sth = $dbh->prepare($sql, array(PDO::ATTR_CURSOR => PDO::CURSOR_FWDONLY));
  15. $sth->execute(array(':test' => 'test'));
  16. $row = $sth->fetch();
  17.  
  18. // compressed result, with first 4 bytes being the length
  19. $result = $row['compressed'];
  20.  
  21. // the length of uncompressed string
  22. $len = substr($result, 0, 4);
  23. // the compressed string
  24. $compressed = substr($result, 4, strlen($result)-4);
  25.  
  26. // uncompress the string
  27. echo gzuncompress($compressed, intval($len)); // test
  28.  
  29. ?>

Edit:

Here is the structure of the table I used in the example:

PHP Syntax (Toggle Plain Text)
  1. +------------+------+------+-----+---------+-------+
  2. | Field | Type | Null | Key | Default | Extra |
  3. +------------+------+------+-----+---------+-------+
  4. | value | text | NO | | NULL | |
  5. | compressed | blob | NO | | NULL | |
  6. +------------+------+------+-----+---------+-------+

And the dump:

PHP Syntax (Toggle Plain Text)
  1. --
  2. -- Database: `test`
  3. --
  4.  
  5. -- --------------------------------------------------------
  6.  
  7. --
  8. -- Table structure for table `compressed`
  9. --
  10.  
  11. CREATE TABLE IF NOT EXISTS `compressed` (
  12. `value` text COLLATE utf8_unicode_ci NOT NULL,
  13. `compressed` blob NOT NULL
  14. ) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
  15.  
  16. --
  17. -- Dumping data for table `compressed`
  18. --
  19.  
  20. INSERT INTO `compressed` (`value`, `compressed`) VALUES
  21. ('test', 0x04000000789c2b492d2e0100045d01c1);
Last edited by digital-ether; Sep 18th, 2009 at 1:47 pm.
Moderator
Reputation Points: 457
Solved Threads: 101
Nearly a Posting Virtuoso
digital-ether is offline Offline
1,250 posts
since Sep 2005

This thread is solved

Either the thread starter or a moderator has marked this thread as solved. You can most likely trust the responses and answers given. There is most likely no reason for any further responses to be posted here. If you have a related question, please start a new thread in this forum instead.

This thread is more than three months old

No one has posted to this discussion for at least three months. Please let old threads die and do not reply to them unless you feel you have something new and valuable to contribute that absolutely must be added to make the discussion complete. Otherwise, please start a new thread in this forum instead.
Message:
Previous Thread in PHP Forum Timeline: .htaccess for short URL redirection on a subdomain?
Next Thread in PHP Forum Timeline: ReGRRtion!





About Us | Contact Us | Advertise | Acceptable Use Policy
Forum Index | Build Custom RSS Feed


Follow us on Twitter


© 2011 DaniWeb® LLC