Memcached Persistent Connections

[Dani]

I'm trying to get Memcached persistent connections to work. We currently have a cluster of four Memcached servers each with 2GB of RAM, and four web servers. My code looks like this:

public function __construct()
{   
    global $config;

    $this->cache = new Memcached('daniweb');

    if ($this->cache->isPristine())
    {
        $this->cache->setOptions(array(
            Memcached::OPT_NO_BLOCK => true,
            Memcached::OPT_BUFFER_WRITES => true,
            Memcached::OPT_BINARY_PROTOCOL => true,
            Memcached::OPT_LIBKETAMA_COMPATIBLE => true,
            Memcached::OPT_DISTRIBUTION => Memcached::DISTRIBUTION_CONSISTENT,
        ));         
    }

    if (count($this->cache->getServerList()) == 0)
    {           
        // Connect to all memcache servers
        $this->cache->addServers($config['Cache']['memcacheserver']);
    }
}

public function __destruct()
{
    $this->cache->resetServerList();
}

If I don't resetServerList(), then the number of connections keeps going up and up until it reaches max allowed connections and freezes the servers. Why does this happen? I was under the impression that if you pass in a persistent ID into the Memcached constructor, it reuses the open connection. I'm only telling it to add servers to the connection pool if the connection pool is currently empty. What am I doing wrong??

[cereal]

Try to add:

Memcached::OPT_TCP_NODELAY => true

It seems to work for me.

@Dani

I assume you are using codeigniter as a connection.

then the number of connections keeps going up and up until it reaches max allowed connections and freezes the servers.

Does it take less than a second to do that?

I was search on the codeigniter forum I found this:

http://ellislab.com/codeigniter/user-guide/libraries/caching.html#memcached

The way it is structure it really looks different from your code.

Here's a plugin but I'm no sure it's gonna work on codeigniter:

https://github.com/php-memcached-dev/php-memcached

The plugin is to speed up the connection.

[Dani]

cereal, I'll give it a shot, thanks.

LastMitch, even though we're using CodeIgniter, we're not using CodeIgniter's memcache wrapper library.

[Dani]

Upon adding Memcached::OPT_TCP_NODELAY => true and commenting out the line $this->cache->resetServerList();, connections per server jumped over the course of about a minute from 20-30 up to 160-180. However, they're now hovering in that range, and don't seem to be going any higher.

Oooh ... I spoke too soon, we're now at 186 connections per server.

[Dani]

It's been awhile, and we're still hovering in the 160-180 connections per server range, so all is good there :) Thanks!

However, I now have another (related) question.

You can see I'm doing $this->cache = new Memcached('daniweb'); ... Is it advantageous to instead do something like $this->cache = new Memcached(rand(1,10)); to create a connection pool of 10 different open connections?

We use Memcached very, very heavily on each page, and we create one Memcached object per page.

[cereal]

Is it advantageous to instead do something like $this->cache = new Memcached(rand(1,10)); to create a connection pool of 10 different open connections?

Sincerly I don't know, I've searched a bit, but I didn't found anything neither in websites nor in books ;(

[veedeoo]

Hi Dani,

My guess is that the constructor of your class is always overloaded. Will it be possible to write a new class that will responsible for checking if there is an instance of Memcached on each page?

You can use a simple singleton to act as the director or instantiator for all Memcached instance request. By doing this, you will be able to remove the object and others from the constructor and replace it with the singleton. So by doing this, the instance requests can be effeciently monitored.

If using the singleton, you will probably need to create a new method to handle all of your conditions that are located on your constructor.

Although Singleton's constructor is also loaded, but it is better to overload one class for monitoring purposes than overloading the constructor of the application class..

I really don't know if this can be of any help, but this is how I would do it on a much bigger application. Most importantly, if we have too many classes overlapping each other. Monitoring them is pretty tricky..

WARNING! singleton codes below are coming from top of my head... I getting ready for my flight to Princeton. I might have to double check on this, once I get into my computer..instead of a netbook. There might be some errors on the codes below..

<?php

final class GetInstance
{

private static $classIntance = array();

## in nature this contructor is overloaded, but only this one
private function __construct(){}

## we can then manage the instatiation of the classes as given by the variable your_class
public static function instantiate($your_class)
{        
    ## we will only instantiate the $class as necessary
    self::allowClass($your_class);    

   ## we can then return the allowed or requested class above
    return self::$classIntance[$your_class];
}


private static function allowClass($your_class)
{
    ## a simple check-point if the requested class even exists.
    if (!array_key_exists($your_class , self::$classIntance))
    {
        self::$classIntance[$your_class] = new $your_class;
    }
}
} 

By implementing a simple singleton above, you new constructor will now look like this..

    public function __construct(){

     $this->cachedInstance = GetInstance::instantiate('Memcached');
    }

Then the new method on the memched class can be something like this

    public function newMethod('daniweb');

Then you add a new method on your posted class above to check all condtions.

    public function checkConditions(){
    $this->cache = $this->cachedInstance->newMethod('daniweb');
     if ($this->cache->isPristine())
    {
    $this->cache->setOptions(array(
    Memcached::OPT_NO_BLOCK => true,
    Memcached::OPT_BUFFER_WRITES => true,
    Memcached::OPT_BINARY_PROTOCOL => true,
    Memcached::OPT_LIBKETAMA_COMPATIBLE => true,
    Memcached::OPT_DISTRIBUTION => Memcached::DISTRIBUTION_CONSISTENT,
    ));
    }

    return true;

    }

Do the same on the other... or they can co-exist in one method..

I am assuming here that Memcached::OPT_NO_BLOCK and the rest of it are static in the Memcached class or this could have been asigned as const... as in

const OPT_NO_BLOCK = array('true','false');

Some people may have disagreement with the singleton approach., this is because they have not tested it throughly, but this is a simple way of looking at the constructor overloading activity..

    class A{
    public function __construct(){
    echo 'constructor of a or instance of a. always active and loaded';
    }

    public static function staticA(){
    echo 'This is an static function and do not need the class A instantiated';
    }
    }

    class B{
    public function __construct(){
    echo 'constructor of b always active and loaded when an instance of b is set';
    }
    }

    ## objects below will ouput the loaded constructors of those classes above.
    $objectB = new B();
    $objectA = new A();

    ## this does not need an instance and overloading of the constructor

    A::staticA();

I really have to go...my flight was cancelled and I hope I get through this time... I already missed a week of school days ..Sorry for the long post.. I am not even sure if it will help or not..maybe for others.. :)

[Larry_Singleton]

@veed The overloading is more problematic, whenever there is a big iteration in the constructor. For iteration in bigger queries, it has to be relocated from constructor to a new method and if ONLY possible and feasible, the method taking the load of the iterating the data should be assign as static. However, if there is some issues on security, the class itself should be extended.

constructor that will kill your application due to excessive inodes.

public function __construct(){

$this->data = dataArray();
$this->someArrayForApp = array();
foreach($this->data as $nonSingletonData){
$this->omeArrayForApp[] = $nonSingletonData;
}
}

The above will create an extreme loading of the constructor equivalient to the items in the array iterated.

[Dani]

I stumbled upon this thread as the result of a Google search, literally struggling with the exact same issue, 7 years later.

I've solved it -- I think, anyways -- by having $this->cache->quit(); in the destructor (as opposed to resetServerList(). Even with quit() it maintains the persistent pool between page loads.