Count Access to Function

[bumsfeld]

I need to know how many times my function has been accessed in my program. Any suggestions other than global variable?

[G-Do]

Hi bumsfeld,

Well, let's think about this. Suppose that we want to create a variable 'count' that records the number of times a function 'myfunc()' is called. If the variable's scope is limited to the function itself, it will disappear off the runtime stack when the function call is completed. Any way you shake it, we're going to need some kind of globalesque way of maintaining count, even if we don't create global variables with the 'global' keyword, per se.

Any of these solutions qualify as globalesque, and most of them are horribly contrived:

* You could keep the count in a text file (ugh! overhead).
* You could keep a separate thread running which polls the main thread of execution.
* You could give each function its own count attribute, and update those attributes in a separate bookie function.

The last solution is essentially the same as collecting global variables, except each is bound to the appropriate function, which is nice. It would look something like this:

def bookie(func):
     func.count += 1
def myfunc():
     bookie(myfunc)
     # Some other code goes here
myfunc.count = 0
for i in range(10):
     myfunc()
print myfunc.count     # Should give you 10

But I can think of no means of doing this without variables belonging to some kind of global scope.

[bumsfeld]

Wow G-Do, your code works great!

I have to think that through to understand it.

[vegaseat]

G-Do,
nice thinking here!

I looked in my code samples and found another way, a little strange at first blush, but it works! I commented it heavily for Bumsfeld so he can follow it. You can put a test print in front of count.append(7) to make it more visible.

# keep track of how many times a function has been accessed

def myfunc(a, count=[]):
    """a default of empty list is used to append an item each time the function is called"""
    # put your function code here, let's do something with a
    func_result = a
    # add this to do the counting, 7 is just a dummy, creates an increasing list of 7s
    count.append(7)
    # return your usual result, and len(count) = number of times accessed
    return func_result, len(count)

for k in range(10):
    a, access = myfunc('something')

print "function has been accessed %d times" % access  # should be 10

[G-Do]

That is bizarre behavior, vegaseat! What is going on there - are lists remembered in a way that other variables are not? Because I can do the following:

>>> def myfunc(s='s'):
...     s += s
...     return len(s)
...
>>> for k in range(10):
...     tally = myfunc()
...
>>> print tally
2

- which is the same thing in principle, but Python doesn't remember 's' and it does remember your 'count' list. It doesn't seem like it should work this way - how are you keeping count from getting wiped off the runtime stack? Surely, the output of len() doesn't function as a reference to count, so how does this work?

[vegaseat]

Looks like a little lecture on argument passing in Python functions is in order, here are some examples I came up with to illustrate the basics:

# do function arguments stay local?
def times3(x):
    """argument x becomes local to function"""
    x = x * 3
    return x

x = 4
y = times3(x)
print "%d times 3 = %d" % (x, y)  # 4 times 3 = 12

print

def exclaim(str1):
    """
    string argument str1 will be local to the function
    a string is immutable, there is no effect on the caller
    """
    str1 = str1 + "!" # creates a new str1
    return str1

str1 = "Hallo"
print "%s is now %s" % (str1, exclaim(str1))  # Hallo is now Hallo!

print

def one_more(list1):
    """
    since list is a mutable object,
    the argument behaves like C's pointer argument
    there is an effect on the caller
    """
    list1.append(4)
    return list1

list1 = [1, 2, 3]
print one_more(list1)  # [1, 2, 3, 4]
print list1            # [1, 2, 3, 4]  oops!
# or
list2 = [1, 2, 3]
print one_more(list2)  # [1, 2, 3, 4]
print list2            # [1, 2, 3, 4]  still an oops!

print

def one_more2(list1):
    """
    since list is a mutable object,
    the argument behaves like C's pointer argument
    to avoid effects on the caller, do operations on a copy
    """
    list1 = list(list1)  # make a simple copy
    list1.append(4)
    return list1

list1 = [1, 2, 3]
print one_more2(list1)  # [1, 2, 3, 4]
print list1             # [1, 2, 3]

Books like Mark Lutz's "Learning Python" talk about this. Really something of which you need to be aware! Some folks think it's a wart, others think it's handy! I used the handy part in my access counter.

[G-Do]

Hi vegaseat,

I understand that immutables are passed by value and mutables are passed by reference, but I fail to see how that rule alone should give rise to the behavior you demonstrated in your first code snippet. If there is no analog to the 'count' list in the outer scope, how can changes to it be remembered at all? Why isn't it simply wiped out of existence every time myfunc() returns? Does this have something to do with closures on the enveloping scope? I have cracked my copy of Learning Python and read in the neighborhood of pp200-230, but things don't seem any clearer :-|

[bumsfeld]

G-Do, I am buffled as much as you.
Does by reference mean that code (originally code=[]) keeps the same memory address, even as it increases?

[vegaseat]

Maybe Bumsfeld is on the right track? The function default assignment count=[] retains the same address to the list object, even though the list may not be empty any longer. I am trying to figure out how to prove that.

The code below shows this to be true, but is not a proof:

# show address of list object count
def funk(count=[]):
    print id(count), count
    count.append(7)

for x in range(10):
    funk()

"""
result =
10312960 []
10312960 [7]
10312960 [7, 7]
10312960 [7, 7, 7]
10312960 [7, 7, 7, 7]
10312960 [7, 7, 7, 7, 7]
10312960 [7, 7, 7, 7, 7, 7]
10312960 [7, 7, 7, 7, 7, 7, 7]
10312960 [7, 7, 7, 7, 7, 7, 7, 7]
10312960 [7, 7, 7, 7, 7, 7, 7, 7, 7]
"""

[mawe]

Hi!

I also got confused once by this "feature" :). After some searching, I found a little hint in the Python docu.

Regards, mawe

[G-Do]

Thanks, bumsfeld, vegaseat, and mawe,

I have done what I always do when I have problems in Python: use the dir() function like a hammer until things make sense again. From the interpreter, I said:

>>> def myfunc(addend, count=[]):
...     count.append(addend)
...     
>>> dir()
['__builtins__', '__doc__', '__name__', 'myfunc']
>>> dir(myfunc)
['__call__', '__class__', '__delattr__', '__dict__', '__doc__', '__get__', '__getattribute__', '__hash__', '__init__', '__module__', '__name__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__', 'func_closure', 'func_code', 'func_defaults', 'func_dict', 'func_doc', 'func_globals', 'func_name']
>>> myfunc.func_defaults
([],)
>>> myfunc('a')
>>> myfunc('b')
>>> myfunc('c')
>>> myfunc.func_defaults
(['a', 'b', 'c'],)

And look what the cat dragged in! a reference to the default variables of myfunc() in the enveloping scope. This explains why 'count' isn't garbage-collected. And using mawe's hint, we now can determine exactly what is going on here:

Python encounters the myfunc() signature. It evaluates all the default arguments and sticks the appropriate values in a tuple, which is then bound to myfunc.func_defaults in the enveloping scope. One of these values is an empty list corresponding to the default argument 'count.'

We call myfunc(). Python doesn't bother evaluating the default arguments again (why would you want to re-evaluate them every time a function is called? that generates a lot of overhead!) - it just calls the existing evaluations from myfunc.func_defaults and sets the arguments of myfunc() to those values. In this way, Python recreates the count reference in every call to myfunc(), but the object it is assigned to already exists elsewhere in memory, preserved after the initial evaluation.

I guess the short version is "default arguments do not belong to a local scope; they are maintained in the enveloping scope, albeit in a more-or-less obfuscated way." Very, very tricky!

[vegaseat]

Seems to be unique to Python. I tried the same thing with Ruby, and it keeps showing the empty array/list. This could be the reason that Ruby is considered mildly slower than Python on function calls.