Default arguments - what is Python doing?

[slunk]

I began to learn about Python perhaps an hour or two ago.

In a Python tutorial, I came across the below text (between lines of equal signs) regarding default function arguments.

This makes no sense to me at all.

What is the scope and lifetime of L? It appears that the scope is within function f, and the lifetime is permanent?

Well, I'll quit trying to suggest all the possibilities. I simply can't think of any kind of implementation that would cause the first version of the function f below to "accumulate" values, while the second version does not accumulate them.

What is happening under the hood? It seems to me that any reasonable implemenation would cause both versions of f to have the same behavior.

===========================================

Important warning: The default value is evaluated only once. This makes a difference when the default is a mutable object such as a list, dictionary, or instances of most classes. For example, the following function accumulates the arguments passed to it on subsequent calls:

def f(a, L=[]):
    L.append(a)
    return L

print f(1)
print f(2)
print f(3)

This will print

[1]
[1, 2]
[1, 2, 3]

If you don't want the default to be shared between subsequent calls, you can write the function like this instead:

def f(a, L=None):
    if L is None:
        L = []
    L.append(a)
    return L

=============================================

[vegaseat]

Just remember that the first time a function is invoked/called, its code gets loaded
into memory. If you give an argument a default that is mutable, you have to be aware of how that will behave. In practice, most of the time you won't do a default with a mutable.

Sort of behaves like a static variable in C. Some folks look at that as a nice feature, others as a "gotcha".

[slunk]

Thanks for your thoughts, Vegaseat.

Actually, on further thought:

It appears to me that the issue is whether the value assigned to the function argument is a value (fundamental type) or a reference (to an object).

Apparently, if a the default value of a function argument is an object, when the function is called for the first time, the object is created and initialized, and the argument variable is assigned a reference to the object.

When the function exits, the object continues to persist.

On subsequent calls to the function, the argument variable is simply assigned a reference to the previously created object.

BY CONTRAST

A statement such as:
L = []

Also creates and initializes an object, and assigns a reference to that object to the variable L.

However, when the function exits, that object is destructed.

[jrcagle]

It appears to me that the issue is whether the value assigned to the function argument is a value (fundamental type) or a reference (to an object).

Exactly so, except for the wording.

replace "value (fundamental type)" with "immutable object"
replace "a reference (to an object)" with "mutable object"

All items in Python are objects, and all variables are handled via references.

Apparently, if a the default value of a function argument is an object, when the function is called for the first time, the object is created and initialized, and the argument variable is assigned a reference to the object.

When the function exits, the object continues to persist.

On subsequent calls to the function, the argument variable is simply assigned a reference to the previously created object.

Sort of. Consider:

# in a file 
class Tmp(object):
    print "Hello!"

def myfunc(a = Tmp(), b="hi there"):
    return a

# now run the file
>>> 
Hello!
>>> c = myfunc()
>>> a

Traceback (most recent call last):
  File "<pyshell#42>", line 1, in -toplevel-
    a
NameError: name 'a' is not defined
>>> c
<__main__.Tmp object at 0x00B84A10>
>>>

Actually, in all cases, when the function is *loaded*, the default object is created and initialized. That's why running the code above produces 'Hello!' on output even *before* myfunc gets called.

Then, c gets assigned a reference to the object a in myfunc; the reference 'a' goes out of scope (hence the error), but the object itself persists (referred to by c).

BTW, the default argument b never gets assigned, but because it's a part of the myfunc code, it persists also.

So now consider this:

>>> def append(L=[],item=None):
    L.append(item)
    return L
                                                 # L gets created when I hit Enter here, before append is called.
>>> append()
[None]                                       # L is modified here
>>> append()
[None, None]
>>> append(L=[1,2,3])              # A different reference is passed, so default is ignored
[1, 2, 3, None]
>>> append()                           # Back to the default.
[None, None, None]
>>>

What happens is that the default object is sitting around in memory. Because it is mutable, the function is able to modify it persistently. Contrast:

>>> def append2(s ="", c=""):
    s = s+c
    return s
>>> append2(c="3")
'3'
>>> append2(c="3")
'3'
>>> append2(c="3")
'3'
>>>

Here, the line s=s+c has the effect of clobbering s. First, the string s + c gets created elsewhere in memory. Then, s gets assigned the reference to the new string. That reference is then returned.

But the original object "" is still sitting around in memory, just as with append() above. So on re-entry, s gets re-assigned to that object, and the result is shown above.

What immutability means for strings is that there are no string methods that can actually change the string itself. Thus, any operation on strings will create a new copy in memory and clobber the reference. That's why string methods, in general, return a copy of the modified string.

By contrast, list methods, in general, return None and instead modify the list in place.

This is Python's subtle way of enforcing the distinction between immutable and mutable objects.

Practical bottom line: DONT USE MUTABLE OBJECTS AS DEFAULT ARGUMENTS UNLESS YOU REALLY WANT THE SIDE EFFECT.

Hope it helps,
Jeff