Posts by hughesadam_87

Try this:

def hailstone(start):

    num = start

    length = 1

    while num != 1:

        if num % 2 != 0:

            num = 3 * num + 1

        else:

            num = num / 2

        length += 1

    return (start, length)

 for i in range(0,100):
    print hailstone(i)

I changed your function so that it remembers the starting value, and then changed the return to send back the starting value and the associated length. Let me know if it works, I can't test it on this computer i'm at right now. Sorry if the formatting is messed up, using crappy windows wordpad.

Your function takes an integer, but you are passing in a list. If you print range(0,100), you will see that this is one object; a list of numbers. Instead, you should pass them one by one, or adjust your function to take in a list of numbers.

for i in range(0,100):
    print hailstone(i)

Thanks for the input!

Hi,

I have several functions that I'd like to operate on both mutable and immutable containers (mostly tuples vs. objects) and I'm looking for the optimal way to test for mutability in these objects. I can think of many ways to do this. For example:

try setattr:
  do mutable stuff
except Error:
  do immutable replacements

I also considered adding a metaattribute to my classes based on mutability; however, this limits the usage of my functions to just the customized objects in my programs.

I could also make 2 functions, one for mutable and one for immutable objects, but then I get lots of extra code.

Does anyone know of a Pythonic, best way to think about this?

Hi,

I'm curious about a few conventions.

If you have a function that can return a or b based on some conditionals, which do you think is the best/accepted notation?

def test(val):
   if val > 10: 
      return a
   else:
      return b

Or

def test(val):
   if val>10:
      return a
   return b

I often see both of these conventions used and wondered what you guys would recommend. Additionally, I commonly see these dual notations:

if x==True:
  do stuff

VS

if x:
  do stuff

Is there a preferred convention in this case?

Thanks.

Can you be specific on where exactly ou need help?

Thanks. I had tried this already and must have had a typo or something!

If you're using linux, this may be handled more simply with a bash script.

Hi,

I have an object subclass that I am using with a cusom repr() method.

class NewClass(object):
    def __repr__(self):
        return ', '.join('%s=%s'%(k, v) for (k,v) in self.__dict__.items())

This works fine (leaving out some more details about my program):

test=NewClass(a=1,b=2)
print test
a=1, b=2

But what I really want for reasons that aren't really worth getting into is for the class name to be printed out in the repr as well. For example,

print test
'NewClass: a=1, b=2'

But python objects don't have a name attribute. I'd prefer not making a separate attribute, aka:

class NewClass(object):
    name='NewClass'
    def __repr__(self):
        return name+', '.join('%s=%s'%(k, v) for (k,v) in self.__dict__.items())

Is there a native way to do this?

Thanks

You can download a new python version and run it separately in ubuntu by putting the new call into your path. Don't uninstall your system version, this will mess up the OS (yes python is integral to ubuntu). By default, your system's python exectuable is installed in /usr/bin/.

So let's say you install python 3.1 to a folder /usr/local/Python3.1/ for example...

Then you want to open up your bashrc file (in terminal sudo gedit ~/.bashrc).

You can then add the new python directory to your path like son:

py3path='/usr/local/Python3.1/bin'
PATH=$py3path:/$PATH
export PATH

Save the file (but don't close it in case you made a typo). Open a new terminal to refresh the changes and then type python. Your new version should be accessed now. You can still keep a link to your old build if you want, and my preferred way to do this is to add an alias in the bashrc file. Like this:

alias python1='/usr/bin/python'

Now when you type "python1" in your terminal, your old version should work.

I am spoiled; I use the enthought python distribution so it's extremely easy for me to update packages; however, when you install from source, I think it should be straightforward to tell setup.py how to know which python you want to install to. That is something someone else can help with probably.

Hope this is helpful; trust me, I learned this the hard way.

This post is a followup to an older dicsussion on recordmanaging. I am writing this to share with colleagues, so it may be a bit fluffy and not straight to the point. I apologize to all the experts.

Good programs always start by managing data in a flexible and robust manner. Python has great builtin container datatypes (lists, tuples, dictionaries), but often, we need to go beyond these and create flexible data handlers which are custom-fitted to our analysis. To this end, the programmer really does become an architect, and a myriad of possible approaches are applicable. This is a double-edged sword, though, as inexperienced coders (like me) will tend to go down the wrong avenues, and implement poor solutions. Emerging from such a trajectory, I really want to share my experience and introduce what I feel is a very easy-to-use and applicable data container.

PyTony in the aforementioned link really turned me on to the underused Python data container,namedtuple. A namedtuple is an immutable array container just like a normal tuple, except namedtuples have field designations. Therefore, elements can be accessed by attribute lookup and itemlookup (ie x.a or x[0]); whereas, tuples have no concept of attribute lookup. Namedtuples are a really great option for storing custom datatypes for these reasons:

• They are lightweight (take up very little memory)
• They allow for manual creation, but are also seemlessly interfaced to file or sqldatabase input (see reference)
• They have many basic utilities builtin, such as the …

Actually, I think I've come up with the best solution for this...

I wrote a wrapper function to return a namedtuple with option defaults and recasting. I would like to post it as a separate code snippet; it is different enough from this one I think to warrant it. Do you mind?

Ya you're right, I had considered that as well. The author of this code actually released a more official version in 2011. Despite the time I put into working with this, I think my best bet would be to make a true namedtuple subclass which typechecks the datafields, passed in the same way I did above. Yes, it's not mutable, but if mutability becomes a concern, I can use the class above as a standin. Unfortunately, collections.namedtuple isn't a subclass; rather, is a factory function so it's harder to go in and add features. I'll get around to doing it though.

So, I noticed the actual nametuple class has a _make method that will return a namedtuple when one passes in an interable.

        @classmethod
        def _make(cls, iterable, new=tuple.__new__, len=len):
            'Make a new %(typename)s object from a sequence or iterable'
            result = new(cls, iterable)
            if len(result) != %(numfields)d:
                raise TypeError('Expected %(numfields)d arguments, got %%d' %% len(result))
            return result \n

So, it can be used for example:

>>> t = [11, 22]
>>> Point._make(t)
Point(x=11, y=22)

I added this line to the codesnippet above, but get an error

TypeError: tuple.__new__(DomainCDD): DomainCDD is not a subtype of tuple

I suppose this makes sense since the record class is mutable, so I replaced tuple.new with object.__new___

But I get an attribute error when I try to pass in my iterables.

AttributeError: Query

This attribute error is being tripped by something in the make function. I don't know how to get _make to work in this case.

Thanks man. I suppose the * notation can be used anytime this is a problem then.

I have a simple class that stores attributes:

In [1]: class Test(object):
   ...:     a=10
   ...:     b=20

I have a function that returns attributes using attrgetter, and it accepts these through the args parameter to allow a user to pass various attribute fieldnames in. For example:

In [4]: def getstuff(*args):
   ...:     return attrgetter(args)

I can manually pass multiple arguments to an attrgetter and return the attributes from my class:

In [10]: f=attrgetter('a','b')

In [11]: f(test)
Out[11]: (10, 20)

However, my getstuff function doesn't work because args is a list and attrgetter takes only comma separated strings:

n [18]: f=getstuff('a','b')

In [19]: f(test)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-19-efe923d88401> in <module>()
----> 1 f(test)

TypeError: attribute name must be a string

Aka, the problem is args is being converted into a list. How do I get a list, say ['a', 'b'] into the attrgetter? attrgetter(['a','b']) ---> attrgetter('a','b')

Thanks

Also I would prefer to use elif for the various ValueError alternative to stress that only maximum one will be raised.

Can you show an example, I'm a bit confused on this.

The error was actually in the textbox, I had posted some numbered, indentied bullets, which was conflicting with the markdown syntax.

Sorry.

This code was inspired after several discussions on the forum, and I'm especially indebted to PyTony for all of his help with these aspects.

The motivation behind this code is quite simple. I wanted a robust, light and general way of storing CSV data that could be used over and over again and had to have the following properties:

The user sets fields and defaults values, and then field typecasting was automatically employed.
The data is stored in a regular, lightweight type (In this case a mutable namedtuple)
The data is managed by a custom dictionary object.

The MutableNamedTuple class is written by another author (http://code.activestate.com/recipes/500261/) via the factory function "recordtype" and can be used as if it were an official NamedTuple object. Lines 8-137 are this source code, and in practical use cases, one would usually save it in a separate module and import it.

The user merely has to change the stict_fields to whatever datatype is intended to be store. This variable implicitly stores types, so the user does not have to do it. Defaults can also be placed in this variable. Once set, a mutablenamedtuple will be generated, and from that, the program makes a subclass called Record, which is effectively a mutablenamedtuple with extra typchecking.

Record objects then behave similar to named tuples. They are exceedingly easy to interface to file I/O and sqlite modules(http://docs.python.org/library/collections.html#collections.namedtuple_).

The RecordManager class is a custom dictionary that is responsible for storing all Record objects. …

Hi,

I've been having trouble posting a code snipet tonight, I keep getting this error:

The code snippet in your post is formatted incorrectly. Please use the Code button in the editor toolbar when posting whitespace-sensitive text or curly braces.

My code is running fine in wingware IDE with 4 space margins everyhwere and no tabs. Perhaps someone who has posted snippets could take the code and description and post on my behalf? I wanet dot post for the contest but its really not important I suppose.

If possible, respond here or pm me.

Thanks.

Thanks Griboulis. Sorry to make you copy that for me, but after programming all day, my mind is shot so that helps a lot just to have someone point me back on the path of sanity.

It seems like, then, to get the behavior I want, I do want to overwrite the update method rather than doing something like:

self.__dict__ = self 

Which will map instance variables that I want to keep separate right into my dictionary. I will drop *args and just use **kwargs. The only appealing thing about *args is that I could send items into my dictionary and have keys auto-generate based on some combination of attributes. In reality, this is probably more harmful than good, as it requires an object in my dictionary to have a method to do this, so I will avoid it.

I'd like to make a dictionary subclass that takes in positional keywords in addition to the standard *args, **kwargs. I found this example on stackoverflow:

class attrdict(dict):
    def __init__(self, *args, **kwargs):
        dict.__init__(self, *args, **kwargs)
        self.__dict__ = self

a = attrdict(x=1, y=2)
print a.x, a.y
print a['x']
b.x, b.y  = 1, 2
print b.x, b.y

I'd like to know how to modfiy it to take in a positional argument. Something like:

class attrdict(dict):
    def __init__(self,positional, *args, **kwargs):
        dict.__init__(self, *args, **kwargs)
        self.positional=positional

### This doesn't
a = attrdict(50, 30, 20)
>>>TypeError: dict expected at most 1 arguments, got 2

This doesn't seem to want to accept multiple args. I was able to rectify this by overwriting the update method, something like:

def update(self, *args, **kwargs):
    'do stuff'

However, it seems like whenever I define the update module, I lose the ability to define the instance variable, self.positional.

For example, if I do:

### Without overwriting update()
a
>>>{}
a.positional
>>>50


### With overwriting update()
a
>>>  #No dictionary!
a.positional
>>> 50

Has anyone ever succesffuly made a custom dict that takes *args, and a positional argument?

Thanks for sharing HiHe.

Your example is the usual way I go about doing things. What motivated me to stray from that is really the idea that I am going to be sharing this code with a larger community (biopython I hope). So the question becomes, what is the most general and robust way to represent all of this data? If I was guaranteed to read it in from the same file type for eternity, then I wouldn't even bother making a custom class. But what happens when you also want the users to be able to enter data manually? Or enter sparse fields (aka the file format changes in the future, will your code still work without rewriting a new file I/O handler).

With all the help I've gotten on the forum as well as some other tips, I think I've come up with a preferred way to do it reliably and robustly. At the end of the day, what you need is a class to manage each record (or row in the data) and a dictionary class that is made to handle multiple records. Depending on if you want mutability and a certain level of type checking, you need to add custom behaviors at both levels.

I will turn this into a code snippet based on namedtuples later.

def setattr(self, key, value):
self.dict[key] = value
getattr = dict.getitem

Hey PyTony, this post makes a lot more sense to me now after the recent discussions we've had. Does enabling dict value access with '.' using the setattr method... does that add a lot extra overhead to the code or not really? It seems a nice convienence to the users, but is there a cost to it?

Yeah, I think there is definately a learning curve to installing packages in Python, mostly because some packages are easy to install and others are strange. What OS do you use?

One thing I do is I use the Enthought Python Distribution, which comes packaged with most of the standard scientific libraries pre-installed (numpy, scipy, biophyton, ipython etc...). I actually have a subscription, but the free version is pretty good anyway. Still, you will probably have the same issues with xephem as I doubt it is natively supported.

Did you read the install directions in the xephem package? If so, maybe email the authors for further help.

I should note that there are some standards to python packages. (http://docs.python.org/install/index.html)

In an ideal world, someone has made a binary which you can basically double click and it will install. These are .exe files on Windows, .deb files on ubuntu and I forget the extension for mac.

If those aren't available, you'd like for someone to have used distilus and made a setup.py file. The link I sent calls this the "standard way" and generally will involve you typing some variation of:

python setup.py install

Unfortunately, the xemphem package seems to be nonstandard. You're probably at the mercy of their installation instructions and maybe you can email and ask why they don't have a setup.py file.

Pandas, the libary I mentioned, for example does use distilus.

http://pandas.pydata.org/getpandas.html

This can be deleted, sorry for the haste.

Geez man, I completely forgot I already had started a thread pre-dating this one on that exact same topic. That's what I get for putting this project down for so long. Sorry, and thanks a lot. I will certainly implement the solution you outlined in that response.

YOu should start using the pandas package. It has explicit support for handling timeseries data and lots of builtin methods to convert. I imagine you're reinventing wheels somewhat.

Hey PyTony,

I hope you win the contest. I started another thread on this, but maybe it isn't really clear as it follows from some info we talked about in here.

In your example with the Info() object, is there a simple way to mandate that certain fields do have a specified type. For example, one of my fields has to be a float? I know that type checking is not always necessary; however, you can imagine certain fields are very prone to errors, so its best to make sure they have the correct type from the start.

As it is right now, I feel like I have to declare the types explicitly in botht he init method and then if I wanted to do more validation, I'd have to overwrite the setattr method. For example, if I wanted to enforce one of my fields is a float:

class Info(object):  
    __slots__ = 'height', 'width'
    def __init__(self, height, width):
        self.height, self.width, = float(height), width

That's no problem, but if the user sets "height", how do I let the program know that this should be a float? Aka, the user wants to do this:

myinfo.width='Sally'

I'd like the program to catch this and barf before any further happenings in the code.

I run into this same issue when I try to use namedTuple as well, namely I don't know how to restrict the field types for a couple of the fields that I know are going to problematic …

Hi,

I have been subclassing namedtuple to handle my custom data type:

fields=['Query', 'u1', 'Accession]

Old = namedtuple('Old', fields, verbose=False)

class New(Old):
    def __repr__(self):
        return 'Overwriting works nicely'

p=New('Hi',2,3)
print p
>>> 'Overwriting works nicely'

What I want to do is to add the behavior that some of my fields must be a certain type. For example, 'Query' must be a string. If the user does something like:

p.Query=50

I want it to raise an error. I tried overwriting the set_item attribute, but to no avail. Is there anyway to make the named tuple aware of what data type each field ought to have outright in the declaration? SOmething like:

fields=[str('Query'), int('u1'), int('Accession')]

Which doens't work. Thanks.

Hmm,

So I think I'm going to adopt this custom "records" class.

http://code.activestate.com/recipes/576555/

Which is effectively a namedtuple with mutability and default values. This seems easier than completely creating my own info object; although, now I understand how to do that thanks to PyTony. This will take care of default values on its own, and I will overwrite set_items and/or init methods to have my own validations in place.

Hi Pytony,

I am finally having some time to go back to this project and implement your suggestions. After reading for hours about named tuples and custom dicts, I see that your implmentation is really intelligent. I would like to adapt it and have a few questions:

First, in terms of validation and default values... I would like to make sure users don't enter incorrect data types. How would I put a stringent requirement on the fields so that users, for example, can't enter string input for a tuple field that I expect to be an integer? I assume I would do the validation at the tuple level. How would you implement this behavior? Also, would you enforce defaults at teh tuple level?

Secondly, can you elaborate a bit on what these methods are doing?

getattr = dict.getitem
def setattr(self, key, value):
self.dict[key] = value

def str(self):
return '\n'.join('%s = %s' % v for v in self.items())

I'm having some trouble finding good information on custom dictionaries in the literature. Like, what is getattr=dict.getitem doing?

Third, let's say I have my dictionary constructed and I'd like to return all values in a field. For example:

def get_widths():
    return [value.width for value in my_data.values]

Is there a better way to do this for all attributes in the dictionary, or should I just make a simple method like this for every field?

Wouldn't it be easier just to test if the first letter is equal to the last letter?

That would be a good idea if the user expected many of the entries were not palindrones. That would possible save a fraction of time; however, it would only catch non-palindrones. Further methods would be implemented to see if it were indeed a palindrone. Have the same first and last letter doesn't guarantee a palindrone.

For example "spoons" has the same first and last letter.

The [::-1] function is already pretty quick/optimized so I'd just use that, or the other suggestion snippsat said.

I would definately implement zztucker's idea. The syntax to reverse a list is:

a='hi there'
print a[::-1]
'ereht ih'

In case that was not clear.

Hi guys,

I was working with the pandas package and was so impressed with how simple something was that I had to share. Have you ever worked with csv data (in excel for example) and wanted to import the data to python, take either a column average or row average, then plot it? For example, imagine I have data formatted like:

    file1   file2   file3
500 50.0    53.3    43.0
550 23.1    32.0    32.5
600 23.0    23.0    35.0    
700 42.0    31.0    44.0

Pandas is a library for handling ordered series data and is GREAT for managing it. It essentially created containers that act as labeled numpy arrays. If I want to read this data in froma file and take the average along the rows (by slicing through names), and output a plot, it is this simple.

from pandas import read_csv
from matplotlib.pyplot import plt

data=read_csv('filename', sep='\t', header=0, index_col=0)
sliceavg=data.ix[550:700].mean(axis=0)

plt.plot(sliceavge)
plt.show()

Pretty impressive if you ask me.

I thought this was such a simple thing to do that I had to share and give some love to this libary. If you work with series data, start using this asap.

Thanks for the heads up. The named tuple is a nice alternative. Would you say that the get_width() function that I presented above is non-pythonic then?

So, in my dictionary, let's say I have a list of values. They each entry represents a datafield. Something like this:

class Test(object)
    mydict={key:[height, width, length]}

The class is built to manage various facets of the dataset, and predominately manipulates the mydict object. It also does things like populate the dictionary from a user-supplied file, for example. If I wanted to return the second column, or "width" entry, for a user-supplied key, or a list of all the values at this index, isn't it neccessary to write a method to do so. Something trivial like:

def get_width(self, key):
    return self.mydict[key][1]

You need to post an attempted effort before anyone will help you with homework.

Thanks PyTony. Let me present a question just to better understand how you would approach the problem under the guidelines you presented.

You have a custom dictionary. You want to write a function the datafield to a given key. Do you make this its own method? You said you don't use getters. Would this be a getter?

Hi guys,

This is more a general discussion of methodology than a specific question.

I have been working on a, the primary purpose of which is to store an oddly formatted data file (aka not you classic CSV). At the end of the day, the data is stored in a custom dictionary (thanks for help with this previously guys). In defining many of the methods, I realized that it's not always clear to me if I should write an instance method, or if I should write general methods and pass objects into them.

def testmethod(self, args...)

def testmethod(args...)

There are certain methods, like getters and setters, which I think intrinsically should be instance methods; however, many methods could easily operate on any object passed into them. I guess at the end of the day, my question is this.

Since I can define methods outside of a class and then pass an object into them, why bother writing instance methods at all? When should I say "yes this method is going to be in the class" as opposed to "no this method is just going to accept objects as its input". In the case of the latter, what's the best way to make sure the user passes the correct object?

Looking forward to your insights. Thanks.

Thanks guys, and thanks for that example drake. I probably will implement it.

Hey pyTony, do you mean something like this:

mylist=[150, 15, 'billy']

try:
    new=[int(mylist[0]), int(mylist[1]), str(mylist[2])
except ...

I was thinking maybe this is stringent enough.

Grib,

I noticed in plain Python that I do something like:

mylist=[int,int,str]

Is there a way to force inputs to adhere to this without defining an entirely new class. For example, if I say:

a=[1,2,'hi']
b=[1,2,5]

And then have the program recvognize that 'a' is valid but 'b' is invalid, without going into the detail of your exapmle? This is what I'm after at the end of the day.

In regard to type checking, I see what you guys are saying and am not trying to validate my types as much as I just want an explicit declaration like

mylist=[int,int,str]

So its clear exactly how the data fields are stored.

Am I making sense or is this not clear?

Or maybe just a method to validate input, rather building a new type. What is the most Pythonic way?

Also, I guess I should be aware that a custom class and a custom type aren't quite the same thing. Do you think it would be a waste of memory for me to create a class for handling this simple data rather than a new type? Like something with a quick initializer?

Damn. That's a great example, thanks!

I am trying to learn it; however, it is a bit involved for me. This is a lot more effort than I had expected. I know if the enthought tool suite, it's fairly straightforward to make custom types, but I guess that's because they already begin by inheriting fomr the HasTraits baseclass to begin with. For example:

List(Int,Int,Str)  

Declared in the contstructor would create a custom class for such a type. I guess I"ll stop taking this for granted.

PyTony's way is a nice solution. I just wanted to add that since you already mentioned trying to do this through bash, have you looked into the awk package? It runs through the shell and is apt at manipulating files, extracting rows, columns etc... on the fly through the terminal. My buddy swears by it for on-the-fly data manipulations.

I'm not quite clear. You're saying you want to be able to open the file on the fly, and based on what you call, it will automatically ignore the first line and column? Or do you want to alter all the files in one go?

Hi,

I think I have a pretty simple question but my google searches are giving me more information than I need. I think my terminology is not clear.

Let's say I'm making a dictionary to hold a datatype very specific to some filetype or data of interest. Let's say I have the following data in a file:

'keith'    100,  50, 'puppy'
'bill'     300,  32, 'cat'

This data is arbitrary, but I will create a dictionary keyed by the first column, and then the other entries will be stored in either a list or tuple. The first two entries of the list/tuple will be integers and the third will be a string. I want to mandate my program to only allow for this ordering to prevent bad input and also clarify to the user exactly how the data is stored. Bastically, I want to tell python "hey make a dictionary, with each value being a list/tuple of three object, and reserve space for two ints and a string. How do I do this?

Something like newtype=type(dict(str:list(int,int,str) )

I have no idea how to start this, although I imagine its very routine. Thanks.

Thanks gribooulilis.