hughesadam_87 54

I use wingware IDE. Worth the money.

hughesadam_87 54

Good to hear. Perhaps mark this thread as solved? If you have further pandas issues, I'd recommend posting to the pandas mailing list, which is very active and extremely helpful.

hughesadam_87 54

The cached property is definatly the best suggestion. If not using that, I would get in the habit of using try: except in these situations. That is syntactially simple and very common, so others reading your code will be able to deduce your intent from the code.

hughesadam_87 54

Look at python library collections, particularlu the Counter class.

hughesadam_87 54

Your best bet is to load all the datafiles into a pandas dataframe. This is the de-facto structure for handling labeled array data, and also has a ton of utilities for data manipulation ESPECIALLY timestamp.

post an example of one of your files, maybe the first 30 lines, including header, and I can help you get it into a pandas dataframe.

http://pandas.pydata.org/

hughesadam_87 54

I can't really digest the entirety of your code just by looking at it, but another thing you might wanna look into is the polymonial support in numpy. There are some functions, mostly for fitting polynomicals of Nth order to other curves, that might be useful to you. For example:
http://docs.scipy.org/doc/numpy/reference/generated/numpy.poly1d.html

hughesadam_87 54

Vega, does the experssion "with open(...) as fout" automatically close the file when the iteration completes?

hughesadam_87 54

outstring=' '.join([str(i) for i in range(0,105,5)])
file.write(outstring)

Also, it's a good habit not to use the word "file" as this is a python reserved file. I'd recommend outfile or just o.

hughesadam_87 54

Can you elaborate on this, your question isn't clear to me.

hughesadam_87 54

If you use numpy, then you want to avoid iterating item by item. In fact, the whole reason to use numpy, is that you don't have to do this. If you do do this, you get no speed increase over plain python (the speed increase is about 1000-fold btw).

You can do linear regression out of the box in python already:

http://glowingpython.blogspot.com/2012/03/linear-regression-with-numpy.html

If you are making your own for a class or something, you still want to avoid itearting over the array itself. It does take some practice to get used to this.

Gribouillis commented: nice help and link +13

hughesadam_87 54

Thanks, this is awesome.

hughesadam_87 54

Can you put all the zipped values into its own list.

allvalues=[zip(ar1.values, br1.values, ar0.values, br0.values), zip(ar2.values, br2.values, ar1.values, br1.values) ... etc]

Then iterate over just this:

for zippedvals in allvalues:
   for w,x,y,z in zippedvals:
        val = (sqrt((w.data**2)+(x.data**2)))-(sqrt((y.data**2)+(z.data**2)))
        files[0].write(('%6.20f\n') % (val))

hughesadam_87 54 Junior Poster

Hi guys,

This feels like something that should be possible in Python, and something that I feel like I should know how to do.

Image I have a list of numbers:

items=[10,20,30]

Let's say I wanted iterate over them and define two new variables, a and b.

a=[]
b=[]
for item in items:
  a.append(item*10)
  b.append(item*20)

I can also do this in two list comprehensions:

a=[item*10 for item in items]
b=[item*20 for item in items]

It feel like I should be able to do this in one single expression. Something like:

a,b=[(item*10),(item*20) for item in items]

Is this possible?

Thanks.

hughesadam_87 54

I actually added that while I was waiting for responses. I have the c2o or o2c as choices. cups2ounces or ounces2cups. Is that not needed? I am just new :\

Also you are saying to start with a "if" statement?

I'm saying you define two variables, c2o and o2c and then never use them.

Yes. Your code would literally only change at the bottom:

main()

becomes:

if __name__=='__main__':
    main()

The code should run fine and you won't notice any differences.

hughesadam_87 54

I noticed a few things.

First, there's no reason to put the entire code into a function (eg main). This is fine, but not mandatory. There's a special line at the bottom of your code that you should add so that if this program is called as an import from another program, then main() won't be executed. It will only be executed if you run the program (ie python thisprogram.py).

Change main() to:

if __name__=='__main__':
    main()

Note that the use of main() and main is just a coincidence. If your main() function was called "main_insane" then it would still be:

if __name__=='__main__':
    main_insane()

Second, why do you have these variables otc and cto?

Third, I would add an error if the use doesn't enter 1 or 2. Something like:

if choice == 1:
    ....
elif choice == 2:
   ...
else:
   raise InputError('Please enter 1 or 2!')

This will let the user know they screwed up.

hughesadam_87 54

Ok thanks guys. Let me look at your suggestions, reevaluate my problem, then come back if I still have questions.

hughesadam_87 54

Why not just split at "enclosure_url" rather than "title" and keep only the half you want?

hughesadam_87 54

Sorry for the late reply. This is exactly what I needed and works great. I honestly think this is the best implementation of property for costly operations. Perhaps thinks of a catchier named than cached_property_with_key2 and put it as a code snippet? I'd like to save this and use it in many future applications, so is there a way you'd like to be credited besided your daniweb account name?

hughesadam_87 54 Junior Poster

Hi guys,

I am running a costly simulation and trying to optimize the output. I'd have a fixed range over which the parameter of interest can vary, and about 100 values it can take in between. We know the result we are looking for and want to find the parameter value that comes closest to this. Instead of iterating the parameter incrementally, from say 0 to 100, I'd like to implement some sort of binary search algorithm. I am sure there is a "best" value on this range, so there's no need to worry about duplicate values.

The algorithm in my head should go something like:

Parameter low bound (p=0): Value known
Parameter high bound (p=100): Value known

Compare=0-50 and 50-100

Choose range at which best estimate is found. For example 0-50 is closest.

Bisect this range again...
0-25 vs. 25-50

Can anyone recommend a module or something that is suited to this type of thing? I know bisect is the module of choice for binary searches, but am having trouble seeing exactly which aspects of it are suitable to my needs.

Thanks.

hughesadam_87 54

Awesome man, thanks for posting this. Let me play with it and bring back some feedback soon.

hughesadam_87 54

I think it's time for me to start using Python3

hughesadam_87 54

Thanks Gribouillis. That property decorator looks nice and is what I'll use if I can modify it to incporporate the following behavior.

class X(object):
   def __init__(self):
       self.a=50
       self.b=30

   @cached_property
   def c:
      '''Return c from pythagorian theorem'''
      return=sqrt(self.a**2 + self.b**2)

I'd like it to cache its value just like you code does; however, if I were to change a or b (eg:

self.a=52

I'd like the cache to be emptied and then recomputed the next time c is called. In essence, the property can check if any dependent variables have changed. If so, it updates c, otherwise it uses cached values. Basically, it's a "smart" cache.

Does behavior like this exist somehow? I know it can be implemented using properties in the enthought tool suite for example... but these are special object types whose purpose is to implement such behavior.

hughesadam_87 54

Thanks PyTony. I'll post the ones I tried, but first let me ask a clarifying question.

I'd like a property to implement the following behavior.

Say a property depends on two variables, a and b.

When I call the property, it returns a X b

If I call it again, it already has this value saved, so it merely returns it without recomputing a X b.

If a or b DO change and I call the property again, it will recompute the product a X b before returning it.

Some of the examples I'm seeing don't to the last part, aka recompute the value if the variable dependencies change.

I will try to implement the example you sent; however, it requires several module imports so I'll sift through all the source code there and take out the ones I need. But offhand, do you know if what I'm asking for is possible?

hughesadam_87 54 Junior Poster

Hi guys,

I've downloaded 2 cached_property implementations from the internet and both did not work correctly for my programs. Does anyone here used a cached_property function that works really well and could share with me?

hughesadam_87 54

Also, python is going to round (9/5) to 2 since these are integers. Get in the habit of using floats: (9.0/5.0)

hughesadam_87 54

Install ubuntuu. It is a free oerating system, basically virus free and better than windows.

hughesadam_87 54

Bump on snippsat's reply...

That is almost certainly the way to do it if you ask me.

hughesadam_87 54

Bump tot he above poster, it is most certainly better to store variables so they may be accessed by attribute (eg movie.director)

hughesadam_87 54

Oh sweet.

To the OP, if you aren't able to get any help here, I expect the appengine has a mailing list and they'd be the best people to help in the scenario that no one here uses the engine.

hughesadam_87 54

Thanks woooee. I hesitate to use eval for the reason you pointed out. I like the idea of using len(); that's smart; however, it does restrict user input style (for example, I'd like 'False' in the field to also work.)

For my use case, the programs actually know ahead of time what field types ought to be. Therefore, I can take advantage of this since my program expects a boolean. Therefore, my solution is to do something like:

_boolean_states = {'1': True, 'yes': True, 'true': True,
                   '0': False, 'no': False, 'false': False}    

for field in expected_fields:
   if field == bool:
      entry=_boolean_states[entry.lower()]  #.lower() for case insensitivity

Stole this idea from the ConfigParser package actually.

I will keep these other methods in mind for other use cases, thanks for helping me out.