Posts by Gribouillis

Hi,

I try to read line by line from a file some data. Each line in that file has the following format:

x = 78.36734693877551, z = 11.428571428571429 -> amplitude: 8.62847057093655E-6 | phase -1.5707968246742405\n

I am trying to extract those four float numbers using regular expression. I came with the following code:

import re

myteststr='x = 78.36734693877551, z = 11.428571428571429 -> amplitude: 8.62847057093655E-6 | phase -1.5707968246742405\n'

rs=re.compile(r'^x = (\S+), z = (\S+) -> amplitude: (\S+) | phase (\S+)(\s*)$')
rss=rs.search(myteststr).groups()

The problem is that only the first 3 floats are extracted. The last one and the '\n' character aren't extracted - they are set to the None object:

>>> rss
('78.36734693877551', '11.428571428571429', '8.62847057093655E-6', None, None)

Please tell me where I do wrong. I want to be able to extract the last number also.

Best regards,

PTS

The vertical bar is a special character in re. You can use

rs=re.compile(r'^x = (\S+), z = (\S+) -> amplitude: (\S+) \| phase (\S+)(?:\s*)$')

Also this looks nice http://stackoverflow.com/questions/385558/python-and-regex-question-extract-float-double-value

It's an ill-posed problem, as mathematicians say. Why do you need your ip address ? Why doesn't 127.0.0.1 suffice ? Do you want the ip address on the local network (you should know it), or your public ip address ? There is an old trick

import urllib2
ip = urllib2.urlopen("http://whatismyip.com/automation/n09230945.asp").read()
print(ip)

but it depends on the server at whatsmyip.com to be up and running...
Other problems, what is your os ? In linux you can get your local ip with sudo /sbin/ifconfig, in windows you could use ipconfig, etc.

k...Grib..thanx for ur effort in helping me...

is there any way to save output of individual commands over telnet session onto seperate variables instead of using Telnet.read_all() at the end? or
is there any way to save the output of Telnet.read_all() to a file , so that the output can be parsed?

Look more carefully in the activestate recipe. At line 46, there is a 'print response' statement. It means that the code displays the result of the commands. You should be able to do the same in your code. Use read_until() instead of read_all().

I think the book means

def Max(L):
    if len(L) <= 1:
        if not L:
            raise ValueError("Max() arg is an empty sequence")
        else:
            return L[0]
    else:
        m = Max(L[1:])
        return m if m > L[0] else L[0]

but don't use it in real code, use the the builtin max function !

Here is a working version

#!/usr/bin/python3
import sys

BUFSIZE = 4096

def make_streams_binary():
    sys.stdin = sys.stdin.detach()
    sys.stdout = sys.stdout.detach()

def main():
    make_streams_binary()
    while True:
        data = sys.stdin.read(BUFSIZE)
        if not data:
            break
        sys.stdout.write(data)

if __name__ == "__main__":
    main()

Test in a console

$ chmod +x streams.py
$ streams.py < dsc01390.jpg > tmp.jpg
$ diff dsc01390.jpg tmp.jpg
$

Write

def make_grid(x, value="0"):
    return [[value] * x for i in range(x)]

grid = make_grid(5)

Thanx Grib.........

I tried it, bu it gives "none" no matter the file is present or not.
Learnt that linux stores the return value of recently executed command in the variable $?. Is there any way of copying the value of $?of remote host onto a variable in the host where i run the script?

I don't think you can get $*. I can't help you because I don't have a telnet server to connect to ...

thanks a lot grib :)

I am able to connect to remote host and change directory. I need to capture the return value of Telnet.write(). What is it's type and value?

I mean to say, what does it return when a path specified with cd is present and when it is not present?

if it works like in a terminal, the answer should be in the string returned by read_until("$ ")

thanks a lot grib....

my guide suggested me to prefer telnet over ssh, as it is widely used. so I am thinking of using the module 'telnetlib'.

I am doing the following:

import telnetlib
...........
........
..........

user=User
password=pass

telnet=telnetlib.Telnet(host)
telnet.write(user+'\r\n')
telnet.write(password+'\r\n')after this, i need to check whether a folder exists on that host and if present, do an FTP upload.. how do I proceed?

Apparently, you must write a command and then read until the prompt is sent, something
like

telnet.write("cd path/to/your/folder\n")
data = telnet.read_until("$ ")

But I think you should have read also with your 2 first writes for the user and the password. You should start by testing this activestate recipe http://code.activestate.com/recipes/52228/
To download a file, try telnet.write("get filename\n"). You should also obtain a list of available commands with telnet.write("help\n") etc. If you never ran telnet by hand in a terminal, it would be a good idea to do it.

write

set1 = Set()
set1.setElements(elements)

I'll try the paramiko module

Here is the code I use with the paramiko module

import paramiko

ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect("computer_name",
                username="username", password="password", allow_agent = False)

def home_dir(user):
    return "/cygdrive/c/Documents*Settings/" + user

# get the content of a user's home dir on the remote machine
i, o, e = ssh.exec_command("ls " + home_dir("username"))

# download a remote file
ftp = ssh.open_sftp()
ftp.get("remote_path/filename" , "local_path/filename")
ftp.close()

after changing your code, I found that

print(*3*(y,))

gives the desired result. What does the '*' before the 3 mean? when I left it out, it returned:
print(3,'%')
('%','%','%')

In the python console

>>> def printSquare(x,y):
...  for z in range(0,x):
...   print(*(y,)*x, sep=",")
... 
>>> printSquare(3, "%")
%,%,%
%,%,%
%,%,%

The * is used when you pass a sequence of arguments to a function. print(*(y,y,y)) is the same as print(y,y,y) ,
but the argument list doesn't need to be written explicitly. For example

my_tuple = ('a', 'b', 'c')
print(*my_tuple) # prints a b c, identical to print('a','b','c')
print(my_tuple) # prints ('a', 'b', 'c'), identical to print(('a','b','c'))

# python 3
print(*(y,) * x, sep=",")

Install a ssh server on the remote machine (eg copssh for windows) and use the module paramiko.

no it doesn't work

OOps, reading further the link I gave, you , it seems that there are no quotes around the %s, try

sql= ("""INSERT INTO phonebook(number, Mobile) VALUES (%s,%s)""", (name, number))
cursor.execute(*sql)

what does python say ?

for only one variable it works but how to handle multiple variables.
i googled this and tried many ways nothing is working for multiple variables.
please help me out

The example I wrote above has 2 variables. Doesn't it work ?

thnx for the info please let me know what changes needs to be done for my code to work

Probably

sql= ("""INSERT INTO phonebook(number, Mobile) VALUES("%s","%s")""", (name, number))
cursor.execute(*sql)

If there is only one value, use sql = ("...", (name,))

line 25 should be while number < 2 or number > 30.

By the way, help is a very poor thread title. Try to be more descriptive of the thread's content.

I'm making a Card class and need to make sure that the suit is valid. When I call on the Card class and construct one setting the suit to 'Jordan' it actually ok's that suit and prints 'Ace of Jordan'. Instead it should become the default, of Spades. What am I doing wrong in my code that will not set it to the default suit? TIA guys

class Card:

    def __init__(self, value, suit):
        validSuit(suit)
        self.suit = suit
        self.face = value
        if self.face >= 1 and self.face <= 13:
            if self.face == 1:
                self.face = 'Ace'
            elif self.face == 11:
                self.face = 'Jack'
            elif self.face == 12:
                self.face = 'Queen'
            elif self.face == 13:
                self.face = 'King'
        else:
            self.face = 'Ace'
            
    def getSuit(self):
        return self.suit

    def getFaceValue(self):
        return self.face
        
    def __str__(self): 
        return str(self.face) + ' of ' + str(self.suit)
    
def validSuit(suit):
    if suit != 'Clubs' or suit != 'Spades' or suit != 'Hearts' or suit != 'Diamonds':
        suit = 'Spades'
        return suit
    else:
        return suit

And here is the driver:

from card import *
def main():

    card = Card(14, 'Jordan')
    print(card)
    # prints 'Ace of Jordan'

main()

Also, validSuit() should be

def validSuit(suit):
    if suit in ('Clubs', 'Spades', 'Hearts', 'Diamonds'):
        return suit
    else:
        return 'Spades'

please let me if there is another way to use python variables in MySQL insert statement

which works!!

See this example page http://mysql-python.sourceforge.net/MySQLdb.html#some-examples

Thanx for the information.
Now i get different error
if i user INSERT statement as
sql= ("""INSERT INTO phonebook(number, Mobile) VALUES(%s,%s)""" , name,number)
cursor.execute(sql)
i get the below error
exceptions:TyprError:execute statemenmt takes atmost 3 arguments (4 given)
and if i modify the insert statement as
sql= ("""INSERT INTO phonebook(number, Mobile) VALUES(%s,%s)""" , %(name,number))

i get another exception as
exceptions:TyprError:execute statemenmt takes atmost 3 arguments (61 given)

Please help me out
thanks in advance

Did you try with *sql as I wrote above ?

Read "dive into python" online.

Isn't it for the java forum ?

You should probably write

cursor.execute(*sql)

because your variable sql is a tuple containing the arguments to cursor.execute.

Also learn about code tags here http://www.daniweb.com/forums/announcement.php?f=8&announcementid=3

word_list = []
for w in book_line.split(" "):
    if w != "":
        word_list.append(w.lower())
print word_list

Indentation is very important.

You can use a regular expression

import re
sp_regex = re.compile(r"[ \t]+$", re.MULTILINE)
filename = "test.txt"
with open(filename) as f_in:
    content = f_in.read()
with open(filename, "w") as f_out:
    f_out.write(sp_regex.sub('', content))

Type pydoc turtle in a terminal.

You can write

with open("output.txt", "w") as fileout:
    fileout.write(stdout_value)

This is really awesome, thanks. I was getting close with a combination of compile, exec, and eval, but this is really helpful. I'll definitely be looking into the ast module.

One question:

def evaluate(expression, *args):
    return evaluator(expression)(*args)

I understand abstractly that *args is passing the sequence args as (positional?) arguments, but how does it work/what is it called? I've been searching for this in Python docs, but there are a lot of syntaxes with asterisks.

It's described in the syntax of function calls in the python documentation http://docs.python.org/reference/expressions.html#calls

For example, using this code snippet http://www.daniweb.com/code/snippet323792.html and the ast module, I obtain the following graph for the parsing of an expression

import ast

node = ast.parse("diff(integ(x**2+y**2, 3, 2), 2)")

def rec_traverse(node):
    for n in ast.iter_child_nodes(node):
        yield node, n
        for item in rec_traverse(n):
            yield item
            
def label(node):
    return node.__class__.__name__
            
from fastgraph import graph
graph(rec_traverse(node), label).draw("graph.png", prog="dot") 
import webbrowser
webbrowser.open("graph.png")

from the nodes Name and BinOp and Call, you should be able to obtain the variable names and the function names, and define your own evaluator for the parsed expression. (Remark: in a graphviz representation, the children of a node are not ordered)

diff(integ(log(anotherCalculatorFunction())))

if I run diff() the way it's written, then it'll try to compile(integ(log(...))) and then eval() it. Unless I'm missing something obvious, it'll choke on this, right?

I tried to play with this code, which evaluates expressions in a fixed namespace, where symbols are added on demand. The function evaluator() take an expression as an argument and returns a function where the unknown variables are ordered alphabetically (for example x and y)

from math import sqrt

EVAL_NAMESPACE = dict()

def varnames(function):
    return func.func_code.co_varnames

def evaluator(expression):
    namespace = EVAL_NAMESPACE
    code = compile(expression, 'expression', 'eval')
    args = []
    for name in (n for n in code.co_names if not n in EVAL_NAMESPACE):
        if name in globals():
            EVAL_NAMESPACE[name] = globals()[name]
        else:
            args.append(name)
    args = tuple(sorted(args))
    func = """
def function(%s):
    return eval(code, namespace, locals())
""" % (", ".join(args))
    exec func in locals()
    function.__name__ = "evaluator(%s)" % repr(expression)
    print "EVALUATOR:", function.__name__
    return function

def foo(expression, a, b):
    func = evaluator(expression)
    return (func(a) + func(b)) * 0.5

def bar(expression, x):
    func = evaluator(expression)
    return func(x+1) - func(x)

def evaluate(expression, *args):
    return evaluator(expression)(*args)

expr = 'foo("x**2 + x", bar("3*x+2", 2), 1)'
print evaluate(expr)

print evaluate("sqrt(evaluate('x**2+y**2', 3, 8))")

For a more complete solution, you should have a look in the ast module. The ast.parse function parses a python expression and returns an abstract syntax tree. You could examine this tree and build your evaluator on this tree.

Ah, you just edited your code for a question I was going to ask.

Now, looking at that, how do I make it deal with nested functions? Or how do I parse the input to extract the nested functions?

what do you mean by nested functions ?

You could write

def evaluator(funct, name='funct'):
    code = compile(funct, name,  'eval')
    def f(x):
        return eval(code, locals())
    f.__name__ = name
    return f

def integ(funct,a,b):
    return gauss(a, b, evaluator(funct))

print integ("3*x++5", 0, 1)

because the expression is evaluated in a dictionary where there is a variable named x (the locals() dict of the function f() above).

Would it help if I gave you access to the whole program? It's very strange as it doesn't really make any sense...

I'm running this on Python 2.7 and will try tweaking it to run on 3.1.2 and keep you posted.

Yes if you can attach a zip with the whole program, I could try it and see if I have the same bug.

I've just discovered something strange, in that if I change the write() line to:

stamp.write(valid + "\n")

This actually shows the correct values, but puts a space in between (which means that other parts of the program fail). I don't know if this helps at all? Here's the output (windows, note the additional line in between):

...and the same in true when run on Linux:

Add assert statements to add more control

assert type(valid) is str
assert len(valid) == 18 # or 19 (with the '\n')

also try to print valid.strip(), then \n

Sorry, it's a mystery. I can't reproduce the bug on my computer, so I can't help you much. I've never heard of such behaviour before :(

Hi Gribouillis

I get this output to screen:

...and this to the file:

If you are on windows, did you try to use the modes 'rb' and 'wb' instead of 'r' and 'w' ?

If you have read my initial post it sais "the code works just fine...". The problem is siply that there are _extra_ spaces that appear when I run the multi threade version. It seems as though one of the threads, for no apparent reason, decides to add a space in front of the print statement.

I have overcome this problem with a rather unoptimized solution, where all the threads store the value to a common array, that is later printed out. This does indeed work, but removes parts of the "paralell-ness" of the program, a part I tried to introduce to gain some performance. It is no longer essential to me to get an answer, because this assignment belongs to the past. But non the less it would be nice to know the reason for this strange behavior.

You could try 2 things:

First idea: define a global Threading.Lock object that each thread must acquire to print a value

lock.acquire()
print value
lock.release()

Second idea: let your threads put the values in a Queue.Queue object, and start a single thread to read values in the queue and print them

# in working threads
thequeue.put(value)

# in the single printing thread
while True:
  value = thequeue.get()
  thequeue.task_done()
  print value

Thanks for taking a look into this :)

The variable that comes back is in fact returned from another function. This function (called snipvalue()) returns either a value (in this case
4CE1FFC64101843801) or False.

When I run the code (I have inserted the line print valid, to print the variable: valid), I get this on screen:

... but I get this in the file:

You will notice the difference between the lines on screen and the lines in the file, they appear to remove the penultimate '0'.

Here's the code that I'm using:

try:
        infile_open = open(k12txt, 'r')
    except IOError:
        print '\nThe file "' + pcapfile + '" does not exist, exiting..\n'
        usage()
        exit(1)
    infile = infile_open.readlines()
    infile_open.close()
    
    stamp = open(stamps, "w")
    
    for line in infile:
        valid = snipvalue(line,size)
        if valid:
            
            # Test code: Debug
            print valid
            
            stamp.write(valid)
            
    stamp.close()

Any ideas?

Thanks again :)

What happens if you print repr(valid) ?

It seems to be an impossible behaviour. To speak about something concrete, I ran the following code

# python 2 and 3
from __future__ import print_function

def main():
    valid = str("4CE1FFC64101843801")
    with open("stamps", "w") as stamps:
        stamps.write(valid)
        
    with open("stamps", "r") as stamps:
        check = stamps.read()
        
    print("valid: ", valid)
    print("check: ", check)
    assert(check == valid)

if __name__ == "__main__":
    main()
    
""" my output -->

valid:  4CE1FFC64101843801
check:  4CE1FFC64101843801
"""

Does it run on your system ?

Here is a good way to do this. The key points are

• use generators
• use list comprehensions
• use sorted() and it's 'key' argument

# python 2 and 3
import os

# write a generator to decouple the code which creates the data
# from the code which uses it

def gen_triples(folder):
    for (path, dirs, files) in os.walk(folder):
        for file in files:
            name = os.path.join(path, file)
            size = os.path.getsize(name)
            repr = "[{n} size = {s}]".format(n = name, s = size)
            yield name, size, repr # we yield a tuple with 3 values
            
def by_name(triple):
    return triple[0]

def by_size(triple):
    return triple[1]

folder = "."

# use the sorted() builtin function to sort the data

name_sorted = [ repr for (name, size, repr) in sorted(gen_triples(folder), key = by_name)]
size_sorted = [ repr for (name, size, repr) in sorted(gen_triples(folder), key = by_size)]

from pprint import pprint
print("BY NAME")
pprint(name_sorted)
print("BY SIZE")
pprint(size_sorted)

ok imported and used ... how would I get the information back to be used in my current program?

If you want information back, there are different ways, you can let your function return a value, then

from mymodule import view

information = view(...)

Or you could make view a method in a class and set instances attributes

class Whatever(object):
  def view(self, ...):
    ...
    self.pic = ...

Then in other files

from mymodule import Whatever

w = Whatever()
w.view(0,0,...)
print w.pic

There are many other ways to get information back...

First I think it can be shortened to

def view(N,W,E,S):
    t = (N, W, S, E)
    st = ("north", "west", "south", "east")
    ft = (fhall, lfhall, frhall, frlhall)

    d = st.index(info_dict["direction"])
    if t[d] != 1:
        pic = fwall
        wall = "yes"
    else:
        x, y = t[(d+1) % 4], t[(d+3) % 4]
        x, y = int(not bool(x-1)), int(not bool(y-1)) # now != 1 means 0
        pic = ft[2 * y + x]
        wall = "no"

Otherwise, to make an importable module, simply save the file as "mymodule.py" and write "import mymodule" in the other files.

Don't program with copy and paste !

Also, use the 'else' keyword in sequences of if statements.

Use

array = [[float(x) for x in line.strip().split()] for line in open("file.txt")]

Start with this documentation page http://docs.python.org/tutorial/floatingpoint.html.

A problem is that python's 0.1 is not the mathematical 0.1 because 0.1 can not be represented exactly as a machine binary number.

Why don't you write

for each in wp_head:
        print "Word Perfect header %s found @ offset %d" % (
                             each.group(), total_data_read + each.start())

?

Another issue to consider is that a match could occur between 2 blocks, for example \xFF\x57 at the end of a 4096 bytes block and \x50\x43 at the beginning of the next block. Currently, your code won't find the match.

When working with bitwise operations (~, &, |, ^), it's better to think of an integer as a semi-infinite sequence of bits. See the examples below

# python 2 and 3
from __future__ import print_function

def tobit(n, width=50):
    m = (1 << (width - 3))
    n = (n & (m-1)) if n >= 0 else ((n & (m-1))+m) & (m-1)
    z = "...{n:0>{w}}".format(n=bin(n)[2:], w=width-3)
    return z

OOPS ! there was an error in the function tobit() above. Replace it with

def tobit(n, width=50, reverse=False):
    m = (1 << (width - 3))
    n = (n % m) & (m-1)
    z = "...{n:0>{w}}".format(n=bin(n)[2:], w=width-3)
    return z[::-1] if reverse else z

The output is now

"""
various numbers [354777, 4191, -24522, 774, -32] :
...00000000000000000000000000001010110100111011001
...00000000000000000000000000000000001000001011111
...11111111111111111111111111111111010000000110110
...00000000000000000000000000000000000001100000110
...11111111111111111111111111111111111111111100000

x, y, x & y:
...00000000000000000000000010001011101101000011110
...00000000000000000000000000010111011101101110111
...00000000000000000000000000000011001101000010110
the bits set in x & y are the common bits set in x and in y

x, ~x:
...00000000000000000000000010001011101101000011110
...11111111111111111111111101110100010010111100001
the bits set in ~x are the bits unset in x

x, ~31, x & ~31:
...00000000000000000000000010001011101101000011110
...11111111111111111111111111111111111111111100000
...00000000000000000000000010001011101101000000000
in ~31, only the five lowest bits are unset
x & ~31 is x with the five lowest bits forced to 0
"""

def round32(x): return (x+16) & ~31

This seems to work right. Could you please explain what the '&' and '~31' are used for?

When working with bitwise operations (~, &, |, ^), it's better to think of an integer as a semi-infinite sequence of bits. See the examples below

# python 2 and 3
from __future__ import print_function

def tobit(n, width=50):
    m = (1 << (width - 3))
    n = (n & (m-1)) if n >= 0 else ((n & (m-1))+m) & (m-1)
    z = "...{n:0>{w}}".format(n=bin(n)[2:], w=width-3)
    return z

L = [354777,4191,-24522,774,-32]
print("various numbers", L, ":")
for n in L:
    print(tobit(n))
    
    
print("")

x, y = 4577822, 768887

print("x, y, x & y:")
for s in (x, y, x & y):
    print(tobit(s))
print("the bits set in x & y are the common bits set in x and in y")

print("")
print("x, ~x:")
for s in (x, ~x):
    print(tobit(x))
print("the bits set in ~x are the bits unset in x")

print("")    
print("x, ~31, x & ~31:")
y = ~31
for s in (x, y, x & y):
    print(tobit(s))
print("in ~31, only the five lowest bits are unset")
print("x & ~31 is x with the five lowest bits forced to 0")

""" my output --->
various numbers [354777, 4191, -24522, 774, -32] :
...00000000000000000000000000001010110100111011001
...00000000000000000000000000000000001000001011111
...11111111111111111111111111111111010000000110110
...00000000000000000000000000000000000001100000110
...11111111111111111111111111111111111111111100000

x, y, x & y:
...00000000000000000000000010001011101101000011110
...00000000000000000000000000010111011101101110111
...00000000000000000000000000000011001101000010110
the bits set in x & y are the common bits set in x and in y

x, ~x:
...00000000000000000000000010001011101101000011110
...00000000000000000000000010001011101101000011110
the bits set in ~x …

I suggest def round32(x): return (x+16) & ~31 . As tonyjv said, we much choose if 16 should be rounded to 0 or 32. This rounds to 32. Otherwise add 15 instead of 16.

The main reason is that a 20 lines python program does more job than a 500 lines C++ program.