Posts by Gribouillis

It goes the other way round. If you're already running python 2, sys.executable is the path to the python 2.7 executable. If you're already running python 3, sys.executable is the path to the python 3.4 executable. If you want to run a script with python 3.4, use the command

python3 myscript.py

in a terminal, or use a shebang line

#!/usr/bin/env python3

at the top of your file and simply call ./myscript.py (without the command python).

I dont understand how you want to get this user input, nor why it could not be possible to append values to a list. You only need to define a global list and call its append() method when needed.

You need to create a Toplevel window. There is a good example in the famous blog Mouse vs Python.

Also study how Mike Driscoll uses of a class App in this example, in order to encapsulate tkinter code.

The example was written for python 2, but it shouldn't be too difficult to adapt it to python 3.

Did you try "%lf " instead of "%lf" ? (see man fscanf)

Replace all the except: pass with except: raise and you'll have a detailed report about what's going wrong.

You could also log the exception tracebacks to files if you prefer.

The problem is that your code works very well for me. I get

<function func at 0x7f48097fd230>
success

so I don't understand the issue.

You could try

NS.update(vars(__builtins__))

I only removed line 7 and it worked.

Edit: You can also use this initialization

NS = {'__builtins__': __builtins__}

Perhaps an error in the computations of y1 an y2 ? Also find the reason of the index error.

it still extends past the dedent in some cases what exactly am I doing wrong??

It is difficult to say without an sufficiently detailed example.

Another issue that you may have is that code in the editor may be invalid python code (with unterminated strings or bad unicode characters, ...). Can your code handle this ?

You could perhaps find some ideas in the source code of the asciitree module, which produces terminal output similar to the linux tree command.

I don't see how tokenize can fail: it generates INDENT and DEDENT tokens that you could use to compute the indentation level. Also note that it is better to indent python code with space characters than tabs. A good python editor needs the 4 space indentation.

Unfortunately, I'm not using windows tools such as notepad++ and VS2010, so I don't really understand what you want to do.

Here is function occur2()

def occur2(n):
    for i, j, a, b in gen_sublist(n):
        s = [x for x in a if str(int(x) - 20) in b]
        for item in s:
            print("%s reoccurs after %d sets" % (item, n))
            print("%s in set%d: %s" % (item, i+1, a))
            print("%s in set%d: %s" % (str(int(item) - 20), j+1, b))

if __name__ == '__main__':
    occur2(2)

The output is

70 reoccurs after 2 sets
70 in set3: ['63', '70', '78', '60']
50 in set6: ['50', '58', '02', '11']
78 reoccurs after 2 sets
78 in set3: ['63', '70', '78', '60']
58 in set6: ['50', '58', '02', '11']
32 reoccurs after 2 sets
32 in set5: ['32', '41', '71', '70']
12 in set8: ['12', '52', '50', '60']
70 reoccurs after 2 sets
70 in set5: ['32', '41', '71', '70']
50 in set8: ['12', '52', '50', '60']

Here is how you could write occur(n) more simply

big_set = [
    ['78','18','79','56'],
    ['13','40','16','04'],
    ['63','70','78','60'],
    ['10','35','66','13'],
    ['32','41','71','70'],
    ['50','58','02','11'],
    ['13','40','41','05'],
    ['12','52','50','60'],
    ['71','13','66','12'],
    ['50','90','73','41'],
    ['09','18','44','54'],
    ['12','41','32','67'],
]

def gen_sublist(n):
    for i, seq in enumerate(big_set[n+1:]):
        yield i, i+n+1, big_set[i], big_set[i+n+1]

def occur(n):
    for i, j, a, b in gen_sublist(n):
        s = set(a) & set(b)
        for item in s:
            print("%s reoccurs after %d sets" % (item, n))
            print("set%d: %s" % (i+1, a))
            print("set%d: %s" % (j+1, b))

if __name__ == '__main__':
    occur(4)

Here is my result (with python 3)

13 reoccurs after 4 sets
set2: ['13', '40', '16', '04']
set7: ['13', '40', '41', '05']
40 reoccurs after 4 sets
set2: ['13', '40', '16', '04']
set7: ['13', '40', '41', '05']
60 reoccurs after 4 sets
set3: ['63', '70', '78', '60']
set8: ['12', '52', '50', '60']
13 reoccurs after 4 sets
set4: ['10', '35', '66', '13']
set9: ['71', '13', '66', '12']
66 reoccurs after 4 sets
set4: ['10', '35', '66', '13']
set9: ['71', '13', '66', '12']
41 reoccurs after 4 sets
set5: ['32', '41', '71', '70']
set10: ['50', '90', '73', '41']
41 reoccurs after 4 sets
set7: ['13', '40', '41', '05']
set12: ['12', '41', '32', '67']

How come you have almost exactly the same issues as one year ago ? Do you attend the same course for the second time ?

You mean class datetime.date where datetime is the module, but as you imported from datetime import datetime, datetime.date is the descriptor date of class datetime. What you can do is

import datetime as dt
day = dt.date(year, month, day)

If it is already a datetime.time instance, you can perhaps skip this step and use strftime() directly.

You could perhaps try

df[...] = df[...].map(lambda x: datetime.strptime(x, '%H:%M:%S'))

Did you run hp-setup ? https://help.ubuntu.com/community/sane

Edit: the hplip page http://hplipopensource.com/hplip-web/models/other/envy_5640_series.html gives a list of distros for which the scanner works. If your distro is in the list, there must be a way to configure this properly.

Did you read this help page https://help.ubuntu.com/community/Mount/USB ?

I had a lot of success using luckybackup instead of rsync (actually, luckybackup wraps rsync but I think it is much easier to use). You can install it with apt-get.

Luckybackup comes with a GUI in which you define job profiles. So here a job profile would contain that you want to backup your /home/garret/Music folder to the destination /media/garrett/6BF6-AC8A and you can exclude files the way you want and include all sorts of parameters in your job profile through the GUI.

Once the profile is written, you can run the job, or run it regularly using a cron task. You can also run the job by invoking luckybackup with convenient parameters on the command line.

Edit: note that you may need a symlink

sudo ln -s /media/6BF6-AC8A /media/garrett/6BF6-AC8A

if luckybackup says that the destination is not mounted.

Sorry I don't understand this code. I think I explained what I meant as clearly as I could, but if you really don't understand it, you can use your own different method. Perhaps someone else can help you with this.

A more complete example would be useful. I don't see where it fails. Did you create the clone function with the NS dictionary ?

One thing to try is

NS['__builtins__'] = sys.modules['__builtin__']

(sys.module['builtins'] in python 3)

I tend to think that the separate exec is not necessary. Try to pass the execution namespace directly when the clone is created.

You need to compute the save_path from the filename argument (which is the path to the source file). The idea is that you dont dictate the path individually, but you dictate a rule to create the save_path.

Here is something

>>> def f2(): print 'test failed'
... 
>>> def f(a1,a2): f2()
... 
>>> function = type(f)
>>> fclone = function(f.__code__, {}) # function with empty func_globals
>>> fclone.__defaults__ = (1,2)
>>> 
>>> exec 'f()' in {'f':fclone}
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 1, in <module>
  File "<stdin>", line 1, in f
NameError: global name 'f2' is not defined

Edit: function() accepts a few other arguments. See the code of func_new() in python's C source code.

There is still a for loop in destination_csv(). You dont need it. This function handles only a single filename which is already a parameter of the function.

do you mean that I dictate the relative path?

Your program takes XML files as input and writes CSV files as output. You are the only one to know where you want to store the csv files on your file system and you must tell this to your program. This is what destination_csv() does.

The computer cannot guess where the output file must be written.

Write a path to one of your xml source files and the corresponding path to the output csv file that you want. destination_csv() must be able to transform the one into the other.

You dont need to copy or move or rename any file. You only need to create a string, the system path to the destination file. You don't need to create or write this destination file. It will be written later by the call to writerows().

Indentation of line 87 is incorrect. Now here is an hypothetical example which shows how destination_csv() should work

>>> obj.destination_csv('C:\\Playground\\Samples\\FOO.xml')
C:\Playground\Samples\CSV_Reports\FOO.csv

Edit: I shortened the path for the example.

You don't need to change the initial findFiles() function which calls open_and_parse(). The destination_csv() function must not rename any file nor iterate over a listdir etc.

In your original code, you only need to replace lines 60 and 61 with

    dest = self.destination_csv(filename)
    with open(dest, 'wt') as fh:
        writer = csv.writer(fh)
        writer.writerows(self.makerows(flatten_dict(root)))

I think you still don't understand what destination_csv() should do.

The directory needs to be made only once. destination_csv() does not rename the XML file. It only creates a new destination filename where the csv data will be written without modifying the source XML file. The name of the destination file is built from the name of the source file, which permits to handle several files which names don't collide.

Edit: destination_csv() is not at all recursive. It handles only a single file name.

You could use

exec "f()" in {'f':f}

You did not understand my advice the function destination_csv() is only supposed to take a filename argument (such as C:\foo\bar\baz\awesomedata.xml) and return another string, such as "C:\\Users\\Desktop\\Playground\\Samples\\CSV_Records\\awesomedata.csv". It is not at all supposed to open or parse the file.

You can write

    dest = self.destination_csv(filename)
    with open(dest, 'wt') as fh:
        writer = csv.writer(fh)
        writer.writerows(self.makerows(flatten_dict(root)))

Then you need a method

    def destination_csv(self, filename):
        """Compute a destination filename from a source filename

        for example if filename is
            C:\foo\bar\baz\awesomedata.xml

        the result could be
            C:\foo\bar\baz\CSV\awesomedata.csv
        """

use function from module os.path and string operations to compute the destination file.

A dictionary is created with {} delimiters, not [].

No problem with firefox in kubuntu.

Do you really mean a distributed denial of service attack ? Where would it be funny or cool ?

The code will not define the rules. It works the other way: you define precise rules, then they can be implemented in code. Otherwise the program will work for this example xml file but not with another.

The question is why MonthDayCount1, MonthDayCount2 etc and not Int321, Int322, etc. By which rule do the Int32 vanish ?

In the xml file, there is no datetime1 datetime2 datetime3. The 1 2 3 etc must be added somewhere. That's what I meant when I said earlier that I dont understand your rules for key generation (or header generation).

When I say what do you want to have instead of 'foo.bar.baz.datetime' and you tell me that you want datetime, you get datetime. If you want datetime1, you must give a very precise and descriptive way to know how 'foo.bar.baz.datetime' becomes datetime1. I can not invent this rule, nor can python.

Because you changed my functions flatten_dict() and flatten_list(). Take the versions I wrote above.

Instead of calling flatten_dict(root), you would call pairs_from_root(root). You need to do this only once at the bottom of the program. This will change the CSV structure in the sense that if you have two columns A.TimeAtPreviousAddress and B.TimeAtPreviousAddress, you will now have only one column TimeAtPreviousAddress.

The question is what do you want to have instead of 'Response.MemberO.PMembers.PMembers.Member.CurrentEmployer.EmployerAddress.TimeAtPreviousAddress'
? If you want to have only 'TimeAtPreviousAddress' you can obtain this without changing the generator, but instead by wrapping it in a modified generator:

def pairs_from_root(element):
    for k, v in flatten_dict(element):
        kk = k.rsplit('.', 1)[-1]
        yield kk, v

This works very well for me: first a file parsexml2.py

# parsexml2.py

import xml.etree.cElementTree as ElementTree 
import csv


def flatten_list(aList, prefix=''): 
    for element in aList:
        if element: 
            # treat like dict 
            if len(element) == 1 or element[0].tag != element[1].tag: 
                yield from flatten_dict(element, prefix)
            # treat like list 
            elif element[0].tag == element[1].tag: 
                yield from flatten_list(element, prefix)
        elif element.text: 
            text = element.text.strip() 
            if text: 
                yield prefix.rstrip('.'), text


def flatten_dict(parent_element, prefix=''):
    prefix = prefix + parent_element.tag + '.'
    if parent_element.items():
        for k, v in parent_element.items():
            yield prefix + k, v
    for element in parent_element:
        eprefix = prefix + element.tag + '.'
        if element:
            # treat like dict - we assume that if the first two tags 
            # in a series are different, then they are all different. 
            if len(element) == 1 or element[0].tag != element[1].tag: 
                yield from flatten_dict(element, prefix=prefix)
            # treat like list - we assume that if the first two tags 
            # in a series are the same, then the rest are the same. 
            else: 
                # here, we put the list in dictionary; the key is the 
                # tag name the list elements all share in common, and 
                # the value is the list itself
                yield from flatten_list(element, prefix=eprefix+element[0].tag+'.')
            # if the tag has attributes, add those to the dict
            if element.items():
                for k, v in element.items():
                    yield eprefix+k, v 
        # this assumes that if you've got an attribute in a tag, 
        # you won't be having any text. This may or may not be a 
        # good idea -- time …

Why not updating the code by yourself ? Here is the solution

def makerows(pairs):
    headers = []
    columns = {}
    for k, v in pairs:
        if k in columns:
            columns[k].extend((v,))
        else:
            headers.append(k)
            columns[k] = [k, v]
    m = max(len(c) for c in columns.values())
    for c in columns.values():
        c.extend('' for i in range(len(c), m))
    L = [columns[k] for k in headers]
    rows = list(zip(*L))
    return rows

Great ! Then mark the thread as solved !

Try what you can. It should be easy to write the csv.

This should work

def makerows(pairs):
    headers = []
    columns = {}
    for k, v in pairs:
        if k in columns:
            columns[k].extend(('', k, v))
        else:
            headers.append(k)
            columns[k] = [k, v]
    m = max(len(c) for c in columns.values())
    for c in columns.values():
        c.extend('' for i in range(len(c), m))
    L = [columns[k] for k in headers]
    rows = list(zip(*L))
    return rows

if __name__ == '__main__':
    XX = [
        [('A', 1), ('B', 2), ('C', 3), ('D', 4),('A', 5), ('B', 6), ('C', 7), ('D', 8)],
        [('A', 1), ('B', 2), ('C', 3), ('D', 4),('A', 5), ('B', 6), ('D', 8)],
        [('A', 1), ('B', 2), ('C', 3), ('D', 4),('D', 5), ('B', 6), ('A', 8)],
    ]
    from pprint import pprint
    for data in XX:
        print(data)
        pprint(makerows(data))

""" my output -->
[('A', 1), ('B', 2), ('C', 3), ('D', 4), ('A', 5), ('B', 6), ('C', 7), ('D', 8)]
[('A', 'B', 'C', 'D'),
 (1, 2, 3, 4),
 ('', '', '', ''),
 ('A', 'B', 'C', 'D'),
 (5, 6, 7, 8)]
[('A', 1), ('B', 2), ('C', 3), ('D', 4), ('A', 5), ('B', 6), ('D', 8)]
[('A', 'B', 'C', 'D'),
 (1, 2, 3, 4),
 ('', '', '', ''),
 ('A', 'B', '', 'D'),
 (5, 6, '', 8)]
[('A', 1), ('B', 2), ('C', 3), ('D', 4), ('D', 5), ('B', 6), ('A', 8)]
[('A', 'B', 'C', 'D'),
 (1, 2, 3, 4),
 ('', '', '', ''),
 ('A', 'B', '', 'D'),
 (8, 6, '', 5)]
 """

Now you work yourself, all this is getting more and more sophisticated.

And if we have

L = [('A', 1), ('B', 2), ('C', 3), ('D', 4),('A', 5), ('B', 6), ('D', 8)]

you want

A B C D
1 2 3 4

A B   D
5 6   8

?

What about

L = [('A', 1), ('B', 2), ('C', 3), ('D', 4),('D', 5), ('B', 6), ('A', 8)]

?

So you mean the headers A B C D, then a first row of values 1 2 3 4, then a blank row, then a row A B C D again and finally a new row with a set of values 5 6 7 8 ?

Do you mean that you now want 2 csv files ?