Odd write() method behaviour

[Simplified]

Hi All

I'm getting very odd behaviour when using file.write(variable) - what happens is that if I print out the variable using the print function I get this: 4CE1FFC64101843801 ... which is what I expect. However, when I try to write this to a file (using the variable 'valid' to write to object 'stamps') I get this:

stamps.write(valid)

4CE1FFC6410184381 This string above is what is written into the 'stamps' file.

The strange thing is that when it writes it basically removes the penultimate 0 and I can't figure out why. It should be noted that 'valid' is already converted to a string, i.e.:

valid = str("4CE1FFC64101843801")

.

Any ideas anyone?

TIA, Simplified

[Simplified]

Nice to see that it has everyone else stumped! I thought I was losing it!

[Gribouillis]

It seems to be an impossible behaviour. To speak about something concrete, I ran the following code

# python 2 and 3
from __future__ import print_function

def main():
    valid = str("4CE1FFC64101843801")
    with open("stamps", "w") as stamps:
        stamps.write(valid)
        
    with open("stamps", "r") as stamps:
        check = stamps.read()
        
    print("valid: ", valid)
    print("check: ", check)
    assert(check == valid)

if __name__ == "__main__":
    main()
    
""" my output -->

valid:  4CE1FFC64101843801
check:  4CE1FFC64101843801
"""

Does it run on your system ?

[Simplified]

Thanks for taking a look into this :)

The variable that comes back is in fact returned from another function. This function (called snipvalue()) returns either a value (in this case
4CE1FFC64101843801) or False.

When I run the code (I have inserted the line print valid, to print the variable: valid), I get this on screen:

4CE1FFC64101843801

4CE1FFC6410187D004

4CE1FFC741018F2801

4CE1FFC7410192C004

4CE1FFC841019A1801

4CE1FFC841019D8804

4CE1FFC94101A50801

4CE1FFC94101A8A004

4CE1FFCA4101AFF801

4CE1FFCA4101B36804

... but I get this in the file:

4CE1FFC6410184381
4CE1FFC6410187D04
4CE1FFC741018F281
4CE1FFC7410192C04
4CE1FFC841019A181
4CE1FFC841019D884
4CE1FFC94101A5081
4CE1FFC94101A8A04
4CE1FFCA4101AFF81
4CE1FFCA4101B3684

You will notice the difference between the lines on screen and the lines in the file, they appear to remove the penultimate '0'.

Here's the code that I'm using:

try:
        infile_open = open(k12txt, 'r')
    except IOError:
        print '\nThe file "' + pcapfile + '" does not exist, exiting..\n'
        usage()
        exit(1)
    infile = infile_open.readlines()
    infile_open.close()
    
    stamp = open(stamps, "w")
    
    for line in infile:
        valid = snipvalue(line,size)
        if valid:
            
            # Test code: Debug
            print valid
            
            stamp.write(valid)
            
    stamp.close()

Any ideas?

Thanks again :)

[Gribouillis]

Thanks for taking a look into this :)

The variable that comes back is in fact returned from another function. This function (called snipvalue()) returns either a value (in this case
4CE1FFC64101843801) or False.

When I run the code (I have inserted the line print valid, to print the variable: valid), I get this on screen:

... but I get this in the file:

You will notice the difference between the lines on screen and the lines in the file, they appear to remove the penultimate '0'.

Here's the code that I'm using:

try:
        infile_open = open(k12txt, 'r')
    except IOError:
        print '\nThe file "' + pcapfile + '" does not exist, exiting..\n'
        usage()
        exit(1)
    infile = infile_open.readlines()
    infile_open.close()
    
    stamp = open(stamps, "w")
    
    for line in infile:
        valid = snipvalue(line,size)
        if valid:
            
            # Test code: Debug
            print valid
            
            stamp.write(valid)
            
    stamp.close()

Any ideas?

Thanks again :)

What happens if you print repr(valid) ?

[Simplified]

Hi Gribouillis

I get this output to screen:

'4CE1FFC64101843801\n'
'4CE1FFC6410187D004\n'
'4CE1FFC741018F2801\n'
'4CE1FFC7410192C004\n'
'4CE1FFC841019A1801\n'
'4CE1FFC841019D8804\n'
'4CE1FFC94101A50801\n'
'4CE1FFC94101A8A004\n'
'4CE1FFCA4101AFF801\n'
'4CE1FFCA4101B36804\n'

...and this to the file:

4CE1FFC6410184381
4CE1FFC6410187D04
4CE1FFC741018F281
4CE1FFC7410192C04
4CE1FFC841019A181
4CE1FFC841019D884
4CE1FFC94101A5081
4CE1FFC94101A8A04
4CE1FFCA4101AFF81
4CE1FFCA4101B3684

[Gribouillis]

Hi Gribouillis

I get this output to screen:

...and this to the file:

If you are on windows, did you try to use the modes 'rb' and 'wb' instead of 'r' and 'w' ?

[Simplified]

I've just tried running this again, using the 'wb' and 'rb' instead of 'w' and 'r'.

On windows I get this on screen:

'4CE1FFC64101843801\r\n'
'4CE1FFC6410187D004\r\n'
'4CE1FFC741018F2801\r\n'
'4CE1FFC7410192C004\r\n'
'4CE1FFC841019A1801\r\n'
'4CE1FFC841019D8804\r\n'
'4CE1FFC94101A50801\r\n'
'4CE1FFC94101A8A004\r\n'
'4CE1FFCA4101AFF801\r\n'
'4CE1FFCA4101B36804\r\n'

On windows I get this in the file:

4CE1FFC6410184381
4CE1FFC6410187D04
4CE1FFC741018F281
4CE1FFC7410192C04
4CE1FFC841019A181
4CE1FFC841019D884
4CE1FFC94101A5081
4CE1FFC94101A8A04
4CE1FFCA4101AFF81
4CE1FFCA4101B3684

I get this on Linux:

'4CE1FFC64101843801\r\n'
'4CE1FFC6410187D004\r\n'
'4CE1FFC741018F2801\r\n'
'4CE1FFC7410192C004\r\n'
'4CE1FFC841019A1801\r\n'
'4CE1FFC841019D8804\r\n'
'4CE1FFC94101A50801\r\n'
'4CE1FFC94101A8A004\r\n'
'4CE1FFCA4101AFF801\r\n'
'4CE1FFCA4101B36804\r\n'
simplified@svr-01:~/temp$ cat stamps.txt
4CE1FFC641018438
4CE1FFC6410187D0
4CE1FFC741018F28
4CE1FFC7410192C0
4CE1FFC841019A18
4CE1FFC841019D88
4CE1FFC94101A508
4CE1FFC94101A8A0
4CE1FFCA4101AFF8
4CE1FFCA4101B368
simplified@svr-01:~/temp$

Both results look the same...

[Simplified]

Actually they don't... I've just noticed that the data written to file in Linux doesn't have the '01' or '04' on the end (compared to the screen output) and that the windows version has '1' or '4'.

Sorry for the confusion...

[Gribouillis]

Sorry, it's a mystery. I can't reproduce the bug on my computer, so I can't help you much. I've never heard of such behaviour before :(

[Simplified]

Thank you anyway for taking the time to look into this all the same :)

I will post the solution if I find it! :D

[Simplified]

I've just discovered something strange, in that if I change the write() line to:

stamp.write(valid + "\n")

This actually shows the correct values, but puts a space in between (which means that other parts of the program fail). I don't know if this helps at all? Here's the output (windows, note the additional line in between):

4CE1FFC64101843801

4CE1FFC6410187D004

4CE1FFC741018F2801

4CE1FFC7410192C004

4CE1FFC841019A1801

4CE1FFC841019D8804

4CE1FFC94101A50801

4CE1FFC94101A8A004

4CE1FFCA4101AFF801

4CE1FFCA4101B36804

...and the same in true when run on Linux:

4CE1FFC64101843801

4CE1FFC6410187D004

4CE1FFC741018F2801

4CE1FFC7410192C004

4CE1FFC841019A1801

4CE1FFC841019D8804

4CE1FFC94101A50801

4CE1FFC94101A8A004

4CE1FFCA4101AFF801

4CE1FFCA4101B36804

[Gribouillis]

I've just discovered something strange, in that if I change the write() line to:

stamp.write(valid + "\n")

This actually shows the correct values, but puts a space in between (which means that other parts of the program fail). I don't know if this helps at all? Here's the output (windows, note the additional line in between):

...and the same in true when run on Linux:

Add assert statements to add more control

assert type(valid) is str
assert len(valid) == 18 # or 19 (with the '\n')

also try to print valid.strip(), then \n

[Simplified]

Would it help if I gave you access to the whole program? It's very strange as it doesn't really make any sense...

I'm running this on Python 2.7 and will try tweaking it to run on 3.1.2 and keep you posted.

[Simplified]

Add assert statements to add more control

assert type(valid) is str
assert len(valid) == 18 # or 19 (with the '\n')

also try to print valid.strip(), then \n

Thanks - I'll give this a go and keep you posted :)

[Simplified]

Thanks - I'll give this a go and keep you posted :)

Just tried the following code:

for line in infile:
        valid = snipvalue(line,size)
        if valid:
            
            # Test code: Debug
            print valid
            assert type(valid) is str
            assert len(valid) == 19
            
            stamp.write(valid)
            
    stamp.close()

This unfortunately didn't work either. I've tried playing with the .strip() method but nothing I do works (i.e. I get syntax errors) - could you possibly tweak the above?

TIA, Simplified.

[Gribouillis]

Would it help if I gave you access to the whole program? It's very strange as it doesn't really make any sense...

I'm running this on Python 2.7 and will try tweaking it to run on 3.1.2 and keep you posted.

Yes if you can attach a zip with the whole program, I could try it and see if I have the same bug.

[Simplified]

Hi Gribouillis

I'm sorry to say but after closer inspection it appears that this was being changed in the written file by another function - I am so sorry to have wasted your time.

Many thanks for the other tips though.

Cheers, Simplified.