lol don't take any offence to the title XD
just being funny :P

anyways...
I'm having a problem with my function...

it's supposed to allow you to read/write float values in any specified byte length.
but there's a problem with the output value >_>

first off though I must state that the equation for processing the exponent field works perfectly for the pre-defined float types:
[ S, E, M ]
[ 1, 3, 4 ] - 8bit
[ 1, 5, 10 ] - 16bit
[ 1, 8, 23 ] - 32bit
[ 1, 11, 52 ] - 64bit

going along with these standards, IDK if the odd types such as 24bit or 48bit are correct.

anyways...
on with the code :P

def f(byte_cnt,val):
    e=int(((byte_cnt*8)-1)/(byte_cnt+1)+(byte_cnt/2 if byte_cnt>2 else 0))

    S,E,M=(val>>((byte_cnt*8)-1))&1,(val>>((byte_cnt*8)-(e+1)))&int('1'*e,2),val&int('1'*((byte_cnt*8)-(e+1)),2)

    print str(byte_cnt*8)+'bits [[1],['+str(e)+'],['+str(((byte_cnt*8)-1)-e)+']]: '+\
          str(S)+' '+('0'*(e-len(bin(E).replace('0b',''))))+bin(E).replace('0b','')+' '+\
          ('0'*(((byte_cnt*8)-(e+1))-len(bin(M).replace('0b',''))))+bin(M).replace('0b','')

    return (pow(-1,S)*(pow(2.0,(E-int('1'*(e-1),2)))*float(('1.' if E!=0 else '0.')+str(M))) if E<int('1'*e,2) else 'NAN') #problem on this line

usage:
f( byte_length, int(float_hex,16) )

IDK what I'm doing wrong, as I've followed the notes of about 8 sites D:
the returned binary values are correct, but the float is not...

anyone mind lending a hand?? :/
many thanx in return :)

Recommended Answers

All 44 Replies

We need test cases input/correct output to help you out, if I catch your meaning, mantissa has assumed 1 as first number except when exponent is 0. What is offset of exponent. I would rewrite your nice formulas by writing one function to pop n bits out from least significant side and working only with numbers, not strings.

def push_bits(n, bits, value):
    return value << n | bits

def pop_bits(n, value):
    return value >> n, value & ((1<<n) -1)

value = 0b11
value = push_bits(3, 0b101, value)
print bin(value)

bits, rest = pop_bits(3, value)
print bin(bits), bin(rest)

yeh... I've tried to keep string usage to a minimum >_<
just couldn't find an easy way to do float('1.'+str(mantissa)) w/o string usage

I'm afraid I'm not quite sure of your meaning to the offset of the exponent :/

e = length of exponent field
shift = (byte_size*8) - (1+e)

E = (val>>shift) & int('1'*e,2)

I already know the ints returned from the input value are correct,
as the binary values are regenerated from the ints (S, E, and M).

NTM I've tested a few examples and compaired my binary values with the example's representations.

here's a few of my tests with the comparison floats:
(what the returned values should be)

f(1,69) = 2.625
f(1,211) = -4.75
f(2,49152) = -2.0
f(2,15361) = 1.0009765625
f(4,1065353216) = 1.0
f(4,3299092992) = -1313.3125

EDIT: only -2.0 and 1.0 are correct

if it helps you out, these were the best notes I could find as to floating-point notation and encoding.
http://en.wikipedia.org/wiki/Half-precision_floating-point_format

MAN do I hate this editor >:O
(I have a connection that cuts every 5 seconds after connecting)
^this editor sometimes misses a paragraph or 2 >_<

I have a starting point using module bitstring (from pypi) and the wikipedia page

import bitstring

test_data = [ 
    ("0 01111 0000000000", 1),
    ("0 01111 0000000001", 1.0009765625),
    ("1 10000 0000000000", -2),
    ("0 11110 1111111111", 65504),
    ("0 00001 0000000000", 2.0 ** (-14)),
    ("0 00000 1111111111", 2.0**(-14) - 2.0**(-24)),
    ("0 00000 0000000001", 2.0**(-24)),
    ("0 00000 0000000000", 0.0),
    ("1 00000 0000000000", -0.0),
    ("0 11111 0000000000", float("infinity")),
    ("1 11111 0000000000", -float("infinity")),
    ("0 01101 0101010101", 1.0/3),
]

fmt = ['uint:1', 'uint:5', 'uint:10']

for u, res in test_data:
    a, b, c = (int(x, 2) for x in u.split())
    s = bitstring.pack(fmt, a, b, c)
    print s.bin
    s, e, m = s.unpack(fmt)
    if e:
        v = 2 ** (e - 25) * (1024 + m)
    else:
        v = 2 ** (-24) * m
    if s:
        v = -v
    print repr(v), repr(float(res))

"""my output -->
0011110000000000
1.0 1.0
0011110000000001
1.0009765625 1.0009765625
1100000000000000
-2.0 -2.0
0111101111111111
65504 65504.0
0000010000000000
6.103515625e-05 6.103515625e-05
0000001111111111
6.097555160522461e-05 6.097555160522461e-05
0000000000000001
5.960464477539063e-08 5.960464477539063e-08
0000000000000000
0.0 0.0
1000000000000000
-0.0 -0.0
0111110000000000
65536 inf
1111110000000000
-65536 -inf
0011010101010101
0.333251953125 0.3333333333333333
"""

Here Gribouillis data and formula for decision of value used with my push_bits and pop_bits if you need to avoid external dependencies. I also left as formulas some Gribouillis constants for easier adaption later for other lengths

def push_bits(n, bits, value):
    return value << n | bits

def pop_bits(n, value):
    return value >> n, value & ((1<<n) -1)

def b(n):
    return ('0' * 16 + bin(n)[2:])[-16:]

def f(val):
    bias, exp, mant = 15, 5, 10

    v, m = pop_bits(mant, val)
    v, e = pop_bits(exp, v)
    v, s = pop_bits(1, v)

    if e == (1<<exp)-1:
        if m:
            return float('NaN')
        else:
            return float('-inf') if s else float('+inf')

    if e:
        v = 2 ** (e - bias - mant) * ((1<<mant) + m)
    else:
        v = 2 ** (1 - bias - mant) * m

    return -v if s else v

for t in [15360, 15361, 49152, 31743, 1024, 1023, 1, 0, 32768, 31744, 64512, 13653]:
    print b(t), repr(f(t))

ahah!
I think I just got what I needed:

    if e == (1<<exp)-1:
        if m:
            return float('NaN')
        else:
            return float('-inf') if s else float('+inf')

    if e:
        v = 2 ** (e - bias - mant) * ((1<<mant) + m)
    else:
        v = 2 ** (1 - bias - mant) * m

    return -v if s else v

but hang on, lemme verify first...

EDIT:
btw, I believe 'pow(-1,s)*v' is faster than '-v if s else v'

pyTony =D
you've done it again ^_^

>>> f(1,69)
8bits [[1],[3],[4]]: 0 100 0101
2.625
>>> f(1,211)
8bits [[1],[3],[4]]: 1 101 0011
-4.75
>>> f(2,49152)
16bits [[1],[5],[10]]: 1 10000 0000000000
-2.0
>>> f(2,15361)
16bits [[1],[5],[10]]: 0 01111 0000000001
1.0009765625
>>> f(4,1065353216)
32bits [[1],[8],[23]]: 0 01111111 00000000000000000000000
1.0
>>> f(4,3299092992)
32bits [[1],[8],[23]]: 1 10001001 01001000010101000000000
-1313.3125
>>> 

thank you :)
that was all I needed.

now I just need to get floats back into 3 ints

You might find it interesting to look for the hex representation of the float by it's hex() method. Or then it could be just confusing.

I have :P
I'd rather use reverse engineering instead of trying to work with that mess >_<

I once wrote this function to get the ieee754 representation of a float

def ieee754(x):
    """Return a string of 0 and 1 giving the ieee754 representation of the float x
    """
    import struct
    from binascii import hexlify
    p = struct.pack("d", x)
    s = bin(int(b"1" + hexlify(p), 16))[3:]
    return " ".join(reversed([s[i:i+8] for i in xrange(0, len(s), 8)]))

The following test shows that the 16 bits version is very close to the 64 bits version. It means that you should be able to find the 3 numbers easily

0011110000000000
00111111 11110000 00000000 00000000 00000000 00000000 00000000 00000000
1.0 1.0
0011110000000001
00111111 11110000 00000100 00000000 00000000 00000000 00000000 00000000
1.0009765625 1.0009765625
1100000000000000
11000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
-2.0 -2.0
0111101111111111
01000000 11101111 11111100 00000000 00000000 00000000 00000000 00000000
65504 65504.0
0000010000000000
00111111 00010000 00000000 00000000 00000000 00000000 00000000 00000000
6.103515625e-05 6.103515625e-05
0000001111111111
00111111 00001111 11111000 00000000 00000000 00000000 00000000 00000000
6.097555160522461e-05 6.097555160522461e-05
0000000000000001
00111110 01110000 00000000 00000000 00000000 00000000 00000000 00000000
5.960464477539063e-08 5.960464477539063e-08
0000000000000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
0.0 0.0
1000000000000000
10000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
-0.0 -0.0
0111110000000000
01000000 11110000 00000000 00000000 00000000 00000000 00000000 00000000
65536 inf
1111110000000000
11000000 11110000 00000000 00000000 00000000 00000000 00000000 00000000
-65536 -inf
0011010101010101
00111111 11010101 01010100 00000000 00000000 00000000 00000000 00000000
0.333251953125 0.3333333333333333

Apparently, removing bits [2, 8[ and truncating brings back the 16 bits float.

Happy log(n, 2)ing then!

can it write 24bit floats?? :P

that's what I'm trying to do...
write floats of the specified byte length

Here is something of "More than you ever wanted to know about floating point numbers" http://www.quadibloc.com/comp/cp02.htm Not helping so much at task at hand but looks really comprehensive computer science basics document for interest and fun.

To post a workin version of multiprecission code for benefit of others, here my function with bits argument instead of bytes (to avoid those 8*byte_cnt parts) Hope it is without bugs, looks to work with the test cases. (result for f(0x7ffeffffffffffffffffffffffffffff, 128) is correct but not maybe what you would expect)

def push_bits(n, bits, value):
    return long(value << n) | long(bits)

def pop_bits(n, value):
    return value >> n, value & ((1<<n) -1)

def b(n, c=16):
    if n < 0:
        # two's complement
        n = ((2**c-1) ^ n) + 1
    return ('0' * c + bin(n)[2:])[-c:]

def f(val, bits=16):
    if bits == 80:
        exp = 15
    else:
        byte_cnt = bits // 8
        exp = (byte_cnt * 8 - 1)//(byte_cnt + 1) + (byte_cnt > 2) * byte_cnt // 2
    mant = bits - 1 - exp
    bias = (1<<(exp-1))-1
    print exp, mant, bias

    v, m = pop_bits(mant, val)
    v, e = pop_bits(exp, v)
    v, s = pop_bits(1, v)

    if e == bias:
        if m:
            return float('NaN')
        else:
            return float('-inf') if s else float('+inf')

    if e:
        v = 2 ** (e - bias - mant) * ((1<<mant) + m)
    else:
        v = 2 ** (1 - bias - mant) * m

    return -v if s else v

for t in [15360, 15361, 49152, 31743, 1024, 1023, 1, 0, 32768, 31744, 64512, 13653]:
    print b(t), repr(f(t))

how does yor code calculate the length of the exponent??

you saw my code, but does yours return the same bit-fields as mine??
http://lh3.ggpht.com/-modh5yIANmA/T6CDQ-67LHI/AAAAAAAADyQ/snPDH-mtSZM/s812/floats.PNG
^just an earlier test I did while developing the equasion :P

but then again, IDK if the format standards for the bit fields may change compaired to mine >_>
(since IEEE 24bit hasn't been dev'd yet)

Your formula, only simplified the integer division and tried to deal with any length not only fixed number of bytes, exactly your results come better with formula:

byte_cnt = bits // 8
exp = ((byte_cnt * 8) - 1)//(byte_cnt + 1)+ (byte_cnt > 2) * byte_cnt // 2

I fixed it in my post.

btw, I believe 'pow(-1,s)*v' is faster than '-v if s else v'

Interesting, but I can not reproduce this result, maybe you could send your data?

import random
import time

ints = [random.randint(0, 1<<62) for count in range(10**5)]
signs = [random.randint(0, 1) for count in range(10**5)]

print('starting test run')
t0= time.clock()
signed1 = [(-1)**s*v for s, v in zip(signs, ints)]
t1= time.clock()
signed2 = [-v if s else v for s, v in zip(signs, ints)]
t2 = time.clock()
print 'if', (t2-t1),'vs', t1-t0, '**'
print ((t2- t1) - (t1- t0))/(t2-t1) * 100, '% speedup'

Output

starting test run
if 0.094547212006 vs 0.183539248703 **
-94.1244430262 % speedup

80 bits has exponent of 15 bits and looks like it has one additional integer part bit in format and is not handled perfectly by the function (http://en.wikipedia.org/wiki/Extended_precision)

interesting find indeed :D
it's not IEEE format though...
but I'll save that for future float format specifications which I do intend to implement.
(all I need is an extra input specifying the format to use)

I might just change my current input from 'signed' to 'format'

current settings:
0 - unsigned
1 - signed
2 - float (IEEE)

and I see the 128bit format has an exponent of len(15)
I'm going to test that now... :P

d for my code...
would you like just the float processing??
or would you like the full function??
^(read/write ints/floats of any specified byte length)

also, why I handle the data by the byte instead of the bit is to leave less room for error.
(I didn't know the difference back when I worked on MAV, and have made that exact mistake) :P
^(specifying 4 bits when needing 4 bytes does not compute)

if you want 4 bits, you'll have to use a bit-field:
data1,data2 = BF( [4,4], bu8() )

note: ^that's how my COMMON module is supposed to be used.

I have a small problem with BF though, and am working on fixing it :P
(it doesn't like being used in loops and returns an errror if so)

I tested 128bit, and it worked perfectly :D
(meaning there may possibly be no change to the IEEE formula)

also, your edit to my formula, I see what you did there. <_<
you've basically just done what I did w/o using if and else.
nice work there =D

with the way this function is used, even 1 small speed-up matters by alot.

hmm...
how would I go about pre-computing it??

and I'm afraid I didn't catch your Q... :P

Something like this (sorry about mixing up the pow and exp function names, I fixed my question):

exponents = [15 if byte_cnt == 10 else (byte_cnt * 8 - 1)//(byte_cnt + 1) + (byte_cnt > 2) * byte_cnt // 2
                for byte_cnt in range(1,128 // 8 + 1)]
fields = [(e, n*8 - 1 - e, (1<<(e-1))-1) for n, e in enumerate(exponents, 1)]

def push_bits(n, bits, value):
    return long(value << n) | long(bits)

def pop_bits(n, value):
    return value >> n, value & ((1<<n) -1)

def gen_bits(value, n=1):
    while value != 0:
        value, bits = pop_bits(n, value)
        yield bits

def differing_bits(a,b):
    from itertools import izip_longest
    return sum(a!=b for a,b in izip_longest(gen_bits(a), gen_bits(b)))

def b(n, c=16):
    if n < 0:
        # two's complement
        n = ((2**c-1) ^ n) + 1
    return ('0' * c + bin(n)[2:])[-c:]

def f(val, byte_cnt=2):
    if not (0 < byte_cnt <= len(fields)):
        raise ValueError('Invalid byte count for float')
    exp, mant, bias = fields[byte_cnt-1]
    #print(fields[byte_cnt])

    v, m = pop_bits(mant, val)
    v, e = pop_bits(exp, v)
    v, s = pop_bits(1, v)

    if e == bias:
        if m:
            return float('NaN')
        else:
            return float('-inf') if s else float('+inf')

    if e:
        v = 2 ** (e - bias - mant) * ((1<<mant) + m)
    else:
        v = 2 ** (1 - bias - mant) * m

    return -v if s else v

for t in [15360, 15361, 49152, 31743, 1024, 1023, 1, 0, 32768, 31744, 64512, 13653]:
    print b(t), repr(f(t))

I wrote a nice class to convert between various ieee754 formats

#!/usr/bin/env python
# -*-coding: utf8-*-
# Title: anyfloat.py
# Author: Gribouillis for the python forum at www.daniweb.com
# Created: 2012-05-02 06:46:42.708131 (isoformat date)
# License: Public Domain
# Use this code freely.

from collections import namedtuple
from math import isnan
import struct
import sys

if sys.version_info < (2, 7):
    raise ImportError("Module anyfloat requires python 2.7 or newer.")


class anyfloat(namedtuple("anyfloat", "sign log2 mantissa")):
    """A class storing real numbers independently from the ieee754 format.

    This class stores a real number as a triple of integers (sign, log2, mantissa)
    where sign is alway 0 or 1 and:
        a) mantissa == -2 is used to represent NaN values. In this case, sign == 0 and log2 == 0.
           There is only one NaN value in this representation.
        b) mantissa == -1 is used to represent +Infinity if sign == 0 and -Infinity if sign == 1.
           In this case, log2 == 0
        c) mantissa == 0 is used to represent 0.0 if sign == 0 and -0.0 if sign == 1. In this
           case, log2 == 0
        d) mantissa > 0 is used to represent any other real number with a finite number of binary
           digits. The real number x corresponding to the anyfloat instance is mathematically
                x = +/- pow(2, log2) * y
           where y is the number in [1, 2[ which binary digits are the binary digits of the mantissa.
           For example the real number corresponding to anyfloat(1, 5, 39) is
                x = - 32 * y
           where y is the real number with binary representation 1.00111, because the binary
           representation of 39 is 100111 and 2**5 = 32 and sign == 1, which means a negative number.

    Arithmetic operations are not implemented for this class, which is meant as a temporary
    class to convert from one ieee754 format to another (for example 64 bits to 32 bits).
    """
    __slots__ = ()
    _b32 = 1 << 32
    _b64 = 1 << 64

    def __float__(self):
        return self.int64_to_float(self.to_ieee())

    @classmethod
    def from_float(cls, x):
        """Create an anyfloat instance from a python float (64 bits double precision number)."""
        u, v = struct.unpack(">LL", struct.pack(">d", x))
        return cls.from_ieee((u << 32) | v)

    @classmethod
    def from_ieee(cls, n, size = (11, 52)):
        """Create an anyfloat from an ieee754 integer.

        Create an anyfloat from an integer which binary representation is the ieee754
        format of a floating point number. The argument 'size' is a tuple (ewidth, mwidth)
        containing the width of the exponent part and the width of the mantissa part in
        this ieee754 format. For example for double precision usual numbers the size
        is (11, 52) (the default), for single precision floating point numbers,
        the size is (8, 23)."""

        _8, _23 = size # local variables are named according to the 32 bits float format.
        _127 = (1 << (_8 - 1)) - 1
        _255 = (_127 << 1) | 1
        r = n >> _23
        m = (r << _23) ^ n
        s = int(r >> _8)
        if not s in (0, 1):
            raise ValueError(("Integer value out of range for ieee754 format", n, size))
        e = (s << _8) ^ r
        if e == _255: # inf and NaN
            args = (0, 0, -2) if m else (s, 0, -1)
        else:
            m, e = ((1 << _23) | m, e -_127) if e else (m, m.bit_length() - _23 - _127)
            bl = (m & -m).bit_length()
            args = (s, int(e), int(m >> (bl - 1))) if bl else (s, 0, 0)
        return cls(*args)

    def to_ieee(self, size = (11, 52)):
        """Convert to an ieee754 integer.

        Convert self to an integer which binary representation is the ieee754 format corresponding
        to the 'size' argument (read the documentation of from_ieee() for the meaning of the size
        argument. Size defaults to (11, 52).
        """
        _8, _23 = size # local variables are named according to the 32 bits float format.
        _127 = (1 << (_8 - 1)) - 1
        _255 = (_127 << 1) | 1

        res = (1 << _8) if self.sign else 0
        m = 0
        if self.mantissa > 0: # general case
            e = self.log2 + _127
            if e > -_23: # otherwise too small (return 0)
                if e >= _255: # too large, return +/-Inf
                    res |= _255
                else:
                    m = self.mantissa
                    c = m.bit_length() - 1
                    lshift = _23 - c
                    if e <= 0: # subnormal number
                        lshift += (e - 1)
                    else: # normal number
                        res |= e
                        m ^= (1 << c)
                    if lshift > 0:
                        m <<= lshift
                    else:
                        m >>= (-lshift)
        elif self.mantissa < 0: # +/-Inf or NaN
            res |= _255
            if self.mantissa == -2: # NaN case
                m = (1 << (_23 - 1)) # create non signaling nan
        return (res << _23) | m

    @classmethod
    def int64_to_float(cls, n):
        """Convert a 64 bits integer to a python float.

        This class method converts an integer representing a 64 bits floating point
        number in the ieee754 double precision format to this floating point number."""

        if not (0 <= n < cls._b64):
            raise ValueError(("Integer value out of range for 64 bits ieee754 format", n))
        u, v = divmod(n, cls._b32)
        return struct.unpack(">d", struct.pack(">LL", u, v))[0]

    def bin(self, size = (11, 52)):
        """Return a binary representation of self.

        The returned string contains only the characters '0' and '1' and shows the
        ieee754 representation of the real number corresponding to self whith the given
        size = (ewidth, mwidth).
        """
        return "{val:0>{width}}".format(val = bin(self.to_ieee(size))[2:], width = sum(size) + 1)


if __name__ == '__main__':
    for x in (1.0, 3.14, 5.0, -34.7, 0.0, -0.0, float("inf"), -float("inf"), float("nan"), 2.0 **(-1030)):
        af = anyfloat.from_float(x)
        print af, x, (x == float(af))
        print af.bin((11, 52))
        print af.bin((8, 23))
        print af.to_ieee((8, 23))

    print "####"
    s1 = (3, 4)
    s2 = (5, 10)
    s4 = (8, 23)
    for size, n in [
        (s1, 69), (s1, 211), (s2, 49152), (s2, 15361), (s4, 1065353216), (s4, 3299092992)
        ]:
        print float(anyfloat.from_ieee(n, size))

If you think it works well, I'll put it in the code snippets.

I fixed one typo in your comments as moderator, maybe you got PM from that, Gribouillis. You have the main test written to Python2, easy to fix by changing print to function and importing from __future__.

Unfortunately Python3 also wants to parse the section of main routine and does not import your function, so for Python3 you need to remove the main part or change the print statements even to only import the code from other modules :(

Did not change that here as tcll seems to stay in Python2 environment, but as code snippet it is good to use function style print.

yea... I now prefer 2.7 over 2.6,
but I heard 3x is a headache and still kinda buggy.

anyways...
sry I havn't been on... IDK WTF's happening over at my end...
either my w/less card is being dumb, or the person I'm connected to didn't pay their bill. >_>
either way, I'm w/o internet on my compy.
(I'm on a friend's laptop atm)

my card says I'm connected with 4 bars, but won't load anything but about 1 packet after 10 minutes.

anyways, I'm looking into the functions right now.

EDIT: lol, yep, I still can barely understand classes XDD
btw, my program returns more precise floats than using the struct module.
(the numbers extend farther)

I tried converting Pichu from Melee and got better results from the bone matrix values. =D

What do you mean when you say that you get more precise floats than using the struct module ? Can you give a code example ?

for Python3 you need to remove the main part or change the print statements

I posted a new version as a code snippet . It includes a bugfix with regard to rounding the last bit.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.