lol don't take any offence to the title XD
just being funny :P

anyways...
I'm having a problem with my function...

it's supposed to allow you to read/write float values in any specified byte length.
but there's a problem with the output value >_>

first off though I must state that the equation for processing the exponent field works perfectly for the pre-defined float types:
[ S, E, M ]
[ 1, 3, 4 ] - 8bit
[ 1, 5, 10 ] - 16bit
[ 1, 8, 23 ] - 32bit
[ 1, 11, 52 ] - 64bit

going along with these standards, IDK if the odd types such as 24bit or 48bit are correct.

anyways...
on with the code :P

def f(byte_cnt,val):
    e=int(((byte_cnt*8)-1)/(byte_cnt+1)+(byte_cnt/2 if byte_cnt>2 else 0))

    S,E,M=(val>>((byte_cnt*8)-1))&1,(val>>((byte_cnt*8)-(e+1)))&int('1'*e,2),val&int('1'*((byte_cnt*8)-(e+1)),2)

    print str(byte_cnt*8)+'bits [[1],['+str(e)+'],['+str(((byte_cnt*8)-1)-e)+']]: '+\
          str(S)+' '+('0'*(e-len(bin(E).replace('0b',''))))+bin(E).replace('0b','')+' '+\
          ('0'*(((byte_cnt*8)-(e+1))-len(bin(M).replace('0b',''))))+bin(M).replace('0b','')

    return (pow(-1,S)*(pow(2.0,(E-int('1'*(e-1),2)))*float(('1.' if E!=0 else '0.')+str(M))) if E<int('1'*e,2) else 'NAN') #problem on this line

usage:
f( byte_length, int(float_hex,16) )

IDK what I'm doing wrong, as I've followed the notes of about 8 sites D:
the returned binary values are correct, but the float is not...

anyone mind lending a hand?? :/
many thanx in return :)

We need test cases input/correct output to help you out, if I catch your meaning, mantissa has assumed 1 as first number except when exponent is 0. What is offset of exponent. I would rewrite your nice formulas by writing one function to pop n bits out from least significant side and working only with numbers, not strings.

def push_bits(n, bits, value):
    return value << n | bits

def pop_bits(n, value):
    return value >> n, value & ((1<<n) -1)

value = 0b11
value = push_bits(3, 0b101, value)
print bin(value)

bits, rest = pop_bits(3, value)
print bin(bits), bin(rest)

Edited 4 Years Ago by pyTony

yeh... I've tried to keep string usage to a minimum >_<
just couldn't find an easy way to do float('1.'+str(mantissa)) w/o string usage

I'm afraid I'm not quite sure of your meaning to the offset of the exponent :/

e = length of exponent field
shift = (byte_size*8) - (1+e)

E = (val>>shift) & int('1'*e,2)

I already know the ints returned from the input value are correct,
as the binary values are regenerated from the ints (S, E, and M).

NTM I've tested a few examples and compaired my binary values with the example's representations.

here's a few of my tests with the comparison floats:
(what the returned values should be)

f(1,69) = 2.625

f(1,211) = -4.75

f(2,49152) = -2.0

f(2,15361) = 1.0009765625

f(4,1065353216) = 1.0

f(4,3299092992) = -1313.3125

EDIT: only -2.0 and 1.0 are correct

Edited 4 Years Ago by Tcll

I have a starting point using module bitstring (from pypi) and the wikipedia page

import bitstring

test_data = [ 
    ("0 01111 0000000000", 1),
    ("0 01111 0000000001", 1.0009765625),
    ("1 10000 0000000000", -2),
    ("0 11110 1111111111", 65504),
    ("0 00001 0000000000", 2.0 ** (-14)),
    ("0 00000 1111111111", 2.0**(-14) - 2.0**(-24)),
    ("0 00000 0000000001", 2.0**(-24)),
    ("0 00000 0000000000", 0.0),
    ("1 00000 0000000000", -0.0),
    ("0 11111 0000000000", float("infinity")),
    ("1 11111 0000000000", -float("infinity")),
    ("0 01101 0101010101", 1.0/3),
]

fmt = ['uint:1', 'uint:5', 'uint:10']

for u, res in test_data:
    a, b, c = (int(x, 2) for x in u.split())
    s = bitstring.pack(fmt, a, b, c)
    print s.bin
    s, e, m = s.unpack(fmt)
    if e:
        v = 2 ** (e - 25) * (1024 + m)
    else:
        v = 2 ** (-24) * m
    if s:
        v = -v
    print repr(v), repr(float(res))

"""my output -->
0011110000000000
1.0 1.0
0011110000000001
1.0009765625 1.0009765625
1100000000000000
-2.0 -2.0
0111101111111111
65504 65504.0
0000010000000000
6.103515625e-05 6.103515625e-05
0000001111111111
6.097555160522461e-05 6.097555160522461e-05
0000000000000001
5.960464477539063e-08 5.960464477539063e-08
0000000000000000
0.0 0.0
1000000000000000
-0.0 -0.0
0111110000000000
65536 inf
1111110000000000
-65536 -inf
0011010101010101
0.333251953125 0.3333333333333333
"""

Here Gribouillis data and formula for decision of value used with my push_bits and pop_bits if you need to avoid external dependencies. I also left as formulas some Gribouillis constants for easier adaption later for other lengths

def push_bits(n, bits, value):
    return value << n | bits

def pop_bits(n, value):
    return value >> n, value & ((1<<n) -1)

def b(n):
    return ('0' * 16 + bin(n)[2:])[-16:]

def f(val):
    bias, exp, mant = 15, 5, 10

    v, m = pop_bits(mant, val)
    v, e = pop_bits(exp, v)
    v, s = pop_bits(1, v)

    if e == (1<<exp)-1:
        if m:
            return float('NaN')
        else:
            return float('-inf') if s else float('+inf')

    if e:
        v = 2 ** (e - bias - mant) * ((1<<mant) + m)
    else:
        v = 2 ** (1 - bias - mant) * m

    return -v if s else v

for t in [15360, 15361, 49152, 31743, 1024, 1023, 1, 0, 32768, 31744, 64512, 13653]:
    print b(t), repr(f(t))

Edited 4 Years Ago by pyTony

ahah!
I think I just got what I needed:

    if e == (1<<exp)-1:
        if m:
            return float('NaN')
        else:
            return float('-inf') if s else float('+inf')

    if e:
        v = 2 ** (e - bias - mant) * ((1<<mant) + m)
    else:
        v = 2 ** (1 - bias - mant) * m

    return -v if s else v

but hang on, lemme verify first...

EDIT:
btw, I believe 'pow(-1,s)*v' is faster than '-v if s else v'

Edited 4 Years Ago by Tcll

pyTony =D
you've done it again ^_^

>>> f(1,69)
8bits [[1],[3],[4]]: 0 100 0101
2.625
>>> f(1,211)
8bits [[1],[3],[4]]: 1 101 0011
-4.75
>>> f(2,49152)
16bits [[1],[5],[10]]: 1 10000 0000000000
-2.0
>>> f(2,15361)
16bits [[1],[5],[10]]: 0 01111 0000000001
1.0009765625
>>> f(4,1065353216)
32bits [[1],[8],[23]]: 0 01111111 00000000000000000000000
1.0
>>> f(4,3299092992)
32bits [[1],[8],[23]]: 1 10001001 01001000010101000000000
-1313.3125
>>> 

thank you :)
that was all I needed.

You might find it interesting to look for the hex representation of the float by it's hex() method. Or then it could be just confusing.

I have :P
I'd rather use reverse engineering instead of trying to work with that mess >_<

I once wrote this function to get the ieee754 representation of a float

def ieee754(x):
    """Return a string of 0 and 1 giving the ieee754 representation of the float x
    """
    import struct
    from binascii import hexlify
    p = struct.pack("d", x)
    s = bin(int(b"1" + hexlify(p), 16))[3:]
    return " ".join(reversed([s[i:i+8] for i in xrange(0, len(s), 8)]))

The following test shows that the 16 bits version is very close to the 64 bits version. It means that you should be able to find the 3 numbers easily

0011110000000000
00111111 11110000 00000000 00000000 00000000 00000000 00000000 00000000
1.0 1.0
0011110000000001
00111111 11110000 00000100 00000000 00000000 00000000 00000000 00000000
1.0009765625 1.0009765625
1100000000000000
11000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
-2.0 -2.0
0111101111111111
01000000 11101111 11111100 00000000 00000000 00000000 00000000 00000000
65504 65504.0
0000010000000000
00111111 00010000 00000000 00000000 00000000 00000000 00000000 00000000
6.103515625e-05 6.103515625e-05
0000001111111111
00111111 00001111 11111000 00000000 00000000 00000000 00000000 00000000
6.097555160522461e-05 6.097555160522461e-05
0000000000000001
00111110 01110000 00000000 00000000 00000000 00000000 00000000 00000000
5.960464477539063e-08 5.960464477539063e-08
0000000000000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
0.0 0.0
1000000000000000
10000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
-0.0 -0.0
0111110000000000
01000000 11110000 00000000 00000000 00000000 00000000 00000000 00000000
65536 inf
1111110000000000
11000000 11110000 00000000 00000000 00000000 00000000 00000000 00000000
-65536 -inf
0011010101010101
00111111 11010101 01010100 00000000 00000000 00000000 00000000 00000000
0.333251953125 0.3333333333333333

Apparently, removing bits [2, 8[ and truncating brings back the 16 bits float.

Edited 4 Years Ago by Gribouillis

can it write 24bit floats?? :P

that's what I'm trying to do...
write floats of the specified byte length

To post a workin version of multiprecission code for benefit of others, here my function with bits argument instead of bytes (to avoid those 8*byte_cnt parts) Hope it is without bugs, looks to work with the test cases. (result for f(0x7ffeffffffffffffffffffffffffffff, 128) is correct but not maybe what you would expect)

def push_bits(n, bits, value):
    return long(value << n) | long(bits)

def pop_bits(n, value):
    return value >> n, value & ((1<<n) -1)

def b(n, c=16):
    if n < 0:
        # two's complement
        n = ((2**c-1) ^ n) + 1
    return ('0' * c + bin(n)[2:])[-c:]

def f(val, bits=16):
    if bits == 80:
        exp = 15
    else:
        byte_cnt = bits // 8
        exp = (byte_cnt * 8 - 1)//(byte_cnt + 1) + (byte_cnt > 2) * byte_cnt // 2
    mant = bits - 1 - exp
    bias = (1<<(exp-1))-1
    print exp, mant, bias

    v, m = pop_bits(mant, val)
    v, e = pop_bits(exp, v)
    v, s = pop_bits(1, v)

    if e == bias:
        if m:
            return float('NaN')
        else:
            return float('-inf') if s else float('+inf')

    if e:
        v = 2 ** (e - bias - mant) * ((1<<mant) + m)
    else:
        v = 2 ** (1 - bias - mant) * m

    return -v if s else v

for t in [15360, 15361, 49152, 31743, 1024, 1023, 1, 0, 32768, 31744, 64512, 13653]:
    print b(t), repr(f(t))

Edited 4 Years Ago by pyTony: same formula result as tcll

Your formula, only simplified the integer division and tried to deal with any length not only fixed number of bytes, exactly your results come better with formula:

byte_cnt = bits // 8
exp = ((byte_cnt * 8) - 1)//(byte_cnt + 1)+ (byte_cnt > 2) * byte_cnt // 2

I fixed it in my post.

Edited 4 Years Ago by pyTony

btw, I believe 'pow(-1,s)*v' is faster than '-v if s else v'

Interesting, but I can not reproduce this result, maybe you could send your data?

import random
import time

ints = [random.randint(0, 1<<62) for count in range(10**5)]
signs = [random.randint(0, 1) for count in range(10**5)]

print('starting test run')
t0= time.clock()
signed1 = [(-1)**s*v for s, v in zip(signs, ints)]
t1= time.clock()
signed2 = [-v if s else v for s, v in zip(signs, ints)]
t2 = time.clock()
print 'if', (t2-t1),'vs', t1-t0, '**'
print ((t2- t1) - (t1- t0))/(t2-t1) * 100, '% speedup'

Output

starting test run
if 0.094547212006 vs 0.183539248703 **
-94.1244430262 % speedup

interesting find indeed :D
it's not IEEE format though...
but I'll save that for future float format specifications which I do intend to implement.
(all I need is an extra input specifying the format to use)

I might just change my current input from 'signed' to 'format'

current settings:
0 - unsigned
1 - signed
2 - float (IEEE)

and I see the 128bit format has an exponent of len(15)
I'm going to test that now... :P

d for my code...
would you like just the float processing??
or would you like the full function??
^(read/write ints/floats of any specified byte length)

also, why I handle the data by the byte instead of the bit is to leave less room for error.
(I didn't know the difference back when I worked on MAV, and have made that exact mistake) :P
^(specifying 4 bits when needing 4 bytes does not compute)

if you want 4 bits, you'll have to use a bit-field:
data1,data2 = BF( [4,4], bu8() )

note: ^that's how my COMMON module is supposed to be used.

I have a small problem with BF though, and am working on fixing it :P
(it doesn't like being used in loops and returns an errror if so)

I tested 128bit, and it worked perfectly :D
(meaning there may possibly be no change to the IEEE formula)

also, your edit to my formula, I see what you did there. <_<
you've basically just done what I did w/o using if and else.
nice work there =D

with the way this function is used, even 1 small speed-up matters by alot.

hmm...
how would I go about pre-computing it??

and I'm afraid I didn't catch your Q... :P

Something like this (sorry about mixing up the pow and exp function names, I fixed my question):

exponents = [15 if byte_cnt == 10 else (byte_cnt * 8 - 1)//(byte_cnt + 1) + (byte_cnt > 2) * byte_cnt // 2
                for byte_cnt in range(1,128 // 8 + 1)]
fields = [(e, n*8 - 1 - e, (1<<(e-1))-1) for n, e in enumerate(exponents, 1)]

def push_bits(n, bits, value):
    return long(value << n) | long(bits)

def pop_bits(n, value):
    return value >> n, value & ((1<<n) -1)

def gen_bits(value, n=1):
    while value != 0:
        value, bits = pop_bits(n, value)
        yield bits

def differing_bits(a,b):
    from itertools import izip_longest
    return sum(a!=b for a,b in izip_longest(gen_bits(a), gen_bits(b)))

def b(n, c=16):
    if n < 0:
        # two's complement
        n = ((2**c-1) ^ n) + 1
    return ('0' * c + bin(n)[2:])[-c:]

def f(val, byte_cnt=2):
    if not (0 < byte_cnt <= len(fields)):
        raise ValueError('Invalid byte count for float')
    exp, mant, bias = fields[byte_cnt-1]
    #print(fields[byte_cnt])

    v, m = pop_bits(mant, val)
    v, e = pop_bits(exp, v)
    v, s = pop_bits(1, v)

    if e == bias:
        if m:
            return float('NaN')
        else:
            return float('-inf') if s else float('+inf')

    if e:
        v = 2 ** (e - bias - mant) * ((1<<mant) + m)
    else:
        v = 2 ** (1 - bias - mant) * m

    return -v if s else v

for t in [15360, 15361, 49152, 31743, 1024, 1023, 1, 0, 32768, 31744, 64512, 13653]:
    print b(t), repr(f(t))

Edited 4 Years Ago by pyTony

I wrote a nice class to convert between various ieee754 formats

#!/usr/bin/env python
# -*-coding: utf8-*-
# Title: anyfloat.py
# Author: Gribouillis for the python forum at www.daniweb.com
# Created: 2012-05-02 06:46:42.708131 (isoformat date)
# License: Public Domain
# Use this code freely.

from collections import namedtuple
from math import isnan
import struct
import sys

if sys.version_info < (2, 7):
    raise ImportError("Module anyfloat requires python 2.7 or newer.")


class anyfloat(namedtuple("anyfloat", "sign log2 mantissa")):
    """A class storing real numbers independently from the ieee754 format.

    This class stores a real number as a triple of integers (sign, log2, mantissa)
    where sign is alway 0 or 1 and:
        a) mantissa == -2 is used to represent NaN values. In this case, sign == 0 and log2 == 0.
           There is only one NaN value in this representation.
        b) mantissa == -1 is used to represent +Infinity if sign == 0 and -Infinity if sign == 1.
           In this case, log2 == 0
        c) mantissa == 0 is used to represent 0.0 if sign == 0 and -0.0 if sign == 1. In this
           case, log2 == 0
        d) mantissa > 0 is used to represent any other real number with a finite number of binary
           digits. The real number x corresponding to the anyfloat instance is mathematically
                x = +/- pow(2, log2) * y
           where y is the number in [1, 2[ which binary digits are the binary digits of the mantissa.
           For example the real number corresponding to anyfloat(1, 5, 39) is
                x = - 32 * y
           where y is the real number with binary representation 1.00111, because the binary
           representation of 39 is 100111 and 2**5 = 32 and sign == 1, which means a negative number.

    Arithmetic operations are not implemented for this class, which is meant as a temporary
    class to convert from one ieee754 format to another (for example 64 bits to 32 bits).
    """
    __slots__ = ()
    _b32 = 1 << 32
    _b64 = 1 << 64

    def __float__(self):
        return self.int64_to_float(self.to_ieee())

    @classmethod
    def from_float(cls, x):
        """Create an anyfloat instance from a python float (64 bits double precision number)."""
        u, v = struct.unpack(">LL", struct.pack(">d", x))
        return cls.from_ieee((u << 32) | v)

    @classmethod
    def from_ieee(cls, n, size = (11, 52)):
        """Create an anyfloat from an ieee754 integer.

        Create an anyfloat from an integer which binary representation is the ieee754
        format of a floating point number. The argument 'size' is a tuple (ewidth, mwidth)
        containing the width of the exponent part and the width of the mantissa part in
        this ieee754 format. For example for double precision usual numbers the size
        is (11, 52) (the default), for single precision floating point numbers,
        the size is (8, 23)."""

        _8, _23 = size # local variables are named according to the 32 bits float format.
        _127 = (1 << (_8 - 1)) - 1
        _255 = (_127 << 1) | 1
        r = n >> _23
        m = (r << _23) ^ n
        s = int(r >> _8)
        if not s in (0, 1):
            raise ValueError(("Integer value out of range for ieee754 format", n, size))
        e = (s << _8) ^ r
        if e == _255: # inf and NaN
            args = (0, 0, -2) if m else (s, 0, -1)
        else:
            m, e = ((1 << _23) | m, e -_127) if e else (m, m.bit_length() - _23 - _127)
            bl = (m & -m).bit_length()
            args = (s, int(e), int(m >> (bl - 1))) if bl else (s, 0, 0)
        return cls(*args)

    def to_ieee(self, size = (11, 52)):
        """Convert to an ieee754 integer.

        Convert self to an integer which binary representation is the ieee754 format corresponding
        to the 'size' argument (read the documentation of from_ieee() for the meaning of the size
        argument. Size defaults to (11, 52).
        """
        _8, _23 = size # local variables are named according to the 32 bits float format.
        _127 = (1 << (_8 - 1)) - 1
        _255 = (_127 << 1) | 1

        res = (1 << _8) if self.sign else 0
        m = 0
        if self.mantissa > 0: # general case
            e = self.log2 + _127
            if e > -_23: # otherwise too small (return 0)
                if e >= _255: # too large, return +/-Inf
                    res |= _255
                else:
                    m = self.mantissa
                    c = m.bit_length() - 1
                    lshift = _23 - c
                    if e <= 0: # subnormal number
                        lshift += (e - 1)
                    else: # normal number
                        res |= e
                        m ^= (1 << c)
                    if lshift > 0:
                        m <<= lshift
                    else:
                        m >>= (-lshift)
        elif self.mantissa < 0: # +/-Inf or NaN
            res |= _255
            if self.mantissa == -2: # NaN case
                m = (1 << (_23 - 1)) # create non signaling nan
        return (res << _23) | m

    @classmethod
    def int64_to_float(cls, n):
        """Convert a 64 bits integer to a python float.

        This class method converts an integer representing a 64 bits floating point
        number in the ieee754 double precision format to this floating point number."""

        if not (0 <= n < cls._b64):
            raise ValueError(("Integer value out of range for 64 bits ieee754 format", n))
        u, v = divmod(n, cls._b32)
        return struct.unpack(">d", struct.pack(">LL", u, v))[0]

    def bin(self, size = (11, 52)):
        """Return a binary representation of self.

        The returned string contains only the characters '0' and '1' and shows the
        ieee754 representation of the real number corresponding to self whith the given
        size = (ewidth, mwidth).
        """
        return "{val:0>{width}}".format(val = bin(self.to_ieee(size))[2:], width = sum(size) + 1)


if __name__ == '__main__':
    for x in (1.0, 3.14, 5.0, -34.7, 0.0, -0.0, float("inf"), -float("inf"), float("nan"), 2.0 **(-1030)):
        af = anyfloat.from_float(x)
        print af, x, (x == float(af))
        print af.bin((11, 52))
        print af.bin((8, 23))
        print af.to_ieee((8, 23))

    print "####"
    s1 = (3, 4)
    s2 = (5, 10)
    s4 = (8, 23)
    for size, n in [
        (s1, 69), (s1, 211), (s2, 49152), (s2, 15361), (s4, 1065353216), (s4, 3299092992)
        ]:
        print float(anyfloat.from_ieee(n, size))

If you think it works well, I'll put it in the code snippets.

Edited 4 Years Ago by pyTony: to_ieee comment sign -> size fix

I fixed one typo in your comments as moderator, maybe you got PM from that, Gribouillis. You have the main test written to Python2, easy to fix by changing print to function and importing from __future__.

Unfortunately Python3 also wants to parse the section of main routine and does not import your function, so for Python3 you need to remove the main part or change the print statements even to only import the code from other modules :(

Did not change that here as tcll seems to stay in Python2 environment, but as code snippet it is good to use function style print.

yea... I now prefer 2.7 over 2.6,
but I heard 3x is a headache and still kinda buggy.

anyways...
sry I havn't been on... IDK WTF's happening over at my end...
either my w/less card is being dumb, or the person I'm connected to didn't pay their bill. >_>
either way, I'm w/o internet on my compy.
(I'm on a friend's laptop atm)

my card says I'm connected with 4 bars, but won't load anything but about 1 packet after 10 minutes.

anyways, I'm looking into the functions right now.

EDIT: lol, yep, I still can barely understand classes XDD
btw, my program returns more precise floats than using the struct module.
(the numbers extend farther)

I tried converting Pichu from Melee and got better results from the bone matrix values. =D

Edited 4 Years Ago by Tcll

What do you mean when you say that you get more precise floats than using the struct module ? Can you give a code example ?

for Python3 you need to remove the main part or change the print statements

I posted a new version as a code snippet . It includes a bugfix with regard to rounding the last bit.

Edited 4 Years Ago by Gribouillis

This question has already been answered. Start a new discussion instead.