[floatingpoint] Forum Topics

while storing the value 1.111 in a float variable the value stored and printed is 1.1109999. why is this loss of data or sometimes the gain of data?.

At work we have a large number of unit tests that try to alert us that we have made changes that have broken our code. Many of these tests work by evaluating functions under known conditions and comparing the result with the known result. The known result is acquired by …

Hello guys, i'm trying to compute the area of a triangle using the Floating-Point stack (FPU87), but i'm having an issue outputting my value. I'm computing the area using Heron's formula: T = sqrt{s * (s - a ) * (s - b) * (s - c) } Note: s …

Hello. I wrote a program to convert decimal fraction values into binary fractions. I seem to be running into some floating point accuracy problems. After multiplying a floating point number by 2 over and over, eventually the number becomes corrupted. Since I'm multiplying so many times, I think the error …

We know that 0.1 + 0.1 + 0.1 - 0.3 = 0.0 But in python 0.1 + 0.1 + 0.1 - 0.3 = 5.5511151231257827e-017 Is there any way I can get 0.0...??

Hi, i'm getting only one digit after floating point after conversion from string to float. Have tryed from standart functions to i don't know.. the problems is that conversion whatever it would be (strtod(), stringstream >> float, atof(), don't even rembember all other wacky ways to do that) brings me …

May I know how to solve floating decimal problems?

I'm in the midst of developing a Python toolset for some proprietary hardware used at NOAO. The code has to write a series of EEPROMS in the hardware to determine conversion factors for hardware attributes. The EEPROM format is 2048 32-bit words each word represents the slope, intercept, minimum or …

How would you safely round a floating point number to the nearest integer ? Python has the built in function [icode]round[/icode], but it returns a floating point number [code=python] >>> round(4.9) 5.0 >>> help(round) Help on built-in function round in module __builtin__: round(...) round(number[, ndigits]) -> floating point number Round …

This program is intended to determine if a fractions is valid (non-zero, non-negative denominator) and to convert an improper fraction into a whole/mixed number. While running the program I get the message "Floating point exception" after it reads the inputs just inside the for loop. Any insight into what is …

Hi, i've just taken higher computing in college and need to know a couple of things before i can proceed further with the course. The first is how to represent floating point numbers in binary. i have a good grasp of binary & data types etc. there are a few …