3
Contributors
6
Replies
25
Views
2 Years
Discussion Span
Last Post by Reverend Jim
0

In the first case, 12.345 is probably being decoded as a double. In the second case, the variable A is a float. Welcome to the world of floating point numbers and rounding errors!

0

Thanks a lot. I tried this to show if A and 12.345 is the same:

Dim A As Single
A = 12.345
Debug.Print A = 12.345
Debug.Print Round(12.345, 2)
Debug.Print Round(A, 2)

and got output:
True
12.34
12.35

0

A quick check in vb.net and vbScript (sorry, do not have vb6 installed) gives the following results

Math.Round(1.5) is 2.0
Math.Round(2.5) is 2.0
Math.Round(3.5) is 4.0    <edit> corrected from 3.0 </edit>
Math.Round(4.5) is 4.0    <edit> corrected from 3.0 </edit>

If the integer portion is odd then it rounds up. If even then it rounds down. Personally I think this is brain dead and as a result I always do my own rounding.

Edited by Reverend Jim

0

Reverend Jim,
I also check it in VB.Net and found you might got mistake. What I got are
Debug.Print(Math.Round(1.5)) 'is 2
Debug.Print(Math.Round(2.5)) 'is 2
Debug.Print(Math.Round(3.5)) 'is 4
Debug.Print(Math.Round(4.5)) 'is 4

0

rubberman,

OK, in Vb.Net it looks better:

Debug.Print(Math.Round(1.5)) 'is 2
Debug.Print(Math.Round(2.5)) 'is 2
Debug.Print(Math.Round(3.5)) 'is 4
Debug.Print(Math.Round(4.5)) 'is 4

Dim A As Single
A = 12.345
Debug.Print (A = 12.345) 'is False! Perhaps A is something like 12.345000001
Debug.Print (A) 'is 12.345 but perhaps it is something like 12.345000001
Debug.Print (Math.Round(12.345, 2)) 'is 12.34
Debug.Print (Math.Round(A, 2)) 'is 12.35

This topic has been dead for over six months. Start a new discussion instead.
Have something to contribute to this discussion? Please be thoughtful, detailed and courteous, and be sure to adhere to our posting rules.