N
N
newuser88882020-10-22 18:45:13
C++ / C#
newuser8888, 2020-10-22 18:45:13

I can't understand the principle of float point rounding, can you explain using a number as an example?

5f91a7f7a1125800167976.png
For example, there is a number 2.999999 - it is not rounded. The accuracy of the number of digits in a float single precison is 7.
Okay, I went beyond the border, 2.999999(9) - the number is rounded to 3. The text says that rounding goes to the nearest (4 - down, 5 - up) then why 2.999999(8) is not 3, and 2.9999997615814208984375. After all, 8 is greater than 5 and the rounding goes up?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
M
Mercury13, 2020-10-22
@newuser8888

1. Rounding to even. This is standard rounding to nearest; if the distance is the same, then to even.
4.4 → 4
4.5 → 4
4.6 → 5
−4.4 → −4
−4.5 → −4
−4.6 → −5
5.4 → 5
5.5 → 6
5.6 → 6
−5.4 → −5
−5.5 → −6
−5.6 → −6
2. Round down to −∞. Rounding down.
4.xxx → 4 (any decimal)
-4.xxx → -5
3. Round up to +∞. Rounding up.
4.xxx → 5
−4.xxx → −4
4. Dropping the fractional part. Rounding down to 0. Rounding down in absolute value.
4.xxx → 4
−4.xxx → −4
Double has so many characters - so somewhere in the bowels of the library, float became double. The float has not exactly 7 characters, but a little more than seven. And this number is really not a triple: the triple has the 16th form 4040.0000, and your number is 403F.FFFF. As you can see, the value of the unit of the least significant digit (ULP) at such values ​​will be about 2.4 10 −7  - there will be more than seven characters, but it does not reach eight.
The IEEE 754 rounding mechanisms have nothing to do with the decimal rounding provided in the language library. IEEE rounding is actually binary and is used when converting from a more precise type to a less precise type, or to round the result of a multiplication/division.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question