I
I
Ilia Zhitenev2018-03-01 14:30:46
C++ / C#
Ilia Zhitenev, 2018-03-01 14:30:46

How to get the required precision when multiplying and dividing double numbers?

There are two numbers of type double - field value and multiplier/divisor. For example, the user writes 4.2 in the input field, this number is multiplied by a factor of 10 or divided by a divisor of 0.1, and then converted to an integer format for sending to another device. The protocol accepts only integers, but the values ​​of the physical quantities behind these values ​​can have tenths and hundredths, so you have to store the divisor \ multiplier variable to show the user "4.2 V" and not "42 V".
However, I ran into an interesting feature that I can not resolve. When multiplying, for example, 4.2 * 10 I get 42, and 4.3 * 10 \u003d 43 - everything is logical and normal. But if I convert the answer from double to uint, then I get two equal answers - 42.
Switching the values, I noticed that odd ones are rounded down to even ones. 8.1 -> 8.0 etc. How can you decide? Precision greater than two decimal places is not required.
ps I write in Qt, i.e. instead of uint I use quint16

Answer the question

In order to leave comments, you need to log in

1 answer(s)
M
Mercury13, 2018-03-01
@ilyazh

The fact is that 4.2 and 4.3 cannot be represented as double. And the system, for example, stores 4.3 as 4.2999999, which when multiplied by 10 becomes 42.999999.
You need to make sure that when converting double → int, round occurs, and not truncation.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question