Answer the question
In order to leave comments, you need to log in
How to get the required precision when multiplying and dividing double numbers?
There are two numbers of type double - field value and multiplier/divisor. For example, the user writes 4.2 in the input field, this number is multiplied by a factor of 10 or divided by a divisor of 0.1, and then converted to an integer format for sending to another device. The protocol accepts only integers, but the values of the physical quantities behind these values can have tenths and hundredths, so you have to store the divisor \ multiplier variable to show the user "4.2 V" and not "42 V".
However, I ran into an interesting feature that I can not resolve. When multiplying, for example, 4.2 * 10 I get 42, and 4.3 * 10 \u003d 43 - everything is logical and normal. But if I convert the answer from double to uint, then I get two equal answers - 42.
Switching the values, I noticed that odd ones are rounded down to even ones. 8.1 -> 8.0 etc. How can you decide? Precision greater than two decimal places is not required.
ps I write in Qt, i.e. instead of uint I use quint16
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question