Answer the question
In order to leave comments, you need to log in
What is the best way to round double with a given error/precision?
In the code I met such a rounding:
static class TraceConvertor
{
public const double ValueTolerance = 1e-2;
public const int ValueDigitTolerance = 2;
[ MethodImpl( MethodImplOptions.AggressiveInlining ) ]
public static double RoundByTolerance( double value )
{
return Math.Round( value / ValueTolerance ) * ValueTolerance;
}
[ MethodImpl( MethodImplOptions.AggressiveInlining ) ]
public static double RoundByDigitTolerance( double value )
{
return Math.Round( value, ValueDigitTolerance );
}
}
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question