H
H
HomoLuden2011-11-22 16:21:18
ruby
HomoLuden, 2011-11-22 16:21:18

[(Iron)Ruby] Problem with Math.cos?

Accidentally stumbled upon an error in Math.cos for the PI/2 argument.

irb(main):002:0&gt; cos(PI/2)<br/>
=&gt; 6.123031769111886e-17

This was discussed on ruby-talk ... as I understand it, this is the problem of "small numbers" and the implementation of trigonometry.
“so of course it's reasonable to say that Math.cos( Math::PI/2 ) â 0.0, but perhaps not exactly. 9e-17 is insignificantly small. „
For me, 9e-17 is a “significant” error, because I use cosine and sine in quaternions to rotate vectors.
Is this error only near the boundary results or over the entire domain of definition?
How correctly to bypass this situation?
Redefine the cosine as I need (for PI / 2 return 0) and assume that for all other arguments the calculations are correct?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
W
Wott, 2011-11-22
@HomoLuden

this is a problem of floating point arithmetic and can pop up in any action
if E-16 is noticeable for your task (well, it happens rarely, but it happens :) ) - find a library that works with precision that suits

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question