Tuesday 12 May 2009

Ruby Math - BigDecimal


Let's have a look at this example:

1.2 + 1.0 == 2.2 (returns true)
but if I do:

1.2 - 1.0 == 0.2 (returns false)
Hmmm ... strange. This comes down to the way computers generally store numbers in binary but not in base10 system. The system that human generally work and think in.

Anyway, to fix this problem, I use Ruby built-in class BigDecimal

require 'bigdecimal'
require 'bigdecimal/util'
1.2.to_d - 1.0.to_d == 0.2.to_d (returns true)

Reference:

http://www.ruby-doc.org/stdlib/libdoc/bigdecimal/rdoc/classes/BigDecimal.html
http://ramblingsof.justinwinkler.com/when-crap-just-doesnt-add-up

I currelty work for UK Business Directory - Touch Local, check out our awesome site written in Ruby on Rails

No comments:

Post a Comment