Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
552 views
in Technique[技术] by (71.8m points)

ruby - Why are my BigDecimal objects initialized with unexpected rounding errors?

In Ruby 2.2.0, why does:

BigDecimal.new(34.13985572755337, 9)

equal 34.0 but

BigDecimal.new(34.13985572755338, 9)

equal 34.1398557?

Note that I am running this on a 64 bit machine.

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

Initialize with Strings Instead of Floats

In general, you can't get reliable behavior with Floats. You're making the mistake of initializing your BigDecimals with Float values instead of String values, which introduces some imprecision right at the beginning. For example, on my 64-bit system:

float1 = 34.13985572755337
float2 = 34.13985572755338

# You can use string literals here, too, if your Float can't be properly
# represented. For example:
#
#    BigDecimal.new("34.13985572755337", 9)
#
# would be safer, but Float#to_s works fine with the provided corpus.
bd1 = BigDecimal.new(float1.to_s, 9)
bd2 = BigDecimal.new(float2.to_s, 9)

bd1.to_s
#=> "0.3413985572755337E2"
bd2.to_s
#=> "0.3413985572755338E2"

bd1.to_f == float1
#=> true
bd2.to_f == float2
#=> true

This is one of those cases where the internal representation of the arguments matter. Therefore, your mileage will vary depending on how you initialize your object.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...