Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
947 views
in Technique[技术] by (71.8m points)

floating point - C# float bug? 0.1 - 0.1 = 1.490116E-08

What's going on?! Subtraction works fine until I get to 0.1 - 0.1. I'm in visual c# 2008 using the nonoba.com API.

Console.WriteLine("hit! " + Users[targetNum].character.health + " : " + player.character.profile.attackPower);

Users[targetNum].character.health -= player.character.profile.attackPower;

Console.WriteLine("health! " + Users[targetNum].character.health);

output:

hit! 0.1 : 0.1
health! 1.490116E-08

Thanks all - I might use the decimal type, as I'm normally adding/subtracting nice "round" numbers. For now I'll just go with:

if (Users[targetNum].character.health <= 0.00001)

By the way I knew this wasn't really going to be a "bug" in c# - I thought it would either by a bug in my code or some lack of understanding, which it was.

Having read all the recommended reading, I'm going to conclude that my folly was due to normally using the ActionScript Number type, which maybe has a decimal rather than binary floating point - anyway, it would never give this output.

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

That seems pretty normal for floating point math... you always have to check against a small delta to account for imperceptible rounding differences. Depending on the scenario, decimal might be what you want.

Basically, unless you can be sure that it is exactly the same 0.1 in both cases (i.e. nothing has been done to them), you aren't likely to get zero; in general you'll get something very nearly zero. With decimal you'll usually get more what you expect intuitively.

See also Jon Skeet's pages here:


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...