3,234 questions
4027
votes
36
answers
617k
views
Is floating-point math broken?
Consider the following code:
0.1 + 0.2 == 0.3 -> false
0.1 + 0.2 -> 0.30000000000000004
Why do these inaccuracies happen?
- 45.9k
317
votes
7
answers
72k
views
Why are these numbers not equal?
The following code is obviously wrong. What's the problem?
i <- 0.1
i <- i + 0.05
i
## [1] 0.15
if(i==0.15) cat("i equals 0.15") else cat("i does not equal 0.15")
## i does not equal 0.15
- 5,453
265
votes
5
answers
98k
views
Why are floating point numbers inaccurate?
Why do some numbers lose accuracy when stored as floating point numbers?
For example, the decimal number 9.2 can be expressed exactly as a ratio of two decimal integers (92/10), both of which can be ...
853
votes
46
answers
785k
views
How can I deal with floating point number precision in JavaScript? [duplicate]
I have the following dummy test script:
function test() {
var x = 0.1 * 0.2;
document.write(x);
}
test();
This will print the result 0.020000000000000004 while it should just print 0....
- 33.1k
1272
votes
16
answers
464k
views
Why not use Double or Float to represent currency?
I've always been told never to represent money with double or float types, and this time I pose the question to you: why?
I'm sure there is a very good reason, I simply do not know what it is.
- 18.7k
2775
votes
34
answers
4.9m
views
How do I parse a string to a float or int?
How can I convert an str to a float?
"545.2222" -> 545.2222
Or an str to a int?
"31" -> 31
For the reverse, see Convert integer to string in Python and Converting a float to ...
- 69.7k
75
votes
5
answers
26k
views
Why is 24.0000 not equal to 24.0000 in MATLAB?
I am writing a program where I need to delete duplicate points stored in a matrix. The problem is that when it comes to check whether those points are in the matrix, MATLAB can't recognize them in the ...
- 8,555
678
votes
32
answers
690k
views
How do you compare float and double while accounting for precision loss?
What would be the most efficient way to compare two double or two float values?
Simply doing this is not correct:
bool CompareDoubles1 (double A, double B)
{
return A == B;
}
But something like:
...
- 6,821
29
votes
7
answers
136k
views
Floating point inaccuracy examples
How do you explain floating point inaccuracy to fresh programmers and laymen who still think computers are infinitely wise and accurate?
Do you have a favourite example or anecdote which seems to get ...
2453
votes
36
answers
6.0m
views
Limiting floats to two decimal points
I want a to be rounded to 13.95. I tried using round, but I get:
>>> a
13.949999999999999
>>> round(a, 2)
13.949999999999999
For the analogous issue with the standard library Decimal ...
kevin
111
votes
7
answers
290k
views
Division of integers in Java [duplicate]
This is a basic question but I can't find an answer. I've looked into floating point arithmetic and a few other topics but nothing has seemed to address this. I'm sure I just have the wrong ...
- 57.6k
797
votes
11
answers
792k
views
How can I force division to be floating point? Division keeps rounding down to 0?
I have two integer values a and b, but I need their ratio in floating point. I know that a < b and I want to calculate a / b, so if I use integer division I'll always get 0 with a remainder of a.
...
- 129k
2019
votes
41
answers
1.8m
views
How do I check if a string represents a number (float or int)?
How do I check if a string represents a numeric value in Python?
def is_number(s):
try:
float(s)
return True
except ValueError:
return False
The above works, but it ...
- 20.7k
2496
votes
19
answers
1.3m
views
Difference between decimal, float and double in .NET?
What is the difference between decimal, float and double in .NET?
When would someone use one of these?
Tom
93
votes
5
answers
65k
views
Fastest way to do horizontal SSE vector sum (or other reduction)
Given a vector of three (or four) floats. What is the fastest way to sum them?
Is SSE (movaps, shuffle, add, movd) always faster than x87? Are the horizontal-add instructions in SSE3 worth it?
What'...
- 3,848
458
votes
16
answers
741k
views
How do I print a double value with full precision using cout?
In my earlier question I was printing a double using cout that got rounded when I wasn't expecting it. How can I make cout print a double using full precision?
user6212
553
votes
18
answers
501k
views
How to compare floats for almost-equality in Python?
It's well known that comparing floats for equality is a little fiddly due to rounding and precision issues. For examples on this, see the blog post Comparing Floating Point Numbers, 2012 Edition by ...
- 11.8k
374
votes
11
answers
82k
views
What is the rationale for all comparisons returning false for IEEE754 NaN values?
Why do comparisons of NaN values behave differently from all other values?
That is, all comparisons with the operators ==, <=, >=, <, > where one or both values is NaN returns false, contrary to ...
- 57.1k
318
votes
22
answers
138k
views
Why can't decimal numbers be represented exactly in binary?
There have been several questions posted to SO about floating-point representation. For example, the decimal number 0.1 doesn't have an exact binary representation, so it's dangerous to use the == ...
- 20.7k
1310
votes
39
answers
2.2m
views
Format number to always show 2 decimal places
I would like to format my numbers to always display 2 decimal places, rounding where applicable.
Examples:
number display
------ -------
1 1.00
1.341 1.34
1.345 1.35
I ...
- 17.2k
43
votes
10
answers
11k
views
Strange output in comparison of float with float literal
Consider:
float f = 0.7;
if (f == 0.7)
printf("equal");
else
printf("not equal");
Why is the output not equal?
Why does this happen?
- 8,605
188
votes
24
answers
269k
views
Retain precision with double in Java
public class doublePrecision {
public static void main(String[] args) {
double total = 0;
total += 5.6;
total += 5.8;
System.out.println(total);
}
}
The above ...
- 4,029
422
votes
12
answers
104k
views
How dangerous is it to compare floating point values?
I know UIKit uses CGFloat because of the resolution independent coordinate system.
But every time I want to check if for example frame.origin.x is 0 it makes me feel sick:
if (theView.frame.origin.x ...
- 40.5k
89
votes
6
answers
53k
views
Large numbers erroneously rounded in JavaScript
See this code:
var jsonString = '{"id":714341252076979033,"type":"FUZZY"}';
var jsonParsed = JSON.parse(jsonString);
console.log(jsonString, jsonParsed);
When I see my console in Firefox 3.5,...
- 17.9k
1025
votes
36
answers
1.2m
views
How do I use a decimal step value for range()?
How do I iterate between 0 and 1 by a step of 0.1?
This says that the step argument cannot be zero:
for i in range(0, 1, 0.1):
print(i)
- 102k
169
votes
9
answers
424k
views
Printf width specifier to maintain precision of floating-point value
Is there a printf width specifier which can be applied to a floating point specifier that would automatically format the output to the necessary number of significant digits such that when scanning ...
- 12.1k
38
votes
7
answers
38k
views
Floating point arithmetic not producing exact results [duplicate]
I need to do some floating point arithmetic in Java as shown in the code below:
public class TestMain {
private static Map<Integer, Double> ccc = new HashMap<Integer, Double>() {
...
user63904
133
votes
13
answers
160k
views
How should I do floating point comparison?
I'm currently writing some code where I have something along the lines of:
double a = SomeCalculation1();
double b = SomeCalculation2();
if (a < b)
DoSomething2();
else if (a > b)
...
- 12.9k
394
votes
27
answers
553k
views
How do I use floating-point arithmetic in bash?
I am trying to divide two image widths in a Bash script, but bash gives me 0 as the result:
RESULT=$(($IMG_WIDTH/$IMG2_WIDTH))
I did study the Bash guide and I know I should use bc, in all examples ...
- 5,173
314
votes
11
answers
325k
views
Biggest integer that can be stored in a double
What is the biggest "no-floating" integer that can be stored in an IEEE 754 double type without losing precision?
In other words, what would the follow code fragment return:
UInt64 i = 0;
...
- 28.8k
19
votes
3
answers
32k
views
Why is floating point arithmetic in C# imprecise?
Why does the following program print what it prints?
class Program
{
static void Main(string[] args)
{
float f1 = 0.09f*100f;
float f2 = 0.09f*99.999999f;
Console....
- 4,097
633
votes
29
answers
871k
views
How to nicely format floating numbers to string without unnecessary decimal 0's
A 64-bit double can represent integer +/- 253 exactly.
Given this fact, I choose to use a double type as a single type for all my types, since my largest integer is an unsigned 32-bit number.
But now ...
- 28.1k
258
votes
22
answers
300k
views
How can I compare two floating point numbers in Bash?
I am trying hard to compare two floating point numbers within a Bash script. I have two variables, e.g.
let num1=3.17648e-22
let num2=1.5
Now, I just want do a simple comparison of these two numbers:
...
- 3,224
245
votes
2
answers
70k
views
Which is the first integer that an IEEE 754 float is incapable of representing exactly?
For clarity, if I'm using a language that implements IEE 754 floats and I declare:
float f0 = 0.f;
float f1 = 1.f;
...and then print them back out, I'll get 0.0000 and 1.0000 - exactly.
But IEEE 754 ...
- 2,713
204
votes
18
answers
154k
views
Compare floats in php
I want to compare two floats in PHP, like in this sample code:
$a = 0.17;
$b = 1 - 0.83; //0.17
if($a == $b ){
echo 'a and b are same';
}
else {
echo 'a and b are not same';
}
In this code it ...
- 4,355
20
votes
1
answer
28k
views
Integer division always zero [duplicate]
Possible Duplicate:
C programming division
probably my question is very simple and stupid.
I would like to store the value of a division, in particular 1 / x where x is a
integer value.
int x = ...
- 12k
49
votes
14
answers
21k
views
Why do I see a double variable initialized to some value like 21.4 as 21.399999618530273?
double r = 11.631;
double theta = 21.4;
In the debugger, these are shown as 11.631000000000000 and 21.399999618530273.
How can I avoid this?
180
votes
6
answers
87k
views
Why is NaN not equal to NaN? [duplicate]
The relevant IEEE standard defines a numeric constant NaN (not a number) and prescribes that NaN should compare as not equal to itself. Why is that?
All the languages I'm familiar with implement this ...
- 52.7k
68
votes
5
answers
92k
views
Is floating point arbitrary precision available?
Just for fun and because it was really easy, I've written a short program to generate Grafting numbers, but because of floating point precision issues it's not finding some of the larger examples.
def ...
- 17.3k
440
votes
8
answers
505k
views
Double vs. BigDecimal?
I have to calculate some floating point variables and my colleague suggest me to use BigDecimal instead of double since it will be more precise. But I want to know what it is and how to make most out ...
- 11.1k
98
votes
8
answers
64k
views
PHP - Floating Number Precision [duplicate]
$a = '35';
$b = '-34.99';
echo ($a + $b);
Results in 0.009999999999998
What is up with that? I wondered why my program kept reporting odd results.
Why doesn't PHP return the expected 0.01?
- 1,325
257
votes
23
answers
468k
views
round() for float in C++
I need a simple floating point rounding function, thus:
double round(double);
round(0.1) = 0
round(-0.1) = 0
round(-0.9) = -1
I can find ceil() and floor() in the math.h - but not round().
Is it ...
- 68.5k
276
votes
17
answers
468k
views
How to suppress scientific notation when printing float values?
Here's my code:
x = 1.0
y = 100000.0
print x/y
My quotient displays as 1.00000e-05.
Is there any way to suppress scientific notation and make it display as
0.00001? I'm going to use the result ...
- 3,676
276
votes
18
answers
1.1m
views
How do I round a number to 2 decimal places in C?
How can I round a floating-point value (such as 37.777779) to two decimal places (37.78) in C?
Tyler
112
votes
8
answers
104k
views
Floating point division vs floating point multiplication
Is there any (non-microoptimization) performance gain by coding
float f1 = 200f / 2
in comparision to
float f2 = 200f * 0.5
A professor of mine told me a few years ago that floating point divisions ...
- 4,547
546
votes
14
answers
1.2m
views
What is the difference between float and double?
I've read about the difference between double precision and single precision. However, in most cases, float and double seem to be interchangeable, i.e. using one or the other does not seem to affect ...
- 7,931
1685
votes
7
answers
167k
views
Why does changing 0.1f to 0 slow down performance by 10x?
Why does this bit of code,
const float x[16] = { 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8,
1.9, 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6};
const float z[...
- 15.1k
231
votes
13
answers
879k
views
Printing the correct number of decimal places with cout
I have a list of float values and I want to print them with cout with 2 decimal places.
For example:
10.900 should be printed as 10.90
1.000 should be printed as 1.00
122.345 should be printed as ...
- 9,562
254
votes
13
answers
674k
views
How to display a float with two decimal places? [duplicate]
I have a function taking float arguments (generally integers or decimals with one significant digit), and I need to output the values in a string with two decimal places (5 → 5.00, 5.5 → 5.50, etc). ...
- 4,331
165
votes
7
answers
383k
views
How to extract a floating number from a string [duplicate]
I have a number of strings similar to Current Level: 13.4 db. and I would like to extract just the floating point number. I say floating and not decimal as it's sometimes whole. Can RegEx do this or ...
- 8,386