Recent questions tagged floating

0 votes

691 views

1 answer

floating point - F# - How to compare floats

In F#. How to efficiently compare floats for equality that are almost equal? It should work for very large and ... and Significand of a float? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

631 views

1 answer

floating point - Why don't operations on double-precision values give expected results?

System.out.println(2.14656); 2.14656 System.out.println(2.14656%2); 0.14656000000000002 WTF? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

657 views

1 answer

floating point - Is it safe to compare a float and an int in Java?

Is it safe to compare a float and an integer like this? private static void foo(float n) { if (n >= 1 ... original float. How can we ensure it? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

625 views

1 answer

floating point - Implementation of nextafter functionality in R

Is there any implementation of functionality in R, such that it is possible to get the next representable floating ... don't work in general. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

887 views

1 answer

floating point - How do you round off decimal places in C++?

I need help rounding off a float value to one decimal place. I know setprecision(x) and cout << precision( ... the decimals to the tenths place. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

731 views

1 answer

floating point precision - PHP - Getting a float variable internal value

I am trying to establish the delta I need when doing float comparison in PHP. I want to take a closer ... var_dump() print the internal value? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

739 views

1 answer

floating point - Ruby Float#round method behaves incorrectly with round(2)

I learned that it's recommended to use BigDecimal instead of Float, but this one is either a bug or highlights ... be a bug in Float#round? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

581 views

1 answer

floating point - correctly-rounded double-precision division

I am using the following algorithm for double-precision division and trying to make it correctly rounded in ... multiplication to 64 bits. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

799 views

1 answer

floating point - Compiling *.vhdl into a library, using Altera Quartus II

I am trying to use 'Floating point and Fixed point package' as a part of my filter design in VHDL. I am ... tool. Any tip would be appreciated. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

514 views

1 answer

floating point - Problem using the find function in MATLAB

I have two arrays of data that I'm trying to amalgamate. One contains actual latencies from an experiment in the ... , I'd be really grateful. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

620 views

1 answer

floating point - Calculation problem in Java

I'm having slight difficulty in performing a calculation in Java. Here is what I'm trying to do - ((0.053800 * ... what can I do to correct it? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

727 views

1 answer

floating accuracy - ruby: converting from float to integer in ruby produces strange results

ree-1.8.7-2010.02 :003 > (10015.8*100.0).to_i => 1001579 ree-1.8.7-2010.02 :004 > 10015.8* ... anybody knows how to eradicate this heresy? =) See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

683 views

1 answer

floating point - Return a float from a 64-bit assembly function that uses x87 FPU

I am trying to make a program that calculates equations (what equation doesn't matter currently) that use 64-bit ... how can I fix it? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

647 views

1 answer

floating point - get the integer representation value of a float type variable in C

I have the number 20 (0x14) stored in a 32-bit register. The register is allocated to a C float variable ... avoiding pointers and working in C? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

704 views

1 answer

floating point - double precision C++

I think the precision of double is causing that problem, as it was described in similiar posts, but I would like to ... with int eq(8*2,16). See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

639 views

1 answer

floating point - Why do Haskell numerical literals need to start and end with digits?

In The Haskell 98 Report it's said that A floating literal must contain digits both before and after the decimal ... to a ., without whitespace? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

494 views

1 answer

floating point - MySQL "greater than" condition sometimes returns row with equal value

I'm running into a baffling issue with a basic MySQL query. This is my table: id | rating 1 | 1317.17 2 ... script or in phpMyAdmin. Any ideas? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

715 views

1 answer

floating point - Is if(double) valid C++?

I just ran into this line of code: if( lineDirection.length2() ){...} where length2 returns a double. ... standard or is it undefined behaviour? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

684 views

1 answer

floating point - What is the smallest number which can be represented in python?

What is the smallest number which can be represented in python? I have seen as small as 2.05623357236e-296 but can there be any smaller? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

715 views

1 answer

floating point - Python array multiply

hh=[[82.5], [168.5]] N=1./5 ll=N*hh What I'm doing wrong? I received error : "can't multiply sequence by non- ... for x in y] for y in hh] ??? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

832 views

1 answer

floating point - How to calculate decimal digits of precision based on the number of bits?

I am learning about floating point formats (IEEE). In the single precision floating point format ,it is ... somebody please enlighten me ? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

846 views

1 answer

floating point - How quickly check whether double fits in float? (Java)

Are there some arithmetic or bitwise operations that can check whether a double fits into a float without loss of ... that works for Java. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

541 views

1 answer

floating accuracy - How to overcome inaccuracy in Java

I came to know about the accuracy issues when I executed the following following program: public static void ... i.e. Double.POSITIVE_INFINITY? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

810 views

1 answer

floating point - Number precision in JavaScript

Why 100 == 99.999 is true, however 100 == 99.99 is false in Javascript? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

585 views

1 answer

floating point - How is fma() implemented

According to the documentation, there is a fma() function in math.h. That is very nice, and I know how ... when the precision is relied upon? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

770 views

1 answer

floating point - Problem with Floats! (in PHP)

What's wrong with php floats? $v = 5.2 * 3; if($v==15.6) { echo 'Everything is fine =)'; ... using this to calculate real money transactions? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

639 views

1 answer

floating point - Return a float from a 64-bit assembly function that uses x87 FPU

I am trying to make a program that calculates equations (what equation doesn't matter currently) that use 64-bit ... how can I fix it? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

604 views

1 answer

floating point - How does R represent NA internally?

R seems to support an efficient NA value in floating point arrays. How does it represent it ... implemented without compromising performance? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

Categories

Just Browsing Browsing

Most popular tags