4,611
questions
-1
votes
0
answers
56
views
Handle small precision in C++ to construct NN biases [closed]
I am trying to recreate a simple Neural Network from scratch in vanilla C++ (no libraries). However, I need to get the sum of the matrix of size (10 x 42000) with value between -1 and 1 (here is the ...
0
votes
0
answers
15
views
Is there a way to increase number precision in qutip functions?
I have a problem where the difference of two relative entropies is finite, but they individually diverge. The function that calculates relative entropies in qutip is qt.entropy_relative(state1, state2)...
- 101
0
votes
0
answers
55
views
Reading Float data with float variable in stored procedure rounds numbers
In SQL Server Management Studio:
DECLARE @Amount FLOAT
SELECT SUM(Amount)
FROM Docs
WHERE DocID IN (104482, 104483, 104484, 104486, 104489, 104491, 104493)
-- displays in results window 178457,05
...
- 11
-2
votes
2
answers
51
views
How many numbers in the range 0.001 .. 0.999 cannot be represented as Floating Point numbers?
Everybody knows, that not all numbers can be represented as Floating Point numbers. For example 0.1 has no exact Floating Point representation.
There is already the question, How many values can be ...
- 1,001
0
votes
0
answers
30
views
Consistently apply Welford Method and Kahan Summation in roll_xxx functions
For the Pandas following functions:
def nancorr
cdef void add_var
cdef void add_skew
cdef void add_mean
It appears that both the Welford method and Kahan summation are taken into account. However, ...
0
votes
2
answers
51
views
GCP Pub/Sub to BigQuery writing incorrect value to NUMERIC column
I'm publishing JSON to Pub/Sub in the following format:
{"reading":"11202.8"}
Numbers are encoded as strings to force explicit precision. This get written to BigQuery using a Pub/...
- 1,050
0
votes
0
answers
44
views
why is it not printing 2.0? Why can't I see a message? [duplicate]
Why doesn't 2.0 work?
My expected output: 1.0 and 2.0
#include <stdio.h>
int main() {
for(float i= 0.0; i<=3.0; i += 0.2){
if(i == 1.0){
printf("1.0\n")...
4
votes
3
answers
107
views
Why does 2x - x == x in IEEE floating point precision?
I would expect this to only hold when the last bit of the mantissa is 0. Otherwise, in order to subtract them (since their exponents differ by 1), x would lose a bit of precision first and the result ...
- 5,450
0
votes
1
answer
114
views
Rounding when ULP 1 - Java and C++
The reason of the question originated from differences in the results that I observed when computing std::exp(x) in C++ against Math.exp(x) in Java (with x of type double). I noticed that the ...
- 29
5
votes
2
answers
133
views
Replicate precise calculation from VBA in Python
I have this code in VBA that needs to be in Python. I'm new to Python, sorry for that.
Sub cdec_test()
Dim val1, val2, result1, result2, x1, x2 As Variant
val = CDec(5930 / 3000) 'val1 : 1....
- 51
-6
votes
0
answers
144
views
Floating point numbers and computer arithmetic - Difference between Java and C++ [duplicate]
In Java, the double result of 1.0 / 20.0 is exactly 0.05.
On the other hand, in C++ the result is 0.050000000000000003.
Could someone help me to understand what is the specific cause of this ...
- 27
-1
votes
0
answers
20
views
Weird behaviour when adding floats in PHP [duplicate]
I came across the following weird result when running this:
<?php
$total = 0;
for($i=0; $i<3; $i++)
{
var_dump($total);
$total += 10.1;
}
var_dump($total);
the final result is 30....
- 29
2
votes
1
answer
53
views
In Python, why does preallocation of a numpy array fail to limit its printed precision?
Here is a minimal example:
import numpy as np
np.set_printoptions(linewidth=1000, precision=3)
# First attempt fails to limit the printed precision of x
x = np.array([None])
x[0] = 1/3
print(x)
# ...
- 227
4
votes
2
answers
136
views
Does the sign affect the precision and accuracy of floating point numbers?
In floating-point arithmetic, if two numbers have the same binary representation, then the result of any operation performed on these numbers should be the same, and equality comparisons using == ...
- 75
2
votes
1
answer
79
views
Why does this Java float addition example behave like the mantissa is 24 bits long?
Intro:
With Java floats, I noticed that when you add 1.0 to a certain range of tiny negative numbers, it equals 1.0. I decided to investigate this and learned a lot about how floats work in my quest ...
- 417