Hello,
using PB 2019 R3 2703, Windows 10 64
in my application I must make use of the LongLong datatype.
Unfortunately I noticed calculation errors on doing a integer division with this datatype.
Here is the code:
longLong x, y
x = 7129750713734852035 // LongLong-Value with 19 digits
y = x / 16 // y gets the value 445609419608428224, which is wrong
The same division done in Windows Calc.exe and other calulators calcuate a value of 445609419608428252
The error seems to happen with large values only
Here are examples
PB:
1) 7.129.750.713.734.852.035 / 16 = 445.609.419.608.428.224 // wrong
2) 712.975.071.373.485.203 / 16 = 44.560.941.960.842.824 // wrong
3) 71.297.507.137.348.520 / 16 = 4.456.094.196.084.282 // OK
Windows Calc.exe:
1) 7.129.750.713.734.852.035 / 16 = 445.609.419.608.428.252
29 712.975.071.373.485.203 / 16 = 44.560.941.960.842.825
3) 71.297.507.137.348.520 / 16 = 4.456.094.196.084.282
Any idea, whats going wrong?
Thank you, Martin