Use Double for (intermediate) grade calculations
We sometimes see floating point errors in grade calculation. Some of the calculations are already using
Submitted by Elmer van Chastelet on 12 July 2021 at 09:34Double
instead ofFloat
, but other calculations only useFloat
, e.g. for collection grades and collection weights.
Change the calculations to useDouble
, so we don’t introduce precision errors.
Issue Log
On 27 July 2021 at 12:08 Elmer van Chastelet commented:
This won’t fix the precision issue. It even makes it worse, because the grade traces (String) would expose the precision errors from the
Double
values, where a stringifiedFloat
would project the least number of decimals to distinguish the number from the next Float value.
The fix would be to limit the precision using BigDecimal (new issue)
On 27 July 2021 at 12:08 Elmer van Chastelet closed this issue.
On 27 July 2021 at 12:08 Elmer van Chastelet removed tag 1.5.0
Log in to post comments