Why is front-end numerical precision lost? (BigInt solution)

I believe that all front-end partners will inevitably be involved in using JavaScript to process numerical values in their daily work, such as numeric calculations, retaining specified decimal places, interface return values that are too large, etc. These operations are all possible. As a result, originally normal values do behave abnormally in JavaScript (ie, precision is lost).

Accuracy loss scenario

Calculation of floating point numbers

1. Addition and subtraction

0.1 + 0.2 // The result is 0.30000000000000004
0.3 - 0.1 // The result is 0.199999999999999996

// This is because the binary representation of floating point numbers cannot accurately represent certain decimal fractions, resulting in slight errors in the calculation results.

2. Multiplication and division

0.1 * 0.2 // The result is 0.020000000000000004
0.3 / 0.1 // The result is 2.99999999999999996

// When performing multiplication and division, the precision problem of floating point calculation results is more prominent, and greater errors may occur.

3. Comparison operations 

0.1 + 0.2 === 0.3 // The result is false

// Direct comparison of floating point numbers may lead to inaccurate results, since small errors in the calculations may make them not exactly equal.

Exceeds the maximum value

The so-called exceeding the maximum value (maximum and minimum value) refers to exceeding Number.MIN_SAFE_INTEGER (- 9007199254740991), that is, + ( 2^53 – 1) or Number.MAX_SAFE_INTEGER (+ 9007199254740991), that is, a value in the range of – (2^53 – 1), which is the most common in the project The following are the situations: 1

  • 1. The value returned by the backend exceeds the maximum value
    • Example 1: The list data returned by the backend usually has a corresponding ID to identify uniqueness, but the backend generates this ID is Long type, then the value is likely to exceed the largest positive integer that can be represented in JavaScript. At this time This results in a loss of precision, that is, the ID value actually obtained by the front end and the one returned by the back end will be inconsistent.
    • Example 2: The backend may need to calculate some values and then return the corresponding result value to the frontend. At this time, if the value exceeds the maximum value, precision loss will also occur.
  • 2. When the front end performs numerical calculations, the calculation result exceeds the maximum value

Keep specified decimal places

In addition to the above scenarios involving floating point calculations that exceed the maximum value, we usually also process the numerical value to retain specified decimal places, and some developers may directly use Number.prototype.toFixed To achieve, but this method does not guarantee the results we expect. For example, when retaining decimal places, there will be problems when rounding is required, as follows:

console.log(1.595.toFixed(2)) // 1.59 --> Expected: 1.60
console.log(1.585.toFixed(2)) // 1.58 --> Expected: 1.59
console.log(1.575.toFixed(2)) // 1.57 --> Expected: 1.58
console.log(1.565.toFixed(2)) // 1.56 --> Expected: 1.57
console.log(1.555.toFixed(2)) // 1.55 --> Expected: 1.56
console.log(1.545.toFixed(2)) // 1.54 --> Expected: 1.55
console.log(1.535.toFixed(2)) // 1.53 --> Expected: 1.54
console.log(1.525.toFixed(2)) // 1.52 --> Expected: 1.53
console.log(1.515.toFixed(2)) // 1.51 --> Expected: 1.52
console.log(1.505.toFixed(2)) // 1.50 --> Expected: 1.51

Causes of loss of accuracy

The computer can actually only storage/recognize binary internally, so documents, pictures, numbers, etc. will be converted to Binary, and for numbers, although we see the representation result of Decimal, in fact, the bottom layer will perform Decimal and < strong>Binary mutual conversion, and this conversion process may cause precision loss, because the conversion from decimal to binary may produce an infinite loop part, and < strong>Actual storage space is limited.

Because the computer internally uses binary floating point number representation, not decimal. This binary representation cannot accurately represent certain decimal fractions in some cases, resulting in a loss of precision, such as:

1. Decimal decimals that cannot be expressed accurately:

Some decimal decimals cannot be accurately represented as finite-length binary decimals. For example, decimal fractions like 0.1 and 0.2 are infinitely recurring decimals in binary representation, so when represented internally by a computer with a limited number of digits, there are rounding errors that cause a loss of precision.

2. Rounding error:

Since floating-point numbers have a limited number of digits, computers round off decimal numbers that cannot be represented accurately to approximate their values. This rounding operation introduces errors and causes differences between calculated results and expected values.

Rounding of Number.prototype.toFixed: Regarding the rounding method of this method, the most common theory currently is the Banker’s algorithm. Indeed, in most cases, it can indeed comply with the rules of the Banker’s algorithm, but in some cases, it does not comply with its rules. Therefore, strictly speaking, Number.prototype.toFixed does not count as using the banker’s algorithm. (Banker’s rounding: The so-called banker’s rounding method is essentially a method of rounding up to five and leaving even (also known as rounding up and leaving even). In simple terms: Consider rounding to five. If the number after five is not zero, then round it up by one. If the number after five is zero, it will be considered odd or even. If the number before five is even, it should be discarded. If the number before five is odd, it should be rounded up by one. See below for specific explanation)

3. Cumulative error of arithmetic operations:

When performing a series of floating point arithmetic operations, rounding errors can accumulate and cause a loss of precision. Each operation introduces some errors, which gradually accumulate over multiple operations, resulting in reduced accuracy of the final result.

4. Inaccuracy of comparison operations:

Due to the limited precision of representation of floating point numbers, directly comparing floating point numbers may lead to inaccurate results. Small rounding errors can cause two floating-point numbers that appear to be equal to be considered unequal when compared.

5. Limitation of numerical range:

The representation range of floating point numbers is limited, and values outside the range may cause overflow or underflow, thus affecting the accuracy of calculation results.

Banker’s Algorithm

The so-called banker’s algorithm can be summarized in one sentence as:
Consider rounding to five, after five if there is a number, round up one, after five if there are countless, look at odd and even , 五前 is odd and should be discarded, 五后 is odd and should be further into one

  • Rounding means that the value < 5 after the reserved digit should be rounded, 4 Just a representative value
  • Six-in refers to the value> 5 after the reserved bit, which should be one, 6 Just a representative value
  • If the value after the reserved digit = 5, check whether there is a number after 5
    • If there are countless numbers after 5, judge by looking at the oddness of the value before 5
      • If the value before 5 is an even number, reject
      • If the value before 5 is odd, then advance
    • If there is a number after 5, then Add one
// Four houses
(1.1341).toFixed(2) = '1.13'

//Sixth entry
(1.1361).toFixed(2) = '1.14'

// After five, there is a count, go to one more
(1.1351).toFixed(2) = '1.14'

// After five, there are countless, look at the odd and even numbers, before five, it is 3, which is an odd number, and then go to one.
(1.1350).toFixed(2) = '1.14'

// after five, there are countless, look at odd and even numbers, before five, it is 0, even number, discard it
(1.1050).toFixed(2) = '1.10'

This seems fine, but:

// After five, if you have a count, you should move on to one more
(1.1051).toFixed(2) = 1.11 (Correct √)
(1.105).toPrecision(17) = '1.1050000000000000' // Precision

//After five, there are countless numbers. Look at odd and even numbers. Before five, it is an even number of 0 and should be discarded.
(1.105).toFixed(2) = 1.10 (Correct √)

// After five, there are countless numbers. Look at the odd and even numbers. The number before five is 2, which is an even number and should be discarded.
(1.125).toFixed(2) = 1.13 (Incorrect ×)
1.125.toPrecision(17) = '1.1250000000000000' // Precision

// After five, there are countless numbers. Look at the odd and even numbers. The number before five is 4, which is an even number and should be discarded.
(1.145).toFixed(2) = 1.15 (Incorrect ×)
1.145.toPrecision(17) = '1.1450000000000000' // Precision

// After five, there are countless numbers. Look at odd and even numbers. Before five, it is an even number of 6 and should be discarded.
(1.165).toFixed(2) = 1.17 (Incorrect ×)
1.165.toPrecision(17) = '1.1650000000000000' // Precision

// After five, there are countless numbers. Look at odd and even numbers. Before five, it is an even number of 8 and should be discarded.
(1.185).toFixed(2) = 1.19 (Incorrect ×)
1.185.toPrecision(17) = '1.1850000000000001' // Precision

toFixed standard defined by ECMAScript

Just explain it briefly:

  1. Let x = target number, such as: (1.145).toFixed(2) where x = 1.145 .
  2. Let f = parameter, such as: f = 2 in (1.145).toFixed(2).
  3. If f = undefined, that is, no parameters are passed, then f = 0.
  4. If f = Infinite, that is, infinite value is passed in, a RangeError exception will be thrown.
  5. If f < 0 or f > 100, the value passed in is not 0 - 100, a RangeError exception is thrown.
  6. If x = Infinite, that is, if you want to operate on inexact values reserved bits, return its string form.
    For example, Infinity.toFixed(2) = 'Infinity', NaN.toFixed(2) = 'NaN' .
  7. Let x = the mathematical value that the computer can represent ?(x)
    The conversion from a number or BigInt x to a mathematical value is expressed as the mathematical value of x, or ?(x)
  8. Let the return value symbol s = ” be the symbol’s initial value .
  9. If x < 0, then s = ‘-‘ and x = -x .
  10. If x ≥ 10^21, then return value m = string represented by scientific notation corresponding to x.

  11. If x < 10^21, then
    a. Let n = an integer, where n / 10^f - x is as close as possible to 0, if there are two such ns, select thelarger n.
    b. If n = Integer 0, then m = "0", otherwise, m = A string value consisting of numbers in the decimal representation of n (in order, without leading zeros).
    c. If exponent f ≠ 0, then k = m.length.
    1. Ifk ≤ f, then
      • z = String consisting of f + 1-k occurrences of code unit 0x0030 (DIGIT ZERO)
      • m = z + m
      • k = f + 1
    2. Let a = the first k-f code unit of m
    3. Let b = the other f coding units of m
    4. Change m = a + "." + b
  12. Returns a string composed of s + m

Example 1: (1.125).toFixed(2) = 1.13

  • Initialx = 1.125, f = 2, s = ”according to the above specification
  • According to specification 7 we know that x = 1.125.toPrecision(53) = 1.125

  • According to the formula provided in specification 11.a: n / 10^f - x ≈ 0 Substitute into the calculation: n ≈ 112.5:
    • At this time, the integer closest to n has two values of 112 and 113, take the largest 113 according to the standard
    • After following the specifications of 11.c, we get m = 1.13
  • Finally returns s + m= 1.13

Example 2: (-1.105).toFixed(2) = -1.10

  • According to the above specification initialx = 1.105, f = 2, s = ‘-‘
  • According to specification 7 we know that x = (-1.105).toPrecision(53) = 1.10499…

  • According to the formula provided in specification 11.a: n / 10^f - x ≈ 0 Substitute into the calculation: n ≈ 110.4...：
    • At this time, the integer closest to n is only one with the value 110 (Because only when the decimal point is 5, there will be two cases of rounding up/down)
    • After following the specifications of 11.c, we get m = 1.10
  • The final return is s + m= -1.13

Solution

Here are some common workarounds:

  1. Calculate using integers (zoom in, then zoom out):
    Since errors only occur when we encounter decimal calculations, we can first convert the decimal into an integer and then convert it into a decimal, so that there will be no accuracy problem. Convert floating point numbers to integers for calculations whenever possible.
    For example, convert a floating point number to an integer by multiplying the number of decimal places by a fixed multiple, perform the calculation, and then convert the result back to a floating point number. This can reduce precision issues in floating point calculations.

    Take 0.1 + 0.2 = 0.30000000000000004 as an example, as follows:

    1. Original formula:0.1 + 0.2 = x
    2. Expand 100 times: 0.1 * 100 + 0.2 * 100 = 100 * x
    3. Variation: 100 * x = 3
    4. Result: x = 0.3
      let num1 = 0.1,num2 = 0.2;
console.log((num1*100 + num2*100)/100); //0.3

      The limitation of this method is that you need to know how many decimal places the calculated number is.
      Andnot all floating point numbers are converted into integers, for example:

      (44.8976).toPrecision(53) There is an error between the four decimal places of the result after taking the precision and the actual value, causing the final product result to be an incorrect value as well.

  2. Use specialized libraries or tools:
    When dealing with scenarios that require high-precision calculations, some specialized libraries or tools can be used. For example, libraries such as Decimal.js, Big.js or BigNumber.js in JavaScript provide high-precision mathematical calculations that avoid the problem of precision loss.

    Check it out if necessary:

    1. math.js
    2. big.js
    3. bignumber.js
    4. decimal.js
  3. When the maximum value is exceeded, the interface returns the corresponding value in the form of String:
    For values beyond the safe range produced by the backend return or frontend calculation mentioned earlier, we can use BigInt To handle, this is a new primitive value type that provides a way to represent integers greater than 2^53 - 1.

    The BigInt here cannot be used to handle values returned by the backend that exceed the safe range (such as id), because when we need to convert these values into BigInt< /strong> Precision loss has already occurred before, so it is meaningless to perform the conversion. The best way at this time is to let the backend process the data so that the interface returns the corresponding value in the form of string.

    When the front end has to temporarily convert to a string, it can be handled like this:

    (1) Regular replacement:
    If we are using axios to request data, Axios provides a custom API for processing the data returned by the original backend: transformResponse, which can be processed like this: 

    axios({
method: method,
url: url,
data: data,
transformResponse: [function (data) {
    //Convert Long type data to string
    const convertedJsonString = data.replace(/"(\w + )":(\d{15,})/g, '"$1":"$2"');
    return JSON.parse(convertedJsonString);
}],
})


// Assume that the JSON data returned by the backend is as follows:
const responseData = {
  id: 12345678901234567890, // This is a Long type data
  name: "John Doe"
};

// Processed json data
console.log(responseData.id); // This will output the string: "12345678901234567890"
console.log(typeof responseData.id); // This will output "string"

    (2) json serialization processing
    You can use the third-party package json-bigint to handle this:
    The parse method in json-bigint will convert numbers beyond the JS safe integer range into a BigNumber type object. The object data is processed by an internal algorithm. All we have to do is convert it into a string when using it. use.
    By enabling the storeAsString option, you can quickly convert BigNumber to a string. The code is as follows: 

     import JSONbig from "json-bigint";
    axios({
    method: method,
    url: url,
    data: data,
    transformResponse: [function (data) {
 + const JSONbigToString = JSONbig({ storeAsString: true });
 + //Convert Long type data to string
 + return JSONbigToString.parse(data);
    }],
    })
    
    
    // Assume that the JSON data returned by the backend is as follows:
    const responseData = {
      id: 12345678901234567890, // This is a Long type data
      name: "John Doe"
    };
    
    // Processed json data
    console.log(responseData.id); // This will output the string: "12345678901234567890"
    console.log(typeof responseData.id); // This will output "string"
  4. Avoid direct comparison of floating point numbers:
    Due to precision issues, directly comparing floating point numbers may lead to inaccurate results. In situations where you need to compare floating point numbers, you can use error bounds for comparison instead of using exact equality judgments.
  5. Limit the number of decimal places:
    For some specific application scenarios, the number of decimal places in floating point numbers can be limited to reduce the impact of precision loss. For example, currency calculations are often limited to two decimal places.
  6. Use appropriate rounding strategies:
    Where rounding is required, choose an appropriate rounding strategy to meet actual needs. Common rounding strategies includerounding, rounding up, rounding down, etc.
    Number.prototype.toFixed=function (d) {
    var s=this + "";
    if(!d)d=0;
    if(s.indexOf(".")==-1)s + =".";
    s + =new Array(d + 1).join("0");
    if(new RegExp("^(-|\ + )?(\d + (\.\d{0," + (d + 1) + "})?)\d*$") .test(s)){
        var s="0" + RegExp.$2,pm=RegExp.$1,a=RegExp.$3.length,b=true;
        if(a==d + 2){
            a=s.match(/\d/g);
            if(parseInt(a[a.length-1])>4){
                for(var i=a.length-2;i>=0;i--){
                    a[i]=parseInt(a[i]) + 1;
                    if(a[i]==10){
                        a[i]=0;
                        b=i!=1;
                    }else break;
                }
            }
            s=a.join("").replace(new RegExp("(\d + )(\d{" + d + "})\d$"),"$1.$2");
 
        }if(b)s=s.substr(1);
        return (pm + s).replace(/\.$/,"");
    }return this + "";
 
}

  7. Note the numerical range:
    When performing floating point calculations, pay attention to the range of values. Values outside the representation range of floating point numbers may cause precision loss or overflow problems.

Reference article:

Online emergency bug: 80% of data accuracy issues that front-ends may encounter

The backend asks why the frontend numerical precision is lost? – Nuggets

How math.js handles decimal problems – Nuggets

Exploring JavaScript precision issues and solutions – Nuggets

[JS] About precision loss, causes and solutions_js addition loss of precision_swimxu’s blog-CSDN blog

syntaxbug.com © 2021 All Rights Reserved.