Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

A lot of Excel's issues are well-known because of how pervasive it is. Python and kin rely on C libraries and other layers of imported packages for numerical work that can lead to all sorts of numerical mischief. Even doing:

  round (2.575, 2)
in Python leads to 2.57 instead of rounding up to 2.58. Everything in our engineering firm is checked by another engineer.


    In [7]: round(2.575, 2)
    Out[7]: 2.58
Umm?


From the Python 3.7 docs (https://docs.python.org/3/library/functions.html#round):

The behavior of round() for floats can be surprising: for example, round(2.675, 2) gives 2.67 instead of the expected 2.68. This is not a bug: it’s a result of the fact that most decimal fractions can’t be represented exactly as a float.


Well that is just normal float fun :)

    In [17]: Decimal(2.675)
    Out[17]: Decimal('2.67499999999999982236431605997495353221893310546875')
So sort of round() is working as expected, but the number you are not inputting is not the one you are expecting.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: