Skip to content

Rounding errors when parsing floats #73

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
abaco opened this issue May 23, 2018 · 2 comments · Fixed by #142
Closed

Rounding errors when parsing floats #73

abaco opened this issue May 23, 2018 · 2 comments · Fixed by #142
Labels
type: bug Something that should function correctly isn't.

Comments

@abaco
Copy link

abaco commented May 23, 2018

I was running some generative tests on my code and I ran into an issue with high precision floats. For example:

> x = 0.7531531167929774
> runParser (show x) testTokenParser.float
(Right 0.7531531167929775)

Note the final 5 instead of 4. The problem seems to be in the code that parses the decimal part of a number. Indeed, the following equivalent code gives the same erroneous result:

> digits = [7,5,3,1,5,3,1,1,6,7,9,2,9,7,7,4]
> foldr (\d f -> (f + toNumber d) / 10.0) 0.0 digits
0.7531531167929775

I'm not sure about the best way to modify fraction so as to avoid rounding errors. Maybe using the native parseFloat?

@jamesdbrock
Copy link
Member

Oh yeah that looks like a bug, thanks for the bug report.

@jamesdbrock jamesdbrock added the type: bug Something that should function correctly isn't. label Oct 8, 2021
jamesdbrock added a commit that referenced this issue Jan 13, 2022
jamesdbrock added a commit that referenced this issue Jan 23, 2022
@jamesdbrock jamesdbrock linked a pull request Jan 23, 2022 that will close this issue
4 tasks
@jamesdbrock
Copy link
Member

Hi @abaco I hope #142 solves your problem. Please reopen this issue if it doesn't.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug Something that should function correctly isn't.
Development

Successfully merging a pull request may close this issue.

2 participants