Please Do This at Home
Once Upon a Time
var x = (4 * y) + "px";
floating-point constant
float-to-string conversion
floating-point multiplication
string concatenation
var x = (4 * y) + "px";
floating-point constant
float-to-string conversion
floating-point multiplication
string concatenation
var x = (4 * y) + "px";
floating-point constant
float-to-string conversion
floating-point multiplication
string concatenation
-
Unknown but used
-
Dragon4 (Steele and White, 1990)
-
Improved by Gay (1990), Burger and Dybvig(1996)
-
-
Grisu family (Loitsch, 2010)
-
Errol3 (Andrysco, Jhala, and Lerner, 2016)
-
Ryu (Ulf Adams, 2018)
-
Schubfach and Dragonbox (2017 to 2021)
History
But Why!?
-
Multiple formats use strings to represent floats
-
SVG
-
CSV
-
JSON
-
-
There are more and more use for these
-
Graphs in liveview
-
Monitoring use a lot of floats
-
IoT
-
ML/Data Science
-
-
CPU bottleneck !
-
not network, disk or memory
-
Does it matters ?
0.3
0
01101
0011001101
0
sign
exponent
mantissa
(-1)^s * (1.mantissa)2 * 2^(exponent - bias)
01101
0011001101
0.300048828125
exactly
0
sign
exponent
mantissa
01101
0011001101
0.300048828125
exactly
0.2999267578125
0.3001708984375
0.3
0.300048828125
0.3?
-
Information preservation: a correct parser must return the original floating point value from the output.
-
Minimum-length output: the output string must be as short as possible.
- Correct rounding: the output string must be as close to the input as possible
Rules
-
Float_to_*/1,2 BIFs
- Do not do shortest, only %f, %e
- Use libc sprintf and not interested in changing
- io_lib_format:fwrite_g/1
- Private API for io_lib:format, shell, pretty printer
-
But used! Poison, Jason, jsx,
- DO NOT USE JSONE, it is wrong there
- Implemented in Erlang!
- Easy to change !
- PR 2960, funded by the EEF
On the BEAM ?
But How!?
-
Unknown but used
-
Dragon4 (Steele and White, 1990)
-
Improved by Gay (1990), Burger and Dybvig(1996)
-
-
Grisu family (Loitsch, 2010)
-
Errol3 (Andrysco, Jhala, and Lerner, 2016)
-
Ryu (Ulf Adams, 2018)
-
Schubfach and Dragonbox (2017 to 2021)
GO FAST
Most libc and stdlib
SUPER FAST
-
Float_to_*/1,2 BIFs
- New BIF option: pretty
- WIP (windows compilation failing)
- Already 4x faster, for a total of 20x before
- hopefully in OTP24.1
- PR 4719, funded by the EEF
-
Look for it and the pretty option
-
Even faster ? Maybe but no real interest for me or OTP right now
-
Would need to replace printf, but maintainance.
-
Dragonbox....
-
Noone want to work on libc
-
Can we go faster ?
-
Would need to replace printf, but maintainance.
-
Dragonbox....
-
Noone want to work on libc
-
String to float maybe....
Even faster and more ?
-
There is lot of dragon area in the BEAM
-
There are totally something you can go in and help !
-
That would help everyone
-
We need funding to help these
What to get from this ?
Dragons are fun and friendly
Please Do This at Home
Sources
-
Stephan T. Lavavej, “Floating-Point ＜charconv＞: Making Your Code 10x Faster With C++17's Final Boss”
-
Ulf Adams, "Ryū: Fast Float-to-String Conversion"
- https://github.com/ulfjack/ryu
- https://github.com/erlang/otp/pull/2960
- https://github.com/erlang/otp/pull/4719
-
C to Erlang is not obvious and not pretty
-
Hopefully the BIF version will help clean this up
-
-
Special case for (small) integers
-
Tests tests tests
-
The erlang version was old and probably written at the same time as the tests
-
The tests did not fail properly in case of failure, the failure was swallowed
-
Saved by compiler test suite....
-
Hopefully better now, we added a lot of tests
-
-
OTP build and tests systems are really really not user friendly
Bonus 1
The Pain points
3000488281250.
2999267578125.
3001708984375.
Bonus 2
3000488281250
2999267578125
3001708984375
Bonus 2
300
299
300
Bonus 2
0.3