Please Do This at Home
Once Upon a Time
var x = (4 * y) + "px";
floating-point constant
float-to-string conversion
floating-point multiplication
string concatenation
var x = (4 * y) + "px";
floating-point constant
float-to-string conversion
floating-point multiplication
string concatenation
var x = (4 * y) + "px";
floating-point constant
float-to-string conversion
floating-point multiplication
string concatenation
-
Unknown but used
-
Dragon4 (Steele and White, 1990)
-
Improved by Gay (1990), Burger and Dybvig(1996)
-
-
Grisu family (Loitsch, 2010)
-
Errol3 (Andrysco, Jhala, and Lerner, 2016)
-
Ryu (Ulf Adams, 2018)
-
Schubfach and Dragonbox (2017 to 2021)
History
But Why!?
-
Multiple formats use strings to represent floats
-
SVG
-
CSV
-
JSON
-
-
There are more and more use for these
-
Graphs in liveview
-
Monitoring use a lot of floats
-
IoT
-
ML/Data Science
-
-
CPU bottleneck !
-
not network, disk or memory
-
Does it matters ?
0.3
0
01101
0011001101
0
sign
exponent
mantissa
(-1)^s * (1.mantissa)2 * 2^(exponent - bias)
01101
0011001101
0.300048828125
exactly
0
sign
exponent
mantissa
01101
0011001101
0.300048828125
exactly
0.2999267578125
0.3001708984375
0.3
0.300048828125
0.3?
-
Information preservation: a correct parser must return the original floating point value from the output.
-
Minimum-length output: the output string must be as short as possible.
- Correct rounding: the output string must be as close to the input as possible
Rules
-
Float_to_*/1,2 BIFs
- Do not do shortest, only %f, %e
- Use libc sprintf and not interested in changing
- io_lib_format:fwrite_g/1
- Private API for io_lib:format, shell, pretty printer
-
But used! Poison, Jason, jsx,
- DO NOT USE JSONE, it is wrong there
- Implemented in Erlang!
- Easy to change !
- PR 2960, funded by the EEF
On the BEAM ?
But How!?
-
Unknown but used
-
Dragon4 (Steele and White, 1990)
-
Improved by Gay (1990), Burger and Dybvig(1996)
-
-
Grisu family (Loitsch, 2010)
-
Errol3 (Andrysco, Jhala, and Lerner, 2016)
-
Ryu (Ulf Adams, 2018)
-
Schubfach and Dragonbox (2017 to 2021)
GO FAST
Most libc and stdlib
SUPER FAST
-
Float_to_*/1,2 BIFs
- New BIF option: pretty
- WIP (windows compilation failing)
- Already 4x faster, for a total of 20x before
- hopefully in OTP24.1
- PR 4719, funded by the EEF
-
Look for it and the pretty option
-
Even faster ? Maybe but no real interest for me or OTP right now
-
Would need to replace printf, but maintainance.
-
Dragonbox....
-
Noone want to work on libc
-
Can we go faster ?
-
Would need to replace printf, but maintainance.
-
Dragonbox....
-
Noone want to work on libc
-
String to float maybe....
Even faster and more ?
-
There is lot of dragon area in the BEAM
-
There are totally something you can go in and help !
-
That would help everyone
-
We need funding to help these
What to get from this ?
Dragons are fun and friendly
Please Do This at Home
Sources
-
Stephan T. Lavavej, “Floating-Point ＜charconv＞: Making Your Code 10x Faster With C++17's Final Boss”
-
Ulf Adams, "Ryū: Fast Float-to-String Conversion"
- https://github.com/ulfjack/ryu
- https://github.com/erlang/otp/pull/2960
- https://github.com/erlang/otp/pull/4719
-
C to Erlang is not obvious and not pretty
-
Hopefully the BIF version will help clean this up
-
-
Special case for (small) integers
-
Tests tests tests
-
The erlang version was old and probably written at the same time as the tests
-
The tests did not fail properly in case of failure, the failure was swallowed
-
Saved by compiler test suite....
-
Hopefully better now, we added a lot of tests
-
-
OTP build and tests systems are really really not user friendly
Bonus 1
The Pain points
3000488281250.
2999267578125.
3001708984375.
Bonus 2
3000488281250
2999267578125
3001708984375
Bonus 2
300
299
300
Bonus 2
0.3
Bonus 2
Here be Floating Dragons: A Tale of Making OTP Faster
By di4nao
Here be Floating Dragons: A Tale of Making OTP Faster
When you spend enough time in the programming community, you learn that a few things are full of dangers. Floating Point ? Here Be Dragons. String representations ? Here Be Dragons. Multi Platform Virtual Machines? Here Be Dragons. Here is the tale of how i replaced a Dragon algorithm with another Dragon algorithm, to convert floating points numbers to string faster inside OTP. Hopefully this will show you that dragons are not impossible to deal with. And that you can contribute to OTP too.
