Best way of ensuring floating point precision?

I need to ensure I get floating point precision for division where the values I'm dividing could both be integers. The problem I'm seeing is if the two numbers are integers, the result isn't as accurate as I need it. I need at least precision to three decimal places.

 

 

 

Ignoring the whole "wide integer" case for the moment, if Tcl sees both sides of the "/" operator as integers, it will do integer division. If either operand is a double, then it will (in a temporary place) promote the other operand to a double if it isn't already and perform division in the usual way. So, to get what you want, you just need to force one side to be a double...

 

 

Here are a few options I've tested and the results:

 

 

 

And as you've found, there are several ways to do this, with varying speeds. Now, there is an important bug in your program that you posted. The last case you display ("5. / 4") is not the last case you test; you're testing "[format %f 5.] / 4" in the loop. You presumably copy/pasted and only changed the calculation once instead of twice. If you change all your log messages to log the actual value of $test, then you only have the actual calculation in one place (inside the for loop), which would avoid that sort of error. I've corrected this and run your program on my (low-end) box; results below.

 

 

There are plenty of references to this in the expr man page and on DC (Click here). No one seems to have solid recommendations for forcing floating precision though.

 

 

 

 

That page actually does have the Tcl way to do what you want; the double() command. I've added it to your test program, also. Here are my results:

 

 

====================================================================================

 

568329753

 

Integer division:

 

[expr {5 / 4}]: 1. Time: 18

 

 

Forcing both operands via format:

 

[expr {[format %f 5] / [format %f 4]}]: 1.25. Time: 182

 

 

Forcing one operand via format:

 

[expr {[format %f 5] / 4}]: 1.25. Time: 119

 

 

Forcing one operand via ".":

 

[expr {5. / 4}]: 1.25. Time: 19

 

 

Forcing one operand the official way, using double():

 

[expr {double(5) / 4}]: 1.25. Time: 18

 

568330109

 

====================================================================================

 

 

Forcing the operand via "." is not a hack, by the way; the dot tells Tcl that it's not an integer, and so it next tries to parse it as a double, and succeeds. That said, the best way is probably using double(), as it's just as fast as anything else and is the most explicit so as to let future maintainers (like maybe you, six months from now) know exactly what you're doing.