Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
double precision or rounding numbers?
#1
Now that I'm pulling my hair out looking -everywhere- for a simple answer.......

If I have double variable with a value of something like 14.94454592, how do I get an output that's rounded off to 14.9.

Thanks for the help! :?
#2
Use round() function of VARIANT.

Code:
Copy      Help
double d=14.94454592
VARIANT v.round(d 1)
d=v
out d

Or use format() function of str.

Code:
Copy      Help
double d=14.94454592
d=val(_s.format("%.1f" d) 2)
out d
#3
wow! are you great or what!

I used the first example you posted with the VARIANT method. One minor question though... If the double variable happens to be a whole number, the output is that number plus a ".". I would prefer no "." or a ".00" on the end, e.g. 14.00. How do I make this happen?

Thanks! Big Grin
#4
If you need only to display the variable, don't use VARIANT etc. To display with 2 digits after decimal, use this:

out "%.2f" d

To display without extra 0 and decimal point, use this:

Create member function (menu File -> New -> New Member Function) str.FormatDouble and paste this code (menu Edit -> Other formats -> Paste escaped):
Code:
Copy      Help
function double'd precision

;Converts double (floating point) value to string so that it will not contain extra 0 or decimal point at the end.

;precision - max number of digits after decimal point.

;EXAMPLE
;double d=14.94454592
;str s.FormatDouble(d 2)
;out s



format(_s.from("%." precision "f") d)
if(precision) rtrim("0"); rtrim(".")

Use it like in the EXAMPLE.


Forum Jump:


Users browsing this thread: 1 Guest(s)