aitchon
Posts: 118 Joined: Mon Jan 22, 2007 10:30 am
Post
by aitchon » Tue Jan 31, 2012 4:15 pm
I'm trying to get the long form of a number after adding 1 to it:
<$id="5835755762">
id:$id
<$maxid=(convert($id, 'double'))>
maxid:$maxid
<$maxid=($maxid+1)>
maxid:$maxid
<$convid=(convert($maxid, 'varchar'))>
convid:$convid
How can I get $convid to display 5835755763? It currently shows as 5.83576e+09.
mark
Site Admin
Posts: 5513 Joined: Tue Apr 25, 2000 6:56 pm
Post
by mark » Tue Jan 31, 2012 4:48 pm
<fmt "%f" $convid> should do it.
John
Site Admin
Posts: 2597 Joined: Mon Apr 24, 2000 3:18 pm
Location: Cleveland, OH
Contact:
Post
by John » Tue Jan 31, 2012 4:51 pm
Also if you are using Texis Version 6 you can use int64 instead of double:
<$id="5835755762">
id:$id
<$maxid=(convert($id, 'int64'))>
maxid:$maxid
<$maxid=($maxid+1)>
maxid:$maxid
John Turnbull
Thunderstone Software
aitchon
Posts: 118 Joined: Mon Jan 22, 2007 10:30 am
Post
by aitchon » Tue Jan 31, 2012 4:52 pm
Adding this gave me:
5835760000.000000
It looks like converting it rounds it to the nearest thousand.
aitchon
Posts: 118 Joined: Mon Jan 22, 2007 10:30 am
Post
by aitchon » Tue Jan 31, 2012 4:53 pm
I'm using 5.01 and would like to see if there's a workaround with this version.
Kai
Site Admin
Posts: 1271 Joined: Tue Apr 25, 2000 1:27 pm
Post
by Kai » Tue Jan 31, 2012 5:44 pm
Print it with <fmt "%.0f\n" $maxid> to suppress/round-off past the decimal.
aitchon
Posts: 118 Joined: Mon Jan 22, 2007 10:30 am
Post
by aitchon » Tue Jan 31, 2012 6:06 pm
I believe the suppressing is happening in the convert($id, 'double').
Kai
Site Admin
Posts: 1271 Joined: Tue Apr 25, 2000 1:27 pm
Post
by Kai » Wed Feb 01, 2012 10:03 am
Doubles can have up to about 16 decimal digits of precision in the mantissa, so the convert varchar to double is not losing any digits; you'll see this if you print $maxid after the convert to double with <fmt "%.0f\n" $maxid>.
I'd have to see your full exact code for msg #4, but there's probably a convert double to varchar (perhaps implicitly) happening, which would use %g, which only uses 6 digits of precision by default -- that would cause the roundoff.
In your msg #1 example, just change:
<$convid=(convert($maxid, 'varchar'))>
convid:$convid
to:
<fmt "%.0f\n" $maxid>
and you'll see the full precision of $maxid.