# Peculiar rounding behavior

Discussion in 'Maple' started by Mr. Neutron, Aug 3, 2004.

1. ### Mr. NeutronGuest

Immediately after starting Maple 9.03, type:

evalf[14](arctan(387.00000001642));

and get: 1.5682123532178

The correct 19 digit value is: 1.5682123532178(50715)

Now edit the "14" in the square brackets and execute again:

evalf[17](arctan(387.00000001642));

and get: 1.5682123532178507

Now re-edit the [17] back to [14] and execute:

evalf[14](arctan(387.00000001642));

and get: 1.5682123532179

What's going on here? Maple forgot how to round until shown what the next 3 digits are?
Then it remembered?

Mr. Neutron, Aug 3, 2004

2. ### Robert IsraelGuest

This is acceptable: the last digit is no more than 1 away
from the correctly rounded result.
"Remembered" is the right word. When evalf calculates something, its
remember table stores the expression calculated, the result and the number
of digits, e.g. arctan(387.00000001642) = (1.5682123532178, 14). If you
later calculate the same expression to more digits, this entry will
be replaced by the more precise one. When you later ask evalf to
calculate the same expression to 14 digits again, Maple first looks
in the remember table, sees that this expression was already calculated
with at least the desired precision, and instead of repeating the
calculation it rounds the remembered result to 14 digits.

Robert Israel
Department of Mathematics http://www.math.ubc.ca/~israel
University of British Columbia

Robert Israel, Aug 3, 2004

3. ### Mr. NeutronGuest

1.5682123532178 is more than 1/2 ULP away from the correct value; 1.5682123532179 is
less than 1/2 ULP away from the correct value. The ...179 result is what the IEEE 754 (or
854) floating point standard requires to be returned. I've not seen any other first-rate
(Mathematica, Matlab, Gauss, etc.) program that doesn't return the more accurate result.
It's not difficult to get better than 1 ULP error.
I'm very disappointed to discover that Maple behaves in this way. I was under the
mistaken impression that Maple conformed to the requirements of the IEEE floating point
standard, from which I quote:

"An implementation of this standard shall provide round to nearest as the default
rounding mode. In this mode the representable value nearest to the infinitely precise
result shall be delivered."

In the case at hand the "representable value" would be the 14-digit result. The
nearest (14 digit) representable value to the infinitely precise result is
1.5682123532179, not 1.5682123532178.

I discovered this because I was checking the performance of a number of hand-held
calculators by comparing the result of a long looped sequence of computations to the
result given by Maple. In one case, the calculators all agreed with each other, but not
with Maple. After several hours of comparing partial results, I discovered that Maple was
not rounding correctly (by "correctly", I mean in conformance to the IEEE standard) at one
point.

Mr. Neutron, Aug 3, 2004
4. ### Robert IsraelGuest

Robert Israel, Aug 4, 2004
5. ### Mr. NeutronGuest

Mr. Neutron, Aug 4, 2004
6. ### Joe RielGuest

You are misapplying the standard. This accuracy is not required for any
mathematical operation, but rather for those mentioned in the standard,
that is, arithmetic operations (+,-,*,/, and rem), square roots,
floating point conversion, and a few others. No mention of trig functions.

Joe Riel

Joe Riel, Aug 11, 2004
7. ### Mr. NeutronGuest

You haven't shown the full context of my quote. I was responding to Prof. Israel who
said:

"I don't see how they could guarantee being less than 1/2 ULP away from
the correct value in all cases. What if the correct value was exactly

1.56821235321785 ?"

And I responded with:

' Expanding the quote from the IEEE standard:

"An implementation of this standard shall provide round to nearest as the default
rounding mode. In this mode the representable value nearest to the infinitely precise
result shall be delivered; if the two nearest representable values are equally near, the
one with its least significant digit even shall be delivered."'

This had nothing to do with trig functions, but just the question of what to do when
the LSD is 5 followed by an infinite number of zeroes.

I was aware that the IEEE standard does not provide specific requirements for functions
such as SIN, EXP, LOG, etc.

And I further said, concerning Maple's returned values for such functions:

" I think Maple is, in fact, rounding properly; in this case it just isn't looking at
enough digits past those requested."

Mr. Neutron, Aug 26, 2004