Opened 11 years ago

Closed 11 years ago

#2517 closed defect (fixed)

wrong computation in OMShell

Reported by: Peter Junglas Owned by: Per Östlund
Priority: high Milestone: 1.9.1
Component: Interactive Environment Version: trunk
Keywords: Cc:

Description

Entering the following in OMShell

a := 294.7;
b := 294.8;
t1 := abs(a-b)/b;
t2 := abs(a-b)/abs(b)

leads to the result

294.7
294.8
0.0003392130257802671
29.480000000006704

which is obviously wrong.

Change History (2)

comment:1 by Per Östlund, 11 years ago

Component: Backend → Interactive Environment
Owner: changed from probably noone to Per Östlund
Status: new → assigned

comment:2 by Per Östlund, 11 years ago

Resolution: → fixed
Status: assigned → closed

Fixed in r18494.

Note: See TracTickets for help on using tickets.