Opened 10 years ago

Closed 10 years ago

#2517 closed defect (fixed)

wrong computation in OMShell

Reported by: peterjunglas Owned by: perost
Priority: high Milestone: 1.9.1
Component: Interactive Environment Version: trunk
Keywords: Cc:

Description

Entering the following in OMShell

a := 294.7;
b := 294.8;
t1 := abs(a-b)/b;
t2 := abs(a-b)/abs(b)

leads to the result

294.7
294.8
0.0003392130257802671
29.480000000006704

which is obviously wrong.

Change History (2)

comment:1 Changed 10 years ago by perost

  • Component changed from Backend to Interactive Environment
  • Owner changed from probably noone to perost
  • Status changed from new to assigned

comment:2 Changed 10 years ago by perost

  • Resolution set to fixed
  • Status changed from assigned to closed

Fixed in r18494.

Note: See TracTickets for help on using tickets.