Opened 6 years ago

Closed 6 years ago

Last modified 6 years ago

#5050 closed defect (fixed)

OMEdit badly truncates numerical values

Reported by: massimo ceraolo Owned by: Adeel Asghar
Priority: normal Milestone: 1.13.2
Component: OMEdit Version:
Keywords: Cc:

Description (last modified by massimo ceraolo)

I have a machine that has as Lstray the value

0.0006480774127832951

If I look at it in the Variable browser while limiting the available horizontal space, the number is truncated in its internal part. For instance it might appear as:

0.000...32951 or 0.0006...832951

which is illogic.

Numerical values should be truncated in their right (not the internal) part.

Change History (12)

comment:1 by massimo ceraolo, 6 years ago

Summary: OMEdit badly truncates nuercial valuesOMEdit badly truncates numercial values

comment:2 by massimo ceraolo, 6 years ago

Description: modified (diff)

comment:3 by massimo ceraolo, 6 years ago

... unless they are in exponential form, in which case dropping the internal part when the available space is insufficient makes sense.

comment:4 by Francesco Casella, 6 years ago

Summary: OMEdit badly truncates numercial valuesOMEdit badly truncates numerical values

comment:5 by Francesco Casella, 6 years ago

Milestone: 1.13.01.14.0

Rescheduled to 1.14.0 after 1.13.0 releasee

comment:6 by Adeel Asghar, 6 years ago

Milestone: 1.14.01.13.2
Resolution: fixed
Status: newclosed

Done in bef7e4f/OMEdit.

comment:7 by massimo ceraolo, 6 years ago

Looking at your code it seems that you did not consider comment3

comment:8 by Adeel Asghar, 6 years ago

Yeah that's true. I didn't even noticed the comment.

Well its hard to know if the number is a regular decimal value or is in exponential form. I could use some regular expressions to check for it but is it really needed?

comment:9 by Francesco Casella, 6 years ago

I guess something like

if strstr(number, "e") || strstr(number,"E")
  {
  // truncate in the middle, keep last 4 characters
  }
else
  {
  // truncate at the end
  }

would do :)

in reply to:  8 comment:10 by massimo ceraolo, 6 years ago

Well its hard to know if the number is a regular decimal value or is in exponential form. I could use some regular expressions to check for it but is it really needed?

It is not very much needed, but would be a nice feature to have. In other words would enhance the quality of OMEdit.

In addition to the proposal by Francesco, a simple option is to use

QString::setNum(double n, char format = 'g', int precision = 6)

with 'g' format and variable precision.
This should be safer, since would work with systems having any number of digits, and whether they use or not a digit for positive signs, as exponent.

Version 1, edited 6 years ago by massimo ceraolo (previous) (next) (diff)

comment:11 by Adeel Asghar, 6 years ago

I don't prefer to modify the values received from omc. I prefer Francesco's solution but of course without assuming that the characters in the number are always same.

ce509a3/OMEdit fixes this.

in reply to:  11 comment:12 by massimo ceraolo, 6 years ago

Replying to adeas31:

I don't prefer to modify the values received from omc. I prefer Francesco's solution but of course without assuming that the characters in the number are always same.

ce509a3/OMEdit fixes this.

Good. I did not know that you receive strings (not numbers)

Note: See TracTickets for help on using tickets.