#5050 closed defect (fixed)
OMEdit badly truncates numerical values
Reported by: | massimo ceraolo | Owned by: | Adeel Asghar |
---|---|---|---|
Priority: | normal | Milestone: | 1.13.2 |
Component: | OMEdit | Version: | |
Keywords: | Cc: |
Description (last modified by )
I have a machine that has as Lstray the value
0.0006480774127832951
If I look at it in the Variable browser while limiting the available horizontal space, the number is truncated in its internal part. For instance it might appear as:
0.000...32951 or 0.0006...832951
which is illogic.
Numerical values should be truncated in their right (not the internal) part.
Change History (12)
comment:1 by , 6 years ago
Summary: | OMEdit badly truncates nuercial values → OMEdit badly truncates numercial values |
---|
comment:2 by , 6 years ago
Description: | modified (diff) |
---|
comment:3 by , 6 years ago
comment:4 by , 6 years ago
Summary: | OMEdit badly truncates numercial values → OMEdit badly truncates numerical values |
---|
comment:6 by , 6 years ago
Milestone: | 1.14.0 → 1.13.2 |
---|---|
Resolution: | → fixed |
Status: | new → closed |
Done in bef7e4f/OMEdit.
follow-up: 10 comment:8 by , 6 years ago
Yeah that's true. I didn't even noticed the comment.
Well its hard to know if the number is a regular decimal value or is in exponential form. I could use some regular expressions to check for it but is it really needed?
comment:9 by , 6 years ago
I guess something like
if strstr(number, "e") || strstr(number,"E") { // truncate in the middle, keep last 4 characters } else { // truncate at the end }
would do :)
comment:10 by , 6 years ago
Well its hard to know if the number is a regular decimal value or is in exponential form. I could use some regular expressions to check for it but is it really needed?
It is not very much needed, but would be a nice feature to have. In other words would enhance the quality of OMEdit.
In addition to the proposal by Francesco, a simple option is to use
QString::setNum(double n, char format = 'g', int precision = 6)
with 'g' format and variable precision.
This should be safer, since would work with systems having any number of digits, and whether they use or not a digit for positive signs, as exponent.
follow-up: 12 comment:11 by , 6 years ago
I don't prefer to modify the values received from omc. I prefer Francesco's solution but of course without assuming that the characters in the number are always same.
ce509a3/OMEdit fixes this.
comment:12 by , 6 years ago
Replying to adeas31:
I don't prefer to modify the values received from omc. I prefer Francesco's solution but of course without assuming that the characters in the number are always same.
ce509a3/OMEdit fixes this.
Good. I did not know that you receive strings (not numbers)
... unless they are in exponential form, in which case dropping the internal part when the available space is insufficient makes sense.