variableFilter sometimes includes aliases of selected variables

[christopher.schoelzel@…]

The simulation option variableFilter sometimes includes variables that do not match the regular expression but that are aliases of a variable that does.

Unfortunately my MWE is still 300 lines of Modelica code, since I found it quite hard to pinpoint why some variables show this behavior and some variables don't. Attached you will find one file where the variable filter works as expected (VariableFilterWorking.mo) and one where it includes four additional variables (VariableFilterBroken.mo). In both cases the outputFormat is "csv" and the variableFilter used is the following:

(n|an_nh)_(vc_v|naca_(i|E[1-4]|k_[1-4][1-4]))

For VariableFilterBroken you find the additional variables an_nh_naca_F1_3n_i, an_nh_naca_F1_3n_o, n_naca_F1_3n_i and n_naca_F1_3n_o, which are equal to one of the variables selected by the filter (e.g. through "Real n_naca_k_34 = n_naca_F1_3n_o;") but do not match the regex. Other variable aliases (e.g. n_naca_F_c_i with "Real n_naca_k_12 = n_naca_F_c_i;"), however, are not selected. I cannot find any pattern here, but I suspect it has something to do with variable naming and ordering since this is the only thing that changed between the working and the broken version.

I suspect that this is related to #3533 and the fact that this happens for some but not all variables and is related to aliasing could explain why this issue could not be reproduced.

This behavior was both tested with OMEdit and by directly calling the OMC through the scripting API. Version numbers are the following (from "About OMEdit" window):

Connected to OMCompiler v1.16.0-dev.252+gebc5dd384
Connected to OMSimulator v2.1.0-dev-155-gf11a7e4-linux

[christopher.schoelzel@…]

variableFilter (n|an_nh)_(vc_v|naca_(i|E[1-4]|k_[1-4][1-4])) selects additional variables

[christopher.schoelzel@…]

variableFilter (n|an_nh)_(vc_v|naca_(i|E[1-4]|k_[1-4][1-4])) only selects expected variables

[casella]

@christopher, #3533 was declared invalid 5 years ago. Are you sure the regular expressions are fine?

[christopher.schoelzel@…]

I double-checked the regex with regex101 before posting. It works as expected: ​https://regex101.com/r/yz3ZXx/2

You can also see this from the structure of the regex. The additional variables contain an uppercase F, which does not occur at all in the regex, and the regex does not have any part that matches multiple characters apart from [1-4].

Therefore the issue cannot be the regex. It must be the handling of variable aliases in OpenModelica. I encountered this problem several times before (also several years ago), but was never able to isolate it well enough to post a bug report. In the past I also was always able to change the regex slightly to achieve the result I wanted (meaning that I used an alternative regex that had the same result on regex101, but different results in OpenModelica). I think given how rarely this issue occurred for me, how difficult it was to reproduce or explain what was wrong, and that most users probably won't have many use cases where they need complex regular expressions in the variableFilter, I think it is plausible that this went unnoticed for so long.

[casella]

OK. We are currently busy with the 1.16.0 release bugfixing, there are many high-priority issues that need to be addressed. I'm scheduling this for the next release, we'll see when we can manage that.

Thanks!

[sjoelund.se]

Aliases of variables are included because aliases don't take any space in the result-file.

[christopher.schoelzel@…]

Replying to sjoelund.se:

Aliases of variables are included because aliases don't take any space in the result-file.

This is true for outputFormat="mat", but in CSV files I get a full duplicated column in the output for each additional variable. Also I do not see any consistent pattern which aliases are included and which are not.

One example: In variableFilterBroken.mo, an_nh_naca_k12 is selected by the regex and defined as "Real an_nh_naca_k_12 = an_nh_naca_F_c_i;", but an_nh_naca_F_c_i is not selected. Conversely, an_nh_naca_k_34 is selected and defined as "Real an_nh_naca_k_34 = an_nh_naca_F1_3n_o;" and here the alias an_nh_naca_F1_3n_o is also selected.

I am completely aware that this is not a high-priority issue by any means. I am also fine with having four additional columns in the data for my regression tests for now. Thank you for your time. =)

[sjoelund.se]

Yeah, we don't care much for the CSV files. If you want to have only a select few signals in a result-file, do some post-processing on it to filter only those ones:

​https://openmodelica.org/doc/OpenModelicaUsersGuide/latest/scripting_api.html#filtersimulationresults

It can take a mat-file and turn it into a CSV-file with only what you specify.

[casella]

Replying to sjoelund.se:

​https://openmodelica.org/doc/OpenModelicaUsersGuide/latest/scripting_api.html#filtersimulationresults
It can take a mat-file and turn it into a CSV-file with only what you specify.

Incidentally, we should probably add a couple of lines of explanation to the documentation of that API function to explain what it does. How can one know it outputs a CSV file?

[casella]

Retargeted to 1.18.0 because of 1.17.0 timed release.

[casella]

Ticket retargeted after milestone closed