Problems with variableFilter

[nimalk@…]

In order to simulate my models and save the resulting plt files, I use a bat script.

However, simulating a Multibody model for one second generates a plt file of almost 100 mb (I have a model with a discrete component and a high sample rate).

As I have to simulate 100 seconds I would have to save 10 GB of unnecessary data. Furthermore, it isn't even possible to simulate 100 s: messages = "Simulation execution failed for model: HangingMass2 assert | assert | Error allocating simulation result data of size 400000000 failed.

In order to reduce the plt file size I tried to use the extra argument variableFilter. Nevertheless, all the variables are still included in the plt file and the size remains unchanged.

To test the variableFilter I have created the following model:

model HangingMass2
  Real x;
  inner Modelica.Mechanics.MultiBody.World world(g = 0) annotation(Placement(visible = true, transformation(origin = {-37.2454,36.0839}, extent = {{-10,-10},{10,10}}, rotation = 0)));
  Modelica.Mechanics.MultiBody.Parts.Body body(r_CM = {0,0.5,0}, m = 16.5, r_0(start = {0,-2.5,0}, each fixed = true)) annotation(Placement(visible = true, transformation(origin = {32.3982,20.6998}, extent = {{-10,-10},{10,10}}, rotation = 0)));
equation
  x = 1;
end HangingMass2;

I then attempted to filter the variable x.

simulate(%MODELNAME%, startTime=%2, stopTime=%3, numberOfIntervals=%INTERVALS%, tolerance=0.000001, method="euler", outputFormat="plt", variableFilter="x");

Please note that the above model is not exactly the same as the one I use to filter, i.e. the 10 GB model is not the one above. Due to the complexity of running my model without the source code and some copyright issues I cannot attach the model.

Furthermore, I would be grateful if you would kindly show me how to filter the variable body.r_0[1] (what should the syntax be?)

[Adrian Pop]

What are the settings you're using?
I mean I need to know %2, %3, %INTERVALS% you use when calling the .bat file.

[Bill Janssen]

Just read this code yesterday. You need to know that the format of the variableFilter is POSIX extended regular expressions. It will add the variables which match the regular expression, plus all alias variables for those variables.

[Adrian Pop]

Note that for .plt files the parameters are put in the file for each time step and that would make the files very large if you have a lot of intervals and a big stopTime.
I talked with Martin (sjoelund.se) about this and we should maybe filter our the parameters too for .plt and .csv. He'll have a look at this tomorrow.

Are you restricted to use .plt files or can you use .mat? Using .mat should work better for you.

[nimalk@…]

Replying to adrpo:

The .mat files will work for me as well. However, I tried to save a .mat file and I managed to reduce the file size. But, there are still some issues to be solved. With the filter most of the variables are still included in the .mat file.

I am trying to filter the variable x with the argument variableFilter = "x". The .mat file without the filter contains all variables and the .mat file with the filter removes the variables $STATESET1.der(x[1:3]). Nevertheless, the variable filter is POSIX. It should only keep the variable x and not remove any variables with the name x,… or…?

Please see the attached files.

What about the variables which are protected like the variables related to the visualization? Will they be removed from the .mat file?

Furthermore, I would be grateful if you would kindly show me how to filter the variable body.r_0[1] (what should the syntax be?). I am actually only interested in the position, toque and force for all my models.

[Martin Sjölund]

In my local copy too many alias variables are no longer output (earlier, even aliases that did not match the regex would be output).

-rw-rw-r-- 1 martin martin  29K Dec 12 09:53 HangingMass2_filtered.mat
-rw-rw-r-- 1 martin martin 2.4M Dec 12 09:53 HangingMass2_res.mat
0
""
"113 variables+parameters"
""
"946 variables+parameters"
""

I will also try to filter out parameters and aliases for csv/plt since aliases and parameters take up more space in these formats

[Martin Sjölund]

Only time+x in the data_2 matrix

octave:1> load HangingMass2_filtered
octave:2> who
Variables in the current scope:

Aclass       ans          dataInfo     data_1       data_2       description  name

octave:3> size(data_2)
ans =

     2   502

octave:4> size(data_1)
ans =

   112     2

[Martin Sjölund]

-rw-rw-r-- 1 martin martin  28K Dec 12 09:59 HangingMass2_filtered.plt
-rw-rw-r-- 1 martin martin 8.7M Dec 12 09:59 HangingMass2_res.plt
0
""
"2 variables+parameters"
""
"835 variables+parameters"
""

-rw-rw-r-- 1 martin martin  10K Dec 12 10:28 HangingMass2_filtered.csv
-rw-rw-r-- 1 martin martin 1.1M Dec 12 10:28 HangingMass2_res.csv
0
""
"2 variables+parameters"
""
"835 variables+parameters"
""

Fixed in r18466.

[Martin Sjölund]

Replying to nimalk@…:

Nevertheless, the variable filter is POSIX. It should only keep the variable x and not remove any variables with the name x,… or…?

Only things that match the full regex

What about the variables which are protected like the variables related to the visualization? Will they be removed from the .mat file?

We don't check for protected variables

[nimalk@…]

Replying to sjoelund.se:

Great!

When can I download a new Windows version of OpenModelica with the new fix?

[Martin Sjölund]

​https://build.openmodelica.org/omc/builds/windows/nightly-builds/ suggests around 4:50 tomorrow morning (Stockholm time)