Wrong simulation results of ScalableTestSuite StringModelica models with the NF

[Francesco Casella]

The StringModelica models of the ScalableTestSuite compute slightly (but distinctly) different simulation results when run with the NF, when compared to the reference results obtained by the old FE, see, e.g. ​StringModelica_N_2.

I checked the simulation results with Dymola and indeed the ones obtained with the old FE are correct, while those obtained with the NF are wrong.

The model is definitely too large to manually compare the output of the flattening directly - I then analyzed the equations after processing in the back-end.

When using the old FE, at the end of the front-end processing there are 164 equations, while the NF leads to 180 equations. The reason of this structural difference is that when the old FE is used, parameters such as bodybox[2].frameTranslation.r[1] are eventually evaluated by the back-end, while if the NF is used, they are not, and since many of them are zero, the end result is a structurally simpler system of equations in the first case.

I tried to make the equations more similar by using -d=evaluateAllParameters, so I could compare the back-end output directly, but unfortunately this doesn't work in the NF, see #5193

When looking at the flattened models, both front-ends produce

  parameter Integer N = 2 \"number of joints\";
  parameter Real L(quantity = \"Length\", unit = \"m\") = 0.5 \"length of the string\";
  parameter Real bodybox[2].frameTranslation.r[1](quantity = \"Length\", unit = \"m\", start = 0.0) = bodybox[2].r[1] \"Vector from frame_a to frame_b resolved in frame_a\";
  parameter Real bodybox[2].r[1](quantity = \"Length\", unit = \"m\", start = 0.1) = l \"Vector from frame_a to frame_b resolved in frame_a\";

however, the OF produces

  final parameter Real l(quantity = \"Length\", unit = \"m\") = L / /*Real*/(1 + N) \"length of each bodbox\";

while the NF produces

  final parameter Real l(quantity = \"Length\", unit = \"m\") = L / 3.0 \"length of each bodbox\";

and I guess the reason is the NF recognizes N as a structural parameter and evaluates it outright, which is correct.

Looking at the output of -d=optdaedump, when the OF is used, the following declarations first appear

173: bodybox[2].frameTranslation.r[1]:PARAM(start = 0.0 unit = "m" )  = bodybox[2].r[1]  "Vector from frame_a to frame_b resolved in frame_a" type: Real  [3,3]
303: bodybox[2].r[1]:PARAM(start = 0.1 unit = "m" )  = l  "Vector from frame_a to frame_b resolved in frame_a" type: Real  [3,3]
581: l:PARAM(unit = "m" final = true )  = L / /*Real*/(1 + N)  "length of each bodbox" type: Real 
582: L:PARAM(unit = "m" )  = 0.5  "length of the string" type: Real 
583: N:PARAM()  = 2  "number of joints" type: Integer 

which then, after the execution of evaluateFunc, become

173: bodybox[2].frameTranslation.r[1]:PARAM(start = 0.0 unit = "m" final = true )  = 0.1666666666666667  "Vector from frame_a to frame_b resolved in frame_a" type: Real  [3,3]
303: bodybox[2].r[1]:PARAM(start = 0.1 unit = "m" final = true )  = 0.1666666666666667  "Vector from frame_a to frame_b resolved in frame_a" type: Real  [3,3]
581: l:PARAM(unit = "m" final = true )  = 0.1666666666666667  "length of each bodbox" type: Real 
582: L:PARAM(unit = "m" final = true )  = 0.5  "length of the string" type: Real 
583: N:PARAM(final = true )  = 2  "number of joints" type: Integer 

with all the parameters having final=true and being evaluated.

When the NF is used, these declarations first appear

173: bodybox[2].frameTranslation.r[1]:PARAM(flow=false start = 0.0 unit = "m" )  = bodybox[2].r[1]  "Vector from frame_a to frame_b resolved in frame_a" type: Real  [3,3]
303: bodybox[2].r[1]:PARAM(flow=false start = 0.1 unit = "m" )  = l  "Vector from frame_a to frame_b resolved in frame_a" type: Real  [3,3]
581: l:PARAM(flow=false unit = "m" final = true )  = L / 3.0  "length of each bodbox" type: Real 
582: L:PARAM(flow=false unit = "m" )  = 0.5  "length of the string" type: Real 
583: N:PARAM(flow=false )  = 2  "number of joints" type: Integer 

and the only differences I can see are that flow=false was added (which shouldn't be relevant) and that N was evaluated to 3.0 already by the front-end.

Despite that, through all the subsequent stages of the front-end, those parameter declarations remain unchanged, none is evaluated, and the final attribute is not added to any of them.

I can't really understand why that happens, unless the presence of the structural parameter N at the root of the parameter dependency chain obtained by the OF somehow triggers the evaluation of all those parameters - but why should that happen, if other non-final parameters such as L are involved?

@perost can you give a look and comment for the people in cc:?

[Per Östlund]

Possibly fixed in 9605802.

The issue seems to have been that bodybox[1].lengthDirection has Evaluate=true, and is thus evaluated by the NF so that we get:

  parameter Real bodybox[1].lengthDirection[1](unit = "1") = 0.1666666666666667;
  parameter Real bodybox[1].lengthDirection[2](unit = "1") = 0.0;
  parameter Real bodybox[1].lengthDirection[3](unit = "1") = 0.0;

The OF does not evaluate these parameters though, so with the OF we instead get:

  parameter Real bodybox[1].lengthDirection[1](unit = "1") = 
    Modelica.SIunits.Conversions.to_unit1(bodybox[1].r[1] - bodybox[1].r_shape[1]);
  parameter Real bodybox[1].lengthDirection[2](unit = "1") = 
    Modelica.SIunits.Conversions.to_unit1(bodybox[1].r[2] - bodybox[1].r_shape[2]);
  parameter Real bodybox[1].lengthDirection[3](unit = "1") = 
    Modelica.SIunits.Conversions.to_unit1(bodybox[1].r[3] - bodybox[1].r_shape[3]);

These parameters are instead evaluated in the backend by EvaluateParameters, which causes e.g. bodybox[1].r[1] to be evaluated and replaced in the DAE, and so on.

However, since these parameters have already been evaluated by the NF they're not evaluated by EvaluateParameters. That shouldn't have been an issue, but the problem was that the NF only marked the parameters with Evaluate=true as structural, it didn't actually mark their dependencies as structural like we do with "real" structural parameters. So neither the frontend nor the backend evaluated e.g. bodybox[1].r[1], the frontend because it didn't consider it to be structural and the backend because any structural parameter depending on it had already been evaluated by the frontend.

Now the NF should evaluate it like it was meant to do, so hopefully that fixes the issue. The model should probably have simulated correctly anyway though, so there might still be some issue in the backend/runtime.

[Francesco Casella]

Replying to perost:
This took some time for me to figure out. Thanks for the quick fix!

Now the NF should evaluate it like it was meant to do, so hopefully that fixes the issue. The model should probably have simulated correctly anyway though, so there might still be some issue in the backend/runtime.

I'll check as soon as asap.openmodelica.org gets back online and your PR is processed. I hope that with proper evaluation in place it becomes easier to spot the difference between OF and NF.

[Per Östlund]

The results are now in, and the issue was indeed fixed by my commit.

[Willi Braun]

Great! Thank you!

[Francesco Casella]

With this last fix, all of the ScalableTestSuite finally works correctly with the NF. There are still a few models that fail, but that is only due to lack of resources for very large models or (in one case) due to numerical noise which is overestimated by the result comparisons routine.

Thanks for the good work!