Avoid bad scalarization of parameter bindings

[Per Östlund]

We currently flatten a model such as this:

function f
  output Real y[3];
external "C" y = f_ext();
end f;

model M
  parameter Real x[:] = f();
end M;

to

function f
  output Real[3] y;

  external "C" f_ext(y);
end f;

class M
  parameter Real x[1] = f()[1];
  parameter Real x[2] = f()[2];
  parameter Real x[3] = f()[3];
end M;

This means the function is called once for each element of the array, so this is obviously something we should try to avoid. This happens particularly with the Table models in the MSL that read input from a file.

One solution is to move the binding to an initial equation, but I'm not sure how safe that is in general. There are many other ways we could handle this in, but most are non-trivial and would probably require extensive changes to the backend.

[Francesco Casella]

@perost, in the future we'll use -nonfScalarize and the new backend, and there won't be any problem with that.

For the time being, when you scalarize to pass the output to the old frontend, you may handle this as the backend does for functions returning records, something like:

class M
  parameter Real $cse1[3] = f();
  parameter Real x[1] = $cse1[1];
  parameter Real x[2] = $cse1[2];
  parameter Real x[3] = $cse1[3];
end M;

Of course this requires the backend to handle array-cse variables besides record-cse ones.

[Per Östlund]

Replying to casella:

@perost, in the future we'll use -nonfScalarize and the new backend, and there won't be any problem with that.

Sure, but that's in the future. We have industry users that have issues with this right now, so if we can fix the issue without too much effort it would be nice to do so.

For the time being, when you scalarize to pass the output to the old frontend, you may handle this as the backend does for functions returning records, something like:

class M
  parameter Real $cse1[3] = f();
  parameter Real x[1] = $cse1[1];
  parameter Real x[2] = $cse1[2];
  parameter Real x[3] = $cse1[3];
end M;

Of course this requires the backend to handle array-cse variables besides record-cse ones.

Yes, that could be a possible solution.

[Francesco Casella]

Replying to perost:

Replying to casella:

@perost, in the future we'll use -nonfScalarize and the new backend, and there won't be any problem with that.

Sure, but that's in the future. We have industry users that have issues with this right now, so if we can fix the issue without too much effort it would be nice to do so.

Absolutely!

[Francesco Casella]

@Karim currently working on that

[Niklas Worschech]

minimal example

[Karim Adbdelhak]

Is the minimal example expected to run completely? It fails because it cannot find the external function. Otherwise it looks good:

FlatModel:                                                                                                                                                                                                 
class initializeParameterArray.Testcase_1                                                                                                                                                                  
  parameter Real t = 1.1;                                                                                                                                                                                  
  parameter Real array[1, 1](fixed = false);                                                                                                                                                               
  parameter Real array[1, 2](fixed = false);                                                                                                                                                               
  parameter Real array[1, 3](fixed = false);                                                                                                                                                               
  parameter Real array[2, 1](fixed = false);                                                                                                                                                               
  parameter Real array[2, 2](fixed = false);                                                                                                                                                               
  parameter Real array[2, 3](fixed = false);                                                                                                                                                               
  parameter Real array[3, 1](fixed = false);                                                                                                                                                               
  parameter Real array[3, 2](fixed = false);                                                                                                                                                               
  parameter Real array[3, 3](fixed = false);                                                                                                                                                               
  parameter Real array[4, 1](fixed = false);                                                                                                                                                               
  parameter Real array[4, 2](fixed = false);                                                                                                                                                               
  parameter Real array[4, 3](fixed = false);                                                                                                                                                               
  parameter Real array[5, 1](fixed = false);                                                                                                                                                               
  parameter Real array[5, 2](fixed = false);                                                                                                                                                               
  parameter Real array[5, 3](fixed = false);                                                                                                                                                               
  parameter Real array[6, 1](fixed = false);                                                                                                                                                               
  parameter Real array[6, 2](fixed = false);                                                                                                                                                               
  parameter Real array[6, 3](fixed = false);                                                                                                                                                               
  parameter Real array[7, 1](fixed = false);                                                                                                                                                               
  parameter Real array[7, 2](fixed = false);                                                                                                                                                               
  parameter Real array[7, 3](fixed = false);                                                                                                                                                               
  parameter Real array[8, 1](fixed = false);                                                                                                                                                               
  parameter Real array[8, 2](fixed = false);                                                                                                                                                               
  parameter Real array[8, 3](fixed = false);                                                                                                                                                               
  parameter Real array[9, 1](fixed = false);                                                                                                                                                               
  parameter Real array[9, 2](fixed = false);                                                                                                                                                               
  parameter Real array[9, 3](fixed = false);                                                                                                                                                               
  parameter Real array[10, 1](fixed = false);                                                                                                                                                              
  parameter Real array[10, 2](fixed = false);                                                                                                                                                              
  parameter Real array[10, 3](fixed = false);                                                                                                                                                              
initial equation                                                                                                                                                                                           
  array = initializeParameterArray.callInit(t);                                                                                                                                                            
end initializeParameterArray.Testcase_1;                   

[Karim Adbdelhak]

For the record, i am talking about the version from ​PR7163. Should be pushed soon and seems to fix the main problem.

[Francesco Casella]

PR 7163 passed the testsuite successfully. @Karim, are you waiting for some review?

[Niklas Worschech]

On Windows the test runs correctly, I could not test it on Linux yet, but there it should also work

[Per Östlund]

PR 7163 caused some regressions in the MSL, see ​report.

[Karim Adbdelhak]

I will have a look at the regressions!

[Francesco Casella]

Multibody models are breaking. Maybe some problems with inlining?

[Karim Adbdelhak]

The problem basically that we converted bindings to initial equations. Seems trivial but causes problems for things like final parameters. Furthermore there is a difference in how C and CPP handles these things.

I think i resolved it in ​PR7271, we will see in the next report.

[Francesco Casella]

The models that were broken in ​PR 7163 were fixed in ​PR 7271. However, the latter commit broke 73 other models, including 8 models from Buildings 7.0.0 and 15 models from ThermoPower, as well as several Modelica and ModelicaTest models.

@Karim, I guess we're still not there.

[Francesco Casella]

Problem identified, @Karim looking for a solution

[Karim Adbdelhak]

I fixed some of it, but there seems to be something left:
​report

[Karim Adbdelhak]

@Francesco I lost track a little, did i fix all of the models?

[Francesco Casella]

This is the gist of the story

• ​PR 7163 broke 87 model, see ​report - only one regression is due to changes in IBPSA
• ​PR 7271 fixed them, but broke another 73, see ​report
• ​PR 7292 fixed 21 of them, see ​report. Most of the cases that were broken by PR 7271 in Buildings and ThermoPower are still broken, and should be fixed with high priority

[Francesco Casella]

Fixed by ​b111336289, see ​report.

Thanks @Karim!