The backend should break down initial record equations into their scalar counterparts

[Francesco Casella]

Consider the following test model.

model M1
  record R
    Real x;
    Real y;
  end R;
  
  Real x;
  Real y;
equation
  R(3,4) = R(x,y);
end M1;

The new front-end flattens it to

function M1.R "Automatically generated record constructor for M1.R"
  input Real x;
  input Real y;
  output R res;
end M1.R;

class M1
  Real x;
  Real y;
equation
  M1.R(3.0, 4.0) = M1.R(x, y);
end M1;

then the backend immediately breaks down the record = record equation into two scalar equations

######################################## 
pre-optimization module normalInlineFunction (simulation) 
######################################## 
 
 
unknown partition 
======================================== 
 
Variables (2) 
======================================== 
1: y:VARIABLE(flow=false )  type: Real  
2: x:VARIABLE(flow=false )  type: Real  
 
 
Equations (2, 2) 
======================================== 
1/1 (1): 4.0 = y   [dynamic |0|0|0|0|] 
2/2 (1): 3.0 = x   [dynamic |0|0|0|0|] 

Consider now this second test case, which has exactly the same structure but involves fixed = false parameters and initial equations instead

model M2
  record R
    Real x;
    Real y;
  end R;
  
  parameter Real x(fixed = false);
  parameter Real y(fixed = false);
initial equation
  R(3,4) = R(x,y);
end M2;

The new frontend flattens it in the same way as M1 to

function M2.R "Automatically generated record constructor for M2.R"
  input Real x;
  input Real y;
  output R res;
end M2.R;

class M2
  parameter Real x(fixed = false);
  parameter Real y(fixed = false);
initial equation
  M2.R(3.0, 4.0) = M2.R(x, y);
end M2;

However, the back-end does not break down the initial equation into its scalar counterparts, and carries the record equation through the whole optimization phase

Initial Equations (1, 2) 
======================================== 
1/1 (2): M2.R(3.0, 4.0) = M2.R(x, y)   [dynamic |0|0|0|0|] 

until eventually failing

Internal error complex equations currently only supported on form v = functioncall(...). Equation: M2.R(3.0, 4.0) = M2.R(x, y) solve for {y,x}

because nonlinear equations in the form record = record are not supported.

The initial record equation should instead be broken into its scalar components at the beginning of the backend operation, in the same way as the regular equations.

Solving this issue should also fix #4354.

[Vitalij Ruge]

The FroneEnd return for M1

Initial Equations (2, 2)
========================================
1/1 (1): 3.0 = x   [initial |0|0|0|0|]
2/2 (1): 4.0 = y   [initial |0|0|0|0|]

and for M2

Initial Equations (1, 2)
========================================
1/1 (2): M2.R(3.0, 4.0) = M2.R(x, y)   [dynamic |0|0|0|0|]

The output is produce with +d=dumpdaelow.
The Backend can inline function e.g. M2.R.`from_real`(3.0,4.0) but the Backend can't inline a constructor.

[Francesco Casella]

Replying to vitalij:

The Backend can inline function e.g. M2.R.`from_real`(3.0,4.0) but the Backend can't inline a constructor.

I'm sorry but I still fail to see why the regular equation and the initial equation should be handled differently. The Modelica source code is exactly the same

  R(3,4) = R(x,y);

and the flattened code produced by the new front-end is exactly the same

  M1.R(3.0, 4.0) = M1.R(x, y);

why should the regular equation be handled as a function and the initial equation as a constructor? What is the difference?

[Vitalij Ruge]

Replying to casella:

  M1.R(3.0, 4.0) = M1.R(x, y);

why should the regular equation be handled as a function and the initial equation as a constructor? What is the difference?

The equation for M1 at the beginning of the backend is different!
If using dumpdaelow

dumpdaelow (default: off)
    Dumps the equation system at the beginning of the back end. 

we get different dump between NF and OF.

[Francesco Casella]

Replying to vitalij:

The equation for M1 at the beginning of the backend is different!
If using dumpdaelow

dumpdaelow (default: off)
    Dumps the equation system at the beginning of the back end. 

we get different dump between NF and OF.

OK, I now I get it, thanks!

Then, I guess the problem is with NFConvertDAE.convert, or with whatever the NF does to prepare the DAE structure after it outputs the flattened model.

Thus, I hand this over to @perost

[Per Östlund]

Replying to casella:

Replying to vitalij:

The equation for M1 at the beginning of the backend is different!
If using dumpdaelow

dumpdaelow (default: off)
    Dumps the equation system at the beginning of the back end. 

we get different dump between NF and OF.

OK, I now I get it, thanks!

Then, I guess the problem is with NFConvertDAE.convert, or with whatever the NF does to prepare the DAE structure after it outputs the flattened model.

Thus, I hand this over to @perost

The difference here is that the old frontend splits the complex equation into scalar equations, while the new frontend doesn't do that. Note that there are no constructor calls in the flat model when using the new frontend, only record expressions, because it inlines default constructors. And the backend can't handle initial equations with record expressions on both sides, so it fails because of that.

However, this is not at all the same issue as in #4354, because Complex has a userdefined constructor and the models also uses non-constant arguments in the calls. As I've mentioned before the old frontend incorrectly uses the default constructor in those cases, but doesn't split them as it does with the models in this ticket since the arguments are non-constant.

The new frontend uses the fromReal constructor instead, and doesn't split them either since it would have to inline the function calls to do that. But then the backend inlines them instead, causing it to fail like in #4354 since it doesn't know how to handle the inlined equations.

In other words: I could make the new frontend split the complex equations for the models in this ticket if you want to, but that won't make any difference whatsoever for #4354.

[Per Östlund]

Possibly fixed in d287947/OMCompiler.

I looked over the code in BackendDAECreate.lowerEqn again, and noticed that the cases for initial and non-initial complex equations were not actually the same. The case for non-initial complex equations had a call to lowerExtendedRecordEqn after doing inlining, while the case for initial complex equations did not.

And that function splits complex equations if both sides are record expressions, as they usually are with the new frontend. So I just changed the initial case to do the same thing, and that seems to have fixed the issue for the new frontend.

[Francesco Casella]

Replying to perost:

Possibly fixed in d287947/OMCompiler.

I looked over the code in BackendDAECreate.lowerEqn again, and noticed that the cases for initial and non-initial complex equations were not actually the same. The case for non-initial complex equations had a call to lowerExtendedRecordEqn after doing inlining, while the case for initial complex equations did not.

:)

And that function splits complex equations if both sides are record expressions, as they usually are with the new frontend. So I just changed the initial case to do the same thing, and that seems to have fixed the issue for the new frontend.

Excellent! This should finally allow us to run some large-scale power system model with the new front-end. I'll report to you about that.

[Francesco Casella]

I confirm that we can now run our power grid models with the new front end. Thank you again for fixing this issue!