Do not scalarize trivial connection equations when using -nonfScalarize

[Francesco Casella]

Once #5643 is fixed, the NF should dump all the scalarized connect equations even in case -nonfScalarize is set. This can be quite inefficient when large number of connect statements are used. Consider the following case:

connector HeatPort
  flow Power Q;
  Temperature T;
end HeatPort;

model Volume
  parameter ThermalConductance G;
  parameter ThermalCapacitance C;

  HeatPort top;
  HeatPort bottom;
  HeatPort left;
  HeatPort right;
  HeatPort front;
  HeatPort back;

  Temperature T;
equation
  C*der(T) = G*((top.T-T) + (bottom.T-T) +
                (left.T-T) + (right.T-T) +
                (front.T-T) + (back.T-T));
end Volume;

model Cube
  parameter Integer N = 100;
  parameter Integer M = 100;
  parameter Integer P = 100;
  Volume vol[N, M, P];
equation
  for i in 1:N-1 loop
    for j in 1:M loop
      for k in 1:P loop
        connect(vol[i,j,k].bottom, vol[i+1,j,k].top);
      end for;
    end for;
  end for;
  for i in 1:N loop
    for j in 1:M-1 loop
      for k in 1:P loop
        connect(vol[i,j,k].right, vol[i,j+1,k].left);
      end for;
    end for;
  end for;
  for i in 1:N loop
    for j in 1:M loop
      for k in 1:P-1 loop
        connect(vol[i,j,k].back, vol[i,j,k+1].front);
      end for;
    end for;
  end for;
end Cube;

The original Cube model has three connect statements inside for loops. If loops are unrolled, about 3 million scalarized connect statements are created, corresponding to about 6 million scalar equations. In addition, there are about 60000 unconnected connectors on the six outer faces of the cube, corresponding to about 60000 additional equations stating the flow variables Q of those connectors are zero. Handling all these equations in the backend would take a lot of time.

However, this model satisfies the following properties:

• each one of the ports vol[i,j,k].bottom, vol[i+1,j,k].top) for i in 1:N-1, j in 1:M, k in 1:P only show up once in one connect statement inside a nested for loop
• each one of the ports vol[i,j,k].right, vol[i,j+1,k].left for i in 1:N, j in 1:M-1, k in 1:P shows only up once in one connect statement inside a nested for loop
• each one of the ports (vol[i,j,k].back, vol[i,j,k+1].front for i in 1:N, j in 1:M, k in 1:P-1 shows only up once in one connect statement inside a nested for loop
• the ports vol[N,:,:].bottom, vol[1,:,:].top), vol[:,M,:].right, vol[:,1,:].left, (vol[:,:,N].back, vol[i:,:,1].front do not show up in any connect statement of the model

Hence, in the first two cases, only connection sets with two connectors will show up, while in the third case, only connection sets with a single connector will show up. The corresponding equations for flow and effort variables can then be easily derived by processing the unexpanded connection statements inside the for loops, leading to:

model Cube
  parameter Integer N = 100;
  parameter Integer M = 100;
  parameter Integer P = 100;
  Volume vol[N, M, P];
equation
  for i in 1:N-1 loop
    for j in 1:M loop
      for k in 1:P loop
        // connect(vol[i,j,k].bottom, vol[i+1,j,k].top);
        vol[i,j,k].bottom.T = vol[i+1,j,k].top.T;
        vol[i,j,k].bottom.Q + vol[i+1,j,k].top.Q = 0;
      end for;
    end for;
  end for;
  for i in 1:N loop
    for j in 1:M-1 loop
      for k in 1:P loop
        // connect(vol[i,j,k].right, vol[i,j+1,k].left);
        vol[i,j,k].right.T = vol[i,j+1,k].left.T;
        vol[i,j,k].right.Q + vol[i,j+1,k].left.Q = 0;
      end for;
    end for;
  end for;
  for i in 1:N loop
    for j in 1:M loop
      for k in 1:P-1 loop
        // connect(vol[i,j,k].back, vol[i,j,k+1].front);
        vol[i,j,k].back.T = vol[i,j,k+1].front.T;
        vol[i,j,k].back.Q + vol[i,j,k+1].front.Q = 0;
      end for;
    end for;
  end for;
  vol[N,:,:].bottom.Q = zeros(M,P);
  vol[1,:,:].top.Q = zeros(M,P);
  vol[:,M,:].right.Q = zeros(N,P);
  vol[:,1,:].left.Q = zeros(N,P);
  vol[:,:,N].back.Q = zeros(N,M);
  vol[i:,:,1].front.Q = zeros(N,M);
end Cube;

which only contains 12 vector or for-loop equations instead of 6 millions.

Summarizing, from what I understand the NF could keep track of connect statements found inside loops as the Modelica model is progressively parsed, taking note of which ports actually show up in the connect statements either in the form of array connections or of for loop connections.

At the end of the parsing, all the connect statements involving ports that only show up once can be expanded into the corresponding flow and effort equations, while array equations corresponding to the trivial boundary conditions can be generated for those ports that do not show up anywhere.

The remaining connection sets involving more than two connectors could be handled in general by unrolling them fully to scalar equations. Since they are expected to be a modest number, particularly if compared to the main loops or arrays, this shouldn't cause any major impact on compile time. Note that there are no such equations in the above-mentioned test case.

@perost, @adrpo, what do you think?

[Per Östlund]

The for-loops are not really necessary in this case, the model can be simplified to:

model Cube
  parameter Integer N = 100;
  parameter Integer M = 100;
  parameter Integer P = 100;
  Volume vol[N, M, P];
equation
  connect(vol[1:N-1,:,:].bottom, vol[2:N,:,:].top);
  connect(vol[:,1:M-1,:].right,  vol[:,2:M,:].left);
  connect(vol[:,:,1:P-1].back,   vol[:,:,2:P].front);
end Cube;

This would probably be easier to handle. The main issue with keeping connector arrays as arrays in the connection handling is when we have several connections to the same array, since different elements of the array might belong to different connection sets.

If both the connections in a connection only shows up in that particular connection we could keep them as arrays. The problem is detecting that before building the connection sets. It's not really hard to do, but would almost certainly give worse performance for "normal" models since it would be fairly expensive to count the number of occurances of each connector.

In other word, such functionality should maybe be optional via a flag, but we'll have to see what the actual impact would be. Another option is to use an annotation to mark connections that shouldn't be scalarized, making it the user's responsibility to make sure it's valid.

[Francesco Casella]

Replying to perost:

The for-loops are not really necessary in this case. This would probably be easier to handle.

Sure. This is of course a limitation in the compiler capability, but I think it is definitely one we can live with. BTW, I guess introducing some pattern matching to transform the for-loops into the sliced array connect statements could be doable later on.

The main issue with keeping connector arrays as arrays in the connection handling is when we have several connections to the same array, since different elements of the array might belong to different connection sets.

That's precisely my point:

• in most applications I can think of, most of the array connections will not have multiple connections to the same array, as connections are mostly one-to-one
• recognizing this simple fact should be straightforward: just check that a certain connector array only shows up in a single connect statement
• exploiting it could provide a huge benefit in most cases, allowing to get rid of most connection sets of cardinality one and two, that correspond to trivial equations.

If both the connections in a connection only shows up in that particular connection we could keep them as arrays. The problem is detecting that before building the connection sets. It's not really hard to do, but would almost certainly give worse performance for "normal" models since it would be fairly expensive to count the number of occurances of each connector.

One could start to do this in a more restrictive way, just by keeping a list of the names of connector arrays that show up in connect statement (not individual scalar components of them). Every time you encounter a new one, you just check if the connector names involved in it have been encountered previously or not. This would work nicely in the above-mentioned example.

In general, once we do not scalarize arrays and for loops, I don't expect that models will have thousands of connect statements. There are of course exceptions (e.g. large scale power system models, which have an irregular pattern of connections that must be explicitly stated one by one), and in that case a flag might do.

In other word, such functionality should maybe be optional via a flag, but we'll have to see what the actual impact would be.

No problem with that.

Another option is to use an annotation to mark connections that shouldn't be scalarized, making it the user's responsibility to make sure it's valid.

This could be a last resort option, but I'd really prefer the compiler to do this automatically.

[Francesco Casella]

Release 1.15.0 was scrapped, because replaceable support eventually turned out to be more easily implemented in 1.16.0. Hence, all 1.15.0 tickets are rescheduled to 1.16.0

[Francesco Casella]

Retargeted to 1.17.0 after 1.16.0 release

[Francesco Casella]

Support for non-scalarized connects was introduced in #5643.