FMU shows parameters that should be hidden

[massimo ceraolo]

Consider model TestFMU below (annotations dropped)

model TestFMU 
  parameter Real K = 1;
  Modelica.Blocks.Interfaces.RealInput u;
  Modelica.Blocks.Interfaces.RealOutput y;
  Modelica.Blocks.Math.Gain gain1(k = K / 2);
Modelica.Blocks.Math.Gain gain(k = 1);
equation
  connect(gain1.u, u);
  connect(gain1.y, gain.u);
  connect(gain.y, y);
end TestFMU;

I exported it as FMU-ME 2.0 and imported into Dymola the parameter window is as in the enclosed file screen.png.
I think the parameter gain1_k shouldn't be visible in the dialog box, since it is internally computed through a binding formula.

[Francesco Casella]

@ceraolo, the binding is not final, so it can still be changed at runtime.

Try with

model TestFMU 
  parameter Real K = 1;
  Modelica.Blocks.Interfaces.RealInput u;
  Modelica.Blocks.Interfaces.RealOutput y;
  Modelica.Blocks.Math.Gain gain1(final k = K / 2);
  Modelica.Blocks.Math.Gain gain(final k = 1);
equation
  connect(gain1.u, u);
  connect(gain1.y, gain.u);
  connect(gain.y, y);
end TestFMU;

[massimo ceraolo]

Two objections to comment 1
1) If @casella is right and the model as written in this ticket's description implies that the parameter gain1.k must be visible from the outside, then we've got a problem with OM: it does not show it! Otherwise it should not appear even when a fmu is created from that model
2) however, even if I add final on binding in gain1:

 Modelica.Blocks.Math.Gain gain1(final k = K / 2);

the fmu still shows this parameter in its dialog box (when imported in Dymola) and this is even more unacceptable.

[Francesco Casella]

Replying to ceraolo:

Two objections to comment 1
1) If @casella is right and the model as written in this ticket's description implies that the parameter gain1.k must be visible from the outside

I didn't write "visible from the outside", I wrote that "it can be changed at runtime". In fact, if you run the original model in OMEdit, you should be able to change all three parameters at run time. Unfortunately, you can only do that for gain.k, not for gain1.k, which is not what I expected.

@Karim, @AnHeuermann, could you check why this happens?

I'm also not sure what is the standard practice with FMU generation. Are you expected to access all the non-final parameters of hierarchically structured models? They could be a lot, but why not?

then we've got a problem with OM: it does not show it! Otherwise it should not appear even when a fmu is created from that model

Show it where, exactly? I'm not sure I understand what you mean here.

2) however, even if I add final on binding in gain1:

 Modelica.Blocks.Math.Gain gain1(final k = K / 2);

the fmu still shows this parameter in its dialog box (when imported in Dymola) and this is even more unacceptable.

I agree. @lochel, would you mind checking why this happens?

[massimo ceraolo]

I didn't write "visible from the outside", I wrote that "it can be changed at runtime".

OK, but from OMEdit perspective visibility through the parameter dialog box is equivalent to changeability (not totally sure, though).

In fact, if you run the original model in OMEdit, you should be able to change all three parameters at run time. Unfortunately, you can only do that for gain.k, not for gain1.k, which is not what I expected.

I' don't know that specification says on this, but for me the current OMEdit behaviour makes sense. Consider for instance a model in which we have two integrators and the time constant of the second must always be twice the second one. In this case, we leave as a changeable (from the outside dialog box) the first time constant and write the formula in the second integral. The model final user should not be able to change it.
This is how OMEdit and Dymola do, at least in the examples I checked. This is particularly important for FMUs, which are often given to people outside the project that has generated it. Yes, maybe we could add the fixed keyword, but I'm at ease with the way the thing is dealt with by OM and Dymola by now.
Except for OM's FMUs.

Show it where, exactly? I'm not sure I understand what you mean here.

I showed this in the attached screen.png, made using Dymola.
I must add that if I do the same with OMEdit, i.e., drag the FMU on an OMEdit diagram, gain1_k does not appear.

[massimo ceraolo]

Btw the descriptive Text in gain k "Gain value multiplied with input signal" is awful! (it is such in MSL)

[Francesco Casella]

Replying to ceraolo:

I' don't know that specification says on this,

​Section 7.2.6:

An element defined as final by the final prefix in an element modification or declaration cannot be modified by a modification or by a redeclaration. All elements of a final element are also final. [Setting the value of a parameter in an experiment environment is conceptually treated as a modification. This implies that a final modification equation of a parameter cannot be changed in a simulation environment].

The normative part regards what you can do with an element that has a final modification (i.e., nothing), when using that model within another Modelica model, e.g., by instantiating it or by extending it.

The non-normative part suggest that final modifiers cannot be overriden, which is what both OpenModelica an Dymola do when simulating the models within the tool. It doesn't say anything about non-final modifiers. In fact, most of the times where you have an expression as a modifier, the intent is that this expression should not be changed, i.e. there is an implicit final there, which modellers (including myself) omit for lazyness.

but for me the current OMEdit behaviour makes sense. Consider for instance a model in which we have two integrators and the time constant of the second must always be twice the second one. In this case, we leave as a changeable (from the outside dialog box) the first time constant and write the formula in the second integral. The model final user should not be able to change it.

I agree. In principle it should be better to put final on the modifiers, but since people don't do that, this approach is sensible.

Hence, for consistency we should have the same behaviour in FMUs.

@lochel, @arun can you please check why this doesn't happen?

[Francesco Casella]

Replying to ceraolo:

Btw the descriptive Text in gain k "Gain value multiplied with input signal" is awful! (it is such in MSL)

You may want to open a pull request on the ​MSL repository where you put a better description. You need to fork the repo to your own account, clone it on your pc, make a branch where you commit your modifications, and push it to your fork on github. Then github automatically recognizes it and gives you a button to open a PR on the master branch of the original repo.

[Andreas Heuermann]

For the above example

model TestFMU 
  parameter Real K = 1;
  Modelica.Blocks.Interfaces.RealInput u;
  Modelica.Blocks.Interfaces.RealOutput y;
  Modelica.Blocks.Math.Gain gain1(final k = K / 2);
  Modelica.Blocks.Math.Gain gain(final k = 1);
equation
  connect(gain1.u, u);
  connect(gain1.y, gain.u);
  connect(gain.y, y);
end TestFMU;

we get

  <!-- Index of variable = "4" -->
  <ScalarVariable
    name="gain.k"
    valueReference="3"
    description="Gain value multiplied with input signal"
    variability="fixed"
    causality="calculatedParameter"
    >
    <Real unit="1"/>
  </ScalarVariable>
  <!-- Index of variable = "5" -->
  <ScalarVariable
    name="gain1.k"
    valueReference="4"
    description="Gain value multiplied with input signal"
    variability="fixed"
    causality="calculatedParameter"
    >
    <Real unit="1"/>
  </ScalarVariable>

And I think this matches with the description in the FMI Standard in 2.2.7 Definition of Model Variables (ModelVariables):

[causality = "calculatedParameter" and causality = "local" with
variability = "fixed" or "tunable" are similar. The difference is that a
calculatedParameter can be used in another model or slave, whereas a local
variable cannot. For example, when importing an FMU in a Modelica environment, a
"calculatedParameter" should be imported in a public section as final
parameter, whereas a "local" variable should be imported in a protected section
of the model.]

Btw., for the original example the parameters gain.k and gain1.k are fixed calculatedParamters as well.

Also I don't think it is a problem that all variables are visible to the importing tool. Visible doesn't mean changable. That is described by variability and causality.
But it should be possible to hide variables from the modelDescription.xml if desired. Use ​-fmifilter=blackBox for this specific case.

[Francesco Casella]

Replying to AnHeuermann:

Also I don't think it is a problem that all variables are visible to the importing tool. Visible doesn't mean changable.

I haven't tried this myself, but from what I understand from the attached screenshots, it seems those parameters are not only "visible", but pretty much changeable, as you get input widgets for all of them. I guess that was what @ceraolo meant.

@ceraolo, could you try to:

• export the same models as FMUs from Dymola
• check that when imported you see the parameter input as you would expect
• post the fragment of XML corresponding to that of comment:10 that you get from Dymola

[massimo ceraolo]

added:
screenDymola.png
modelDescription.xml

[massimo ceraolo]

Further addition.
I'm not sure whether the visualization of gain1.k is an OM or Dymola issue.
I've just checked with Matlab/Simulink.
The result is that both FMUs (the one created by OM and the one created by Dymola) do not show gain2.k in Matlab (screenMatlab.png)

[Andreas Heuermann]

The problem is, I couldn't find an option in the FMI 2.0 Standard that will describe these parameters better.
The variability is not constant, because for our example the parameter gain.k is allowed to change if parameter K is allowed to change. So it has variability fixed.
The causality calculated parameter is correct as well.

I'm not sure if the FMI standard says anything about if the user is allowed to change the value of calculated parameters. Per definition they are calculated from other variables, so it doesn't make much sense to change them directly.

[massimo ceraolo]

After looking at the xmls I also envisaged that there could be some weakness in the FMU standard on this ticket's issue.

If the standard does not help us, usefulness and consistency should guide our actions.

For consistency with OMEdity present behaviour, gain1.k should not be accessible from the FMU parameter dialogue box. The problem we have is that the FMU created by OM shows this parameter when loaded from Dymola and does not when loaded from Matlab!

Even if I add fixed to gain1.k, still the FMU from Dymola shows it (and allows modifying it).

But it is of the outmost importance to be able to forbid the final user to change some parameters. In production models there are many parameters, and the user can easily wreak havoc to any if changes some of them that should indeed be a function of (user modifiable) others

For me, to consider this ticket closed or at least 90% satisfied, it is sufficient that some way is found to obtain this behaviour. A possible way is to require fixed keyword to ensure that a parameter is not modifiable by the final FMU user (as I said this is not presently the case when the user uses Dymola)

In case of test is needed, I can check with Dymola and Matlab/Simulink.

[Francesco Casella]

Replying to AnHeuermann:

I'm not sure if the FMI standard says anything about if the user is allowed to change the value of calculated parameters. Per definition they are calculated from other variables, so it doesn't make much sense to change them directly.

If think this is the point.

In Modelica, you can have calculated parameters with non-final bindings containing expressions. In principle, these could be changed at runtime (because they are not final) but in practice they really can't, because you may need to change the computation flow if they become dependent on other parameters. That requires to re-compile the model, you can't do that on an FMI which is pre-compiled.

Hence, it makes sense to allow changing non-final parameter values that are given by numerical literal constants (that you can change at runtime to other numerical literal constants), while parameters that actually need to be computed from other parameters in the original models will not be changeable at run time in the FMI.

Hence, we should get a parameter input field for gain_k but not for gain1_k, in case we use the original model of this ticket.

If you put final on the modifiers, none of them should be changeable, and the fact that gain_k still is in Dymola-generated FMUs is, IMHO, a Dymola bug that should be reported. The Modelica Specification, section 7.2.6, is pretty clear about that

This implies that a final modification equation of a parameter cannot be changed in a simulation environment

even though this statement is placed in the non-normative part.

@ceraolo, does this make sense to you?

[massimo ceraolo]

Hence, it makes sense to allow changing non-final parameter values that are given by numerical literal constants (that you can change at runtime to other numerical literal constants), while parameters that actually need to be computed from other parameters in the original models will not be changeable at run time in the FMI.

This is the rationale of this ticket

If you put final on the modifiers, none of them should be changeable, and the fact that gain_k still is in Dymola-generated FMUs is, IMHO, a Dymola bug that should be reported.

Dymola applies the final keyword correctly.

After so many messages and experiments, maybe it is better to summarize the situation:

1. Dymola is consistent with the FMUs it creates
2. FMUs from OM when imported in Dymola allow changing Parameters that have a formula binding in the original model like gain1.k, which is illogical. However, the same FMUs, when imported into Matlab, correctly do not allow changing gain1.k
3. FMUs from OM having the final keyword on gain.k and gain1.k, when imported into Dymola, allow changing parameters, which is even more illogical than point 2

I conclude that there is some incompatibility from OM created FMUs and Dymola that cannot be solved by resorting to FMU standards (comment 14), but, I hope, can somehow be addressed using common sense.

Let us compare the xmls produced by the original model of this ticket.

modelDescription.xml from Dymola:

   <ScalarVariable
      name="gain.k"
      valueReference="16777217"
      description="Gain value multiplied with input signal"
      causality="parameter"
      variability="tunable">
      <Real
        unit="1"
        start="1"/>
    </ScalarVariable>
    <ScalarVariable
      name="gain1.k"
      valueReference="100663296"
      description="Gain value multiplied with input signal"
      variability="tunable">
      <Real
        unit="1"/>
    </ScalarVariable>

modelDescription.xml from OM:

  <ScalarVariable
    name="gain.k"
    valueReference="4"
    description="Gain value multiplied with input signal"
    variability="fixed"
    causality="parameter"
    >
    <Real start="1.0" unit="1"/>
  </ScalarVariable>
  <ScalarVariable
    name="gain1.k"
    valueReference="5"
    description="Gain value multiplied with input signal"
    variability="fixed"
    causality="calculatedParameter"
    >
    <Real unit="1"/>
  </ScalarVariable>

Maybe looking at these snippets, the developers could envisage a way that, while sticking with the standard, could make OM's FMUs correctly read by Dymola. Maybe omitting casualty keyword? Maybe requiring to have in the original model the final keyword before calculated parameters such as gain1.k?

[Andreas Heuermann]

Omitting the causality in the xml from Dymola

    <ScalarVariable
      name="gain1.k"
      valueReference="100663296"
      description="Gain value multiplied with input signal"
      variability="tunable">
      <Real
        unit="1"/>
    </ScalarVariable>

means that causality has its default value local.

Local variable that is calculated from other variables or is a continuous-time state (see section 2.2.8). It is not allowed to use the variable value in another model or slave.

---

Maybe that is the way to go. Whenever we have a binding we make the variable causality=local. If it has fixed we make it variability=fixed and otherwise variability=tunable.

From the FMI Standard:
A fixed local variable means:

Local variable that depends on fixed parameters only and is computed in the FMU. Cannot be used in another model. After initialization, the value of this local variable cannot change.

A tunable local variable means:

Local variable that depends on tunable parameters only and is computed in the FMU. Cannot be used in another model. The value of this local variable can only change during initialization and at event instants, provided a tunable parameter was changed.

[Francesco Casella]

Retargeted to 1.18.0 because of 1.17.0 timed release.

[Francesco Casella]

Ticket retargeted after milestone closed