Issue with FMI import

[massimo ceraolo]

Consider the package fbSystemTest.mo added to ticket #5581 created on 2019-07-12.
That ticket was created using OMEdit v1.14.0-dev-5826 (64 bit, win), which was able to create an fmu, and run a system containing that fmu, with a small accuracy issue detailed in that ticket.
I repeated that procedure today, using OpenModelica v1.14.0-dev-26545 and got a worse result. In fact, upon trying to run fbSystemFMU one receives the following error message:

[fbSystemPkg: 364:9-364:39]: Operator reinit may only be used in the body of a when equation.

the error message points to the first of the reinit rows in following code snippet, automatically generated when creating the fmu:

    when {triggerDSSEvent > flowStatesInputs, pre(nextEventTime) < time, terminal()} then
      newStatesAvailable := fmi2Functions.fmi2EventUpdate(fmi2me);
      nextEventTime := fmi2Functions.fmi2nextEventTime(fmi2me, flowStatesInputs);
      if newStatesAvailable then
        fmi_x_new := fmi2Functions.fmi2GetContinuousStates(fmi2me, numberOfContinuousStates, flowStatesInputs);
        reinit(fmi_x[3], fmi_x_new[3]);
        reinit(fmi_x[2], fmi_x_new[2]);
        reinit(fmi_x[1], fmi_x_new[1]);
      end if;
    end when;

Appearently, those reinits were well accepted up to last July.

Since the issue comes from a VERY basic modelica model, since it used to work just a few weeks ago, I set milestone for this ticket to 1.14.0

[Francesco Casella]

@ceraolo, the definition of reinit() in Section 3.7.3 of the specification states:

It can only be applied in the body of a when clause in an equation section.

From the code snippet you posted, it seems you are trying to use reinit in an algorithm section. I guess the old front-end didn't catch this error, while the new one, which is now used by default, does.

[massimo ceraolo]

Replying to casella:

@ceraolo, the definition of reinit() in Section 3.7.3 of the specification states:

It can only be applied in the body of a when clause in an equation section.

From the code snippet you posted, it seems you are trying to use reinit in an algorithm section. I guess the old front-end didn't catch this error, while the new one, which is now used by default, does.

In the ticket description I said where that snipped came from: it is not modelica code created by me, it is, instead, what OMEdit shows in an error message when trying to use an FMU it had created.

It is "modelica-like", but I don't know which relation actually has with Modelica specification.

So, to me, it is not clear if this ticket issue comes from a wrong fmu creation or interpretation. A point that seems important to me is that the same example worked "well" (except for the precision issue pointed out in ticket #551) last July, while generates an error today.
The model from which the fmu is created is valid Modelica, and reported as such by both OF an NF.

[massimo ceraolo]

I made a further check: I ran the system containing fmu (file "fbSystemFMU) using as simulation tool OM 1.14.0-dev14-26545, but as fmu one created using 1.14.0-dev14-234 (from last April).

The result is:

• the simulation runs without errors
• the fmu from dev-234 contains the same code snippet shown in this ticket's description

So one might conclude that even though that code snippet is (probably) valid fmu code, there is something in the fmu created using dev14-26545, that triggers an error message pointing to the snippet used in this ticket's description.

A further note: I checked with Dymola and got the same results: the FMU created using dev14-26545 triggers an error message that is, more or less, the same from OM, i.e. related to the usage of reinit inside an algorithm section. The fmu created using dev-234 runs smoothly.

[Francesco Casella]

Releasing 1.14.0 which is stable and has many improvements w.r.t. 1.13.2. This issue is rescheduled to 1.16.0

[Adrian Pop]

The only question remains if this is equivalent or not, any comments?

OMCompiler/Compiler/Template/CodegenFMU.tpl

@@ -1667 +1667 @@
     <%if not boolAnd(stringEq(booleanOutputVariablesNames, ""), stringEq(booleanOutputVariablesVRs, "")) then "{"+booleanOutputVariablesNames+"} = fmi1Functions.fmi1GetBoolean(fmi1me, {"+booleanOutputVariablesVRs+"}, flowStatesInputs);"%>
     <%if not boolAnd(stringEq(stringOutputVariablesNames, ""), stringEq(stringOutputVariablesVRs, "")) then "{"+stringOutputVariablesNames+"} = fmi1Functions.fmi1GetString(fmi1me, {"+stringOutputVariablesVRs+"}, flowStatesInputs);"%>
     <%dumpOutputGetEnumerationVariables(fmiModelVariablesList, fmiTypeDefinitionsList, "fmi1Functions.fmi1GetInteger", "fmi1me")%>
-  algorithm
+  equation
   <%if intGt(listLength(fmiInfo.fmiNumberOfEventIndicators), 0) then
   <<
     when {<%fmiInfo.fmiNumberOfEventIndicators |> eventIndicator =>  "change(fmi_z_positive["+eventIndicator+"])" ;separator=" or "%>, triggerDSSEvent > flowStatesInputs, nextEventTime < time, terminal()} then
@@ -1677 +1677 @@
     when {triggerDSSEvent > flowStatesInputs, nextEventTime < time, terminal()} then
   >>
   %>
-      newStatesAvailable := fmi1Functions.fmi1EventUpdate(fmi1me, intermediateResults);
+      newStatesAvailable = fmi1Functions.fmi1EventUpdate(fmi1me, intermediateResults);
   <%if intGt(listLength(fmiInfo.fmiNumberOfContinuousStates), 0) then
   <<
       if newStatesAvailable then
-        fmi_x_new := fmi1Functions.fmi1GetContinuousStates(fmi1me, numberOfContinuousStates, flowStatesInputs);
+        fmi_x_new = fmi1Functions.fmi1GetContinuousStates(fmi1me, numberOfContinuousStates, flowStatesInputs);
         <%fmiInfo.fmiNumberOfContinuousStates |> continuousStates =>  "reinit(fmi_x["+continuousStates+"], fmi_x_new["+continuousStates+"]);" ;separator="\n"%>
       end if;
   >>
@@ -2037 +2037 @@
     <%if not boolAnd(stringEq(stringOutputVariablesNames, ""), stringEq(stringOutputVariablesVRs, "")) then "{"+stringOutputVariablesNames+"} = fmi2Functions.fmi2GetString(fmi2me, {"+stringOutputVariablesVRs+"}, flowStatesInputs);"%>
     <%dumpOutputGetEnumerationVariables(fmiModelVariablesList, fmiTypeDefinitionsList, "fmi2Functions.fmi2GetInteger", "fmi2me")%>
     callEventUpdate = fmi2Functions.fmi2CompletedIntegratorStep(fmi2me, flowStatesInputs+flowTime);
-  algorithm
+  equation
   <%if intGt(listLength(fmiInfo.fmiNumberOfEventIndicators), 0) then
   <<
     when {<%fmiInfo.fmiNumberOfEventIndicators |> eventIndicator =>  "change(fmi_z_positive["+eventIndicator+"])" ;separator=" or "%>, triggerDSSEvent > flowStatesInputs, pre(nextEventTime) < time, terminal()} then
@@ -2047 +2047 @@
     when {triggerDSSEvent > flowStatesInputs, pre(nextEventTime) < time, terminal()} then
   >>
   %>
-      newStatesAvailable := fmi2Functions.fmi2EventUpdate(fmi2me);
-      nextEventTime := fmi2Functions.fmi2nextEventTime(fmi2me, flowStatesInputs);
+      newStatesAvailable = fmi2Functions.fmi2EventUpdate(fmi2me);
+      nextEventTime = fmi2Functions.fmi2nextEventTime(fmi2me, flowStatesInputs);
   <%if intGt(listLength(fmiInfo.fmiNumberOfContinuousStates), 0) then
   <<
       if newStatesAvailable then
-        fmi_x_new := fmi2Functions.fmi2GetContinuousStates(fmi2me, numberOfContinuousStates, flowStatesInputs);
+        fmi_x_new = fmi2Functions.fmi2GetContinuousStates(fmi2me, numberOfContinuousStates, flowStatesInputs);
         <%fmiInfo.fmiNumberOfContinuousStates |> continuousStates =>  "reinit(fmi_x["+continuousStates+"], fmi_x_new["+continuousStates+"]);" ;separator="\n"%>
       end if;
   >>

[Adrian Pop]

I will push this in a PR an we'll see if the tests we have are passing.

[Adrian Pop]

PR is running: ​https://github.com/OpenModelica/OpenModelica/pull/613

[Francesco Casella]

Replying to adrpo:

The only question remains if this is equivalent or not, any comments?

I guess so. The only thing that changes if you turn those algorithms into equations is that the order of execution of the assignments is now determined by the BLT. From what I see, dependencies are such that the order remains the same, so I don't see any problem with this.

[Francesco Casella]

Replying to adrpo:

I will push this in a PR an we'll see if the tests we have are passing.

Do we have tests that actually try to run FMUs?

[Adrian Pop]

It seems we do, about 30 tests are failing now:
​https://test.openmodelica.org/jenkins/blue/organizations/jenkins/OpenModelica/detail/PR-613/1/tests

[Adrian Pop]

The backend does't like the equation form we have now:

"[openmodelica/fmi/ModelExchange/2.0/initializationTestsFMI_initial_equation1_me_FMU.mo:53:3-61:11:writable] Error: Internal error BackendDAECreate.lowerWhenEqn: equation not handled:
equation
  when {triggerDSSEvent > flowStatesInputs, pre(nextEventTime) < time, terminal()} then
  newStatesAvailable = initializationTestsFMI_initial_equation1_me_FMU.fmi2Functions.fmi2EventUpdate(fmi2me);
  nextEventTime = initializationTestsFMI_initial_equation1_me_FMU.fmi2Functions.fmi2nextEventTime(fmi2me, flowStatesInputs);
  if newStatesAvailable then
  fmi_x_new = initializationTestsFMI_initial_equation1_me_FMU.fmi2Functions.fmi2GetContinuousStates(fmi2me, 2, flowStatesInputs);
  reinit(fmi_x[2],fmi_x_new[2]);
  reinit(fmi_x[1],fmi_x_new[1]);
  end if;
  end when;
"

[Adrian Pop]

I guess I could try a different strategy, have only the reinit in an equation section:

algorithm
  when {triggerDSSEvent > flowStatesInputs, pre(nextEventTime) < time, terminal()} then
    newStatesAvailable := initializationTestsFMI_initial_equation1_me_FMU.fmi2Functions.fmi2EventUpdate(fmi2me);
    nextEventTime := initializationTestsFMI_initial_equation1_me_FMU.fmi2Functions.fmi2nextEventTime(fmi2me, flowStatesInputs);
    if newStatesAvailable then
      fmi_x_new := initializationTestsFMI_initial_equation1_me_FMU.fmi2Functions.fmi2GetContinuousStates(fmi2me, 2, flowStatesInputs);
      doReinit := true;
    else
      doReinit := false;
    end if;
  end when;
equation
  when doReinit then
    reinit(fmi_x[2],fmi_x_new[2]);
    reinit(fmi_x[1],fmi_x_new[1]);
  end when;

I'll give this a try.

[Francesco Casella]

Rescheduled to 1.15.0 due to lack of time

[Lennart Ochel]

This functionality isn't maintained and should be used any longer. Instead, OMSimulator and its integration into OMEdit should be used.

Instead of fixing the old implementation, we would rather like to focus on the new implementation. Can you tell us more about your use cases? Do you mainly simulate single FMUs or composite models? Do you need to connect Modelica models to FMUs? Do you use the graphical user interface or the scripting environment.

[massimo ceraolo]

Thank you for asking.
When I opened thicket, and now again more strongly, my group is asked to create causal modelica models and then convert them into FMUs (ME, 2.0).
These FMUs will be used in the final user's computing environment, but should also work when inserted into modelica simulation models as (input-output) submodels.

I tried OM simulator, read the available documentation and papers on it, but could not find, not in papers nor in documentation, how to simulate composite Modelica models containing as some input-output blocks FMUs.

So, for me is rather vital to be able to import ME FMUs, and connect them with hosts constituted by modelica models within OMEdit. This will enable us to check from within OMEdit the behaviour of the FMUs we create before trying to use them in the final user's software environment.

[Lennart Ochel]

I think we need major improvements to the graphical user interface (OMEdit) to make these kind of tasks as intuitive as possible. From a technical point of view, however, I think it is already possible and it doesn't require to wrap FMUs with some kind of Modelica code as we did earlier (with the "old approach").

Instead, you could export the Modelica part as a separate FMU and connect it to your other FMU in OMEdit and simulate the FMU-based composite model in OMSimulator. Do you think this approach would work for you?

[massimo ceraolo]

Instead, you could export the Modelica part as a separate FMU and connect it to your other FMU in OMEdit and simulate the FMU-based composite model in OM Simulator. Do you think this approach would work for you?

I think that being able to use imported FMUs just as normal Modelica blocks in a Modelica simulation model is a very important feature and would be a real pity to lose it.

I mean, we already have this! I would miss it a lot if it is dropped.

I have OM v1.14.0-dev-234-g5ef43cce8 (64-bit) in which this old approach works, and keep it installed on my PC to avoid losing this functionality. But, obviously, I cannot do this forever.

[Lennart Ochel]

I see! Can you explain why it is useful for you? As far as I can tell, it doesn't add anything to the simulation capability.

[Francesco Casella]

Replying to lochel:

I see! Can you explain why it is useful for you? As far as I can tell, it doesn't add anything to the simulation capability.

Consider this hypothetical case: a Modelica model M, containing three sub-models M.A, M.B, M.C. Inside M.C, one component is a block that is specified by an FMU.

I see no obvious practical way to handle this case using OMSimulator.

[massimo ceraolo]

In addition to Casella's example I must add that when I create an FMU I would test it thoroughly, and my best option is to simulate it inside Modelica Models whose behaviour I know very well.

Creating a block with everything external to the FMU adds more uncertainty to my analysis and check.

[Lennart Ochel]

I think it's rather the other way around. Wrapping an FMU with Modelica code, with all the external functions and the initialization section, is introducing uncertainty to your simulation. Instead exporting the Modelica model as ME-FMU with its well-definined interface would be the more reliable approach to test FMUs.

[Francesco Casella]

I agree with @lochel that testing of an FMU is best done without a Modelica wrapper.

However, the use case of an FMU embedded deep in a Modelica model hierarchy is still significant, and is not covered by OMSimulator, unless some ugly hacks are performed, like exposing its interfaces at the top level.

[Francesco Casella]

@ceraolo, could you comment on what kind of functionality do you need support for in those FMUs? Are they continuous-time models, do they generate events, do you need to reinitialize state variables inside them?

Maybe it would be possible to keep supporting FMI import for a subset of FMI which is good enough for the kind of application you have in mind.

[massimo ceraolo]

Thank you for this opportunity!

I'm rather surprised that no other people seem to be interested in this function of "hybridising" Modelica with FMUs.

For the foreseeable future, I only need to create continuous-time FMUs being a combination of blocks from the MSL blocks folder.

Among them I need to use blocks from Sources folder, some blocks from the Continuous folder (TransferFunctions, Limintegrators, PI), some from the non-linear folder (Limiter, VariableLimiter, FixedDelay).

Since I need to create non-windup controllers with variable limits, I plan to build them modifying the LimIntegrator block: for it, I need events, which occur when outMin or outMax is hit; I don't see now the need of using the reset signal of LimIntegrator.

So:

1. I need events
2. I probably won't need the ability to reset state variables, at least in the coming few months.

To conclude, in case OM becomes capable of mixing simple FMUs containing continuous-time linear blocks, some non-linear blocks such that those listed earlier, and some events, it would be a good leap forwards for me.

[Francesco Casella]

If I am not missing something, it looks like we almost have everything in place, except reinit, which from @ceraolo's comment could be not that important.

Regarding reinit, @adrpo had a suggestion in comment:15. Could we try that out?

[Adrian Pop]

@casella: I've tried to fix that in PR:
​https://github.com/OpenModelica/OpenModelica/pull/613
The result is not too good:
​https://test.openmodelica.org/jenkins/blue/organizations/jenkins/OpenModelica/detail/PR-613/31/tests

I would really suggest to just allow reinit in algorithm sections in NF for 1.16 and then properly fix it via OMSimulator or whatever for 1.17.

See also:
​https://github.com/modelica/ModelicaSpecification/issues/1727

I wonder how other tools are doing this import, I guess they also generate some kind of Modelica wrapper model that use external functions or external objects to tap the FMU.

[Per Östlund]

Replying to adrpo:

@casella: I've tried to fix that in PR:
​https://github.com/OpenModelica/OpenModelica/pull/613
The result is not too good:
​https://test.openmodelica.org/jenkins/blue/organizations/jenkins/OpenModelica/detail/PR-613/31/tests

The problem with that PR is that doReinit is only changed when the when-statement in the algorithm section triggers, and the when-equation that does the reinit:ing will only trigger when doReinit becomes true. As long as doReinit then remains true it will not do the reinit again even if there are new states available, which is what happens for the BouncingBall example at least.

So putting the reinits in an equation section might work, but we need to find some way to trigger the when-equation with the reinits each time there are new states available. I tried just setting doReinit to false at the top of the algorithm section, but then the model becomes mixed-determined for some reason. Maybe someone who knows more about modeling knows how to handle such situations?

[massimo ceraolo]

@adrpo, @perost, thank you for taking time in these issues.

As I said, it is not very important for me to have reinit now.

So, it's up to you whether to spend time to allow more sophisticated FMI-Modelica interactions. In case this is low priority for anyone except me I can, at least for the time being, limit myself to systems without it.

[Francesco Casella]

Replying to adrpo:

I wonder how other tools are doing this import, I guess they also generate some kind of Modelica wrapper model that use external functions or external objects to tap the FMU.

I have no idea, but I think using external functions/objects sounds good.

Replying to ceraolo:

@adrpo, @perost, thank you for taking time in these issues.

As I said, it is not very important for me to have reinit now.

So, it's up to you whether to spend time to allow more sophisticated FMI-Modelica interactions. In case this is low priority for anyone except me I can, at least for the time being, limit myself to systems without it.

For 1.16.0, definitely. We can reconsider this issue in September, after the 1.16.0 release and the summer break. Maybe I can help with that at that point.

@ceraolo, are other FMU import examples without reinit working for you with the current nightly? If so, I'd suggest to close this ticket and open another one on the topic of supporting reinit in FMU imports.

[massimo ceraolo]

@ceraolo, are other FMU import examples without reinit working for you with the current nightly? If so, I'd suggest to close this ticket and open another one on the topic of supporting reinit in FMU imports.

Actually, the Modelica code from which I created this ticket's FMU does not contain any Modelica reinit (I've checked the flat model). It doesn't even contain any when equation (one of which instead appears in this ticket's description in the error message).

The Modelica code from which I created the fmu is as follows (except for annotations):

model fbSubSysDX
  Modelica.Blocks.Continuous.PI PI(T = 1);
  Modelica.Mechanics.Rotational.Components.Inertia inertia1(J = 0.1);
  Modelica.Mechanics.Rotational.Sources.Torque torque1;
  Modelica.Blocks.Interfaces.RealInput PIu;
  Modelica.Blocks.Interfaces.RealOutput w;
 Modelica.Mechanics.Rotational.Sensors.SpeedSensor wSens;
equation
  connect(wSens.w, w);
  connect(inertia1.flange_b, wSens.flange); 
  connect(inertia1.flange_a, torque1.flange);
  connect(PI.y, torque1.tau);
  connect(PI.u, PIu);
end fbSubSysDX;

When I say that I don't need reinit for the time being I mean in the Modelica code.

[Francesco Casella]

OK, so I guess we need to do something about it :)

[Adrian Pop]

The Modelica wrapper code that we generate to wrap the FMUs has reinit in it. Strangely enough if you select the old front-end you can import the FMU with no issues as the old front-end doesn't check for reinit in algorithm sections.

[Francesco Casella]

We should have a closer look at that wrapper code. I can't do that right now, hopefully in month or so I could take care of that

[Francesco Casella]

Release 1.15.0 was scrapped, because replaceable support eventually turned out to be more easily implemented in 1.16.0. Hence, all 1.15.0 tickets are rescheduled to 1.16.0

[Francesco Casella]

Retargeted to 1.17.0 after 1.16.0 release

[Francesco Casella]

Rescheduled to 1.18.0

[Francesco Casella]

Ticket retargeted after milestone closed

[Francesco Casella]

1.18.0 blocker tickets moved to 1.19.0