OMEdit does not import initial conditions correctly

[Francesco Casella]

Problem: I want to restart a simulation from the state of a previously saved simulation.

OMEdit allows to specify in Simulation Setup | Simulation Flags an Equation System Initialization file and an Equation System Intialization Time. I report two problems:

• if I use the .mat file which is generated by default, OMC then tries to write to it and read to it simulataneously and it crashes. So, I need to copy or rename that result file, which is a bit inconvenient
• once I copy the file, OMEdit seems to ignore the intial conditions from the .mat file anyway.

Test case:

model foo
  Real x(start = 2, fixed = true);
equation 
  der(x) = -x;
end foo;

[Lennart Ochel]

The imported values from the mat files are used as start-values. If they should be used as initial solution you have to set the initialization method to 'none'. Otherwise, the initial equations get not ignored, but solved with the imported start-values.

[Lennart Ochel]

This is the simulation flag: -iim=none
I think there is also a drop-down box in the GUI.

[andrea.bartolini@…]

test case

[anonymous]

test result plot

[andrea.bartolini@…]

the proposed solution works fine with the example from Francesco, but with the attached example Test2.mat (built with modelica standard library blocks) it doesn't work...
I've run the model for 2 second first and then I've renamed the model and run it for 2 second more, using the previous mat result as startup values and with initialization method set to "none".

The attached png file shows the result.

[Andrea Bartolini]

I've some news about the problem, I think to have found a possible cause of it.

One of the model that I used for my test (the "first order" filter from the modelica standard library, here renamed as "myfirstorder1") contains the following declaration:

extends Modelica.Blocks.Interfaces.SISO(y(start = y_start));

and, during the initialization of the simulation, the modifier "y(start = y_start)" seems to overwrite the "y" value read from the .mat file with the value from "y_start", as per log in the following:


LOG_INIT | info | ### START INITIALIZATION ###
......
LOG_INIT | info | import real variables
LOG_INIT | info | | myfirstorder1.y(start=0.393469)
......
LOG_INIT | info | updating start-values
| | | | | myfirstorder1.y(start=0)
LOG_INIT | info | initialization method: none [sets all variables to their start values and skips the initialization process]
.......
LOG_SOTI | info | ### SOLUTION OF THE INITIALIZATION ###
| | | | | states variables
| | | | | | [1] Real myfirstorder1.y(start=0, nominal=1) = 0 (pre: 0)


Replacing the previous declaration with the following:

extends Modelica.Blocks.Interfaces.SISO;

then the initialization from the .mat file works properly, as per following log:

LOG_INIT | info | ### START INITIALIZATION ###
LOG_INIT | info | import real variables
LOG_INIT | info | | myfirstorder1.y(start=0.393469)
......
LOG_INIT | info | updating start-values
LOG_INIT | info | initialization method: none [sets all variables to their start values and skips the initialization process]
......
LOG_SOTI | info | ### SOLUTION OF THE INITIALIZATION ###
| | | | | states variables
| | | | | | [1] Real myfirstorder1.y(start=0.393469, nominal=1) = 0.393469 (pre: 0)

Unfortunately, this solution requires the modification of models in the modelica standard library....

[Francesco Casella]

I have investigated the matter with Andrea. We have double-checked that the back-end works as explained by Lennart: take the model

  model Test1
    Real x;
    Real y(start = 0);
  equation
    der(x) = 10;
    der(y) = 10;
  initial equation
    x = 5;
    (y - 1) * (y - 12) + 1e-6 * sin(y) = 0;
  end Test1;

run it for 1 second and then load the initialization file at t = 1.

If we set the initialization mode to 'none', then the simulation restarts from the previous end states (x = 15 and y = 11),

If we set the initialization mode to 'symbolic', the simulation starts from x = 5, because the initial equation for x can be solved explicitly and the imported start value is ignored, and from y = 12, because the imported start value causes the initial equation for y to converge on y = 12 and not on y = 1, as it would with the default start value. So far so good.

However, if you take this model:

model Test2
  parameter Real x_start = 5;
  Real x(start = x_start, fixed = true);
  Real y(fixed = true);
equation
  der(x) = -x + 1;
  der(y) = -y + 1;
end Test2;

run it for 1 second, load the initialization file at t = 1, and set initialization mode to 'none', the initial value of x is still set to 5 (i.e., x_start) and not to 2.45 (i.e., the imported value).

It seems that this behaviour is deliberate, as we see in the log the following lines:

LOG_INIT		| info	 | updating start-values
|		        | |	 | | x(start=5)

but this makes no sense at all: if one wants to start from a previous state, this should override any start value set in the code, even those based on parameters.

[Lennart Ochel]

Good analysis, thanks. This need to be changed.

[Lennart Ochel]

I changed the behavior in r22479. Please check if it behaves now as expected.
Models that are using fixed=true conditions to specify the initial problem may behave now different compared with those who are using actual initial equations. This can cause confusion. Maybe –iif should be only allowed to be used in combination with –iim=none.

[Lennart Ochel]

The combination of the flag –iif and –iim=symbolic is useful. Because it should not make any difference if the initial equations are specified using fixed attribute or initial equations, the following output is intended:

./Test2 -iif=_Test2_res.mat -iit=1.0 -lv=LOG_SOTI
LOG_SOTI          | info    | ### SOLUTION OF THE INITIALIZATION ###
|                 | |       | | states variables
|                 | |       | | | [1] Real x(start=2.47151, nominal=1) = 5 (pre: 2.47151)
|                 | |       | | | [2] Real y(start=0.632122, nominal=1) = 0 (pre: 0.632122)
|                 | |       | | derivatives variables
|                 | |       | | | [3] Real der(x) = -4 (pre: -1.47151)
|                 | |       | | | [4] Real der(y) = 1 (pre: 0.367878)

./Test2 -iif=_Test2_res.mat -iit=1.0 -lv=LOG_SOTI -iim none
LOG_SOTI          | info    | ### SOLUTION OF THE INITIALIZATION ###
|                 | |       | | states variables
|                 | |       | | | [1] Real x(start=2.47151, nominal=1) = 2.47151 (pre: 0)
|                 | |       | | | [2] Real y(start=0.632122, nominal=1) = 0.632122 (pre: 0)
|                 | |       | | derivatives variables
|                 | |       | | | [3] Real der(x) = -1.47151 (pre: 0)
|                 | |       | | | [4] Real der(y) = 0.367878 (pre: 0)

[Lennart Ochel]

I added the model from above as test case in r22546.

[Francesco Casella]

Not done yet... Consider this test case:

package Test
  model M
    parameter Real p(fixed = false);
  initial equation 
    20 = p*10 + 10;
  end M;

  model Q
    M m;
  end Q;
end Test;

Test procedure:

• Simulate Q for one second. In the result files, p = 1 as expected.
• Copy result file to xxx.mat
• Set Initialization method = none
• Set Equation System Initialization file = xxx.mat
• Set Equation System Initialization time = 1
• Simulate. In the results, p = 0 (wrong result!)

Excerpt from the log:

LOG_INIT		| info	 | initialization method: none            [sets all variables to their start values and skips the initialization process]
LOG_INIT		| info	 | parameter values
|		| |	 | | real parameters
|		| |	 | | | [1] parameter Real m.p(start=1, fixed=false) = 0

Apparently, all variables are set to their start values and the initialization process is skipped, but some computations are still performed for the parameters. This is obviously wrong, and doesn't work in particular when the paramaters have fixed = false and are computed together with the initial states.

Suggested fix: when Initialization method = none, also load all parameter values from the .mat file, and skip all parameter computations.

[Martin Sjölund]

This ticket was not closed for 1.9.1, which has now been released. It was batch modified for milestone 1.9.2 (but maybe an empty milestone was more appropriate; feel free to change it).

[Martin Sjölund]

Milestone changed to 1.9.3 since 1.9.2 was released.

[Martin Sjölund]

Moved to new milestone 1.9.4

[Francesco Casella]

Lennart, now that you have extensively reworked the initialization part of the back-end, would you mind having one more look at comment:12?

Thanks!

[Lennart Ochel]

Replying to casella:

Lennart, now that you have extensively reworked the initialization part of the back-end, would you mind having one more look at comment:12?

Thanks!

The initialization process considers now overwritten start values properly. Hence you do not need the -iim=none flag anymore. However, parameters get still not imported from the mat file.

[Lennart Ochel]

Sorry that this took so long. Now I have fixed the mat import for parameters (b635a2).

[Lennart Ochel]

Replying to casella:

Not done yet... Consider this test case:

package Test
  model M
    parameter Real p(fixed = false);
  initial equation 
    20 = p*10 + 10;
  end M;

  model Q
    M m;
  end Q;
end Test;

Test procedure:

• Simulate Q for one second. In the result files, p = 1 as expected.
• Copy result file to xxx.mat
• Set Initialization method = none
• Set Equation System Initialization file = xxx.mat
• Set Equation System Initialization time = 1
• Simulate. In the results, p = 0 (wrong result!)

I added this as test case with ​PR146.

[Martin Sjölund]

Milestone renamed

[Martin Sjölund]

Milestone renamed

[Lennart Ochel]

The changes in ​OpenModelica#629 will introduce slightly different behavior for the following model pattern:

package initializationTests
  model bug_2673
    parameter Real x_start = 5;
    Real x(start = x_start, fixed = true);
    Real y(fixed = true);
  equation
    der(x) = -x + 1;
    der(y) = -y + 1;
  end bug_2673;
end initializationTests;

Overwriting the start value of x (or using the runtime flag -iif) has no longer an impact on the initialization of x and der(x). This makes it consistent with the definition of the initial condition x = x_start inside the model. Therewith, only changing x_start will have an impact on the initialization of x unless the initialization is deactivated with the runtime flag -iim none.

[Francesco Casella]

@lochel, I'm sorry, but I'm afraid there is a misunderstanding; this is not what I was asking for, and honestly I have no use for it. I'm not sure if you have some use case for it - if you do, can you please explain it to me?

Unfortunately, for some reason, the Modelica community (including FMI) has some issues with the concept of "snapshot", which was commonplace in older simulation environment of the 20th century. Bear patience with an old guy like me that still need old-fashioned functionality. BTW, this is exactly what Dymola's "Import Initial" functionality does.

The use case I have in mind is the following:

• I have a certain Modelica model of a system, with certain initial equations and start values.
• Using those initial equations and start values, I run a certain simulation and produce a result file.
• Afterwards, I want to resume the simulation from a certain state that was reached during the previous simulation (the final state, or possibly any intermediate state, specified by the value of time) and continue it further. At this point, the original start values and the parameters used to specify them are completely irrelevant.

For example, I start the simulation of a building on Jan 1, 00:00 and run it through Jan 31 24:00. Then I later want to resume it from the same state and continue it until Feb 28 24:00.

This require to load the following information from the saved results:

• the values of all state variables
• the values of all iteration variables of nonlinear implicit systems

in principle, the former are only necessary; however, in practice the latter are also required becaus the original start values, which were OK to solve the algebraic loops in the initial state, may no longer be OK to solve the algebraic loops in the state you want to restart from.

Again, the main requirement of this functionality is that it overrides completely all the information about initial equations and start values of the model. The rationale is that you need that information to get the initial state of the first simulation, but once you get going, you no longer care about it at all.

As I understand it, your PR goes in the opposite direction, and I fail to understand the rationale of doing that. x_start is only useful to compute the initial state, but it will not (and should not!) be updated with the value of x at the end of the previous simulation, where you want to restart from. Why bother? If you have 600 states which are set by 150 start parameters, which should you update each and every one of them? Just reload the previously state exactly as it was, and get the simulation going again.

BTW, that also has a 100% guarantee of success in terms of solver convergence, as long as all the information stated above was saved and can be retrieved.

Thanks!

[Francesco Casella]

BTW, I also think this feature is also extremely useful for FMI.

Currently, there is no easy way to just import the initial state from a previous simulation result (e.g. of the same model run in OpenModelica) and use it as an initial state wholesale. Instead, you need to painstakingly adjust a potentially very large number of x_start parameters, which is tedious and inconvenient. Furthermore, there is no easy way to override the initial equations embedded in the FMU. The way to do that is to modify the original Modelica code by removing all initial equations, which is sometimes either extremely complicated or not possible at all.

One would like to just say: forget about all those initial equations, and just restart from a previously saved state, with just one boolean choice.

I understand FMI 2.0 at last introduced this possibility, though for some reason I cannot fathom it made it optional, so nobody's implementing it.

I read at page 119 of FMI 2.0.1 that

The complete FMU state can be saved, restored, and serialized to a byte vector (that can be stored on file). As a result, a simulation (both for Model Exchange and for Co-Simulation) can be restarted from a saved FMU state.

This "snapshot feature" is extremely useful, e.g., in the case of power plant models, where getting a good initial condition is a lengthy and painful process, possibly including homotopy and other black magic, which involves setting a large number of parameters. Ideally, you should be able to load this complete state not only from a saved FMU state, but also from a saved simulation of the Modelica model that was used to generate the FMU. The typical use case is then the following:

• an expert generates the Modelica model
• the same expert generates a number of initial conditions through some non-trivial process
• the expert then ships the FMU and the set of saved initial condition
• the final user can simply load them and get going, without getting involved in any complicated and potentially failure-prone initialization procedures

I have been looking for this functionality since the early 2000s, when I was using the Dymola block in Simulink which did not, unfortunately, provide this functionality and proved such a pain that I swore to myself never to attempt using it again for this purpose. I understand it's still not there in OMC-generated FMUs. I would strongly suggest to get it implemented. I am sure lots of users (including myself) will appreciate it immensely.

Thanks!

[Lennart Ochel]

@casella, as far as I can tell, the changes do not break any of the use cases you just described. It might just be a big misunderstanding. I tried to make it very clear in last comment, but I probably just confused you. In order to import a previous model state, you have to disable the initialization. This is still possible and I've actually tested it before pushing the changes. Also overwriting start values works just fine and as expected in OMEdit and from command line.

If I overlooked something major here, please let me know.

The motivation for the changes is indeed the FMI export. It is embarrassingly wrong and buggy, and canot be used for any practical application. My goal for the next year is to make it one of our strong features in OpenModelica. Depending on the amount of resources that we can use for this development, it may even include the FMI "snapshot feature" which indeed is crucial for many co-simulation applications.

[Lennart Ochel]

I am sorry for the confusion. Please let me try to make it very clear now: My intention was just to report slightly different behavior of one of the test cases related to this ticket, i.e. ​bug_2673.mos. It is actually correct now and has been wrong before (see the ​diff).

It is the model that I posted earlier on this ticket:

package initializationTests
  model bug_2673
    parameter Real x_start = 5;
    Real x(start = x_start, fixed = true);
    Real y(fixed = true);
  equation
    der(x) = -x + 1;
    der(y) = -y + 1;
  end bug_2673;
end initializationTests;

The test case simulates the model from time=0.0 to 1.0 and then imports the result file for another two simulation runs, with different runtime settings:

First: -iif=bug_2673.mat -iit=1.0 -lv=LOG_SOTI

This imports start values from the original result file at time=1.0. Then, it executes the initialization (using the imported start values). Therefore, x has to be 5.0:

LOG_SOTI          | info    | ### SOLUTION OF THE INITIALIZATION ###
|                 | |       | | states variables
|                 | |       | | | [1] Real x(start=2.47151, nominal=1) = 5 (pre: 2.47151)
|                 | |       | | | [2] Real y(start=0.632122, nominal=1) = 0.632122 (pre: 0.632122)
|                 | |       | | derivatives variables
|                 | |       | | | [3] Real der(x) = -4 (pre: -1.47151)
|                 | |       | | | [4] Real der(y) = 0.367878 (pre: 0.367878)

Second: -iif=bug_2673.mat -iit=1.0 -lv=LOG_SOTI -iim none

This imports start values from the original result file at time=1.0. Then, it skips the initialization in order to start from a "snapshot". Therefore, x has to be 2.47...:

LOG_SOTI          | info    | ### SOLUTION OF THE INITIALIZATION ###
|                 | |       | | states variables
|                 | |       | | | [1] Real x(start=2.47151, nominal=1) = 2.47151 (pre: 0)
|                 | |       | | | [2] Real y(start=0.632122, nominal=1) = 0.632122 (pre: 0)
|                 | |       | | derivatives variables
|                 | |       | | | [3] Real der(x) = -1.47151 (pre: 0)
|                 | |       | | | [4] Real der(y) = 0.367878 (pre: 0)

[Martin Sjölund]

It might be possible to skip initialization in this simple case, but in the general case we need to initialize external objects and other external state present in the initialization equations. That's where the hard part lies.