Allow to set -d=evaluateAllParameters from the OMEdit GUI

[Francesco Casella]

If one doesn't need to change parameters at runtime without recompiling the model, which is often the case, setting -d=evaluateAllParameters can dramatically improve the compile time, numerical robustness and simulation speed.

Unfortunately, this trick is only known to a handful of advanced users, which is too bad. I myself only discovered this a couple of months ago, and it really made a difference in some critical models. I wish I had known about it before.

I would recommnd to make it possible to select this flag via some checkbox with a clear plain English description such as

Evaluate all parameters at compile time

so that more people can exploit this nice feature, and explain the trade-off (more efficiency vs. need to recompile if you want to change parameter values and re-simulate) in the online documentation.

It would also be nice if one could save this option within a commandLineOptions annotation, so you don't have to set it all the time.

@adeas31, how could this be implemented?

[massimo ceraolo]

I have a proposal and two bug reports about this ticket.

Proposal.
I tried -d=evaluateAllParameters for the first time today, on Windows with

OMEdit v1.13.0-dev-95-g4eb7a19 (64-bit)
Connected to v1.13.0-dev-380-g8b177b9 (64-bit)

and got spectacular timing improvements. I tried just one mildly complex model and the whole compilation+simulation time went from around 20s to #around 6s. I cannot be more accurate, nor mention other tests because of issues 1 and 2 below.

The advantage is so huge that I wonder whether it should even be the default option.
According to my experience, the basic usage of OM is to create models and simulate them. Only when one has reached a stable model he needs to re-simulate it many times changing parameters.
Otherwise one could use other people's ready models, to simulate several times and in this case these models could contain indication that all parameters should not be evaluated inside the commandLineOptions annotation proposed in this ticket.

So, having as default all parameters evaluated should enhance perceived quality of OM significantly, and I propose this.

Issue 1.
Under the above windows version I found that, once set, this option is always active, even after closing and reopening OMEdit! It must somehow have been written on the registry.

Issue 2
I tried -d=evaluateAllParameters under Linux with:

OMEdit 1.13.0~dev-208-g609455a
Connected to OpenModelica 1.13.0~dev-1445-ge8110d1
Connected to OMSimulator HEAD-HASH-NOTFOUND-linux

and dit did not work: times were roughly the same and parameters could still be changed for resimulation.

[Francesco Casella]

@ceraolo, did you set the -d=evaluateAllParameters option in the Options|Simulation|OMC Command Line Options window?

[massimo ceraolo]

Replying to casella:

@ceraolo, did you set the -d=evaluateAllParameters option in the Options|Simulation|OMC Command Line Options window?

Yes, except for the fact that in the Windows version I used its name was still OMC flags.

Have you tried my the issues I mentioned?, they are very strange, but I carefully checked them before posting. In case of need I can do additional tests from my Work PC, which has both versions mentioned in my comment (the linux in a Virtual box virtual machine).

[Francesco Casella]

After the discussion on Monday's devmeeting, there is consensus on making this option available in the Simulation Setup window. I would suggest to put it in the General tab, immediately below Simulation interval.

We could have a checkbox named Evaluate All Parameters with a comment such as: Faster simulation, cannot change parameter values and re-simulate. This would add the -d=evaluateAllParameters flag when building the model.

I think this setting should remain the same on the specific model during the session, so that if you re-compile and simulate multiple times, you don't have to set this each and every time. If you want to make this permanent, I'm not sure how to proceed. Currently, you can save simulation flags via the GUI, but not command line options. We could add a checkbox to just save this specific setting, but I guess we should probably try to have a consistent way to deal with other such command line options.

[Adeel Asghar]

Replying to casella:

After the discussion on Monday's devmeeting, there is consensus on making this option available in the Simulation Setup window. I would suggest to put it in the General tab, immediately below Simulation interval.

I go against this consensus.

We could have a checkbox named Evaluate All Parameters with a comment such as: Faster simulation, cannot change parameter values and re-simulate. This would add the -d=evaluateAllParameters flag when building the model.

The handling of command line options is not via the Simulation Setup Window.

I think this setting should remain the same on the specific model during the session, so that if you re-compile and simulate multiple times, you don't have to set this each and every time.

If you just want it at the model level you should add it to the model. In OMEdit right click inside the icon/diagram view of the model and choose Properties. Then OMC Command Line Options and in the text field write -d=evaluateAllParameters. ​More information is here.

If you want to make this permanent, I'm not sure how to proceed.

You should add it to Options|Simulation|OMC Command Line Options then all models will use it.

Currently, you can save simulation flags via the GUI, but not command line options. We could add a checkbox to just save this specific setting, but I guess we should probably try to have a consistent way to deal with other such command line options.

We do save the command line options if you add them to the model as explained above.

I think there is nothing that should be done for this ticket.

[Francesco Casella]

I try to give some arguments to my proposal.

For the tool developers, who have been working for years with concepts like front-end, back-end, Susan, run-time, etc, the distinction between the various stages of building and simulation is second nature. From this point of view, the current status is fine: omc command line flags are set in Tools|Options, simulation flags (i.e., arguments to the simulation executable command line) are set in Simulation Options.

My argument is that this is not the mindset of the vast majority of our users, which can be domain experts or students, but are not simulation tool experts in most cases. For them, OMEdit is a kind of magic box that turns equation-based object-oriented models into simulation results, using smart methods and state-of-the art tools. People using OMEdit may not even know about command lines (they may actually not even know what a command line is), nor they should, in order to use the tool. In fact, the whole point of a GUI is to forget about command lines.

Of course, more advanced users will better understand the various transformations that are carried out in the process, but we cannot expect all users to have this kind of awareness.

Now, from an end-user perspective, there is a simple fact: most non-trivial models compile faster, simulate faster and are more numerically robust if all parameters are evaluated, because then OMC can apply a lot more smart optimizations; the drawback is that you cannot change parameters and re-simulate quickly, which is a big deal in some cases, but absolutely irrelevant in many other cases. Unfortunately, most end-users (including many veterans, see comment:1) are simply not aware of this. I think you don't (and you shouldn't!) need to be an advanced or expert user in order to get this benefit, which can be really substantial. The only thing to be understood is that you have a trade-off between speed and robustness on one side, and the possibility of changing parameters without recompiling on the other side.

This leads to Requirement no. 1: This option should be very visible even to beginner users, so they understand the trade-off, experiment with it and understand how effective it can be.

However, the choice of evaluating or not evaluating is not a choice that you make for life: it is typically model-specific, and it can also change during the lifetime of the model. For example, when you start building it you really change stuff all the time, so you need to recompile every time you apply some changes, hence evaluating parameters is a good idea. Later on, you may want to fine-tune the model by only changing parameters, and then not evaluating them is the way to go.

This leads to Requirement no. 2: The choice to evaluate or not to evaluate should be performed on a model-by-model basis.

I understand @adeas31's concern of mixing up things, and I am completely open to suggestions so as to how to implement this feature, but I am convinced that requirements 1. and 2. should be fulfilled in some way. The current situation, that one needs to add an obscure debug flag, possibly adding it to other existing ones (which requires understanding the syntax of command line parameters) in the Tools | Options | Simulation window is not really addressing those requirements.

BTW, again from an end-user perspective, if I am not aware of the technical details of how omc handles things, I could have difficulties understanding the difference between Tools | Options | Simulation and Simulation Setup, as they seem to point to the same kind of thing - what is the difference between Options and Setup, after all? The only difference I can think of (unless I have been introduced to the sacred mysteries of how omc works internally) is that Tools | Options | Simulation is to set global options affeting all models, while Simulation Setup is meant for model-specific choices that can change every time you simulate a model. Given requirements 1, and 2., after all Simulation Setup seems to me a more fit place to put this option.

[Francesco Casella]

Replying to adeas31:

If you just want it at the model level you should add it to the model. In OMEdit right click inside the icon/diagram view of the model and choose Properties. Then OMC Command Line Options and in the text field write -d=evaluateAllParameters. ​More information is here.

I see, I was not aware of this. Again, I find it quite confusing to have model-specific Simulation Options and Simulation Setup handled in a completely different way. The only reason they are different is that one affects omc command line options, the other one affects the runtime simulation executable command line options.

My point is that this is a low-level technical detail, that people using a GUI are not (and should probably not necessarily be) aware of. I think a good GUI should support usage patterns effectively, not simply be a thin wrapper around command-line-based tools.

BTW, I'm not even sure why evaluateAllParameters is a debug flag at all: evaluating all parameters has nothing to do with debugging, nor with advanced development activity. It's just getting more efficient simulation if you don't care about changing parameters at runtime. I guess this should be made a full-fledged flag, but that's a minor issue.

If you want to make this permanent, I'm not sure how to proceed.

In fact, you answered to this question, see above.

[Francesco Casella]

Performance-wise, this feature has quite some impact, but debug flags are normally used by developers and experts.

I think a user-friendly solution is needed for v1.14.0.

[Adeel Asghar]

Fixed in f818ba8/OMEdit.

[Per Östlund]

I noticed that -d=evaluateAllParameters is turned on by default in OMEdit now. That's probably going to cause a lot of issues with the NF currently, but those will have to be fixed anyway I guess.

However, have we done any proper testing of this? I.e. has anyone run the library coverage tests with the option turned on (it's still disabled by default in OMC after all)?

[Francesco Casella]

Replying to perost:

I noticed that -d=evaluateAllParameters is turned on by default in OMEdit now. That's probably going to cause a lot of issues with the NF currently, but those will have to be fixed anyway I guess.

However, have we done any proper testing of this? I.e. has anyone run the library coverage tests with the option turned on (it's still disabled by default in OMC after all)?

I agree 100% with this comment.

@sjoelund.se, could you manage to get add one test case on the MSL, so we can see if there are regressions?

[Francesco Casella]

In #5436, @ceraolo notices that this option is by default true, and argues against that. I agree with this remark, and I also think this option should be false by default, so that first-time users can actually modify parameters.

I would also propose to change the text of the option from

Evaluate all parameters at compile time

to

Evaluate all parameters (faster simulation, cannot change them at runtime)

[Adeel Asghar]

I thought we had a consensus to make this default and ceraolo was the one who proposed it. Anyways, may bad for wrong interpretation. I will fix it real quick.

[Adeel Asghar]

Fixed in f1f591f/OpenModelica.

[Francesco Casella]

Thanks Adeel!

In fact, there is one last pending issue. When parameters are evaluated, the input fields for the parameters in the Variables Browser are still available. You can actually change the values in there, but as soon as you re-run the simulation, they are reverted to the evaluated variable. This behaviour is a bit odd, and in fact it looks a bit like a bug, tough there is some logic to it.

Could you instead issue some warning in a pop-up window when people try to modify the parameters but the evaluate parameters flag is switched on? The message could be something as:

The "Evaluate parameters" option is turned on, so changing parameter values at
runtime has no effect. If you want to change the value of the parameters and re-simulate without re-compiling, please uncheck "Evaluate parameters" in Tools|Options|Simulation. Note that this may make the simulation slower.

[Adeel Asghar]

See ticket:5377#comment:27