Online documentation of OMC flags

[casella]

The OMC compiler and the executable codes it generates have a lot of optional flags. The documentation of these flags in the User's Guide and System Documentation is inevitably going to be incomplete and outdated, because new flags are constantly being added or removed during the development process.

Furthermore, the exhaustive documentation of these flags in those two documents makes them bulky and awkward-looking.

Currently, some basic information is available at run-time by running the appropriate executables with the appropriate help flags. This is not very convenient, in particular for OMEdit and OMNotebook users, which are not familiar with the settings required for the command line usage. They might even ignore that a command-line interface actually exist, and they should not be forced to learn how to use that in order to use the GUI - what would a GUI be good for, in that case?

Proposal: automatically gather that information and generate some reports that can be published online and included in the installation package as well. Then, the bulk of this information can be removed from the User's Guide and System Documentation, which should only refer to this automatically generated documentation.

Requirements:

• The documentation should be automatically generated from information present in the source code (à la Doxygen).

• The document generation should be handled by Hudson, so it is always up-to-date.

• The generated documentation should be available in platform-independent formats, e.g. HTML for online viewing and PDF for those who prefer an old-fashioned comprehensive printed reference.

• It would be good to have some kind of tags, so that the appropriate sections of the generated documentation could be referred to. For instance, when setting up a simulation in OMEdit, the "method" field should have a "Help" button close to it; clicking on this button should open an HTML page with all the information about integration methods. Ditto for all other flags.

• Links to this documentation should also be included in the Help menu of all the OMC suite tools (OMEdit, OMShell, OMNotebook), and a local copy included in the released versions and nightly builds

Suggestions are welcome for the best choice of tools to meet these requirements with the least possible effort by the developers.

[lochel]

One way to solve this in a short term would be an automatically generated documentation of all omc and simulation flags by omc itself. I guess, this could be implemented rather easy by a mos-script in combination with a couple of new api calls:

getBriefOmcFlagDescription("flag"); // returns a string
getBriefSimFlagDescription("flag"); // returns a string
getOmcFlagDescription("flag");      // returns a string
getSimFlagDescription("flag");      // returns a string
getAllOmcFlags("flag");             // returns a list of strings
getAllSimFlags("flag");             // returns a list of strings

This need probably to be extended to also cover omc debug flags and so on.

With this functionality it should be possible to just write a mos-script that generates the documentation as html and tex file.
Depending on how quick this get realized, we could add this documentation as appendix to the User's Guide before the final 1.9.1 release. Any comments on this approach?

[sjoelund.se]

You can already do help("debug"), help("profiling"), etc. If you had something like help("debug", format="latex"), we could generate documentation using that as well.

[lochel]

Yes, that's true. But it is not possible to generate a help text of a specific flag right now. That would be necessary to generate a user-friendly index page with cross references.

[lochel]

It would be also helpful in the OMNotebook environment, to bring up the needed information for a specific flag.

[lochel]

I have something in mind like ​https://build.openmodelica.org/Documentation/OpenModelica.Scripting.html.
There, you have an index page with a brief description and one can click on a specific item to get detailed information.

[sjoelund.se]

Replying to lochel:

Yes, that's true. But it is not possible to generate a help text of a specific flag right now. That would be necessary to generate a user-friendly index page with cross references.

Why would it not be possible to generate cross-references for example for each individual debug-flag when you do help("debug", format="latex")?

It seems help() only generates info for multi-option flags. But we could easily change it to also print help-text for int/bool flags, etc.

[lochel]

Replying to sjoelund.se:

Replying to lochel:

Yes, that's true. But it is not possible to generate a help text of a specific flag right now. That would be necessary to generate a user-friendly index page with cross references.

Why would it not be possible to generate cross-references for example for each individual debug-flag when you do help("debug", format="latex")?

Well, that might be possible. But how do you want to generate the detailed description for flags with generated cross references in the main help page?

[sjoelund.se]

Replying to lochel:

Well, that might be possible. But how do you want to generate the detailed description for flags with generated cross references in the main help page?

The index would correspond to help(format="latex"), right? That is the "index page" right now.

[lochel]

Replying to sjoelund.se:

Replying to lochel:

Well, that might be possible. But how do you want to generate the detailed description for flags with generated cross references in the main help page?

The index would correspond to help(format="latex"), right? That is the "index page" right now.

Yes, but of course for all kind of available topics (omc, simulation, …).

[sjoelund.se]

So structure the help system to output the sections you want. We have sections for help("simulation") as well after all.

[casella]

Replying to lochel:

I have something in mind like ​https://build.openmodelica.org/Documentation/OpenModelica.Scripting.html.
There, you have an index page with a brief description and one can click on a specific item to get detailed information.

Incidentally, I was not aware of the existence of such a page. I searched for "scripting" in the User's Guide and found nothing. It is available as a separate document here ​https://www.openmodelica.org/useresresources/userdocumentation, but it is not cross-referenced in the User's Guide, which IMHO is a mistake. I guess this is mostly a duplicate of pages 155-176 of the User's Guide, which I'm afraid have been added manually (God forbid!)

Also in this case, we should generate those pages automatically and then incorporate them in the manual (possibly as an appendix, I would say), or, maybe, just refer to something that can be browsed online/offline and is thus always up-to-date.

Should I open a ticket for this part of the documentation as well? I think we need to do some extra work to rationalize the whole documentation and make it more consistent, accessible and up-to-date.

[sjoelund.se]

Replying to casella:

Replying to lochel:

I have something in mind like ​https://build.openmodelica.org/Documentation/OpenModelica.Scripting.html.
There, you have an index page with a brief description and one can click on a specific item to get detailed information.

Incidentally, I was not aware of the existence of such a page. I searched for "scripting" in the User's Guide and found nothing.

The OM library documentation is even linked from ​https://modelica.org/libraries :(

It is available as a separate document here ​https://www.openmodelica.org/useresresources/userdocumentation, but it is not cross-referenced in the User's Guide, which IMHO is a mistake. I guess this is mostly a duplicate of pages 155-176 of the User's Guide, which I'm afraid have been added manually (God forbid!)

Some of them were automatically generated and then manually copy-pasted from some latex to html if I remember correctly...

Also in this case, we should generate those pages automatically and then incorporate them in the manual (possibly as an appendix, I would say), or, maybe, just refer to something that can be browsed online/offline and is thus always up-to-date.

Should I open a ticket for this part of the documentation as well? I think we need to do some extra work to rationalize the whole documentation and make it more consistent, accessible and up-to-date.

Adrian played with some word-to-latex conversion.
​https://dev.openmodelica.org/libraries/OpenModelicaUsersGuide.linux.pdf

But I think what we need most is generating online documentation as it is easily indexed by Google. book.xogeny.com is generated by sphinx, which can do both html and latex.

[lochel]

Replying to sjoelund.se:

It seems help() only generates info for multi-option flags. But we could easily change it to also print help-text for int/bool flags, etc.

help() should also be extended to work for simulation flags as well. It could be something like help("<flagname>", "Simulation");.

[sjoelund.se]

Why is this needed? Is it not enough to print all simflags using help("simulation")?

[lochel]

Replying to sjoelund.se:

Why is this needed? Is it not enough to print all simflags using help("simulation")?

No, that is not enough. What we should have is a list with a brief description of all flags as overview (that is already available). Moreover it is necessary to get a detailed description of a specific flag. That is just the same as for the omc flags where this is already supported somehow (at least for enumeration flags).

The simulation runtime contains already a brief and a detailed description for each flag. That information should be accessible from the omc. This would be good for OMNotebook/OMShell usage as well as for generating a suitable documentation of all flags.

[sjoelund.se]

Split into help("simulation") and help("simulationdetailed")?

[lochel]

Replying to sjoelund.se:

Split into help("simulation") and help("simulationdetailed")?

I think it will be annoying to always dump all detailed flag descriptions at once. So why should we not provide the same functionality for all flags as we already do for the omc enumeration flags?

[sjoelund.se]

Because the simflags are not really part of omc? Soon people will want the scripting help to give information for cppruntime flags and C#, and... everything.

Besides, if you put it all in nice generated html, will it matter?

[casella]

Replying to sjoelund.se:

Because the simflags are not really part of omc?

Yes, they are. Simulation executables are generated by omc and simflags are actually defined inside the omc source code, so I don't see why they should be treated differently.

Soon people will want the scripting help to give information for cppruntime flags and C#, and... everything.

Absolutely! That is precisely my point. When people in the OSMC eco-system implement some feature that has a flag or a function to use it, they write (or should write) some help info in the code base. IMHO, we need to make this surface out and be readily accessible without the need of belonging to some inner circle, or the need of bothering the gurus over forums. OMC is not meant for geeks, it is meant for engineers!

The documentation of all flags and functions whatsoever are used by or related somehow to omc should be readily accessible, coherently organized, hyperlinked and, last but not least, always up to date. Front-end flags, back-end flags, code generation flags, run-time flags, flags which activate fancy experimental features, omc API functions, you name them. Why do we want to end-users to play treasure hunt in order to find this out? Do we want Hidden flags as well? Otherwise, most of the capabilities of omc will never be tested and used by people doing actual modelling, which ultimately means that all the developers' efforts will be wasted.

Also, there should be one access point from which all of this documentation can be reached, and this access point needs to be widely advertised both online and in the User's Guide and System Documentation. Of course this doesn't mean that this entry point should be the only way to access such information, but IMHO it is important, in particular for newcomers, that there is a single access point from which the entire documentation can be easily reached.

Besides, if you put it all in nice generated html, will it matter?

I agree. I guess that if there is a nice html documentation containing the same information which is currently avaliable by asking for help from the command line, nobody (save a maybe a few die-hards which insist on using ed to write Modelica code) will ever use the command line option again. It is important that the way this information is dumped by omc makes it easy to organize and hyperlink it nicely in OMC. Maybe an XML file would do the job better than the current fixed-font more-or-less-pretty-printed one does, and would be more easily transformed into HTML.

[sjoelund.se]

Replying to casella:

Soon people will want the scripting help to give information for cppruntime flags and C#, and... everything.

Absolutely! That is precisely my point. When people in the OSMC eco-system implement some feature that has a flag or a function to use it, they write (or should write) some help info in the code base. IMHO, we need to make this surface out and be readily accessible without the need of belonging to some inner circle, or the need of bothering the gurus over forums. OMC is not meant for geeks, it is meant for engineers!

The documentation of all flags and functions whatsoever are used by or related somehow to omc should be readily accessible, coherently organized, hyperlinked and, last but not least, always up to date. Front-end flags, back-end flags, code generation flags, run-time flags, flags which activate fancy experimental features, omc API functions, you name them. Why do we want to end-users to play treasure hunt in order to find this out? Do we want Hidden flags as well? Otherwise, most of the capabilities of omc will never be tested and used by people doing actual modelling, which ultimately means that all the developers' efforts will be wasted.

They are not really part of omc. omc can be built entirely without the cppruntime. The C# runtime isn't even in our svn! With the cruntime simulations, we are able to link in the actual simulation flags and query them. Which is what we do, and why help("simulation") works.

With cppruntime, we cannot do this and make it part of the omc help system. We could generate the documentation from them, sure, but it couldn't really be part of the omc help...

We should document them, but I am not sure we need everything inside the omc executable itself.

Also, there should be one access point from which all of this documentation can be reached, and this access point needs to be widely advertised both online and in the User's Guide and System Documentation. Of course this doesn't mean that this entry point should be the only way to access such information, but IMHO it is important, in particular for newcomers, that there is a single access point from which the entire documentation can be easily reached.

I think the user's guide should be an online reference if we want to link everything together. You would also need to get the cppruntime people to generate documentation in the same format as the cruntime, but this should be doable.

Besides, if you put it all in nice generated html, will it matter?

I agree. I guess that if there is a nice html documentation containing the same information which is currently avaliable by asking for help from the command line, nobody (save a maybe a few die-hards which insist on using ed to write Modelica code) will ever use the command line option again. It is important that the way this information is dumped by omc makes it easy to organize and hyperlink it nicely in OMC. Maybe an XML file would do the job better than the current fixed-font more-or-less-pretty-printed one does, and would be more easily transformed into HTML.

[casella]

Do we have any (possibly partial) plan regarding this issue for the 1.9.2 release?

The situation is actually getting worse. For example, I need to do some debugging of models involving homotopy, I don't remember the name of the flag (it's NLS_SOMETHING), omc +help=debug doesn't work anymore (see #3075), this option is not included in the Simulation Flags tab of OMEdit, a text search for the "NLS" string on both the User's Guide and System's Guide returns nothing. How can I find a complete list of all the logging options, if I'm not part of the inner circle of developers?

We should really give our users a better access to all the supported feature of the compiler (including the features of the generated code!)

[lochel]

Replying to casella:

We should really give our users a better access to all the supported feature of the compiler (including the features of the generated code!)

You get a list of all flags of the generated executable with:

./executable -help

A detailed explanation should be available for each flag with

./executable –help=flagname

[casella]

Replying to lochel:

You get a list of all flags of the generated executable with:

./executable -help

A detailed explanation should be available for each flag with

./executable –help=flagname

There are a few problems with this:

• This information is not available in the documentation. Can you please add it as a new Section 2.1.5 in the OpenModelica System Documentation (or rename 2.1.4 and make it more general, besides initialization)?
• If you try this in Windows from the regular command line prompt, you get an error because of a missing pthreadGC2.dll. You need to checkout OMDev, run msys and then run the executable from there. This should be explained in the documentation. Maybe there is a more practical way, in that case please state it in the system documentation.
• Anyway, if I do this I get info about many interesting flags, but not the logging flags (the ones listed in OpenModelica/SimulationRuntime/c/util/omc_error.h). How do you get those ones? Can you add this info to the new section of the System Guide?

[lochel]

Replying to casella:

• Anyway, if I do this I get info about many interesting flags, but not the logging flags (the ones listed in OpenModelica/SimulationRuntime/c/util/omc_error.h). How do you get those ones? Can you add this info to the new section of the System Guide?

You should get a list of all logging flags with ./executable –help=lv.

[ceraolo]

I'm also interested in creating that list of flags.
Can you please clarify what you mean when you say "./executable"?

I tried (from Windows powershell, while in OPENMODELICA\BIN):
./omc.exe -help=lv
but it did not work ("unknown option -help").

Thanks.

[lochel]

Replying to ceraolo:

Can you please clarify what you mean when you say "./executable"?

./executable is the generated simulation executable.

[ceraolo]

ok, thanks. At least now I understand the language.
It did not work (the exe crashed) but it is not very important now, for me.
In the future I may try again.

[sjoelund.se]

Milestone changed to 1.9.3 since 1.9.2 was released.

[sjoelund.se]

We have the API calls documented in the User's Guide now:
​https://openmodelica.org/doc/OpenModelicaUsersGuide/latest/scripting_api.html#all-openmodelica-api-calls

Time for someone to play with automatically generating documentation for OMC flags and simulation flags :)

[sjoelund.se]

Simulation flags documented here: ​https://openmodelica.org/doc/OpenModelicaUsersGuide/latest/simulationflags.html

@adeel

OMEdit links against simulation_options.o, right? If so, it can now generate the drop-down menus for the solvers from the used simulation options. And on mouse-over, it can display the same description text that you get from the documentation. The same goes for dassl options, linear/non-linear solvers (OMEdit is missing a lot of these). You can even generate the matrix of log levels+descriptions.

You could even look for new simulation flags/options and create a new tab if either of the options are missing in the GUI, to show the user that they are there.

[sjoelund.se]

OMC help text, flags, and options:
​https://openmodelica.org/doc/OpenModelicaUsersGuide/latest/omchelptext.html

[adeas31]

OMEdit now reads the integration methods, initilization methods, linear & non-linear solvers from simulation_options.h

[sjoelund.se]

Moved to new milestone 1.9.4

[lochel]

@casella
Can we close this ticket?

[casella]

Absolutely!

The latest developments went beyond my wildest dreams :)