Issues with MSL library handling in Linux

[Francesco Casella]

Two more models fail in ​MSL 3.2.3/newInst than in ​MSL_trunk/newInst.

The failures happen at runtime. ​Modelica.Utilities.Examples.readRealParameterModel returns:

assert            | debug   | Not possible to construct full path name of "modelica://Modelica/Resources/Data/Utilities/Examples_readRealParameters.txt"
|                 | |       | No such file or directory

while ​Modelica.Electrical.Digital.Examples.RAM returns

assert            | debug   | Not possible to construct full path name of "modelica://Modelica/Resources/Data/Electrical/Digital/Memory_Matrix.txt"
|                 | |       | No such file or directory

I'm not sure if something went wrong with the MSL codebase, or if the lack of the statement

if not setCommandLineOptions("--std=3.2") then exit(1); end if;

in the simulation flags is relevant.

[Francesco Casella]

Both models use Modelica.Utilities.Files.loadResource, which extends ModelicaServices.ExternalReferences.loadResource, maybe something's wrong with ModelicaServices 3.2.3?

[Francesco Casella]

Fixed by the appropriate patches, see ​report.

[Christian Kral <dr.christian.kral@…>]

I was just running into an issue with reading a file in the ​PhotoVoltaics library. On the search on the cause I found this ticket. So I tried to run Modelica.Utilities.Examples.readRealParameterModel and failed. I got the following error message:

Not possible to construct full path name of "modelica://Modelica/Resources/Data/Utilities/Examples_readRealParameters.txt"<br>
No such file or directory

This issue was caused in:

OMEdit 1.14.0~dev-48-g43ebf04
Connected to OpenModelica 1.14.0~dev-354-g4a303b3
Connected to OMSimulator unknown-linux

[Christian Kral <dr.christian.kral@…>]

I also tried this with -d=newInst and this did not change anything.

[Christian Kral <dr.christian.kral@…>]

Investigated with MSL 3.2.3

[Martin Sjölund]

Do you use the OpenModelica patched version of ModelicaServices?

[Christian Kral <dr.christian.kral@…>]

This is a good question. I actually do not know. I installed everything through the nightly repository, so I would have expected, that a valid version of the Modelica Standard Library (which includes ModelicaServices) is installed.

When looking into the package manager, I see that I have version 3.2.3-rc.2-5-g51471b3-om2-1 installed which seems strange to me, as I would not have expected to have a release candidate version installed.

[Martin Sjölund]

The latest version should be:

Package: omlib-modelicaservices-3.2.3
Version: 3.2.3-4-gab2d460-om1-1

Which should be available through a normal apt-get upgrade. The release candidate had a broken patch from us and indeed won't work.

[Christian Kral <dr.christian.kral@…>]

Could it be the case that 3.2.3-4-gab2d460-om1-1 should be in the repository, but isn't? If I run apt list | grep omlib-modelicaservices it lists

omlib-modelicaservices-1.0/bionic,bionic,now 1.0-20140322-174148~git~maint-3.1-om1-1 all [installed,automatic]
omlib-modelicaservices-3.2.1/bionic,bionic 3.2.1+build.4-18-g60cb4e4-om1-1 all
omlib-modelicaservices-3.2.2/bionic,bionic,now 3.2.2-237-gd992c34-om1-1 all [installed]
omlib-modelicaservices-3.2.3/now 3.2.3-rc.2-5-g51471b3-om1-1 all [installed,local]
omlib-modelicaservices-trunk/bionic,bionic,now 20190307-080313~git~master-om1-1 all [installed]

See also the attached screenshots.

[Christian Kral <dr.christian.kral@…>]

Packages installed

[Christian Kral <dr.christian.kral@…>]

MSL pacakges installed through repository

[Martin Sjölund]

It might simply be that apt considers 3.2.3-rc.2-5 to be > 3.2.3-4 :(

[Christian Kral <dr.christian.kral@…>]

I found two ways to compare these version numbers on ​stackoverflow:

Pyhton

import apt_pkg
apt_pkg.init_system()
old = '3.2.3-rc.2-5'
new = '3.2.3-4'
vc = apt_pkg.version_compare(old,new) 
if vc > 0:
	print('version old > version new')
elif vc == 0:
	print('version old == version new')
elif vc < 0:
	print('version old < version new')

The result of this Python script is:

version old > version new

DPKG

Also dpkg --compare-versions 3.2.3-rc.2-5 gt 3.2.3-4 && echo true shows true, but shouldn't.

So apparently it is the case that 3.2.3-rc.2-5 > 3.2.3-4 :-(

[Francesco Casella]

Oh gosh, I thought we had a version numbering system that worked well with lexicographical ordering. Isn't that the case?

[Christoph Buchner <buchner@…>]

I think the problem is that what you expect/define about precedence differs from what dpkg expects, which can lead to surprises, see e.g. this snippet from ​https://debian-handbook.info/browse/stable/sect.manipulating-packages-with-dpkg.html:

$ dpkg --compare-versions 2.6.0pre3-1 lt 2.6.0-1
$ echo $?
1
Note the unexpected failure of the last comparison: for dpkg, pre, usually denoting a pre-release, has no particular meaning, and this program compares the alphabetic characters in the same way as the numbers (a < b < c ...), in alphabetical order. This is why it considers “0pre3” to be greater than “0”. When we want a package's version number to indicate that it is a pre-release, we use the tilde character, “~”:

Debian sorting policy here: ​https://www.debian.org/doc/debian-policy/ch-controlfields.html#version
I think (but can't check without a linux machine) that in the above case (3.2.3 being equal, an -4/-5being the respective debian revisions), the substring -rc.2 is being compared against the empty string (as its counterpart does not exist in the second version string), and because of the leading hyphen gets sorted later.
It would be interesting to check if 3.2.3rc.2-5 would yield the expected behaviour.

[Christoph Buchner <buchner@…>]

Well actually (thank you, repl.it and your bash support), replacing the hyphen with a tilde seems to do the trick:

GNU bash, version 4.4.12(1)-release (x86_64-pc-linux-gnu)
 dpkg --compare-versions 3.2.3-rc.2-5 gt 3.2.3-4 && echo true
true
 dpkg --compare-versions 3.2.3rc.2-5 gt 3.2.3-4 && echo true
true
 dpkg --compare-versions 3.2.3-rc.2 gt 3.2.3 && echo true
true
 dpkg --compare-versions 3.2.3-rc2 gt 3.2.3 && echo true
true
 dpkg --compare-versions 3.2.3-rc2 gt 4.2.3 && echo true
 dpkg --compare-versions 3.2.3rc2 gt 3.2.3 && echo true
true
 dpkg --compare-versions 3.2.3-rc2 gt 3.2.3 && echo true
true
 dpkg --compare-versions 3.2.3rc2 gt 3.2.4 && echo true
 dpkg --compare-versions 3.2.3r gt 3.2.4 && echo true
 dpkg --compare-versions 3.2.3r gt 3.2.3 && echo true
true
 dpkg --compare-versions 3.2.3~r gt 3.2.3 && echo true
 dpkg --compare-versions 3.2.3~rc.2-5 gt 3.2.3-4 && echo true
 ^C

(tilde sorts before anything)

[Christoph Buchner <buchner@…>]

Actually, that ​seems to be the canonical way to specify versions for Debian:

[7] One common use of ~ is for upstream pre-releases. For example, 1.0~beta1~svn1245 sorts earlier than 1.0~beta1, which sorts earlier than 1.0.

[Christian Kral <dr.christian.kral@…>]

From my current update maintained by the OpenModelica repository I received the version 3.2.3-4-gab2d460-om1-1 of ModelicaSerives. Therefore, example Modelica.Electrical.QuasiStationary.SinglePhase.Examples.SeriesBode as of #4793 is now working as expected.

[Christian Kral <dr.christian.kral@…>]

Replying to Christian Kral <dr.christian.kral@…>:

From my current update maintained by the OpenModelica repository I received the version 3.2.3-4-gab2d460-om1-1 of ModelicaSerives. Therefore, example Modelica.Electrical.QuasiStationary.SinglePhase.Examples.SeriesBode as of #4793 is now working as expected.

Please ignore the second part of this message.

[Francesco Casella]

@sjoelund.se, is there any way to fix this?

[Martin Sjölund]

I'm not actually sure how to deal with it as each release does things slightly differently. You have:

3.2.3
3.2.3+build.2 (semver says sorted exactly as 3.2.3; no precedence)
3.2.3-rc.2 (semver says sorted as a pre-release)

I suppose I could change all dashes to ~. But any existing pre-releases would be sorted as post-releases and need to be uninstalled anyway.

[Francesco Casella]

How do similar projects deal with this issue?

[Martin Sjölund]

Manually updating release numbers I suppose :D

[Christian Kral <dr.christian.kral@…>]

I checked my actual nightly build version

Connected to OpenModelica 1.14.0~dev-26494-g4d4d809
Connected to OMSimulator unknown-linux

and it still shows MSL 3.2.3 as a release candidate version number (see attachment). I suppose that I cannot check any fixed tickets such as #4297, since an obsolete Modelica library is loaded.

[Christian Kral <dr.christian.kral@…>]

Modelica 3.2.3 installed version

[Martin Sjölund]

You will need to manually remove it for Ubuntu to prefer the newer version. The fix is just that this shouldn't occur in the future

[Christian Kral <dr.christian.kral@…>]

Thanks for the hint. I re-installed the modelica 3.2.3 package and got

3.2.3+build.2~5~g2556186~om2-1

installed. So this seems to work now.

[Francesco Casella]

Replying to sjoelund.se:

You will need to manually remove it for Ubuntu to prefer the newer version. The fix is just that this shouldn't occur in the future

This looks a bit ugly for the general public. Say I had formerly installed 1.13.0 on Linux and I'm going to install 1.14.0. Will I need to do that? If so, we have to state that very clearly in the release notes.

[Martin Sjölund]

Well, I cannot really change the packages installed locally on someone's system. And I cannot change the version number of MSL. And I don't really want to bump the epoch version which causes every single library to upgrade either.

What I want is a separate library manager for Modelica packages that we control and that will pull libraries from github and compile the libraries at the same time.

[Francesco Casella]

Replying to sjoelund.se:

What I want is a separate library manager for Modelica packages that we control and that will pull libraries from github and compile the libraries at the same time.

Any plans for that? Can't we really re-use existing package managers out there?

[Martin Sjölund]

Well. We can. But I want something that knows Modelica. Most Modelica libraries are hosted on GitHub without binaries, but requires compiling binaries to work. We also want to be able to install multiple versions of the same library at the same time.

And I don't want the Windows installer to have to include all the libraries when most of them remain unused.

My plan was to have a BSc student write a Modelica package manager in Julia that handled everything, but I fear writing the requirements takes as long time as writing the package manager would...

[Francesco Casella]

Replying to sjoelund.se:

Well. We can. But I want something that knows Modelica. Most Modelica libraries are hosted on GitHub without binaries, but requires compiling binaries to work.

Ah, ok, I missed this point.

In fact, I always posed this issue to Mike and Dietmar as a requirement for their otherwise excellent impact tool, but they insisted it was out of scope, which for me makes the whole thing unusable. My use case was ExternalMedia, but I understand MSL has the same issues (possibly worse) because of tables, Modelica.Math, and so forth.

And I don't want the Windows installer to have to include all the libraries when most of them remain unused.

My plan was to have a BSc student write a Modelica package manager in Julia that handled everything, but I fear writing the requirements takes as long time as writing the package manager would...

Yeah...

[Francesco Casella]

Releasing 1.14.0 which is stable and has many improvements w.r.t. 1.13.2.

This issue, previously marked as blocker for 1.14.0, is rescheduled to 1.15.0

[Francesco Casella]

Release 1.15.0 was scrapped, because replaceable support eventually turned out to be more easily implemented in 1.16.0. Hence, all 1.15.0 tickets are rescheduled to 1.16.0

[Francesco Casella]

This problem should be hopefully solved in 1.17.0 by the new package manager.

[Francesco Casella]

Rescheduled to 1.18.0

[Francesco Casella]

Ticket retargeted after milestone closed

[Francesco Casella]

1.18.0 blocker tickets moved to 1.19.0