Accessing older Linux/MacOSX nightly builds

[Francesco Casella]

Until we succeed to manage a tighter release schedule, there is a significant number of users who have to rely on the nightly builds to get some critical bugfixes, and when something bad as #5068 happens they are left with no alternatives. This is the feedback I got from one of our users

It would be great if the 'stable' and 'release' packaging of OpenModelica could be freshened up a bit... we have been forced to use Nightly packages for some months now, and there is no fallback for us in the case where the nightly version becomes broken.

Windows users have access to all the past nightly build installers, but Linux and Mac users, who rely on services like APT, don't.

I would suggest that we find a solution for this. I'm not sure what is the easiest way to provide this kind of service. For example, we may branch off master every 30 days and make a separate installation available. Any simple solution would work, as long as we have one.

[Francesco Casella]

BTW, I just checked ​https://build.openmodelica.org/omc/builds/windows/nightly-builds/64bit/older/ and to my disappointment I only found the penultimate version there. In the past we kept a lot more, I'm not sure it was a good move to remove all of them and only keep the last. I think we should keep at least one every other week or something like that. With 2TB of HDD costing less then 100 €, this shouldn't be a big deal...

[Francesco Casella]

Adrian's proposal for the Windows nightly

• last week, all
• each 1st of the month for the last year
• That would be ~20GB.

That's fine for me

[Francesco Casella]

This feature is essential if two or more people or organizations need to work together on a specific project: if a certain model or library works well with a certain specific version, we should enable all partners to install the same version, so that they get consistnt behaviour, and be able to fall back to that if the latest nightly gets broken for any reason. This is currently not possible.

[Francesco Casella]

As regards Debian Linux releases on the APT server, we currently have three repositories: nightly , stable, and release. My proposal is to add a few repositories based on the nightly builds between two stable releases, to be tagged out of master every few months when the development process is relatively stable, e.g., before commits such as the addition of OMSimulator or the new API in the case of 1.13.0, with unique names such as v1.13.0-dev-885-g33b800de6.

These should be advertised as a means to get critical bugfixes applied to master, even though there are no guarantees (i.e., no beta release process) that some other features have been broken in the meantime.

Once the stable version has been released, we should delete those repositories.

[Adrian Pop]

Strangely enough we have older versions of packages for the nightly-builds, going back to 2018-03-15. This is just an example for Ubuntu 18.0.4 LTS (Bionic)
​http://build.openmodelica.org/apt/pool/contrib-bionic/
For example:
openmodelica_1.13.0~dev-1191-g5e27ad0-1*.deb are the packages before OMSimulator was added.
The problem seems to be that we only advertise the last version in:
​http://build.openmodelica.org/apt/dists/bionic/nightly/binary-amd64/Packages

[Adrian Pop]

It seems we don't link the dependencies in the openmodelica glue package so is impossible to downgrade currently. It would be possible manually but one would need to get all the deb for a specific revision.

I will create maintenance/v1.13 branch and tag 1.13.0-dev.alpha1 on it and we will switch stable to that branch.

[Francesco Casella]

Replying to adrpo:

I will create maintenance/v1.13 branch and tag 1.13.0-dev.alpha1 on it and we will switch stable to that branch.

I'm not sure I understand this. Does it mean that stable no longer points to 1.12.0?

[Adrian Pop]

Replying to casella:

Replying to adrpo:

I will create maintenance/v1.13 branch and tag 1.13.0-dev.alpha1 on it and we will switch stable to that branch.

I'm not sure I understand this. Does it mean that stable no longer points to 1.12.0?

Yes. 1.12.0 is pointed by release anyway, no? Hm, not sure about maintenance/v1.12.

[Francesco Casella]

I got to the [linux download page], where I read the feature of the stable package

• contains new features to be part of the next release and are to be validated (in the beta phase)
• interfaces may have changed but shall remain stable during beta phase, changes are accepted only to fix critical issues
• intended to prepare a mature release, enable integration tests of partners, not productive usage also contains commits to the maintenance branch before new releases are made

In fact, this corresponds quite well with what we are looking for, except that we can only have one such "stable" release for 1.13.0, which means it may change over time and may still potentially break something, which would be bad for people relying on it.

On the other hand, the process of deciding to upgrade the stable release to a later version in the development process can be negotiated with people using it, and they can experiment a bit with the latest nightly to check if anything is actually broken, so it should probably fit most people and organization's needs. We could certainly negotiate this with all OSMC consortium members via an announcement on the Consortium's mailing list.

Regarding the recent developements, there has been a lot of turmoil in the last few weeks because of package dependencies introduced by OMSimulator, and though at last all the ​nightly builds now actually build, they haven't been used much, so they could still have issues. I would propose to pick one of the nightly builds of late June, before the OMSimulator commits, for the stable release, as nobody was complaining back then of recently introduced issues. Besides, there's not been much bug-fixing since then, except for OMSimulator-introduced issues.

@john.pye, I understand from your e-mails this fits exactly your needs. Can you please confirm?

People having more specific needs (i.e., needing a very specific release) on Linux can still build OMC from sources, which is a relatively straightforward process (see ​here) once you have all the package installed.

[Francesco Casella]

For the record, the nightly builds started breaking after ​PR #74 on July 12.

A few days earlier, on July 5 and 6, there were some fixes to FMU generation that it would be nice to have.

My proposal is to tag ​2e7f9fc as 1.13.0-dev-stable and make it available on the apt-server on the stable channel, as well as for Windows and MacOSX.

Once Adrian has pushed in the new API, I would branch off 1.13.0-beta, so that development of new features continues on 1.14.0, but I would wait a bit before releasing the beta as the new stable release - some testing of the new API, including possible dependency issues on all the released versions, should be performed on the nightly, otherwise we risk breaking the stable release just after we released it.

I'm a bit unsure about Willi's parallel stuff. I would feel more comfortable if we commit it to master after branching off 1.13.0, and release it in 1.14.0, but we can discuss that.

[Martin Sjölund]

Replying to casella:

My proposal is to tag [...] 1.13.0-dev-stable [...]
Once Adrian has pushed in the new API, I would branch off 1.13.0-beta

Except if you call a release -stable now it will prevent you from installing -beta later (because s comes after b). I should be an alpha release now and beta later. Or a first beta followed by a second. Like we've done in the past.

[Adrian Pop]

Replying to sjoelund.se:

Replying to casella:

My proposal is to tag [...] 1.13.0-dev-stable [...]
Once Adrian has pushed in the new API, I would branch off 1.13.0-beta

Except if you call a release -stable now it will prevent you from installing -beta later (because s comes after b). I should be an alpha release now and beta later. Or a first beta followed by a second. Like we've done in the past.

It will be called 1.13.0-dev.alpha1. I branched from revision: 5e27ad03e1b7a1c5c1ad7ba86e4a6592e6dbce4d/OpenModelica which is from 11 July just before 12 July when OMSimulator was added. I will tag this and the submodules as well and then start the builds to see how it goes.

[Francesco Casella]

If there are requirements on tag names due to lexicographical ordering, so be it. But I would try to avoid making confusion in communicating our release cycle to a wider audience.

Until we prove that we can have a tighter release cycle (since 2016 it took us between 7 and 11 months between two official releases), the requirement is to have some releatively stable release in-between that people can use if they need crucial bug fixes that cannot easily be patched onto maintenance release.

From what I understand (e.g., ​here), an alpha release is the first step towards the final official release, which is meant for testing and implies a feature freeze (i.e, a branch off master with our github management style), after which only bugfixes are applied. If we tag the 6 July 2018 version of alpha-something, we generate a lot of confusion, because the key feature expected for 1.13.0, replaceable support, is not there and will be added later.

I'm open to any tag that allows to later install alphas and betas on the same channel, but I would avoid using "alpha" to characterize this release. For example, we could have 1.13.0-xxxx-dev, where xxxx is a four-digit numerical code corresponding to the specific tagged release, e.g. 1.13.0-0895-dev for what we currently call 1.13.0-dev-895.

[Francesco Casella]

Replying to adrpo:

It will be called 1.13.0-dev.alpha1.

If -d comes after -b, I understand from comment:12 that it won't be possible to later install the beta on the same channel. Do I miss something?

[Adrian Pop]

@casella the beta will be called 1.13.0-dev.beta1, dev will not be removed until the final release. From my side I really don't care how we call it.

[Francesco Casella]

OK, then 1.13.0-dev-01 or something like that, which comes before both 1.13.0-dev.alpha1 and 1.13.0-dev.beta1, so it will be replaced by them, and does not contain the misleading 'alpha' concept

[Adrian Pop]

Ok, v1.13.0-dev.01 is now tagged on maintenance/v1.13 and stable branch is set to it. Hopefully the Linux/Mac builds will be fine. I will play with the Windows build later today.

[Francesco Casella]

From the ​Hudson build log it seems that everything went fine. The APT_SERVER job has already run, so the release should already be available.

@john.pye, @ali.shirazi, would you mind trying out the stable Linux release, check if 1.13.0-dev.01 is fine for you and report?

[Martin Sjölund]

apt-mirror script

[Martin Sjölund]

A small comment. If you really want to depend on pre-release software of a very particular version and make sure everyone uses the same version, you can mirror an apt repository using for example the script attached (set section=nightly). Then just point to whatever http URL online that your output directory points to.

[ali.shirazi@…]

Hello,

I tested out "1.13.0-dev.01" stable and all our SolarTherm models were run successfully.

Cheers,
Ali

[Francesco Casella]

Replying to sjoelund.se:

A small comment. If you really want to depend on pre-release software of a very particular version and make sure everyone uses the same version, you can mirror an apt repository using for example the script attached (set section=nightly). Then just point to whatever http URL online that your output directory points to.

You mean that if today's nightly is good, I run this script once, create a mirror apt repo, and than point to that for future use?

[Francesco Casella]

Replying to ali.shirazi@…:

Hello,

I tested out "1.13.0-dev.01" stable and all our SolarTherm models were run successfully.

Good!

I think that with the stable release available and @sjoelund's suggestion on how to create a mirror we can successfully close this issue :)

[Martin Sjölund]

Replying to casella:

You mean that if today's nightly is good, I run this script once, create a mirror apt repo, and than point to that for future use?

Yes. It's how the old release repos are created: ​https://build.openmodelica.org/omc/builds/linux/releases/