Get OpenModelica back into FMI Cross-checker

[Francesco Casella]

We should get back into the FMI Cross-checker logs. We can use this ticket to keep track of required activities

[Francesco Casella]

Duplicate of #6009 :)

[Lennart Ochel]

Doesn't seem to be the same issue though.

[Andreas Heuermann]

Sorry, I thought you were refering to the Docker FMU stuff. So reopening it again.

[Lennart Ochel]

This is the corresponding GitHub issue for OMSimulator: ​https://github.com/OpenModelica/OMSimulator/issues/739

[Andreas Heuermann]

Fmi-Cross-Check repository:
​https://github.com/modelica/fmi-cross-check

[Andreas Heuermann]

I created a new Jenkins file that runs a new Docker image with OpenModelica nightly and all tools needed for the fmi-cross-check based on Ubuntu Focal Fossa and added a mos-script to generate our FMUs.

At the moment I run the mos-script locally for generating linux64 FMUs. That's working just fine.

My next steps:

• Enhance script to be (more) OS independent.
• Use Jenkins + Docker image (based on Ubuntu Focal Fossa)for 64bit FMU generation and FMU validation using ​FMPy 0.2.10.
• Post first results here and see if we are passing the test.
• Integrate Jenkinsfile to OpenModelica CI/CD.

After that all other OS need to be checked:

• Creating a 32bit Docker image with OpenModelica and FMPy based on Ubuntu Bionic Beaver (Focal Fossal will not support many 32bit features) should be easy.
• Cross-compilation for 64bit Windows FMUs using some Docker image and installing FMPy via conda should be possible on my local machine. With some help of @sjoelung even on our test machines that run Windows.
• For Windows Docker images could also be an option, but I never tested that with OpenModelica. Running it without Docker directly on our test machine is probably better. We(and with we I mean @adpro?) need to install FMPy on them though.
• For macOS there is no Docker available, so doing it directly on our test machine is the only option. Integrating it to Jenkins should be possible, if we install FMPy on those machnines as well.

Do we have Darwin 32bit and Windows 32bit available?

PS: Adding OMSimulator to this should be fairly easy. Just a few more lines of code to add to the mos-script.

[Andreas Heuermann]

All of this is independent of the OpenModelica/OpenModelica github project. So if version v1.16.0 is available for install we can run the check and upload the results to ​https://github.com/modelica/fmi-cross-check.

[Martin Sjölund]

Do we have Darwin 32bit and Windows 32bit available?

There is no Darwin 32-bit anymore. Apple stopped supporting 32-bit (like they do with everything).

[Andreas Heuermann]

My Jenkins script is running now and validating the FMUs.

+ grep OpenModelica

+ find /fmi-cross-check/fmus -name *.fmu

/fmi-cross-check/fmus/1.0/me/linux64/OpenModelica/1.16.0-dev-479-g8630a30/ControlledTemperature/ControlledTemperature.fmu
/fmi-cross-check/fmus/1.0/me/linux64/OpenModelica/1.16.0-dev-479-g8630a30/CoupledClutches/CoupledClutches.fmu
/fmi-cross-check/fmus/1.0/me/linux64/OpenModelica/1.16.0-dev-479-g8630a30/Rectifier/Rectifier.fmu
/fmi-cross-check/fmus/1.0/me/linux64/OpenModelica/1.16.0-dev-479-g8630a30/BouncingBall/BouncingBall.fmu
/fmi-cross-check/fmus/2.0/cs/linux64/OpenModelica/1.16.0-dev-479-g8630a30/ControlledTemperature/ControlledTemperature.fmu
/fmi-cross-check/fmus/2.0/cs/linux64/OpenModelica/1.16.0-dev-479-g8630a30/CoupledClutches/CoupledClutches.fmu
/fmi-cross-check/fmus/2.0/cs/linux64/OpenModelica/1.16.0-dev-479-g8630a30/Rectifier/Rectifier.fmu
/fmi-cross-check/fmus/2.0/cs/linux64/OpenModelica/1.16.0-dev-479-g8630a30/BouncingBall/BouncingBall.fmu
/fmi-cross-check/fmus/2.0/cs/arm-linux-gnueabi/OpenModelica/v1.11.0/RoomHeating_OM_RH/RoomHeating_OM_RH.fmu
/fmi-cross-check/fmus/2.0/cs/arm-linux-gnueabi/OpenModelica/v1.11.0/FmuExportCrossCompile/FmuExportCrossCompile.fmu
/fmi-cross-check/fmus/2.0/cs/arm-linux-gnueabi/OpenModelica/v1.11.0/WaterTank_Control/WaterTank_Control.fmu
/fmi-cross-check/fmus/2.0/cs/arm-linux-gnueabi/OpenModelica/v1.11.0/WaterTank_TestSingleWaterTank/WaterTank_TestSingleWaterTank.fmu
/fmi-cross-check/fmus/2.0/me/linux64/OpenModelica/1.16.0-dev-479-g8630a30/ControlledTemperature/ControlledTemperature.fmu
/fmi-cross-check/fmus/2.0/me/linux64/OpenModelica/1.16.0-dev-479-g8630a30/CoupledClutches/CoupledClutches.fmu
/fmi-cross-check/fmus/2.0/me/linux64/OpenModelica/1.16.0-dev-479-g8630a30/Rectifier/Rectifier.fmu
/fmi-cross-check/fmus/2.0/me/linux64/OpenModelica/1.16.0-dev-479-g8630a30/BouncingBall/BouncingBall.fmu
/fmi-cross-check/fmus/2.0/me/arm-linux-gnueabi/OpenModelica/v1.11.0/RoomHeating_OM_RH/RoomHeating_OM_RH.fmu
/fmi-cross-check/fmus/2.0/me/arm-linux-gnueabi/OpenModelica/v1.11.0/FmuExportCrossCompile/FmuExportCrossCompile.fmu
/fmi-cross-check/fmus/2.0/me/arm-linux-gnueabi/OpenModelica/v1.11.0/WaterTank_Control/WaterTank_Control.fmu
/fmi-cross-check/fmus/2.0/me/arm-linux-gnueabi/OpenModelica/v1.11.0/WaterTank_TestSingleWaterTank/WaterTank_TestSingleWaterTank.fmu

+ make -C /fmi-cross-check/ validate-files
make: Entering directory '/fmi-cross-check'
python3 -m fmpy.cross_check.validate_vendor_repo

#################################
3 problems found in /fmi-cross-check
Validated 1245 FMUs and 10008 results
#################################

/fmi-cross-check/fmus/1.0/me/linux64/OpenModelica/1.16.0-dev-479-g8630a30/Rectifier/Rectifier_ref.csv is larger than 1 MB (3.49007 MB)

/fmi-cross-check/fmus/2.0/cs/linux64/OpenModelica/1.16.0-dev-479-g8630a30/Rectifier/Rectifier_ref.csv is larger than 1 MB (3.49007 MB)

/fmi-cross-check/fmus/2.0/me/linux64/OpenModelica/1.16.0-dev-479-g8630a30/Rectifier/Rectifier_ref.csv is larger than 1 MB (3.49007 MB)

make: *** [Makefile:12: validate-files] Error 3
make: Leaving directory '/fmi-cross-check'
script returned exit code 2

I only need to filter the csv files. Sadly filterSimulationResultscan't handle CSV.

And since I know that the results for BouncingBall are incorrect until ​PR 959 gets merged
we should also validate our results in this script.

I'll use OMSimulator or FMUCheck for that.

[Francesco Casella]

When this is finished, do we need to do something specific on the FMU Compliance Checker server/website?

[Andreas Heuermann]

Not really.
We can add a for to ​https://github.com/OpenModelica/fmi-cross-check and have a Jenkins pipeline that runs everything after every release.
Then someone can open a PR on ​https://github.com/modelica/fmi-cross-check and when it gets accepted OpenModelica should become green on ​https://fmi-standard.org/tools/ and listed in ​https://fmi-standard.org/cross-check/fmi2-cs-linux64/.

And if that is not the case we'll just ask.

[Andreas Heuermann]

I'll creating tickets for all problems I'm finding while doing the fmi-cross-tests.

Here is the first one:

• Simulation in clean testing environment impossible: #6017

[Andreas Heuermann]

More problems:

• I had to reopen #5367
• The binary files are to big

  /fmi-cross-check/fmus/2.0/me/linux64/OpenModelica/v1.16.0-dev.521+geba516059d/BouncingBall/FMU/binaries/linux64/BouncingBall.so is larger than 1 MB (1.12493 MB)
  

• Names of variables marked as output changed names: der(h) -> der(_D_outputAlias_h):

  Error in /fmi-cross-check/fmus/2.0/me/linux64/OpenModelica/v1.16.0-dev.521+geba516059d/BouncingBall/BouncingBall_ref.csv. Variable 'der(h)' does not exist in the FMU
  

  That is not a big problem but inconvenient as I need to change all output variable names manually.

[Francesco Casella]

Replying to AnHeuermann:

More problems:

• I had to reopen #5367

Please see my comments there.

• The binary files are to big

  /fmi-cross-check/fmus/2.0/me/linux64/OpenModelica/v1.16.0-dev.521+geba516059d/BouncingBall/FMU/binaries/linux64/BouncingBall.so is larger than 1 MB (1.12493 MB)
  

Q1: Why is the dll so large for such a small model?
Q2: Why is this limit enforced by the cross-checker (maybe we should ask).

• Names of variables marked as output changed names: der(h) -> der(_D_outputAlias_h):

  Error in /fmi-cross-check/fmus/2.0/me/linux64/OpenModelica/v1.16.0-dev.521+geba516059d/BouncingBall/BouncingBall_ref.csv. Variable 'der(h)' does not exist in the FMU
  

  That is not a big problem but inconvenient as I need to change all output variable names manually.

I don't get this. We should always get the right names automatically. I'm not sure why we have an alias for the output, it is fine to use it for internal purpose, but we shouldn't expose these internal aliases outside the model.

[Andreas Heuermann]

Q1: Why is the dll so large for such a small model?

Probably due to my configuration of my build. I have debug symbols enabled. When i strip the executable it will be reduced to something around 400KB. I need to test this with the release version or simply strip everything before uploading it to fmi-cross-check.

Q2: Why is this limit enforced by the cross-checker (maybe we should ask).

If using the release version and stripping does not help I would ask them.

I don't get this. We should always get the right names automatically. I'm not sure why we have an alias for the output, it is fine to use it for internal purpose, but we shouldn't expose these internal aliases outside the model.

I agree completely. I'm opening a new ticket about it, see #6040.

[Andreas Heuermann]

We need to use CVODE for model Modelica.Electrical.Analog.Examples.Rectifier. Explicit Euler is not working very well for Co-Simulation. And with "not very well" I mean not at all.

But for that I need to fix some dependencies to get CVODE into the FMU without any missing libraries.
See ticket #6041.

[Francesco Casella]

Replying to AnHeuermann:

Q1: Why is the dll so large for such a small model?

Probably due to my configuration of my build. I have debug symbols enabled

I guess production-grade FMI shouldn't have that, right?

[Francesco Casella]

Replying to AnHeuermann:

We need to use CVODE for model Modelica.Electrical.Analog.Examples.Rectifier. Explicit Euler is not working very well for Co-Simulation. And with "not very well" I mean not at all.

I guess this is due to poor event handling, the system should not be particularly stiff.

[Francesco Casella]

Retargeted to 1.18.0 because of 1.17.0 timed release.

[Francesco Casella]

Ticket retargeted after milestone closed