FMUs don't export libopenblas.so

[Andreas Heuermann]

I'm building a static C FMU 2.0 for ModelExchange with OpenModelica and try to simulate it with FMPy in a clean testing environment without omc available.

Our OpenModelica FMUs can't be simulated with FMPy since they are missing openblas.so

How to reproduce
Note: You will need omc and docker in your path.
Also you have to trust my docker container or build your own ;-)

Run in your shell:

$ omc buildFMU.mos
true
""
"/home/.../ticket6010/BouncingBall.fmu"
""
$ docker run -it --rm -v $(pwd):/testDir anheuermann/fmpy:focal-arm64 /bin/bash
root@19cd9af7df5a:/# cd /testDir/
root@19cd9af7df5a:/testDir# ls *.fmu
BouncingBall.fmu
root@19cd9af7df5a:/testDir# python3
Python 3.8.2 (default, Apr 27 2020, 15:53:34)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from fmpy import *
>>> fmu='BouncingBall.fmu'
>>> result = simulate_fmu(fmu)
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/fmpy/fmi1.py", line 142, in __init__
    self.dll = cdll.LoadLibrary(libraryPath)
  File "/usr/lib/python3.8/ctypes/__init__.py", line 451, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib/python3.8/ctypes/__init__.py", line 373, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libopenblas.so.0: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/fmpy/simulation.py", line 557, in simulate_fmu
    fmu = instantiate_fmu(unzipdir, model_description, fmi_type, visible, debug_logging, logger, fmi_call_logger, use_remoting)
  File "/usr/local/lib/python3.8/dist-packages/fmpy/simulation.py", line 646, in instantiate_fmu
    fmu = FMU2Model(**fmu_args)
  File "/usr/local/lib/python3.8/dist-packages/fmpy/fmi2.py", line 409, in __init__
    super(FMU2Model, self).__init__(**kwargs)
  File "/usr/local/lib/python3.8/dist-packages/fmpy/fmi2.py", line 85, in __init__
    super(_FMU2, self).__init__(**kwargs)
  File "/usr/local/lib/python3.8/dist-packages/fmpy/fmi1.py", line 144, in __init__
    raise Exception("Failed to load shared library %s. %s" % (libraryPath, e))
Exception: Failed to load shared library /tmp/tmprujxrrsu/binaries/linux64/BouncingBall.so. libopenblas.so.0: cannot open shared object file: No such file or directory
>>>

buildFMU.mos

loadString("
model BouncingBall
  parameter Real e=0.7 \"coefficient of restitution\";
  parameter Real g=9.81 \"gravity acceleration\";
  Real h(fixed=true, start=1) \"height of ball\";
  Real v(fixed=true) \"velocity of ball\";
  Boolean flying(fixed=true, start=true) \"true, if ball is flying\";
  Boolean impact;
  Real v_new(fixed=true);
  Integer foo;
equation
  impact = h <= 0.0;
  foo = if impact then 1 else 2;
  der(v) = if flying then -g else 0;
  der(h) = v;
  when {h <= 0.0 and v <= 0.0,impact} then
    v_new = if edge(impact) then -e*pre(v) else 0;
    flying = v_new > 0;
    reinit(v, v_new);
  end when;
end BouncingBall;");
getErrorString();

buildModelFMU(BouncingBall, version="2.0", fmuType="me", fileNamePrefix="BouncingBall", platforms={"static"});
getErrorString();

[Andreas Heuermann]

Same problem for fmuChecker:

$ docker run -it --rm -v $(pwd):/testDir docker.openmodelica.org/fmuchecker:v2.0.4 /bin/bash
root@f7f16bbcefb2:/# cd /testDir/
root@f7f16bbcefb2:/testDir# ls *.fmu
BouncingBall.fmu
root@f7f16bbcefb2:/testDir# fmuCheck.linux64 -f -h 0.002 -n 0 -k me BouncingBall.fmu
[INFO][FMUCHK] FMI compliance checker 2.0.4 [FMILibrary: 2.0.3] build date: Nov  6 2017
[INFO][FMUCHK] Called with following options:
[INFO][FMUCHK] fmuCheck.linux64 -f -h 0.002 -n 0 -k me BouncingBall.fmu
[INFO][FMUCHK] Will process FMU BouncingBall.fmu
[INFO][FMILIB] XML specifies FMI standard version 2.0
[INFO][FMUCHK] Model name: BouncingBall
[INFO][FMUCHK] Model GUID: {26e75b47-c6e0-4645-bf77-9ef0c8e991af}
[INFO][FMUCHK] Model version:
[INFO][FMUCHK] FMU kind: ModelExchange
[INFO][FMUCHK] The FMU contains:
0 constants
2 parameters
4 discrete variables
4 continuous variables
0 inputs
0 outputs
8 local variables
0 independent variables
0 calculated parameters
7 real variables
1 integer variables
0 enumeration variables
2 boolean variables
0 string variables

[INFO][FMUCHK] Printing output file header
"time","h","v","der(h)","der(v)","v_new","e","g","foo","flying","impact"
[INFO][FMUCHK] Model identifier for ModelExchange: BouncingBall
[INFO][FMILIB] Loading 'linux64' binary with 'default' platform types
[FATAL][FMICAPI] Could not load the DLL: libopenblas.so.0: cannot open shared object file: No such file or directory
[FATAL][FMUCHK] Could not create the DLL loading mechanism(C-API) for ME.
FMU check summary:
FMU reported:
        0 warning(s) and error(s)
Checker reported:
        0 Warning(s)
        2 Error(s)
        2 Fatal error(s) occurred during processing
root@f7f16bbcefb2:/testDir#

[Andreas Heuermann]

Also not working are

• 2.0 cs static
• 2.0 me dynamic
• 2.0 me_cs static
• 2.0 me_cs dynamic
• 2.0 me_cs static,dynamic

For version="2.0", fmuType="me", fileNamePrefix="BouncingBall", platforms={"dynamic"} libSimulationRuntimeC.so can't be loaded.

root@55bc5352204b:/testDir# fmuCheck.linux64 -f -h 0.002 -n 0 -k me BouncingBall.fmu
[INFO][FMUCHK] FMI compliance checker 2.0.4 [FMILibrary: 2.0.3] build date: Nov  6 2017
[INFO][FMUCHK] Called with following options:
[INFO][FMUCHK] fmuCheck.linux64 -f -h 0.002 -n 0 -k me BouncingBall.fmu
[INFO][FMUCHK] Will process FMU BouncingBall.fmu
[INFO][FMILIB] XML specifies FMI standard version 2.0
[INFO][FMUCHK] Model name: BouncingBall
[INFO][FMUCHK] Model GUID: {3252d5f7-7575-4404-adfd-e4eb0e6a1c77}
[INFO][FMUCHK] Model version:
[INFO][FMUCHK] FMU kind: ModelExchange
[INFO][FMUCHK] The FMU contains:
0 constants
2 parameters
4 discrete variables
4 continuous variables
0 inputs
0 outputs
8 local variables
0 independent variables
0 calculated parameters
7 real variables
1 integer variables
0 enumeration variables
2 boolean variables
0 string variables

[INFO][FMUCHK] Printing output file header
"time","h","v","der(h)","der(v)","v_new","e","g","foo","flying","impact"
[INFO][FMUCHK] Model identifier for ModelExchange: BouncingBall
[INFO][FMILIB] Loading 'linux64' binary with 'default' platform types
[FATAL][FMICAPI] Could not load the DLL: libSimulationRuntimeC.so: cannot open shared object file: No such file or directory
[FATAL][FMUCHK] Could not create the DLL loading mechanism(C-API) for ME.
FMU check summary:
FMU reported:
        0 warning(s) and error(s)
Checker reported:
        0 Warning(s)
        2 Error(s)
        2 Fatal error(s) occurred during processing

[Andreas Heuermann]

@sjoelund What am I doing wrong here?
We are testing this on every commit with ./single-fmu-run.sh linux64 'cat VERSION', see​Jenkinsfile#L499.

[Francesco Casella]

It wouldn't be bad if we could fix it in 1.16.0 already

[Martin Sjölund]

The lapack/blas should be statically linked (but are not, it seems). I am not sure if we should link to them by default though... I think that is done because of:

// Lapack is always included
case Absyn.STRING("lapack") then ({},{});
case Absyn.STRING("Lapack") then ({},{});

I guess it's good to test some model calling Modelica.Math.Matrices.solve as well as ModelicaTest.Tables.CombiTable1Ds.Test1

[Martin Sjölund]

I think the problem was that pretty much no Linux comes with statically linkable lapack/blas and everyone has liblapack.so/libblas.so so treat them as system libraries and just link them.

But if they are symbolic links to openblas that sort of fails as well..

Maybe the best solution would be to not remove those lapack libraries when creating an FMU and only have a dynamic lapack/blas when an MSL/etc model needs it.

[Francesco Casella]

Replying to sjoelund.se:

Maybe the best solution would be to not remove those lapack libraries when creating an FMU and only have a dynamic lapack/blas when an MSL/etc model needs it.

At least as a temporary workaround, I would do that right away.

For the long term, it depends how large those libraries are, @sjoelund.se can you please report that?

[Andreas Heuermann]

I removed the default linking to openblas / lapack for source-code FMUs.

In that case BouncingBall works, but models like
TestMatricesExamplesSolveLinearEquations.mo

model TestMatricesExamplesSolveLinearEquations
    extends Modelica.Icons.Example;
    equation
      Modelica.Math.Matrices.Examples.solveLinearEquations();
    annotation (experiment(StopTime=0));
  end TestMatricesExamplesSolveLinearEquations;

are needing LAPACK and BLAS.

root@0d4e1c9eb4fc:/ticket6017# fmuCheck.linux64 -f -h 0.002 -n 0 -k me lapackExample.fmu
[INFO][FMUCHK] FMI compliance checker 2.0.4 [FMILibrary: 2.0.3] build date: Nov  6 2017
[INFO][FMUCHK] Called with following options:
[INFO][FMUCHK] fmuCheck.linux64 -f -h 0.002 -n 0 -k me lapackExample.fmu
[INFO][FMUCHK] Will process FMU lapackExample.fmu
[INFO][FMILIB] XML specifies FMI standard version 2.0
[INFO][FMUCHK] Model name: TestMatricesExamplesSolveLinearEquations
[INFO][FMUCHK] Model GUID: {74641164-6787-4529-ad7d-19c6731f6b01}
[INFO][FMUCHK] Model version:
[INFO][FMUCHK] FMU kind: ModelExchange
[INFO][FMUCHK] The FMU contains:
0 constants
0 parameters
0 discrete variables
0 continuous variables
0 inputs
0 outputs
0 local variables
0 independent variables
0 calculated parameters
0 real variables
0 integer variables
0 enumeration variables
0 boolean variables
0 string variables

[INFO][FMUCHK] Printing output file header
"time"
[INFO][FMUCHK] Model identifier for ModelExchange: lapackExample
[INFO][FMILIB] Loading 'linux64' binary with 'default' platform types
[FATAL][FMICAPI] Could not load the DLL: liblapack.so.3: cannot open shared object file: No such file or directory
[FATAL][FMUCHK] Could not create the DLL loading mechanism(C-API) for ME.
FMU check summary:
FMU reported:
        0 warning(s) and error(s)
Checker reported:
        0 Warning(s)
        2 Error(s)
        2 Fatal error(s) occurred during processing

So I moved the problem from basically all models to only those that actually need LAPACK. And those are only a few.
I'm not sure it this will work for models with linear and non-linear loops in them. The FMIRuntime should be statically compiled and contain the needed solvers, including LAPACK.

But we need a smart way to detect what system libraries dependencies a model has, copy dynamic libraries or compile them statically into the <fmiPrefix>.so. On Mac we need to add a bunch of install_name_tool and other magic.
That's not easy, especially since we want to be able to cross-compile for different platforms.
A solid solution will need some work and time.

And we can't forget models with libraries that are not system libraries. Like his*her own doCoolStuff.so library that needs to be compiled into the FMU.

I think we have three options:

a) Leave it as it is. So every FMU user needs lapack/openblas. And nearly everbody does.
b) Use my compromise for 1.16 and only someone who want's to use a OpenModelica FMU with LAPACK/BLAS/openblas in it needs those installed in his*her PATH.
c) Spend some time and fix it properly for 1.17-

Of course we can go with b) and c) combined.

[Francesco Casella]

As soon as there is an implicit equation in the model (which happens quite often in OO modelling) these libraries will be needed. I don't really like solutions whereby if you don't have the right library installed in the right place, the FMU will fail with an obscure linker messages complaining about a missing library, without giving a hint about how to fix the problem. IMHO this makes the whole thing look crappy and unprofessional. A new user tries FMU generation out with a simple model with some algebraic loops, gets this, and gives up immediately. I would do that myself.

I would suggest to put the libraries in each and every FMU, which is safe, foolproof, and only somewhat inefficient in terms of memory usage. No big deal, unless you want to have 100 FMUs in your model. Our FMU generation is still not top notch, so I wouldn't be too worried about that for 1.16.0. Then, fix the thing properly in 1.17.0

@AnHeuermann, what do you think?

BTW, FMI is great, but it somehow pushes us back squarely in the '90s, with linker errors, missing libraries, and 32- vs. 64 bit issues. I sometimes have mixed feelings about it :)

[Francesco Casella]

Retargeted to 1.17.0 after 1.16.0 release

[Francesco Casella]

Rescheduled to 1.18.0

[Francesco Casella]

Ticket retargeted after milestone closed

[Francesco Casella]

1.18.0 blocker tickets moved to 1.19.0