The enclosed code translates, Test2 can be instantiated, but compilation fails. Test runs

[Anton.Soppelsa@eurac.edu]

Dear all,

trying to encapsulate an external object class (TSModelHandle) within an interface class (TSModelC, allowing, for example, to derive its icon from the graphic classes) I get a compilation error building Test2 (Test runs correctly). Despite my environment is complex and you will not be able to reproduce the problem yourself (the external object is an interface layer to Python3 code), I think that the compiler complains could be indicative enough for somebody of you to see immediately if my attempt is fundamentally flawed. If not, there could be a problem in the code generation.

This happen with version 1.14.1 and 1.16.0-dev.03. IF you need me to do some testing I will be happy to do it.

Sincerely,

Anton

within SHCControls;

package TSModel
  extends Modelica.Icons.TypesPackage;

  class TSModelHandle
    extends ExternalObject;

    function constructor "Gets an handle to a TSModel object loaded from file."
      extends Modelica.Icons.Function;
      input String fileName;
      input String modelName;
      output TSModelHandle tsmh;
    
      external "C" tsmh = initTSModel(fileName, modelName) annotation(
        Library = "TSModel-Interface",
        LibraryDirectory = "file:///C:/Development/shc-controls/SHCLib/C-Cpp/Cpp-ModelicaInterface/Build/Release");
    end constructor;

    //C:\\Development\\shc-controls\\SHCLib\\C-Cpp\\Cpp-ModelicaInterface\\Build\\Release

    function destructor "Release storage of the TSModel object."
      extends Modelica.Icons.Function;
      input TSModelHandle tsmh;
    
      external "C" finalTSModel(tsmh) annotation(
        Library = "TSModel-Interface",
        LibraryDirectory = "file:///C:/Development/shc-controls/SHCLib/C-Cpp/Cpp-ModelicaInterface/Build/Release");
    end destructor;

    //      external "C" finalTSModel(tsmh) annotation(Library="TSModel-Interface"); // If not specified looks in the default dir Library/win64
  end TSModelHandle;

  function tsmEval "Evaluate model at point u"
    extends Modelica.Icons.Function;
    input TSModelHandle tsmh;
    input Real u[:];
    output Real y[ysize];
    input Integer ysize;
  
    external "C" tsmEval(tsmh, u, size(u, 1), y, size(y, 1)) annotation(
      Library = "TSModel-Interface",
      LibraryDirectory = "file:///Development/shc-controls/SHCLib/C-Cpp/Cpp-ModelicaInterface/Build/Release");
  end tsmEval;

  function invTsmEval "Evaluate inverse model at point y"
    extends Modelica.Icons.Function;
    input TSModelHandle tsmh;
    input Real u0[:];
    input Real y[:];
    output Real u[size(u0, 1)];
  
    external "C" tsmEvalInv(tsmh, u0, size(u0, 1), y, size(y, 1), u, size(u, 1)) annotation(
      Library = "TSModel-Interface",
      LibraryDirectory = "file:///Development/shc-controls/SHCLib/C-Cpp/Cpp-ModelicaInterface/Build/Release");
  end invTsmEval;

  model Test "Test of the TSModel package"
    import Modelica.Blocks.Sources.{Step,Constant};
    import Modelica.Blocks.Routing.{Multiplex2};
    TSModelHandle tsmh = TSModelHandle(fileName = "C:\\Development\\shc-controls\\SHCLib\\Python\\tests\\cannonau2x2.mat", modelName = "Cannonau");
    // call initTSModel
    Real u0[2] = {0.4, 75e3};
    Real u[2];
    Real y[2];
    Real uTilde[2];
    Step T1R(height = 5, offset = 273.15 + 50, startTime = 10) annotation(
      Placement(visible = true, transformation(origin = {-70, 70}, extent = {{-10, -10}, {10, 10}}, rotation = 0)));
    Step M1R(height = 0.03, offset = 0.3, startTime = 20) annotation(
      Placement(visible = true, transformation(origin = {-70, 30}, extent = {{-10, -10}, {10, 10}}, rotation = 0)));
    Step U(height = 0.05, offset = 0.5, startTime = 15) annotation(
      Placement(visible = true, transformation(origin = {-70, -30}, extent = {{-10, -10}, {10, 10}}, rotation = 0)));
    Step DP(height = 1000, offset = 50000, startTime = 15) annotation(
      Placement(visible = true, transformation(origin = {-70, -70}, extent = {{-10, -10}, {10, 10}}, rotation = 0)));
    Multiplex2 muxR annotation(
      Placement(visible = true, transformation(origin = {-30, 50}, extent = {{-10, -10}, {10, 10}}, rotation = 0)));
    Multiplex2 muxU annotation(
      Placement(visible = true, transformation(origin = {-30, -50}, extent = {{-10, -10}, {10, 10}}, rotation = 0)));
  equation
    u = muxU.y;
    y = tsmEval(tsmh, u, size(y, 1));
    uTilde = invTsmEval(tsmh, u0, muxR.y);
    connect(U.y, muxU.u1[1]) annotation(
      Line(points = {{-58, -30}, {-50, -30}, {-50, -44}, {-42, -44}, {-42, -44}}, color = {0, 0, 127}));
    connect(DP.y, muxU.u2[1]) annotation(
      Line(points = {{-58, -70}, {-50, -70}, {-50, -56}, {-42, -56}, {-42, -56}}, color = {0, 0, 127}));
    connect(M1R.y, muxR.u2[1]) annotation(
      Line(points = {{-59, 30}, {-50, 30}, {-50, 44}, {-42, 44}}, color = {0, 0, 127}));
    connect(T1R.y, muxR.u1[1]) annotation(
      Line(points = {{-59, 70}, {-50, 70}, {-50, 56}, {-42, 56}}, color = {0, 0, 127}));
    annotation(
      experiment(StartTime = 0, StopTime = 50, Tolerance = 1e-06, Interval = 0.1));
  end Test;

  class TSModelC
    parameter String fileName = "C:\\Development\\shc-controls\\SHCLib\\Python\\tests\\cannonau2x2.mat";
    parameter String modelName = "Cannonau";
    parameter TSModelHandle tsmh = TSModelHandle(fileName, modelName);

    function eval "Evaluate model at point u"
      extends Modelica.Icons.Function;
      input Real u[:];
      output Real y[ysize];
      input Integer ysize;
    algorithm
      y := tsmEval(tsmh, u, size(y, 1));
    end eval;

    function evalInv "Evaluate inverse model at point y with initial guess u0"
      extends Modelica.Icons.Function;
      input Real u0[:];
      input Real y[:];
      output Real u[size(u0, 1)];
    algorithm
      y := invTsmEval(tsmh, u0, y);
    end evalInv;
    annotation(
      Icon(graphics = {Rectangle(fillColor = {255, 255, 255}, fillPattern = FillPattern.Solid, extent = {{-100, 100}, {100, -100}}), Text(origin = {0, -130}, lineColor = {0, 0, 255}, fillColor = {0, 0, 255}, extent = {{-100, -20}, {100, 20}}, textString = "%name"), Text(origin = {0, -40}, extent = {{-100, -20}, {100, 20}}, textString = "Ext Obj"), Text(origin = {0, 40}, extent = {{-100, -20}, {100, 20}}, textString = "TSModel")}, coordinateSystem(initialScale = 0.1)));
  end TSModelC;

  model Test2 "Test of the TSModelC class"
    SHCControls.TSModel.TSModelC tsm(fileName = "C:\\Development\\shc-controls\\SHCLib\\Python\\tests\\cannonau2x2.mat", modelName = "Cannonau") annotation(
      Placement(visible = true, transformation(origin = {0, 0}, extent = {{-20, -20}, {20, 20}}, rotation = 0)));
    Real u[2] = {0.4, 75e3};
    Real y[2];
    Real uTilde[2];
  equation
    y = tsm.eval(u, size(y, 1));
    uTilde = tsm.evalInv({0.5, 75e3}, y);
  end Test2;
end TSModel;

Compilation errors:

C:/Program Files/OpenModelica1.16.0-dev.03-64bit/share/omc/scripts/Compile.bat SHCControls.TSModel.Test2 gcc mingw64 parallel 6 0
PATH = "C:\PROGRA~1\OPENMO~1.03-\tools\msys\mingw64\bin;C:\PROGRA~1\OPENMO~1.03-\tools\msys\mingw64\bin\..\..\usr\bin;"
mingw32-make: Entering directory 'C:/Users/ASOPPE~1/AppData/Local/Temp/OPENMO~1/OMEdit/SHCCON~2.TES'
gcc  -Os -falign-functions -fno-ipa-pure-const -mstackrealign -msse2 -mfpmath=sse     -I"C:/Program Files/OpenModelica1.16.0-dev.03-64bit/include/omc/c" -I. -DOPENMODELICA_XML_FROM_FILE_AT_RUNTIME -DOMC_MODEL_PREFIX=SHCControls_TSModel_Test2 -DOMC_NUM_MIXED_SYSTEMS=0 -DOMC_NUM_LINEAR_SYSTEMS=0 -DOMC_NUM_NONLINEAR_SYSTEMS=0 -DOMC_NDELAY_EXPRESSIONS=0 -DOMC_NVAR_STRING=0  -c -o SHCControls.TSModel.Test2.o SHCControls.TSModel.Test2.c
gcc  -Os -falign-functions -fno-ipa-pure-const -mstackrealign -msse2 -mfpmath=sse     -I"C:/Program Files/OpenModelica1.16.0-dev.03-64bit/include/omc/c" -I. -DOPENMODELICA_XML_FROM_FILE_AT_RUNTIME -DOMC_MODEL_PREFIX=SHCControls_TSModel_Test2 -DOMC_NUM_MIXED_SYSTEMS=0 -DOMC_NUM_LINEAR_SYSTEMS=0 -DOMC_NUM_NONLINEAR_SYSTEMS=0 -DOMC_NDELAY_EXPRESSIONS=0 -DOMC_NVAR_STRING=0  -c -o SHCControls.TSModel.Test2_functions.o SHCControls.TSModel.Test2_functions.c
gcc  -Os -falign-functions -fno-ipa-pure-const -mstackrealign -msse2 -mfpmath=sse     -I"C:/Program Files/OpenModelica1.16.0-dev.03-64bit/include/omc/c" -I. -DOPENMODELICA_XML_FROM_FILE_AT_RUNTIME -DOMC_MODEL_PREFIX=SHCControls_TSModel_Test2 -DOMC_NUM_MIXED_SYSTEMS=0 -DOMC_NUM_LINEAR_SYSTEMS=0 -DOMC_NUM_NONLINEAR_SYSTEMS=0 -DOMC_NDELAY_EXPRESSIONS=0 -DOMC_NVAR_STRING=0  -c -o SHCControls.TSModel.Test2_records.o SHCControls.TSModel.Test2_records.c
gcc  -Os -falign-functions -fno-ipa-pure-const -mstackrealign -msse2 -mfpmath=sse     -I"C:/Program Files/OpenModelica1.16.0-dev.03-64bit/include/omc/c" -I. -DOPENMODELICA_XML_FROM_FILE_AT_RUNTIME -DOMC_MODEL_PREFIX=SHCControls_TSModel_Test2 -DOMC_NUM_MIXED_SYSTEMS=0 -DOMC_NUM_LINEAR_SYSTEMS=0 -DOMC_NUM_NONLINEAR_SYSTEMS=0 -DOMC_NDELAY_EXPRESSIONS=0 -DOMC_NVAR_STRING=0  -c -o SHCControls.TSModel.Test2_01exo.o SHCControls.TSModel.Test2_01exo.c
gcc  -Os -falign-functions -fno-ipa-pure-const -mstackrealign -msse2 -mfpmath=sse     -I"C:/Program Files/OpenModelica1.16.0-dev.03-64bit/include/omc/c" -I. -DOPENMODELICA_XML_FROM_FILE_AT_RUNTIME -DOMC_MODEL_PREFIX=SHCControls_TSModel_Test2 -DOMC_NUM_MIXED_SYSTEMS=0 -DOMC_NUM_LINEAR_SYSTEMS=0 -DOMC_NUM_NONLINEAR_SYSTEMS=0 -DOMC_NDELAY_EXPRESSIONS=0 -DOMC_NVAR_STRING=0  -c -o SHCControls.TSModel.Test2_02nls.o SHCControls.TSModel.Test2_02nls.c
gcc  -Os -falign-functions -fno-ipa-pure-const -mstackrealign -msse2 -mfpmath=sse     -I"C:/Program Files/OpenModelica1.16.0-dev.03-64bit/include/omc/c" -I. -DOPENMODELICA_XML_FROM_FILE_AT_RUNTIME -DOMC_MODEL_PREFIX=SHCControls_TSModel_Test2 -DOMC_NUM_MIXED_SYSTEMS=0 -DOMC_NUM_LINEAR_SYSTEMS=0 -DOMC_NUM_NONLINEAR_SYSTEMS=0 -DOMC_NDELAY_EXPRESSIONS=0 -DOMC_NVAR_STRING=0  -c -o SHCControls.TSModel.Test2_03lsy.o SHCControls.TSModel.Test2_03lsy.c
SHCControls.TSModel.Test2_functions.c: In function 'omc_SHCControls_TSModel_Test2_tsm_eval':
SHCControls.TSModel.Test2_functions.c:99:68: error: '_tsm' undeclared (first use in this function)
   copy_real_array_data(omc_SHCControls_TSModel_tsmEval(threadData, _tsm._tsmh, _u, tmp1), &_y);
                                                                    ^
SHCControls.TSModel.Test2_functions.c:99:68: note: each undeclared identifier is reported only once for each function it appears in
SHCControls.TSModel.Test2_functions.c: In function 'omc_SHCControls_TSModel_Test2_tsm_evalInv':
SHCControls.TSModel.Test2_functions.c:122:71: error: '_tsm' undeclared (first use in this function)
   copy_real_array_data(omc_SHCControls_TSModel_invTsmEval(threadData, _tsm._tsmh, _u0, _y), &_y);
                                                                       ^
<builtin>: recipe for target 'SHCControls.TSModel.Test2_functions.o' failed
mingw32-make: *** [SHCControls.TSModel.Test2_functions.o] Error 1
mingw32-make: *** Waiting for unfinished jobs....
mingw32-make: Leaving directory 'C:/Users/ASOPPE~1/AppData/Local/Temp/OPENMO~1/OMEdit/SHCCON~2.TES'
Compilation process failed. Exited with code 2.

[Adrian Pop]

The problem seems to be the parameter external object used directly in a function in TSModelC.
I'm not even sure this is legal if the external object is not a constant, I'll leave that for Per to decide.

As a workaround for now, try to send TSModelHandle tsmh as input to functions TSModelC.eval and TSModelC.evalInv as you already did for tsmEval.

[Francesco Casella]

Replying to adrpo:

The problem seems to be the parameter external object used directly in a function in TSModelC.
I'm not even sure this is legal if the external object is not a constant, I'll leave that for Per to decide.

I guess it isn't, please check the Modelica Language Specification, ​section 12.9.7 for the expected usage pattern. I guess this should be caught by the compiler.

[Anton.Soppelsa@eurac.edu]

Well, I don't see which rule this code would be breaking.

Luckily, the specification is short enough that it can analysed clause by clause. Leaving out statements that are requirement to the compiler, they are:

1. since the class is partial, it is not possible to define an instance of this class

I'm not istantiatin the ExternalObject class directly, only a class derived from it which is legal and the intended usage.

2. An external object class shall be directly extended from ExternalObject, shall have exactly two function definitions, called ”constructor” and ”destructor”, and shall not contain other elements.

TSModelHandle complies with this.

3. The functions ”constructor” and ”destructor” shall not be replaceable.

TSModelHandle complies with this.

4. The constructor shall have exactly one output argument in which the constructed instance derived from ExternalObject is returned.

TSModelHandle complies with this.

5. The destructor shall have no output arguments and the only input argument of the destructor shall be of the type derived from ExternalObject.

TSModelHandle complies with this.

6. It is not legal to call explicitly the constructor and destructor functions.

Test2 (and the entire code) never calls the constructor or destructor functions directly.

7. The constructor shall initialize the object, and must not require any other calls to be made for the initialization to be complete (e.g., from an initial algorithm or initial equation)

TSModelHandle complies with this.

8. The destructor shall delete the object, and must not require any other calls to be made for the deletion to be complete (e.g., from a ’when terminal()’ clause).

TSModelHandle complies with this.

9. The constructor may not assume that pointers sent to the external object will remain valid for the life-time of the external object. 10. [An exception is that if the pointer to another external object is given as argument to the constructor, that pointer will remain valid as long as the other external object lives.]

This is not fully clear to me. What pointers is the specification referring to? I guess it is a protection against external code that may free some memory. My dll does not delete the pointers passed by Modelica.

11. External objects may be a protected component (or part of one) in a function

This is not a limitation, however, it is not the my case.

12. External objects may be an input (or part of an input) to a function, in that case the destructor is not called (since the external object is active before and after the function call).

13. Normally this is an external function, but it could be a non-external function as well (e.g. calling external functions one or more times)

These are not a limitations, however, it is not the my case.

14. The function input may not have a default value using the constructor.

I interpret this as "If you are using the constructor you can avoid to specify the default value for the function input", but it is not clear to me what it means.

15. Classes derived from ExternalObject can neither be used in an extends-clause nor in a short class definition.

TSModelC class and Test2 comply with this. They are not extending TSModelHandle or doing short class definitions from it.

16. Only the constructor may return external objects

TSModelHandle, TSModelC and Test2 comply with this.

17. external object can only be bound in component declarations and neither modified later nor assigned to

TSModelC and Test2 comply with this. TSModelC bounds and external object as a component delaration (very similarly to what Test does). Test2 bounds in the same way an instance of class TSModelC.

18. No function may return a component containing an external object

TSModelHandle, TSModelC and Test2 comply with this.

19. External functions may be defined which operate on the internal memory of an ExternalObject.

This is not a limitation and is exactly what the underlying dll does. Functions used data stored in the external object to process the inputs passed by Modelica.

What am I getting wrong?

If my analysis is correct and still there are technical reasons to exclude that programming pattern, then the specification should make the point. Misunderstanding could come from the fact that the mechanism could be designed for "functions with memory" but the wording and even the example drive the user strongly towards "objects with methods".

Let me know what do you think.

[Adrian Pop]

The problem is not from external object, is the fact that you're using it as a global variable which is not constant (tsmh in your case). You can see this pattern in the model below. Unfortunately OpenModelica doesn't detect this and generates code where this parameter component is missing.

  model M
    parameter Real a = 2;

    function f
      input Real u1;
      input Real u2;
      output Real o;
    algorithm 
      // a is a global non-constant parameter used in a function
      o := u1 + u2 + a;
    end f;

    Real x(start=1);

  equation 
    der(x) = f(x, 2);
  end M;

The model below, however, works fine because the variable is sent as a function input:

  model M
    parameter Real a = 2;

    function f
      input Real u1;
      input Real u2;
      output Real o;
    algorithm 
      o := u1 + u2;
    end f;

    Real x(start=1);

  equation 
    der(x) = f(x, a);
  end M;

[Adrian Pop]

For example, this is how Dymola complains on the first model, and we should do that too:

Check of TestExternal.M:
Variables found in types or in global scope must be constant, but a wasn't.
In class TestExternal.M.f.
Modelica Text: line 7
Errors found in: o = u1+u2+a
Modelica Text: line 12
The model contained invalid expressions.
Check aborted.
ERRORS have been issued.

[Anton.Soppelsa@eurac.edu]

Dear Adrian,

thank you for your clarifications. It seems that from one way or the other I always slam my face with this idiom. I have changed the structure of my classes to accomodate you observation.

For the records, I write here how I proceeded. The point is making eo, an external object of type EO with methods m1, m2, m3, ..., work as much as possible as a normal class. I came up with two use methods, based on the following "layout":

class MyEO
  class Handle
    extends ExternalObject;
    
    function constructor
      output Handle eoh; 
      ...
      extern "C" init(...);
    end constructor

    function destructor
      input Handle eoh;
      ...
      extern "C" final(...);
    end destructor
  end Handle;

  // Public interface
  function m1
    input Handle eoh;
    ...
    m1_(...);
  end m1;

  function m2
    input Handle eoh;
    ...
    m2_(...);
  end m2;

  // Private interface
  function m1_
    input Handle eoh;
    ...
    extern "C" m1(...);
  end m1_;

  function m2_
    input Handle eoh;
    ...
    extern "C" m2(...);
  end m2_;

  ...

end MyEO;

Method 1, where I landed, is the following:

model M
  import MyEO;

  MyEO.Handle eoh = MyEO.Handle(...);


  MyEO.m1_(eoh, ...);
  MyEO.m2_(eoh, ...);

  ... 
end M;

Method 2, where I started, with the idea of instantiating only EO, without passing the handle to each function (like in C++, for example) can be made to work only applying such modifications that render it useless.

model M
  import MyEO;

  MyEO.Handle eoh = MyEO.Handle(...); // Would like to avoid this, but it's a constrain of the language that this has to be declared at global scope to be accessible by functions.
  MyEO eo;


  eo.m1(eoh, ...); // Would like to avoid passing eoh, but can't (for the same reason as above)
  eo.m2(eoh, ...); // Would like to avoid passing eoh, but can't (for the same reason as above)

  ... 
end M;

Method 2 shonw above also works but since it requires an additional declaration without solving the problem of passing the handle, I got rid of it and removed the private interface.

Sincerely,

Anton