Better support for generic types

[Per Östlund]

We currently don't seem to have any good way of implementing generic types, such as dynamic arrays, hashtables, etc. It should be possible to use redeclare to do this, e.g.:

package SomeTypePackage
  replaceable type T = ?;
  
  uniontype SomeType
    // Or maybe define T here?
    record TYPE
      T data;
    end TYPE;

    function new
      input T inValue;
      output SomeType outType = TYPE(inValue);
    end new;

    function get
      input SomeType inType;
      output T outValue;
    algorithm
      TYPE(outValue) := inType;
    end get;
  end SomeType;
end SomeTypePackage;

Trying to use this doesn't quite work though:

// Doesn't work, MetaModelica package dependency stuff fails.
package MyTypePackage = SomeTypePackage(redeclare type T = Integer);

// Same name as the import => ok. But can only be used once in each package.
package SomeTypePackage = SomeTypePackage(redeclare Type T = Integer);
import MyType = SomeTypePackage.SomeType;

MyType t = MyType.new(17); // ok
Integer i = MyType.get(t); // Doesn't work, the compiler ignores the redeclared type and uses the declared type instead.

So trying to use redeclare almost works, but has too many issues currently. Also, the whole uniontype-in-package deal is perhaps not the best way of doing this. It would be nice to allow top-level uniontypes, or something equivalent, since packages are mostly just in the way for these kind of things (uniontypes are not strictly necessary either). Or we can just handle it in some different way, as long as it's possible to implement generic types in a convenient way.

[Martin Sjölund]

attachment:AvlTree.mo​ contains my first attempt at this. It seems that in the generated code, we only include functions for the redeclared functions and not the inherited functions. I am not sure why, but I think I only traverse the Absyn to decide which functions to generate (and partial functions should not be generated).

[Martin Sjölund]

One disadvantage is that the file needs to contain all used AvlTrees in this case, including mixing backend and frontend datatypes in the same file. I should at the same time make it possible to keep the whole file as an interface file (for BaseAvlTree).

[Martin Sjölund]

Does the following look like an acceptable use pattern?
​BaseAvlTree.mo
​AvlTreeString.mo

[Per Östlund]

How flexible are the derived packages? Can they be declared anywhere you like, or do they need to be top-level (or even in their own file)?

[Martin Sjölund]

It should be possible to declare them anywhere you like. There might be some bug where uniontypes cannot be inherited unless they are inherited at the top level... We need to fix the hacks surrounding uniontypes sometime in the future as well ;)

[Per Östlund]

Ok, seems promising. I'll have to play around a bit with it and see how it works.

[Martin Sjölund]

It's still in my pull request as I am unsure if more functionality is needed or if it affects performance in a very bad way.

[Martin Sjölund]

For the bootstrapping to work, there seems to be some problems with the cache (which I have fixed locally), and some problems generating code for the uniontypes (because I disabled the cache)...

[Martin Sjölund]

I have merged my commit, but not created a new bootstrapping tarball (as I want to make some other, bigger changes first, to not move metarecords from their original location and requiring imports to be made to metarecords).

[Per Östlund]

I implemented a dynamic array base class, Util/BaseVector.mo (merged into master, but not added to LoadCompilerSources.mos yet). There seems to be some issues with declaring a new extending package inside another package though. Allowing this would be nice, since otherwise you'll have to add a new file for each derived type, and the declaration is usually very small as seen below.

I tried the obvious first, by adding this inside a top-level package to try it out (see comment in BaseVector for info on DUMMY):

encapsulated package StringVector
  import BaseVector;
  extends BaseVector(redeclare type T = String,
                     redeclare constant String DUMMY = "");
end StringVector;

This causes the compiler to fail with "class StringVector.Vector not found ..." when declaring a component of type StringVector.Vector. Adding import BaseVector in the top-level package fixes the issue. The import inside StringVector is actually not needed in that case, the compiler seems to ignore the encapsulation. Not really a big issue, just a bit backwards.

However, no code is generated for StringVector in this case, so you get undefined references for any functions that are used from it.

[Martin Sjölund]

You do want to create a new file for each package, because you cannot mix front-end and back-end specializations in the same file.

The vector should probably have used type Vector = array<InternalVector> as its internal type (to prevent the "semi-mutable" problem of the hashtable we have). Adding an element now does not cause "old" vectors to contain this element, but a new add would overwrite the old one.

The following works better for me (but still no code...):

encapsulated package StringVector
  import BaseVector;
  extends BaseVector(redeclare type T = String,
                     redeclare constant String DUMMY = "");
annotation(__OpenModelica_Interface="util");
end StringVector;

[Martin Sjölund]

The following also generates code:

encapsulated package StringVector
  import BaseVector;
  extends BaseVector(redeclare type T=String, DUMMY = "XXX");
annotation(__OpenModelica_Interface="util");
end StringVector;

[Martin Sjölund]

I have a mostly working prototype for this now. I guess what is missing is allowing a uniontype as a top-level package in the build system...

[Martin Sjölund]

My testcase:

encapsulated uniontype Deque<T> "Implementation of a mutable double-ended queue (a list that can append to both front and back in linear time)"

record DEQUE
  array<Integer> length;
  array<list<T>> front, back;
end DEQUE;

protected
import GC;
import MetaModelica.Dangerous;

public

impure function empty
  input T dummy;
  output Deque<T> deque;
algorithm
  deque := DEQUE(arrayCreate(1,0),arrayCreate(1,{}),arrayCreate(1,{}));
end empty;

impure function new
  input T first;
  output Deque<T> deque;
protected
  list<T> lst={first};
algorithm
  deque := DEQUE(arrayCreate(1,1),arrayCreate(1,lst),arrayCreate(1,lst));
end new;

function push_front
  input Deque<T> deque;
  input T elt;
protected
  Integer length=arrayGet(deque.length,1);
  list<T> lst;
algorithm
  arrayUpdate(deque.length, 1, length+1);
  if length==0 then
    lst := {elt};
    arrayUpdate(deque.front, 1, lst);
    arrayUpdate(deque.back, 1, lst);
    return;
  end if;
  lst := arrayGet(deque.front,1);
  arrayUpdate(deque.front, 1, elt::lst);
  arrayUpdate(deque.length, 1, length+1);
end push_front;

function push_back
  input Deque<T> deque;
  input T elt;
protected
  Integer length=arrayGet(deque.length,1);
  list<T> lst;
algorithm
  arrayUpdate(deque.length, 1, length+1);
  if length==0 then
    lst := {elt};
    arrayUpdate(deque.front, 1, lst);
    arrayUpdate(deque.back, 1, lst);
    return;
  end if;
  lst := {elt};
  Dangerous.listSetRest(arrayGet(deque.back,1), lst);
  arrayUpdate(deque.back, 1, lst);
end push_back;

function toListAndClear
  input Deque<T> deque;
  output list<T> res;
algorithm
  res := arrayGet(deque.front,1);
  arrayUpdate(deque.back, 1, {});
  arrayUpdate(deque.front, 1, {});
  arrayUpdate(deque.length, 1, 0);
end toListAndClear;

function test
  output Boolean b=true;
protected
  Deque<Integer> d;
algorithm
  d := new(1);
  push_back(d, 2);
  print(stringDelimitList(list(String(j) for j in toListAndClear(d)), ","));
  print("\n");
  push_back(d, 3);
  print(stringDelimitList(list(String(j) for j in toListAndClear(d)), ","));
  print("\n");
end test;

protected

annotation(__OpenModelica_Interface="backend");
end Deque;

[Martin Sjölund]

Fixed in 7e8eeef.

[Martin Sjölund]

Milestone renamed

[Martin Sjölund]

Milestone renamed