String GC issues

[Per Östlund]

It seems we have some kind of garbage collection issue with strings. For example with this model:

model M
  String str[:] = Modelica.Utilities.Streams.readFile("M.mo"); // Reads itself
algorithm
  for i in 1:1000 loop
    print(String(i) + "\n");
    for j in 1:size(str, 1) loop
      print(str[j] + "\n");
    end for;
  end for;
end M;

Compiling this with +d=noevalfunc so the readFile isn't constant evaluated, this model segfaults in stringAppend every time I run it. It prints itself out ~990 times or so (it's slightly different each execution), and then it starts to drop some of the newlines until it finally segfaults a few iterations later.

Running the model with GC_DONT_GC=1 works perfectly fine, so it seems to be some kind of GC issue. I also tried it with an older revision than r23321 (before the change to MetaModelica strings), and the only difference was that it started printing garbage after a while instead of segfaulting. Turning off the GC solved the issue in that case also.

This is the last remaining issue with the wind power model I'm trying to get running.

[Martin Sjölund]

For how long did you run this? I killed it manually after 8m22s because it just wouldn't crash.

[Per Östlund]

Replying to sjoelund.se:

For how long did you run this? I killed it manually after 8m22s because it just wouldn't crash.

It segfaults in less than a second. Actually, running with GC_DONT_GC=1 it finishes the simulation successfully in ~0.8 seconds for me, producing 112 MB of output. So it seems odd that it's still running after more than 8 minutes for you.

[Martin Sjölund]

Odd. I piped it to /dev/null to speed it up. But then it just doesn't finish. If I grep 999 or something, it does segfault.

[Martin Sjölund]

On my laptop I can never reproduce this. Only sometimes on my desktop. Seems very hard to fix due to this.

[Per Östlund]

Replying to sjoelund.se:

On my laptop I can never reproduce this. Only sometimes on my desktop. Seems very hard to fix due to this.

Try it on test.openmodelica.org, it segfaults 100% of the time for me on that machine. Also, I'm using clang, GCC might behave differently.

[Per Östlund]

Also tried with GCC, as well as GC_MARKERS=1 with both Clang and GCC, but the same issue happens in all cases.

[Adrian Pop]

This seems to be a problem with dll load/unload several times combined with GC.
I'll try some things and see if I can get it fixed.

[Martin Sjölund]

Replying to adrpo:

This seems to be a problem with dll load/unload several times combined with GC.

Replying to perost:

Compiling this with +d=noevalfunc

I doubt dll load/unload is the problem since there is no dll loading and only a simulation running.

[Adrian Pop]

Yes. I got sidetracked by the fact that I ran it with +d=noevalfunc which made some dll loading of readFile, but the actual crash is inside the simulation executable.

If I compile omc with CFLAGS=-g -falign-functions then this problem does not happen anymore, so it
seems to be some sort of optimization issue.

[Adrian Pop]

I tried with both -O3, -O2 and it worked fine when calling:

./omc gc.mos

where gc.mo is:

model gctest
  String str[:] = Modelica.Utilities.Streams.readFile("gc.mo"); // Reads itself
algorithm
  for i in 1:1000 loop
    print(String(i) + "\n");
    for j in 1:size(str, 1) loop
      print(str[j] + "\n");
    end for;
  end for;
end gctest;

and gc.mos is:

loadModel(Modelica); getErrorString();
loadFile("gc.mo"); getErrorString();
simulate(gctest); getErrorString();

Strangely I can only reproduce this when I compile the model with

./omc +d=noevalfunc gc.mos

because then the readFile function is compiled in and the str array is populated via:

/*
 equation index: 1
 type: SIMPLE_ASSIGN
 str[10] = Modelica.Utilities.Streams.readFile("gc.mo")[10]
 */
void gctest_eqFunction_1(DATA *data)
{
  const int equationIndexes[2] = {1,1};
  TRACE_PUSH
  $Pstr$lB10$rB = string_get(omc_Modelica_Utilities_Streams_readFile(threadData, _OMC_LIT0), 9);
  TRACE_POP
}

which means a call for each str element (via ASUB).
So it seems the issue has to do with arrays of strings and function calls.

[Per Östlund]

I did some more testing. This works:

model test
  String str;
algorithm
  for i in 1:1000 loop
    str := String(time);
    print(str + "\n");
  end for;
end test;

This segfaults everytime in GC_malloc_atomic after 0.062s:

model test
  String str[1000];
algorithm
  for i in 1:1000 loop
    str[i] := String(time);
    print(str[i] + "\n");
  end for;
end test;

[Adrian Pop]

I also debugged this last night. you can do:

export GC_PRINT_STATS=

before running it to see that a collection happens just before it crashes.
It seems that string arrays are not handled very well in the GC.

[Adrian Pop]

This doesn't seem to happen with the previous GC we had in OpenModelicaExternal (7.2f or so).
I will revert to that version but the problem is that it doesn't have the GC_get_prof_stats functionality.

[Per Östlund]

Replying to adrpo:

This doesn't seem to happen with the previous GC we had in OpenModelicaExternal (7.2f or so).
I will revert to that version but the problem is that it doesn't have the GC_get_prof_stats functionality.

I tried it with the old version, it made no difference at all for me.

[Adrian Pop]

Partial fix in r23562. The initial given model works, the last one given by Per in the comments, not yet.

[Martin Sjölund]

I'm not sure why those fixes would help. size_alloc only contains integers and string_alloc should only be used for strings (no pointers), right?
Changing from malloc_atomic to malloc_string changes nothing if we use the pooled interface by default. But we should be using the GC interface, right?

[Martin Sjölund]

Also, if the strings are allocated using libGC and the arrays using the memory pool, the strings cannot be found when searching the roots.

[Adrian Pop]

Well, my latest changes doesn't seem to affect anything as the bug was partial fixed by one of my previous commit anyway (r23549 i think).

As to string_alloc, it is used in string_array.c to build array of strings,
so I guess some things were not properly adapted when you changed from Modelica
to MetaModelica strings.

As far as I can tell is an entire mess which allocation interface is used:

• GC*
• GC* via omc_alloc_interface
• direct calloc & malloc

Some of the pooled interface seems to be using omc_alloc_interface???!!
What's up with that? Shouldn't be directly malloc, etc?

[Martin Sjölund]

It's sort of ok, because we have: omc_alloc_interface = omc_alloc_interface_pooled; in generated code.

I guess if you disable that everything will magically start working.

[Adrian Pop]

Now I'm more confused than before.
We use omc_alloc_interface_pooled in the generated Modelica code??!!

[Adrian Pop]

No true, omc_alloc_interface_pooled is only used for HPCOM, ParModelica, and JavaScript.

[Martin Sjölund]

Then I'm at a loss why this doesn't work :)

[Adrian Pop]

Fixed in r23564.

[Dietmar Winkler]

It doesn't make sense to keep closed ticket in the "Future" milestone that were simply forgotten to assign to the correct milestone in the past.

[Martin Sjölund]

Removing the pre1.9.4 milestone in favor of 1.9.4.