Translation does not terminate for large models

[kkanso@…]

I have simple, but large files that consist of arrays of values used to provide realistic input for a simulation.

The attached file House6.mo contains 36 models, each of which consists of 3 arrays of values. The three Test*.mo files import various combinations of these models.

Test.mo imports all of these models, but does not finish the "translating" phase when simulating in OMEdit, no errors are reported but simply does not terminate, it was left running for approx 12 hours then killed. The Test2.mo and Test3.mo import the first 18 and last 18 models, respectively. Both of these test models simulate without problems, translating phase takes approx 1 minute.

This was observed on version: OpenModelica 1.9.1+dev (r20821) (RML version), Ubuntu x64 installed from the repository.

[perost]

I get very different behaviour. Each house instance increases instantiation with ~20 sec for me, and the whole Test model takes about 14 min to instantiate. This is with r21019, bootstrapped version.

Are you sure that you can instantiate Test2 or Test3 in 1 minute? They each take about 7 min for me (with a Core i7 930 @ 3.5 GHz). I will check out the version you're using and see if it behaves differently for me, but please try out the latest nightly build if you can and see if it works better for you.

[kkanso@…]

I have just updated to the latest version in the Ubuntu repositories (r21018).

The translation phase of Test2 takes approx 3 minutes (from OMEdit), I cancelled the compilation after the translating completed. I didn’t time it very well before. Instantiating Test2 takes approx 3 minutes as well.

However, Test still does not finish the translating phase after 20 minutes. The above comment suggests that it should be linear, so should have finished by now. Instantiating Test takes about 6 minutes.

The computer I am using is an Intel i3 2.2Ghz. Could the time difference be related to running a 64bit version?

[perost]

It seems I ran my tests with OMC compiled without optimizations, with the default -O2 it takes about half as long for me (~3.5 for Test). I tested r20821, but got the same performance.

Then I realised that OMEdit is not using the same terminology as I do, and that translating actually means going through the entire compiler and generating a simulation executable. I've only been running the model through the front-end of the compiler, i.e. the instantiation phase.

So I tried using the buildModel API call on Test, and ran out of memory. So I tried it on another machine with more memory, which finished building the simulation executable in just above 7 min. In the process it used up ~8 GB of memory though. How much memory do you have? Maybe you run out of memory and start swapping, which will probably kill the performance of the back-end which is very memory heavy.

[kkanso@…]

The machine is not swapping, it has 16GB. After running the translation on Test for 1 hour I have the following from top:

KiB Mem:  16327772 total, 14682588 used,  1645184 free,   236208 buffers
KiB Swap: 25484284 total,    52976 used, 25431308 free,  3830732 cached

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND           
29100 kazza     20   0 1953m 990m 9232 R  99.8  6.2  52:00.62 omc               
28313 kazza     20   0 2049m 949m  11m R  98.8  6.0 152:29.22 omc 
...
29093 kazza     20   0 2209m 118m  41m R  10.9  0.7   5:12.89 OMEdit 

The memory usage has been stable for a while, it just eats up CPU and appears to do nothing.

[kkanso@…]

If you can give me instructions how to call buildModel directly, I can try that if it will help?

[perost]

Replying to kkanso@…:

If you can give me instructions how to call buildModel directly, I can try that if it will help?

Certainly, just create a .mos-file that looks like this:

loadModel(Modelica);
loadFile("Test.mo");
loadFile("Houses6.mo");
buildModel(Test);
getErrorString();

Then run it as omc scriptname.mos. It might be that there is some issue with the RML-version of the compiler. Maybe someone who still uses RML could check? Alternatively you could try the bootstrapped version, which we will hopefully soon switch to anyway. I think you just need to install the omc-bootstrapped package and use omc-bootstrapped instead of omc.

Edit: Changed Test2 => Test in the script.

[kkanso@…]

Thanks, it worked!

As you suggested it takes 6 to 7 minutes to execute. The only thing is that the stack size needed to be increased.

I'm using: -stack-size=1073741824

I tried to copy this over to OMEdit in the compiler settings, but it made no difference. It still does not terminate for this model.

Maybe this bug should be moved to OMEdit than the frontend?

[perost]

Replying to kkanso@…:

Thanks, it worked!

As you suggested it takes 6 to 7 minutes to execute. The only thing is that the stack size needed to be increased.

What happens if you don't increase the stack size? Does omc exit with an error message or just continue running forever? If omc exits then OMEdit should handle it.

I tried to copy this over to OMEdit in the compiler settings, but it made no difference. It still does not terminate for this model.

Maybe this bug should be moved to OMEdit than the frontend?

OMEdit starts omc at startup time, and then communicates with omc via CORBA. So the stack size can't be set separately on a simulation, but would need to be set by OMEdit at startup time when omc is started. It would probably be relatively little work to add an option in the settings for the stack size though. On the other hand we will soon™ switch to using the bootstrapped compiler exclusively, which does not have (nor need) a flag for setting the stack size. I'm assigning the ticket back to Adeel and we'll see what happens.

[kkanso@…]

Replying to perost:

Replying to kkanso@…:

Thanks, it worked!

As you suggested it takes 6 to 7 minutes to execute. The only thing is that the stack size needed to be increased.

What happens if you don't increase the stack size? Does omc exit with an error message or just continue running forever? If omc exits then OMEdit should handle it.

It exits and complains about stack overflow.

Stack overflow! Failing the current function stack chain until the stack overflow signal is caught!

"Error: Error occurred while flattening model Test. The compiler got into Stack Overflow!

[sjoelund.se]

This ticket was not closed for 1.9.1, which has now been released. It was batch modified for milestone 1.9.2 (but maybe an empty milestone was more appropriate; feel free to change it).

[adeas31]

Now since we have moved to bootstrapped compiler, what is the status of this ticket now?

[sjoelund.se]

Milestone changed to 1.9.3 since 1.9.2 was released.