Intermediate compilation files should be removed at the end of code generation

[Francesco Casella]

When generating simulation code, omc outputs a large number of possibly big .c, .h, and .o files which are used to finally obtain the simulation executable file. Unless they are regularly deleted, these files are going to bloat the temporary working directory indefinitely, until the HD is filled to capacity.

As we (still) don't have separate compilation of models, I understand these files cannot be reused a second time. Besides, now that we have extensive documentation on the solved model structure in the XML files and in the interactive debugger, I guess very few people will actually be interested at reading the source code, once the executable has been built.

Therefore I would recommend that, unless a specific omc flag is turned on (say, for debugging purposes), all those files are automatically removed at the end of the code generation process.

[Martin Sjölund]

Milestone changed to 1.9.3 since 1.9.2 was released.

[Martin Sjölund]

Moved to new milestone 1.9.4

[Martin Sjölund]

I'm not sure about this. Maybe it would make sense for OMEdit to clean up after itself. But for OMC... Should for example translateModel() clean up everything after generating code when the user probably meant to generate code and compile it manually?

[Francesco Casella]

I always use omc from the command line for testing, not OMEdit, and all these leftovers are very inconvenient.

I would introduce a flag --removeIntermediateFiles which defaults to true. If you want to keep the files, just use --noremoveIntermediateFiles. I am sure this is of interest for less than 5% of users, and those who are interested can sure enough set a flag. For everybody else, all those files are just an inconvenience.

In fact, if the default is true, we are introducing a non-backwards compatible change, for those people who use omc to generate C code and want to use that. If that is a problem, then it's ok for me to make this flag false by default. Then, OMEdit and OMNotebook should set it to true by default themselves. And I could easily do it from the command line :)

[Martin Sjölund]

Milestone pushed to 1.9.5

[Martin Sjölund]

Milestone renamed

[Martin Sjölund]

Ticket retargeted after milestone closed

[Martin Sjölund]

Milestone moved to 1.12.0 due to 1.11.0 already being released.

[Francesco Casella]

These are the statistics of the OMEdit working directory on my PC, after one year of moderate use (I mostly use command line and/or a Linux server): about 4800 files and 1.5 GB. About 650 MB are occupied by 82 executable files, about 250 MB are as many result files and another about 250 MB are twice as many json and xml files. Though the chances are quite low, these could actually be of some use.

On the other hand, 4180 files are totally useless intermediate .c, .h, and .o files. Their combined size is "only" 275 MB, but they make up 87% of the files, which is a pain in case of backups or similar, add to disk fragmentation, and in general unecessarily clutter the file system.

@adeas31, would you mind adding an option to the Options|General tab, named "Delete intermediate compilation files", checked by default, which causes OMEdit to delete *.c, *.o, and *.h files after the compilation is complete? Shouldn't take much to implement.

You may also want to remove the .makefile files; they're not many, but they are basically useless for the OMEdit user, particularly once the source files are gone.

Would be nice to also cherry pick this fix for 1.12.0 :)

Thanks!

[Adeel Asghar]

I have added an option named Delete intermediate compilation files to Options|Simulation tab since it is more suited there instead of Options|General tab.

If checked all the generated *.c, *.h, *.o and .makefile are deleted after compilation.
The .exe, _info.json, _init.xml, _prof.intdata, _prof.realdata and result file are not deleted since they are needed for simulation and plotting.

126db4/OMEdit.

[Adeel Asghar]

IMO we should only port bug fixes to 1.12 and not the enhancements.

[Martin Sjölund]

@Adeel: If you want to make the clutter less apparent: create a separate directory to hold the files (files under $TEMP/ModelName/ModelName.*); makes it easier to cleanup the files when exiting OMEdit if you want such an option. Especially if the C++ runtime is used since it generates files with a different pattern (like OMCppModelName).

[massimo ceraolo]

Consider also #3788 where a similar discussion took place.

There I proposed four directories. Naturally if the user selects the new option Delete intermediate compilation files
the first of those four folders is not needed (could be automatically deleted when this option is selected and active).

I agree to comments 6 in #3788: this new option should be active by default

[Francesco Casella]

Replying to ceraolo:

Consider also #3788 where a similar discussion took place.

There I proposed four directories. Naturally if the user selects the new option Delete intermediate compilation files
the first of those four folders is not needed (could be automatically deleted when this option is selected and active).

Given that this is also a matter of taste, I prefer @ceraolo's option, i.e. to have four sub-directories in the working directory, rather than one for each model. If you run 10 simulations there will be 10 files for the results, which is not at all cluttered, as they will nicely show up in alphabetical order within any file browser, or in time of generation order, depending on the user's choice.

So, I'd propose four subdirectories of OMEdit's working directory:

• DataFiles
• IntermediateFiles (will be empty by default)
• SimulationResults
• SimulationExecutables

Would that be ok?

[Adeel Asghar]

That is not ok. In order to get such a directories structure OMEdit needs to do post processing on the files i.e., moving the files to the appropriate folders and of course such operations will depend on the size of the files.

Martin's suggestion is more easier and efficient. OMEdit just need to tell OMC where to dump the files i.e., calling cd() (change directory) and it is always very fast operation.

[Francesco Casella]

Sounds good to me, I have no strong opinions here.

As long as the bulk of useless output files is deleted (no reason to keep them except for very sophisticated debugging activity or for didactical purposes on simple examples), there is really not much difference.

[Martin Sjölund]

cd() is not needed. Just use fileNamePrefix=.... Setting the directory for result-files can be done with the appropriate flag to simulation executables (and could be stored in a separate location if desired).

[massimo ceraolo]

Replying to adeas31:

That is not ok. In order to get such a directories structure OMEdit needs to do post processing on the files i.e., moving the files to the appropriate folders and of course such operations will depend on the size of the files.

I don't know how OMC creates names for files, but I never thought of moving around files.
I just thought that there must be some way to tell OMC where to write its files.

If it is not, it I were in our position I would at least think on how to implement this in the future. For the file locations used, when files might interest the final user, should not create constraints to the program's user interface.

I'm not a professional programmer, so I cannot teach anyone. However, AFAIK, separating inputs, from programs, from outputs is a somewhat golden rule never to be violated in IT.

[Adeel Asghar]

For each model a separate directory is created and files are dumped in it bc6e18/OMEdit.

[Francesco Casella]

Works as required in 1.13.0

[Francesco Casella]

If you turn on profiling=blocks+html, some of the .svg files are generated in the working directory and not in the subdirectory with the model name

[Adeel Asghar]

Its already fixed few days when I fixed #4583.