Make it possible to actually inspect real-life debug logs in OMEdit

[Francesco Casella]

The models most likely to need debugging are non-trivial ones with significant size. The experience with OMEdit in this case should be improved, because it is currently unacceptable IMHO.

As a test case, please consider

ScalableTestSuite.Electrical.DistributionSystemDC.ScaledExperiment.DistributionSystemModelica_N_10_M_10

Note that this is not really a large model, it is just not a toy-sized one.

Turn on the LOG_LS_V flag and run the simulation.

The first problem is that sometimes the log messages are larger than the window, which unfortunately has no horizontal scroll bar, which prevents looking at the full log message. Please add a horizontal scrollbar to this window.

Now, assume I want to do some analysis on the matrix A reported by the log, using Matlab, e.g. to figure out why it is singular. I expand matrix A, select the rows of the matrix and hit Ctrl-C. The hourglass shows up, and it takes about one minute before I regain control of the whole thing. Now, this is really annoying. For larger matrices, it just takes forever, and the only way out is to send a SIGTERM signal to the process and restart the whole thing over. Please make sure this operation doesn't take more than a few seconds in the worst case.

Then, I try to paste these rows in Matlab, but a lot of extra stuff like LOG_LS_V | info | <p> and </p> shows up. Ok, I can paste this in a text editor and use global find and replace to get rid of it. However, it would be nicer to have WYSIWYG here and only get the numbers, not the extra stuff.

I now copy the cleaned-up numbers in Matlab, type A=[ and paste the stuff, then type ] and hit return. I get the following error message:

Error: The input character is not valid in MATLAB statements or expressions.

It took me a while to figure out why, including using a hex editor. The bottom line is, all the spaces there are not the standard ASCII 32 characters, but rather more exotic ASCII 160, i.e., non-breaking space. Which Matlab doesn't like. Could we please just get plain spaces here when we copy to the clipboard?

Every now and then I would then eventually prefer to just load the whole log file in a good old text editor and work from there (e.g. to perform search operations). I tried to figure out where the corresponding .log file is in the working directory, but failed miserably. There's plenty of totally useless files (see #2909 on this topic), but I can't find that. Please add some way to easily access the textual log file from this window, e.g. starting from the context menu or adding a button to the window, or by adding a link, so that one can open it easily with his/her (serious) text editor of choice.

Last, but not least, when the log file gets really long, this windows basically hangs up, it does not allow to press the cancel button (which does not respond to user input) and the whole thing gets stuck, which is really annoying. You can test this with the N_20_M_20 model, for example. I would suggest to put an upper limit to the size of the log file that is represented here (e.g., 1 MB, or something similar) and then put a pointer to the textual .log file at the end if the size exceeds the limit.

If you get 100+ MB log (which is not unheard of if you turn on verbose logging on iterative solvers) that's the only sensible way to operate, this window simply cannot handle such large amounts of data. Killing the process with the Task Manager and restarting the analysis from scratch shouldn't be the only way out of this situation.

Thanks!

[massimo ceraolo]

regarding the working directory, there is also #3788 that, for me, is very important and has high priority.

[Francesco Casella]

I just discussed this with @andrea.bartolini. He just ran a simulation of a hydraulic circuit with digital control with LOG_NLS_V turned on to understand what's going wrong at some point. As the digital controller samples very often, the log file is huge. Very soon, the hard disk started swapping and he had a hard time stopping the process before it completely hanged the OS, forcing a machine shutdown.

I don't think this is acceptable, software may crash, but it shouldn't lead to situations where the entire machine is doomed and should be rebooted, only because of a log flag. Note that we are not talking about fancy experimental models of very large systems, where you can run out of memory anyway; you may end up in this situation with run-of-the-mill models that take many simulation steps and involve algebraic loops.

In fact, there is a very likely scenario, which happened to me many times. A complex model fails initialization. You turn on LOG_NLS_V to figure out why. You catch the bug and solve it. Then you re-run the simulation, and when you realize that you should have turned off the log flag, the machine is already swapping like mad and it's basically lost. Shutdown, reboot, and some not-so-nice thoughts :)

My recommendation: the log window in OMEdit should display at most X MB of data (the default could be set at 10 and be user-manageable). If the size exceeeds that, it would be nice to display the last X MB of data (as in Linux's tail) and eventually point the end user to the log file on mass storage with an indication of its size, so you open it with the appropriate tool (can't use Notepad for 1.5 GB logs). An interim solution could be that if X MB are exceeded, there is no tail display but only an indication to look at the file. This is not ideal, but at least it prevents dooming the entire machine.

BTW, if I run the simulation in OMEdit, there is no log file at all saved to disk. In many cases it is convenient to inspect the log in the log window, but it should also be possible to have a good old log file on disk to be used for further post-processing.

The attached example demonstrates a situation where the log output is barely manageable in OMEdit. If you change stopTime to 1 or 10 seconds, the need of this kind of handling becomes apparent.

[Francesco Casella]

Please implement at least a basic fix to this issue: if the log grows over a certain size stop displaying it to the OMEdit window and start dumping to a plain old text file on disk, which should be then referenced on the OMEdit window.

[Francesco Casella]

Rescheduled to 1.14.0

[Francesco Casella]

@adeas31, this issue should really be taken care of. Some of the features mentioned in the ticket would be nice to have but can wait; however, we need to put in place some solutions to avoid OMEdit (or possibly the whole OS) to hang.

Can you take care of this before the 1.14.0 release?

[Adeel Asghar]

This is now done in 3cd9411/OpenModelica.

You can set the display limit via Tools->Options->Simulation->Output->Display Limit. A message with a link to the simulation log file is shown once the display limit is reached. I have also added a button in the simulation window to open the log file.

I have fixed copying of text as well so it doesn't take forever to copy. Also it is copied as a plain text now so should work fine with other tools like Matlab etc.

[Francesco Casella]

Thanks, this is really appreciated!