Improve how equations are shown in the declarative debugger when tearing is applied

[casella]

When tearing is applied to systems of equations, the way they are shown in the Equations windows of the declarative debugger is not immediately clear (understatement). Unfortunately, the ​user's guide does not explain how this works at all.

After some investigation through examples, I understood that:

• when you click on the system of equations, only the tearing variables are shown in the Defines window
• the first equations shown when you open the system are the torn equations and when you click on one, the Defines window shows the corresponding torn variable
• then, the residual equations are shown, which are marked with a (residual) indication after the body of the equation. If you click on them, no equation shows up in the "Defines" section
• in some cases, after the residual equations some other equations appear, defining variables such as $pDERLSJac18.dummyVarLSJac18 which seem to be related to analytical Jacobian computations

Eventually, the whole thing makes sense, but it can be extremely puzzling at first sight, particularly since there is no documentation at all on this topic.

My first suggestion is to make this clearer at first sight, by:

• adding some indication that tearing has been applied to the description of the equation that represents the whole system, e.g., nonlinear, size 8, tearing applied
• moving the (residual) marker before the body of the equation, otherwise it is not seen immediately if the equation lenght exceeds the (small) window width. For example, change x + y - z (residual) into (residual) x + y - z = 0. Note the addition of = 0 which, IMHO, makes the whole thing clearer
• adding a (torn) marker before the body of torn equations
• adding a (Jacobian) marker before the equations to compute Jacobian terms showing up after the residual equations
• adding the indication Tearing variables to the list of equations shown in the Define window when clicking on the system of equations, so it is immmediately clear why not all the unknowns of the system are shown
• showing some meaningful message, e.g., Part of an implicit system of equations in the Define window when clicking on a residual equations, so that one does not have the impression that nothing is shown because of a bug

My second suggestion is to add some documentation of this thing to the user's guide. I can volunteer to write this once the GUI is updated.

[adeas31]

I think first we need to add this information to the json/xml file.

[lochel]

Replying to adeas31:

I think first we need to add this information to the json/xml file.

I am fully occupied at the moment and I am afraid that Martin isn't available either. How urgent is this and can it be done by someone else?

[casella]

I have no strong opinions on how to implement this, but my understanding is that the current json/xml files already contain the required information. It's just not encoded in a from immediately readable for humans, but humans are not expected to read json/xml files either.

I would suggest to implement these changes as a post-processing step in OMEdit, at least for the time being. At the end of the day, all that's requird is just some simple string manipulation operations, such as adding some characters to torn variables names, removing the trailing (residual) string and adding it as a leading prefix, add a = 0 postfix to residuals, etc. It shouldn't take much time to implement this in OMEdit.

Also, by doing this we ensure that we are not breaking third party code that may already be using the json/xml files in its current form for some purpose.

My understanding is that doing this would boost the usability (and thus the use) of the debugger substantially. Today pepole probably try it out (by clicking on the debug more link), but then quickly get frustrated by not understanding the whole thing actually means, and stop using it. The fact that nobody complained about the broken Transformation windows for such a long time points to this. My impression whenever I tried to use the debugger was of some half-finished thing, which is a shame, as there is actually a lot of very good conceptual work behind it, which is also only available in OMC and not in other commercial tools.

So, although this is not really a top priority task, I think that the usefulness/effort ratio of doing this right away is very favourable.

My two cents :)

[adeas31]

Replying to casella:

I have no strong opinions on how to implement this, but my understanding is that the current json/xml files already contain the required information. It's just not encoded in a from immediately readable for humans, but humans are not expected to read json/xml files either.

I would suggest to implement these changes as a post-processing step in OMEdit, at least for the time being. At the end of the day, all that's requird is just some simple string manipulation operations, such as adding some characters to torn variables names, removing the trailing (residual) string and adding it as a leading prefix, add a = 0 postfix to residuals, etc. It shouldn't take much time to implement this in OMEdit.

Also, by doing this we ensure that we are not breaking third party code that may already be using the json/xml files in its current form for some purpose.

My understanding is that doing this would boost the usability (and thus the use) of the debugger substantially. Today pepole probably try it out (by clicking on the debug more link), but then quickly get frustrated by not understanding the whole thing actually means, and stop using it. The fact that nobody complained about the broken Transformation windows for such a long time points to this. My impression whenever I tried to use the debugger was of some half-finished thing, which is a shame, as there is actually a lot of very good conceptual work behind it, which is also only available in OMC and not in other commercial tools.

So, although this is not really a top priority task, I think that the usefulness/effort ratio of doing this right away is very favourable.

My two cents :)

OK. I will take a look at it tomorrow.
Do you have a test model for it?

[casella]

Test case

[casella]

You can have a look at the attached test case.

BTW, there is also another problem with the Equation Operation window, but I'll report it in a separat ticket.

[casella]

Any chance of also getting this fixed in 1.12.0 beta1?

[adeas31]

• moving the (residual) marker before the body of the equation, otherwise it is not seen immediately if the equation lenght exceeds the (small) window width. For example, change x + y - z (residual) into (residual) x + y - z = 0. Note the addition of = 0 which, IMHO, makes the whole thing clearer
• showing some meaningful message, e.g., Part of an implicit system of equations in the Define window when clicking on a residual equations, so that one does not have the impression that nothing is shown because of a bug

Done in 375731a/OMEdit.

• adding some indication that tearing has been applied to the description of the equation that represents the whole system, e.g., nonlinear, size 8, tearing applied
• adding a (torn) marker before the body of torn equations
• adding a (Jacobian) marker before the equations to compute Jacobian terms showing up after the residual equations
• adding the indication Tearing variables to the list of equations shown in the Define window when clicking on the system of equations, so it is immmediately clear why not all the unknowns of the system are shown

There is no information in the model_info.json file from which I can deduce that tearing is applied or a particular equation is torn or jacobian equation.

[casella]

Webmeeting scheduled Jun 28 11:00 on the topic

[casella]

Plan:

• Patrick adds the required information to the json file, making sure that the equations to compute the jacobian of a nonlinear system show up as children of the nonlinear system (currently this does not happen, see the attached test case, equation system 22)
• Adeel implements the rendering in the OMEdit declarative debugger

[ptaeuber]

There is one more point that should be changed: The system size is a quite confusing information. If tearing is applied it does not show the real system size but the original one. If this is intended, e.g. to show how many variables are solved in that system, it is still not correct for systems with complex equations.
In the TestFalseIteration model from #4046 there is a system with one complex equation and one residual equation. So the actual size of the system is 1. On the other hand there are 9 variables solved in that system. Nevertheless, the transformational debugger shows a system size of 2, which is not correct.

[casella]

Replying to ptaeuber:

Thanks Patrick for pointing this out, I left this somewhat implicit in my original description.

There is one more point that should be changed: The system size is a quite confusing information. If tearing is applied it does not show the real system size but the original one.

I think this is the right thing to do, because eventually all the variables of the system are solved for; personally, I find reporting only the number of tearing variables as the size also quite confusing.

My proposal, if tearing is applied, is to report both the (full) system size and the number of tearing variables, e.g. nonlinear, size: 8, tearing variables: 2

If this is intended, e.g. to show how many variables are solved in that system, it is still not correct for systems with complex equations.
In the TestFalseIteration model from #4046 there is a system with one complex equation and one residual equation. So the actual size of the system is 1. On the other hand there are 9 variables solved in that system. Nevertheless, the transformational debugger shows a system size of 2, which is not correct.

Given the example, I guess by 'complex equations' you mean equations involving functions with multiple outputs, not equations involving complex numbers.

This is an interesting corner case, because there is no way to rewrite this equation as a set of scalar equations. On the other hand, the number of unknowns is definitely obtained by counting all the scalarized outputs. So, maybe we could solve this issue by mentioning # of unknowns instead of size. What do you think?

[ptaeuber]

Replying to casella:

Given the example, I guess by 'complex equations' you mean equations involving functions with multiple outputs, not equations involving complex numbers.

Yes, for some reason they are called complex equations in the compiler...

This is an interesting corner case, because there is no way to rewrite this equation as a set of scalar equations. On the other hand, the number of unknowns is definitely obtained by counting all the scalarized outputs. So, maybe we could solve this issue by mentioning # of unknowns instead of size. What do you think?

I think this is the best way to do it. We could also think about # of iteration variables (rather than tearing variables), because in case of other systems, e.g. one with "complex equations", it is not a classic torn sytem.

Could be like this for torn systems then:
nonlinear (torn), unknowns: 8, iteration variables: 2

[casella]

Replying to ptaeuber:

Could be like this for torn systems then:
nonlinear (torn), unknowns: 8, iteration variables: 2

Excellent!

For systems that are not subject to tearing, we should still mention unknowns rather than size, to maintain consistency.

[ptaeuber]

Replying to casella:

• ...making sure that the equations to compute the jacobian of a nonlinear system show up as children of the nonlinear system (currently this does not happen, see the attached test case, equation system 22)

Done in ​f30ae8a.

[ptaeuber]

All the remaining needed information are added in ​90441d8.

[adeas31]

And I have also updated OMEdit accordingly 3082898/OMEdit.