Performance issues with medium-sized linear systems

[Francesco Casella]

Please consider the performance of the following models

ScalableTestSuite.Electrical.DistributionSystemAC.ScaledExperiments.DistributionSystemLinear_N_10_M_10
ScalableTestSuite.Electrical.DistributionSystemAC.ScaledExperiments.DistributionSystemLinear_N_14_M_14
ScalableTestSuite.Electrical.DistributionSystemAC.ScaledExperiments.DistributionSystemLinear_N_20_M_20
ScalableTestSuite.Electrical.DistributionSystemAC.ScaledExperiments.DistributionSystemLinear_N_28_M_28

reported ​here.

N	M	Time
10	10	5.8
14	14	1961
20	20	FAIL
28	28	38

With the current default settings --maxSizeLinearTearing = 4000 and -lssMinSize=4001, the first three are solved using tearing and the dense solver, while the fourth is solved without tearing and with the sparse solver. The diagnostic messages of the N = 14 and N = 20 cases report multiple messages such as

warning | The default linear solver fails, the fallback solver with total pivoting
is started at time 0.004864. That might raise performance issues,
for more information use -lv LOG_LS.

The sparse solver without tearing obviously performs much better. I'm not sure if the problem with the other combination is that the dense solver is less efficient or that the tearing is questionable and leads to singularities, or both.

Please investigate. If the tearing is bad, we should fix it, otherwise we may need to revise the default values of those two flags. I think it would be better to use defaults that optimize the performance of this batch of test cases, rather then using the arbitrary value 4000 set so far. We may revise the default later when we have more diverse test cases.

[Patrick Täuber]

I did some investigations. The tearing is ok, I think. The problem is that the systems are still very sparse after tearing.
Some tearing heuristics lead to tearing sets, that cannot be solved with LAPACK because the residuals remain too big. So the "default linear solver fails". But this is not really the problem, because using other tearing sets, which can be solved with LAPACK, the simulation time is similar.
Furthermore, it does not take much time to solve the linear systems, but the same system is solved over and over again, because the solver is repeating the step.
With a sparse solver the simulation is faster. Without tearing and a sparse solver the simulation is a lot faster (as shown in the table).

I did some testing with the ScalableTestsuite and came to the result that the thresholds --maxSizeLinearTearing=200 and -lssMinSize=201, would improve the situation a lot. Even relatively small sparse systems benefit from the sparse solvers. Thus, I suggest these as a new default. And then let's see what impact it has on other libraries.

[Francesco Casella]

Replying to ptaeuber:

Thanks for the analysis, I agree on setting the default limit at 200.

We also need to deal with the nonlinear solver. In the case of nonlinear systems, there is another reason to use tearing besides exploiting sparsity, that is, reducing the number of variables that need a good initialization (i.e., good start attributes). For thermo-hydraulic models the difference can be massive, like reducing the number of start attributes by one order of magnitude or more. So, the criteria there to switch on and off the tearing are a bit different.

BTW, we should probably have different criteria for initialization and simulation. Initialization favours robusness and few start values, simulation favours performance, usually without too many convergence problems, unless the model has discontinuous behaviour.

That said, given the results obtained here, I wonder if the current limits for disabling tearing and using the sparse nonlinear solver (10000 equations) aren't way too large.

Unfortunately we still don't have test cases in the ScalableTestSuite for large nonlinear systems. I will provide some in the future, but it will be hard to be general enough. With the grid models, we succeded without tearing and with sparse solvers, but given the size of the problem (well above 10000) there was no alternative. What is the optimum for medium-sized systems (say from 200 to 5000 unknowns) is unclear to me, though my gut feeling is that 10000 is too large a default.

[Rüdiger Franke]

Siehe #4751 for the treatment of dense matrices.

[Francesco Casella]

Rescheduled to 1.14.0 after 1.13.0 releasee

[Francesco Casella]

Releasing 1.14.0 which is stable and has many improvements w.r.t. 1.13.2. This issue is rescheduled to 1.16.0

[Francesco Casella]

Retargeted to 1.17.0 after 1.16.0 release

[Francesco Casella]

Retargeted to 1.18.0 because of 1.17.0 timed release.

[Francesco Casella]

Ticket retargeted after milestone closed