Issues with start values, aliases and linear dependencies in the backend

[casella]

We are finalizing the PowerGrids library with RTE, the plan is to go open source next year. We are currently testing the classical example of the IEEE 14 bus system. We are running into a very serious problem, which is currently partially solved by homotopy, but needs to be addressed very urgently.

As you know from our recently submitted paper, convergence of Newton-Raphson's algorithm requires that the variables influencing the Jacobian get a good initial value, while for the other ones one zero may be used safely.

We designed the library carefully so that all the variables appearing non-linearly in the system get a good start value based on the values of active and reactive power, voltage magnitude and voltage angle at each component port, which are supplied as parameters and are computed by previously solving a load-flow problem, either in Modelica with simplified static load flow models, or by means of some external specialized software.

The idea is that this should be enough guarantee fast and painless convergence to the solution. Unfortunately, the solver went crazy and eventually converged to a completely wrong solution. Homotopy allows to get the required solution, but the basic question is, why should the direct solver fail if the relevant start values have been selected?

It turns out, they actually haven't.

If you run the PowerGrids.Examples.IEEE14bus.IEEE14busStaticNetwork model with LOG_NLS_V, the report of the first iteration of the big system to be solved at initialization is

Iteration: 1
newton step
variables
[ 1]             GEN6.terminal.i.im  =        106.94919         step =        106.94919         old =                0
[ 2]            Load6.terminal.i.im  =       -339.33637         step =       -339.33637         old =                0
[ 3]             bus6.terminal.i.re  =                0         step =               -0         old =                0
[ 4]             GEN6.terminal.i.re  =        16.904257         step =        16.904257         old =                0
[ 5]             bus9.terminal.i.im  =                0         step =               -0         old =                0
[ 6]           Cbank9.terminal.i.im  =        724.41317         step =        1803.9553         old =       -1079.5421
[ 7]             bus9.terminal.i.re  =                0         step =               -0         old =                0
[ 8]           Cbank9.terminal.i.re  =        196.86999         step =       -3845.1486         old =        4042.0186
[ 9]                    Tgen3.vA.re  =        10655.283         step =        -608.4801         old =        11263.763
[10]                    Tgen3.vz.re  =        132.06637         step =        132.06637         old =                0
[11]             bus3.terminal.i.im  =                0         step =               -0         old =                0
[12]             bus3.terminal.i.re  =   5.2041704e-014         step =   5.2041704e-014         old =                0
[13]                    Tgen3.iA.re  =       -204.54852         step =       -48.305226         old =       -156.24329
[14]                    Tgen3.iB.im  =        457.69766         step =         258.9994         old =        198.69826
[15]                    Tgen1.vA.im  =        212.95535         step =        212.95535         old =                0
[16]                    Tgen1.vz.re  =        119.06993         step =        119.06993         old =                0
[17]                    Tgen8.vA.im  =       -2304.5147         step =        99.470998         old =       -2403.9857
[18]                    Tgen8.vz.im  =       -1.2337234         step =       -1.2337234         old =                0
[19]             bus2.terminal.i.re  =                0         step =               -0         old =                0
[20]             bus2.terminal.i.im  =                0         step =               -0         old =                0
[21]                    Tgen2.vA.im  =       -1073.4827         step =        129.24559         old =       -1202.7283
[22]                   Tgen2.vAt.re  =        37431.923         step =        37431.923         old =                0
[23]             bus4.terminal.i.re  =                0         step =               -0         old =                0
[24]             bus4.terminal.i.im  =                0         step =               -0         old =                0
[25]             bus5.terminal.i.im  =                0         step =               -0         old =                0
[26]             bus5.terminal.i.re  =                0         step =               -0         old =                0
[27]                    Tgen1.iA.re  =        4477.2403         step =       -1112.7023         old =        5589.9426
[28]                    Tgen1.iB.im  =        302.86441         step =        439.14628         old =       -136.28187
[29]                   L1to2.vAt.re  =        39244.512         step =       -592.65653         old =        39837.169
[30]                   L1to2.vAt.im  =  -1.7967683e-012         step =  -1.7967683e-012         old =                0
[31]                   L3to4.vAt.re  =        36628.659         step =       -2231.3127         old =        38859.971
[32]                   L3to4.vAt.im  =       -8631.0278         step =        138.38691         old =       -8769.4147
[33]                    T5to6.vA.im  =       -5605.7303         step =        481.76865         old =        -6087.499
[34]                    T5to6.vA.re  =         36493.37         step =       -2875.9371         old =        39369.307
[35]                   L2to3.vAt.im  =       -3188.0453         step =        269.79843         old =       -3457.8437
[36]                   L2to3.vAt.re  =        37511.036         step =        -2175.779         old =        39686.815
[37]                   L4to5.vAt.re  =        36234.932         step =       -2957.7933         old =        39192.725
[38]                    T4to7.vz.im  =        411.59155         step =        411.59155         old =                0
[39]            bus13.terminal.i.im  =                0         step =               -0         old =                0
[40]            bus13.terminal.i.re  =                0         step =               -0         old =                0
[41]           Load11.terminal.i.re  =         110.9733         step =        -7978.482         old =        8089.4553
[42]           Load11.terminal.i.im  =         -91.3697         step =        2047.3252         old =       -2138.6949
[43]            bus12.terminal.i.re  =                0         step =               -0         old =                0
[44]            bus12.terminal.i.im  =                0         step =               -0         old =                0
[45]            bus14.terminal.i.re  =                0         step =               -0         old =                0
[46]            bus14.terminal.i.im  =                0         step =               -0         old =                0
[47]            bus10.terminal.i.re  =                0         step =               -0         old =                0
[48]            bus10.terminal.i.im  =                0         step =               -0         old =                0
[49]                    Tgen8.iA.re  =       -12.416707         step =        115.98136         old =       -128.39806
[50]                    Tgen8.iB.im  =        119.41884         step =       -526.77567         old =        646.19451
[51]                    T4to7.iB.re  =       -1033.5054         step =        3036.7185         old =       -4070.2239
[52]                    T4to7.iB.im  =         179.3075         step =       -788.47519         old =        967.78268
[53]                    Tgen8.vB.im  =       -1765.5617         step =        77.483667         old =       -1843.0453
[54]                    Tgen8.vB.re  =        7363.5488         step =       -387.78558         old =        7751.3343
[55]                 L12to13.vAt.re  =        7210.8136         step =       -481.77997         old =        7692.5936
[56]                 L12to13.vAt.im  =       -1972.3931         step =        102.21634         old =       -2074.6094
[57]                  L13to14.vB.re  =        7061.8655         step =       -595.06812         old =        7656.9336
[58]                  L13to14.vB.im  =       -2075.2896         step =        127.29235         old =       -2202.5819
[59]                   L7to8.vAt.im  =       -1760.1287         step =        82.916676         old =       -1843.0453
[60]                   L7to8.vAt.re  =        7323.4885         step =       -427.84582         old =        7751.3343
[61]                 L10to11.vAt.im  =       -1983.4854         step =        93.851808         old =       -2077.3372
[62]                  L9to10.vAt.im  =       -1973.2581         step =        82.621951         old =         -2055.88
[63]                  L9to10.vAt.re  =        7260.9044         step =       -436.71579         old =        7697.6202
[64]                 L10to11.vAt.re  =        7214.1145         step =       -477.74285         old =        7691.8574
[65]                  L10to11.vB.re  =        7248.8912         step =       -453.88815         old =        7702.7793
[66]                  L10to11.vB.im  =       -1944.8966         step =        91.568737         old =       -2036.4653
[67]                 L13to14.vAt.re  =        7176.8892         step =       -512.72849         old =        7689.6177
[68]                 L13to14.vAt.im  =       -1978.1204         step =         107.4921         old =       -2085.6125
[69]                  L6to11.vAt.re  =        7341.9428         step =       -380.68415         old =         7722.627
[70]                  L6to11.vAt.im  =       -1880.2338         step =        79.621435         old =       -1959.8552
[71]                  GEN2.GEN.idPu  =       -128464.74         step =        -128464.7         old =     -0.038382685
[72]                  GEN2.GEN.iqPu  =        157913.07         step =         157913.1         old =     -0.031224884
[73]                  GEN1.GEN.idPu  =       -6102670.6         step =       -6102670.6         old =     -0.050185524
[74]                  GEN1.GEN.iqPu  =        1753180.3         step =        1753180.5         old =      -0.17469147
[75]                  GEN6.GEN.idPu  =     -0.032213372         step =       0.14817089         old =      -0.18038426
[76]                  GEN6.GEN.iqPu  =    -0.0021349681         step =    -0.0020216829         old =   -0.00011328523
[77]                  GEN8.GEN.iqPu  =   -0.00040584505         step =   -0.00039190632         old =  -1.3938727e-005
[78]                  GEN8.GEN.idPu  =     -0.011465974         step =      0.051954414         old =     -0.063420388
[79]                  GEN3.GEN.iqPu  =   -0.00024220295         step =   -0.00024154975         old =  -6.5319823e-007
[80]                  GEN3.GEN.idPu  =     -0.033283719         step =     -0.018550966         old =     -0.014732753
[81]             GEN2.GEN.port.v.im  =       -1073.4827         step =        187.41755         old =       -1260.9002
[82]             GEN2.GEN.port.v.re  =        13019.937         step =       -1451.8315         old =        14471.768
[83]             GEN1.GEN.port.v.im  =        212.95535         step =        212.95535         old =                0
[84]                    Cbank9.v.re  =        7260.9044         step =       -436.71579         old =        7697.6202
[85]                    Cbank9.v.im  =       -1973.2581         step =        82.621951         old =         -2055.88
[86]             GEN6.GEN.port.v.re  =        7341.9428         step =       -903.75222         old =        8245.6951
[87]             GEN6.GEN.port.v.im  =       -1880.2338         step =        212.36612         old =       -2092.5999
[88]             GEN8.GEN.port.v.re  =        9616.4901         step =       -1418.5454         old =        11035.035
[89]             GEN8.GEN.port.v.im  =       -2304.5147         step =        319.30071         old =       -2623.8154
[90]             GEN3.GEN.port.v.im  =       -2506.7059         step =        64.257695         old =       -2570.9636
[91]             GEN3.GEN.port.v.re  =        10655.283         step =       -737.44697         old =         11392.73
[92]                GEN2.AVR.gain.y  =        1.1437764         step =        1.1437764         old =                0
[93]                GEN1.AVR.gain.y  =        1.0984072         step =        1.0984072         old =                0
[94]                GEN6.AVR.gain.y  =       0.97540271         step =       0.97540271         old =                0
[95]                GEN8.AVR.gain.y  =       0.96909303         step =       0.96909303         old =                0
[96]                GEN3.AVR.gain.y  =        1.0414639         step =        1.0414639         old =                0
[97]                Load2.port.v.re  =        37511.036         step =       -3961.6857         old =        41472.722
[98]                Load2.port.v.im  =       -3188.0453         step =         425.4014         old =       -3613.4467
[99]                Load5.port.v.re  =         36493.37         step =       -3706.2301         old =          40199.6
[100]                Load5.port.v.im  =       -5605.7303         step =        610.15312         old =       -6215.8835
[101]                Load9.port.v.re  =        7260.9044         step =       -881.05813         old =        8141.9626
[102]                Load9.port.v.im  =       -1973.2581         step =        201.29688         old =        -2174.555
[103]               Load10.port.v.re  =        7214.1145         step =       -881.34241         old =        8095.4569
[104]               Load10.port.v.im  =       -1983.4854         step =         202.8518         old =       -2186.3372
[105]               Load14.port.v.re  =        7061.8655         step =       -875.93333         old =        7937.7989
[106]               Load14.port.v.im  =       -2075.2896         step =        208.08561         old =       -2283.3752
[107]                Load6.port.v.re  =        7341.9428         step =       -921.26804         old =        8263.2109
[108]                Load6.port.v.im  =       -1880.2338         step =         216.8113         old =       -2097.0451
[109]               Load13.port.v.re  =        7176.8892         step =       -938.10923         old =        8114.9984
[110]               Load13.port.v.im  =       -1978.1204         step =        222.86577         old =       -2200.9862
[111]               Load11.port.v.re  =        7248.8912         step =       -898.13757         old =        8147.0287
[112]               Load11.port.v.im  =       -1944.8966         step =        209.01966         old =       -2153.9162
[113]               Load12.port.v.re  =        7210.8136         step =       -871.42656         old =        8082.2402
[114]               Load12.port.v.im  =       -1972.3931         step =        207.29982         old =       -2179.6929
[115]                Load4.port.v.re  =        36234.932         step =       -3682.3475         old =        39917.279
[116]                Load4.port.v.im  =       -6607.8176         step =        660.66173         old =       -7268.4794
[117]                Load3.port.v.re  =        36628.659         step =       -2619.9124         old =        39248.571
[118]                Load3.port.v.im  =       -8631.0278         step =        226.08106         old =       -8857.1088
[119]                 bus2.port.v.im  =       -3188.0453         step =        269.79843         old =       -3457.8437
[120]                 bus2.port.v.re  =        37511.036         step =        -2175.779         old =        39686.815
[121]                 bus5.port.v.re  =         36493.37         step =       -2875.9371         old =        39369.307
[122]                 bus5.port.v.im  =       -5605.7303         step =        481.76865         old =        -6087.499
[123]                 bus4.port.v.re  =        36234.932         step =       -2957.7933         old =        39192.725
[124]                 bus4.port.v.im  =       -6607.8176         step =        528.72871         old =       -7136.5463
[125]                 GEN3.GEN.theta  =     -0.027330813         step =       0.19466277         old =      -0.22199359
[126]                 bus3.port.v.re  =        36628.659         step =       -2231.3127         old =        38859.971
[127]                 bus3.port.v.im  =       -8631.0278         step =        138.38691         old =       -8769.4147
[128]             Load3.port.iGen.re  =       -702.27764         step =        24.151301         old =       -726.42894
[129]             Load3.port.iGen.im  =        312.02122         step =       -14.123819         old =        326.14504
[130]             Load4.port.iGen.re  =       -373.47377         step =        18.614983         old =       -392.08875
[131]             Load4.port.iGen.im  =        40.037653         step =        1.2101318         old =        38.827521
[132]                 bus9.port.v.im  =       -1973.2581         step =        82.621951         old =         -2055.88
[133]                 bus9.port.v.re  =        7260.9044         step =       -436.71579         old =        7697.6202
[134]                 bus7.port.v.im  =       -1760.1287         step =        82.916676         old =       -1843.0453
[135]                 bus7.port.v.re  =        7323.4885         step =       -427.84582         old =        7751.3343
[136]                 bus8.port.v.im  =       -1765.5617         step =        77.483667         old =       -1843.0453
[137]                 bus8.port.v.re  =        7363.5488         step =       -387.78558         old =        7751.3343
[138]                 GEN8.GEN.theta  =      -0.22212469         step =      0.011531756         old =      -0.23365644
[139]                bus10.port.v.im  =       -1983.4854         step =        93.851808         old =       -2077.3372
[140]                bus10.port.v.re  =        7214.1145         step =       -477.74285         old =        7691.8574
[141]                bus14.port.v.re  =        7061.8655         step =       -595.06812         old =        7656.9336
[142]                bus14.port.v.im  =       -2075.2896         step =        127.29235         old =       -2202.5819
[143]                bus12.port.v.re  =        7210.8136         step =       -481.77997         old =        7692.5936
[144]                bus12.port.v.im  =       -1972.3931         step =        102.21634         old =       -2074.6094
[145]            Load12.port.iGen.re  =       -207.65405         step =         10.27925         old =        -217.9333
[146]            Load12.port.iGen.im  =         112.2076         step =        -12.55496         old =        124.76256
[147]                 GEN6.GEN.theta  =      -0.24444982         step =     0.0047121785         old =        -0.249162
[148]                 bus6.port.v.im  =       -1880.2338         step =        79.621435         old =       -1959.8552
[149]            Load11.port.iGen.im  =        91.369701         step =       -12.851701         old =         104.2214
[150]            Load11.port.iGen.re  =        -110.9733         step =        4.6740752         old =       -115.64738
[151]                bus11.port.v.im  =       -1944.8966         step =        91.568737         old =       -2036.4653
[152]                bus11.port.v.re  =        7248.8912         step =       -453.88815         old =        7702.7793
[153]                bus13.port.v.im  =       -1978.1204         step =         107.4921         old =       -2085.6125
[154]                bus13.port.v.re  =        7176.8892         step =       -512.72849         old =        7689.6177
[155]            Load13.port.iGen.re  =       -446.58549         step =        21.234935         old =       -467.82042
[156]            Load13.port.iGen.im  =        320.46061         step =       -44.665726         old =        365.12633
[157]             Load6.port.iGen.re  =       -340.02524         step =        12.304477         old =       -352.32972
[158]             Load6.port.iGen.im  =        339.33649         step =       -52.623896         old =        391.96038
[159]                 bus6.port.v.re  =        7341.9428         step =       -380.68415         old =         7722.627
[160]            Load14.port.iGen.im  =        322.09652         step =       -38.055119         old =        360.15164
[161]            Load14.port.iGen.re  =       -497.90906         step =         24.18849         old =       -522.09755
[162]            Load10.port.iGen.re  =       -275.10071         step =        10.173073         old =       -285.27378
[163]            Load10.port.iGen.im  =        276.00619         step =       -39.854678         old =        315.86086
[164]             Load9.port.iGen.im  =        821.06724         step =       -114.37473         old =        935.44197
[165]             Load9.port.iGen.re  =       -921.07738         step =        36.819839         old =       -957.89722
[166]             Load5.port.iGen.re  =       -57.000049         step =        2.5437477         old =       -59.543797
[167]             Load5.port.iGen.im  =        20.186497         step =       -2.2876227         old =         22.47412
[168]              bus1.port.iGen.im  =                0         step =                0         old =               -0
[169]                 bus1.port.v.re  =        39244.512         step =       -592.65653         old =        39837.169
[170]                 GEN1.GEN.theta  =         34933993         step =         34933992         old =       0.35390883
[171]              bus1.port.iGen.re  =        204.68197         step =        1041.4215         old =       -836.73952
[172]             Load2.port.iGen.im  =        100.80948         step =        -15.57831         old =        116.38779
[173]             Load2.port.iGen.re  =        -157.4795         step =         6.791659         old =       -164.27116
[174]                 GEN2.GEN.theta  =        4114175.4         step =        4114175.4         old =     -0.010419249

Let's focus on variables 1 and 4 for simplicity. The backend reports

Nonlinear iteration variables with default zero start attribute in NLSJac401. (26)
========================================
...

23: GEN6.terminal.i.im:VARIABLE(flow=true unit = "A" nominal = 10000.0 )  "Imaginary part of complex number"PowerGrids.Examples.IEEE14bus.IEEE14busStaticNetwork, PowerGrids.Examples.IEEE14bus.ControlledGeneratorIEEE, PowerGrids.Interfaces.TerminalAC, PowerGrids.Types.ComplexCurrent type: Real
24: GEN6.terminal.i.re:VARIABLE(flow=true unit = "A" nominal = 10000.0 )  "Real part of complex number"PowerGrids.Examples.IEEE14bus.IEEE14busStaticNetwork, PowerGrids.Examples.IEEE14bus.ControlledGeneratorIEEE, PowerGrids.Interfaces.TerminalAC, PowerGrids.Types.ComplexCurrent type: Real
...
Linear iteration variables with predefined start attributes that are unrelevant in NLSJac401. (243) 
======================================== 
...
39: GEN6.GEN.port.iGen.re:VARIABLE(start = -148.5096000353568 unit = "A" nominal = 3346.9580822587 )  "Real part of complex number"PowerGrids.Examples.IEEE14bus.IEEE14busStaticNetwork, PowerGrids.Examples.IEEE14bus.ControlledGeneratorIEEE, PowerGrids.Electrical.Machines.SynchronousMachine4Windings, PowerGrids.Electrical.BaseClasses.PortAC, PowerGrids.Types.ComplexCurrent type: Real 
40: GEN6.GEN.port.iGen.im:VARIABLE(start = -585.1882455910535 unit = "A" nominal = 3346.9580822587 )  "Imaginary part of complex number"PowerGrids.Examples.IEEE14bus.IEEE14busStaticNetwork, PowerGrids.Examples.IEEE14bus.ControlledGeneratorIEEE, PowerGrids.Electrical.Machines.SynchronousMachine4Windings, PowerGrids.Electrical.BaseClasses.PortAC, PowerGrids.Types.ComplexCurrent type: Real 
...
Info: Only non-linear iteration variables in non-linear eqation systems require start values.
All other start values have no influence on convergence and are ignored. Use "-d=dumpLoops" to show all loops. 
In OMEdit Tools->Options->Simulation->OMCFlags, in OMNotebook call setCommandLineOptions("-d=dumpLoops") 

Now, it turns out GEN6.GEN.port.iGen and GEN6.terminal.i are alias variables, being one the opposite of the other.

So, in the library models we set the right start attribute on GEN6.GEN.port.iGen (in fact, we did it on its alias GEN6.GEN.port.i) and expected this to be propagated automatically to all aliases. This variable happens to appear linearly in the system equations, though its alias GEN6.terminal.i doesn't. Unfortunately, the backend failed to propagate this alias information correctly, so it ignored the carefully selected attribute of GEN6.GEN.port.iGen and instead gave a default zero start attribute to GEN6.terminal.i, causing all kind of convergence problems. The same issue happens with other variables.

The success of the strategy that we described in the Newton initialization paper crucially depends on start values being correctly propagated to all aliases, since setting them all manually is not a feasible task.

Can you please check what's going wrong here and fix it?

Thanks!

[casella]

We did some further analysis, maybe I understood what the problem is. Please wait until I post a new comment.

Thanks!

[casella]

I found what the problem was: the aliases were not actually aliases.

We recently introduced this pattern in a base model

equation
  if initial() and localInit == LocalInitializationOption.PV then
    // During local initialization, P,V is enforced at the connector towards
    // the grid, while Q,phase(V) is enforced at the port towards the component
    // Further two initial equations will be needed (e.g. in the controller)
    // to enforce P,V at the port or achieve actuator saturation if that is not possible
    CM.real(terminal.v*CM.conj(terminal.i)) = port.PStart;
    CM.'abs'(terminal.v) = port.VStart;
    port.Q = port.PStart;
    port.v.re*port.vStart.im = port.v.im*port.vStart.re;
  elseif initial() and localInit == LocalInitializationOption.PQ then
    // During local initialization, P,Q is enforced at the connector towards
    // the grid, while phase(V), phase(I) is enforced at the port towards the component
    // FurthThe initial equation of the specific component that extends
    // this base class will try to enforce P,Q at the port or achieve actuator
    // saturation if that is not possible
    terminal.v*CM.conj(terminal.i) = Complex(port.PStart, port.QStart);
    port.v.re*port.vStart.im = port.v.im*port.vStart.re;
    port.i.re*port.iStart.im = port.i.re*port.iStart.im;
  else
    // In all other cases, the port voltage and current are equal to
    // the connector voltage and current
    port.v = terminal.v;
    port.i = terminal.i;
  end if;

If localInit is not evaluated, then the alias equation is conditional. So, we need to a) evaluate it, so the conditional equations can be when they are never needed, and b) propagate the start values to terminal, to deal with the cases at initialization when terminal.{v,i} and port.{v,i} are not identically equal.

I just wonder, in case localInit=true, if we could separate the "regular equations for initialization" from the "regular equations for simulation", instead of keeping the conditional equation at runtime.