Deactivate synchronous replication for cluster consisting of 2 nodes in case the slave fails

[ghost]

Hi Sorintlab,
please consider the enhancement:
In 2 nodes cluster it makes sense to deactivate the synchronous replication from the master to slave in case the slave fails (or network is splitted) and restore it when the slave backs online.
Seems it should be the option.

Basic ideas:

1. Though the slave fails the master still contains the correct data and database clients are connected to the master. From their point of view the database is still online.

[sgotti]

@KimSS If i got this right you're asking for a "best effort" synchronous replication. Use sync repl if possible but downgrade to async repl if not possible.

As a note, the current sync repl logic is a strict one because the main use of sync repl is to avoid losing data and transactions when electing a new master. So if someone is going to implement this it should be an option disabled by default.

[ghost]

@sgotti
Yes, correct. I need the synchronous replication will be deactivated in the 2 DB nodes environment if the slave DB disappears.

Would it be possible to implement such a feature?

[sgotti]

@KimSS it should be possible, it requires some changes in the sentinel code and to add/change the related tests. I'll keep it disabled by default and add a cluster spec option (like 'strictSyncRepl') which defaults to true. I cannot provide an estimate on when it'll be implemented, if no one is going to implement it I can do it after other older opened issues are implemented.

[ghost]

@sgotti this is very positive!
Probably it sounds not good, if we disable sync replication in 2 nodes cluster in a moment of master or slave failure we will face the risk of the absent data redundancy. This is not good from the data safety perspective, but from the service perspective it is normal => DB clients will continue their activities.

In out case at this moment infrastructure guys will immediately start working (probably they will be waken up). This reaction might take ~30 minutes. But the applications will continue operate smoothly.

Also the environment (2 nodes) may face another instance of the problem: failure of the master. In this case stolon promotes the slave into master - this is magnificent. But the sync replication is still enabled. So the new master is not master in fact. It is not available for write operations.

[sgotti]

@KimSS I'm not sure if I understand your statements but your proposal will obviously trade consistency for availability (in the CAP theorem). For this reason I'll keep this option disabled by default.

If you want consistency you should use sync repl with at least 3 nodes (and 2 sync repl standbys) to be available with one node loss.

There are other solutions outside stolon if you want consistency with only 2 nodes using a shared storage (and this doesn't have the downside of the pg synchronous replication) like pacemaker with fencing, kubernetes with a stateful set using a single replica and persistent volumes (but you'll need a fence controller if the node is partitioned). See also the FAQ.

[ghost]

@sgotti
No, no, the question is not about consistency. With this option (disabled by default) I'm primarily focused on the Availability. Your arguments are completely correct!
In fact Stolon already provides the required level of availability in 2 nodes cluster (with sync replication enabled). Since the slave is already promoted into master if the previous master fails. But it is true only for read operations. The feature(option) will make the new master completely available, since it will be available for write operations.

[arssher]

Hi,

I actually suspect that just setting minSynchronousStandbys to 0 with maxSynchronousStandbys set to 1 will do what @saranhada (the topic starter, later opened #410) wished. With this config,

• If replica dies, stolon will reconfigure the master, disabling sync replication
• If master dies with live replica, stolon will choose replica as new master and disable sync replication until old master comes back.
• However, if the following happens, with initially A master and B replica
  • A dies
  • B is chosen as new master, sync replication disabled
  • B dies
  • A comes back
    A will not be elected as new master because it will not contain transactions completed on B.

So it should be a safe setup. In short, the rule is 'we will allow single node to work as long as it is proper one, that is, it contains all transactions. ' Availability is traded over consistency here: it is possible that one node is alive, but it won't be elected as master, as shown above. This is still useful because it allows to use synchronous replication with only two copies of data safely, while simultaneous failure of both instances should be rare.

Now, curiously, 0 is forbidden value for minSynchronousStandbys (I changed this and tested the above scenarios; they work). @sgotti is there any particular reason for this, probably I just don't know something?

[rg2011]

Hi, I needed this and since #410 seems to have a larger scope, I decided to submit a smaller PR #841 just for this.

I was not sure of the side effects of allowing minSynchronousStandbys to be 0, so I followed @sgotti's suggestion (adding a StrictSyncRepl cluster parameter).

[rg2011]

After a review of PR #841, removed StrictSyncRepl option in favour of minSynchronousStandbys = 0 as @arssher suggested. Running the tests.