Skip to content

Make FailoverClient(sentinel mode) more robust #1655

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Feb 10, 2021
Merged

Make FailoverClient(sentinel mode) more robust #1655

merged 19 commits into from
Feb 10, 2021

Conversation

wangrzneu
Copy link
Contributor

@wangrzneu wangrzneu commented Feb 7, 2021

Make FailoverClient(sentinel mode) more robust

  1. add failover option AllowDisconnectedSlaves
  2. shuffle the sentinel addresses
  3. query other sentinels when get empty slave addresses

sentinel.go Outdated
var addresses []string
var err error

for _, allowDisconnected := range []bool{false, true} {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we simplify this? It is getting late here and I am having troubles trying to understand what this does...

If it is not possible - please add a comment explaining what is happening.

sentinel.go Outdated
@@ -553,9 +591,9 @@ func (c *sentinelFailover) getSlaveAddrs(ctx context.Context, sentinel *Sentinel
return parseSlaveAddrs(addrs)
}

func parseSlaveAddrs(addrs []interface{}) []string {
func parseSlaveAddrs(addrs []interface{}, allowedFlags ...string) []string {
Copy link
Collaborator

@vmihailenco vmihailenco Feb 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I read this code correctly allowedFlags should be replaced with keepDisconnected bool.

@@ -576,8 +614,10 @@ func parseSlaveAddrs(addrs []interface{}) []string {
}

for _, flag := range flags {
switch flag {
case "s_down", "o_down", "disconnected":
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please keep this switch but add a separate clause like:

case "disconnected":
    if !keepDisconnected {
        isDown = true
    }

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fix it.

@@ -509,8 +529,20 @@ func (c *sentinelFailover) slaveAddrs(ctx context.Context) ([]string, error) {
return addrs, nil
}
_ = c.closeSentinel()
} else {
// shuffle the sentinelAddrs to make the load more balanced
rand.Shuffle(len(c.sentinelAddrs), func(i, j int) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this code really be in else clause?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this code really be in else clause?

I add QuerySentinelRandomly option to decide whether to shuffle.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure this is an essential option so I probably going to remove it. Let me know if you think it is essential.

I guess your original code was correct and we only want to shuffle this list once.

sentinel.go Outdated
@@ -36,6 +36,10 @@ type FailoverOptions struct {
// Route all commands to slave read-only nodes.
SlaveOnly bool

// Allow to access slaves disconnected with master when cannot get connected slaves
// Now, this option only works in RandomSlaveAddr function.
AllowDisconnectedSlaves bool
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Allow is a bit vague here. I would rename to UseDisconnectedSlaves.

Also perhaps disconnected slaves should be used only when there are no healthy slaves?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is renamed to UseDisconnectedSlaves.

Refine code

See merge request gray-gateway/go-redis!6
checkout unrelated changes

See merge request gray-gateway/go-redis!8
@wangrzneu
Copy link
Contributor Author

@vmihailenco PTAL

@vmihailenco
Copy link
Collaborator

Looks good. I am going to tweak code a bit myself but but nothing big.

@vmihailenco vmihailenco merged commit 8b19c31 into redis:master Feb 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants