I’ve encountered an issue with my validator node when deploying it in a sentry node architecture on the Cosmos Hub Mainnet using Gaia release 14.1.0. The specific problem is that while the full nodes work seamlessly when launched individually, the validator node experiences halting with timeout errors when attempting to connect in a sentry node setup (consisting of 2 sentry nodes and 1 validator).
I would appreciate any insights or guidance from the community regarding this issue. The full nodes and sentry nodes seem to be functioning correctly in isolation, but the validator node is facing challenges in this particular architecture. Has anyone experienced a similar situation or could provide recommendations for troubleshooting and resolving this timeout error?
Thank you for your assistance!
Please find below Logs of validator for your reference
7:26AM INF Replay: Timeout dur=966.940586 height=18869094 module=consensus round=0 step=1
Jan 25 07:26:17 ip-101.0.1.1 gaiad[20827]: 7:26AM INF Replay: New Step height=18869094 module=consensus round=0 step=RoundStepPropose
Jan 25 07:26:17 ip-101.0.1.1 gaiad[20827]: 7:26AM INF Replay: Timeout dur=3000 height=18869094 module=consensus round=0 step=3
Jan 25 07:26:17 ip-101.0.1.1 gaiad[20827]: 7:26AM INF Replay: New Step height=18869094 module=consensus round=0 step=RoundStepPrevote
Jan 25 07:26:17 ip-101.0.1.1 gaiad[20827]: 7:26AM INF Replay: New Step height=18869094 module=consensus round=0 step=RoundStepPropose
Jan 25 07:26:17 ip-101.0.1.1 gaiad[20827]: 7:26AM INF Replay: New Step height=18869094 module=consensus round=0 step=RoundStepPrevote
Jan 25 07:26:17 ip-101.0.1.1 gaiad[20827]: 7:26AM INF Replay: Timeout dur=3000 height=18869094 module=consensus round=0 step=3
Jan 25 07:26:17 ip-101.0.1.1 gaiad[20827]: 7:26AM INF Replay: New Step height=18869094 module=consensus round=0 step=RoundStepPropose
Jan 25 07:26:17 ip-101.0.1.1 gaiad[20827]: 7:26AM INF Replay: New Step height=18869094 module=consensus round=0 step=RoundStepPrevote
Jan 25 07:26:17 ip-101.0.1.1 gaiad[20827]: 7:26AM INF Replay: Timeout dur=3000 height=18869094 module=consensus round=0 step=3
Jan 25 07:26:17 ip-101.0.1.1 gaiad[20827]: 7:26AM INF Replay: Done module=consensus
Jan 25 07:26:19 ip-101.0.1.1 gaiad[20827]: 7:26AM INF service start impl=“Peer{MConn{110.1.4.253:26656} f649a3aaab5b65dc000cf98ca6e3ddd0febout}” module=p2p msg={} peer={“id”:“f649a3aaab5b65dc000cf98ca6e3ddd0febout”,“ip”:“110.1.4.253”,“port”:26656}
Jan 25 07:26:19 ip-101.0.1.1 gaiad[20827]: 7:26AM INF service start impl=MConn{110.1.4.253:26656} module=p2p msg={} peer={“id”:“f649a3aaab5b65dc000cf98ca6e3ddd0febout”,“ip”:“110.1.4.253”,“port”:26656}
Jan 25 07:26:20 ip-101.0.1.1 gaiad[20827]: 7:26AM INF Timed out dur=3000 height=18869094 module=consensus round=0 step=3