818 has gone up without a forum venue for community dialogue. LOA Labs validator is opening it up here to complement the discussion on Twitter and elsewhere.
Why this is such a fascinating and consequential governance proposition? See @dimiandre twitter post:
"… Prop 818 on
is so much fun Assuming that no one votes for the best of the network but only for their own interests we have: - Validators voting yes to free some space in set and harm a competitor - Validators voting no so that they expect others to do the same in case they mess up something - Validators voting yes/no because they are enemy/friends - Validators voting abstain cause they don’t want to jump on either side and play fake friends with everyone - Delegators not affected voting yes to decrease supply of atom - Delegators affected voting veto for their own interest - Delegators not affected voting no/veto chasing a possible airdrop And many more! Do you see any other conflict of interest possibilities? The usd value of the slashed tokens would be around 2.3M"
Why this needed to go onchain without coming to the Forum first as is normally expected as outlined by Informal:
“…Governance proposals must go through a 2 week voting period before they are passed. The unbonding period, after which delegators and validators can unbond their stake is 3 weeks. This means that if malicious validators participate in an equivocation attack and then immediately unbond their stake, and then the governance proposal is not put up for voting for at least a week, the validators could avoid the 5% slashing penalty. They would still be tombstoned, meaning that they could never validate again with the same keys.”
Make it harder for malicious validators to organize a cartel to double spend by showing different chain states to multiple victims.
Act as a disincentive for ambivalent validators to sign on any chain state they see, for example signing on to multiple states created by a double spending attack (this is called the nothing at stake problem).
On the first point, it’s not possible for validators to double spend without at least 1/3 of the voting power participating. They simply won’t have enough signatures to create a valid alternate block. In this case, it doesn’t appear that the validators were actually attempting to double spend. A real double spending attempt would have required a much larger number of validators to participate, and they would also have had to have a plan to show alternate histories to multiple clients, for example sending the same tokens through multiple bridges or exchanges. There’s an extremely low probability that this is what they were doing.
The second point is a little more subtle, but it doesn’t appear that this is what happened either. The nothing at stake problem is a scenario where multiple forks are proliferating and validators are just signing everything they can get their hands on. This is not the scenario that we saw.
Double sign slashing is not intended to punish validators who make mistakes. This is why slashes are only for 5% of a validator’s stake, even though this means that the economic security of Cosmos chains with this setting is only 5% of the total staked. This is also why there have been many proposals throughout the years in Cosmos and other PoS ecosystems to limit slashing amounts when only a small number of validators double sign (Sunny had a proposal about this, and I think Ethereum 2.0 may have actually implemented this).
So to the question of whether these validators should be slashed:
NO if we are actually worried about an attempted attack having taken place
YES if we want to “prove” that ICS governance gated slashing really works
I feel like I am out of the loop and I am not sure what has happened here so can anyone explain? There doesn’t seem to be a lot of details in the prop. The only detail is to use neutron cli to see the data.
Once you understand how to use any cosmos cli you should be able to get any data from any cosmos chain but I don’t think many people know how to use cli + not everyone has linux to install cli and use it just to check the necessary data for this prop.
So you did a double sign but weren’t slashed? I know ICS is still new but this seems like a bug or missing feature. Currently looking at mintscan and you are still in active set.
As for reason of slashing I agree that this wasn’t intentional and sometimes with rushed cli updates this is bound to happen. Realaty is every time swiching to newer version without a proper vote, time, cosmovisor setup and a planed schedule of release can lead to human error, even though it is considered a soft fork it usually has to be done within limited time. But unfortunately “it is a part of the job”.
Maybe some softer punishment should be enacted instead of tombstone like 0.1% slash. I think tombstone punishment is too hars for something like this. Some other solution should be considered for this problem. I am aware this is more of a safety feature but this is something to consider.
Also I would like to point out similar thing have happened on Comdex and Huahua chain and they reverted tombstone. I can’t say that I know the technical behind how they did this I just wanted to point out that this problem exist on every chain based on cosmos SDK chain. A different solution must be created for this problem.
I am still undecided how to vote on this prop. On one side the chain should have slashed on the other there wasn’t any malicious attempt.
When optimistic slashing was introduced, my understanding was that the reason to gov gate slashing was to prevent false positives (consumer chains triggering slashes through code) and protect the Hub. It seems like the definition of false positive is shifting also to include real equivocation that wasn’t intended to be malicious.
This seems reasonable, especially if the framing is that these are training wheels to allow for ICS to be live in production (I could see validators not being willing to run ICS chains without this). Or maybe optimistic slashing is just better because validators mostly double-sign on accident, and other chains should add it.
I do have some questions about this from the consumer chain POV
Are these just training wheels? Or will optimistic slashing also be added to the Hub?
Will governance be able to accurately judge whether future double signs are accidental?
Does this change the incentive structure for validators to double-sign?
ICS double signing is made that way because it’s new. It was originally intended to be designed via a proposal to be able to understand situations like this one (imo).
Yes, that’s correct, on a couple of chains, the networks were reverted 1 block to save validators in similar cases
Yes I digged a bit around docs still going through so this is actualy a normal feature than a gov vote needs to be made so “social slashing” could happen leaving power to slash in the hands of the people. Since this is the case i will go through the evidence on neutron when I have the time and make a decision. Since this currently looks as a human error rather than malicious attack i will probably vote a no on this prop.
According to the ‘Slashing updates in replicated security post’: “We received feedback from some validators and community members that it would be too risky to slash based solely on information transmitted from the consumer chain. The concern is that some malicious code on a consumer chain could send fake slash packets and slash a validator that had not committed any infractions.”
So this double-sign slashing implementation done via governance instead of directly is because of this risk of potential fake slash packets from consumer chains mentioned above, therefore for the Cosmos Hub itself since this risk doesn’t exist double-sign slashing would continue as always.
Also, it is mentioned “Instead of slashing and tombstoning validators who double sign on consumer chains immediately, it will go through a governance vote first. This modification will add an extra layer of safety, while still punishing validators who violate the rules of consensus. Double signing is extremely rare in practice.”
Citizen Cosmos and Pupmos add a lot of value to the Cosmos Hub and are well-known in the community, clearly it wasn’t an attack but related to some upgrade coordination issues of Neutron that they ackownledge themselves and said they will improve the process for future upgrades. I don’t think voting yes and tombstoning Citizen Cosmos and Pupmos to ‘prove’ that ICS governance gated slashing really works is the correct action. So we voted no for this proposal.
These are just training wheels. Work is in progress here to make slashing automatic on the Hub, and end governance gating. There are not plans to add anything like this to the Hub. However, I would be in support of a mechanism like Anoma’s cubic slashing, or Ethereum’s correlation penalty to let validators who make mistakes and are not part of a coordinated attack get off with a smaller slash.
If there’s any question that a double sign could have been part of a double spend attack attempt, I think that governance will slash them.
I don’t think so. There is no benefit to double signing unless you are part of a cartel of 1/3+ double signing on the same block, as part of a double spend attack with carefully crafted alternate histories. In this case I think that the validators involved would be slashed. There is no reason for a validator to just double sign randomly, and really no direct harm if they do so.
That being said, I wouldn’t mind if these validators are slashed just to “prove that it works”, if that’s what people need to see. At the same time, I can also understand if people want to let them off easy for this obviously non-malicious operational error.
To me another goal of slashing in Cosmos is to strongly incentivize a culture of operation excellence among Cosmos Hub validators.
Cosmos validators have always been advised to run TMKMS or horcrux in production. Each of these applications provides additional double sign protection but also protects against a host of other potential threats to the network like malicious node binaries getting access to keys or an attacker getting access to keys via a compromising the only validator node.
The double spend incidents are indicative a significant decay in validator operational practices.
@David_Fortson I do not view this governance proposal as discretionary. Once I was aware of the equivocation, I feel obligated to put it up for a vote.
There is also the issue that evidence must be processed promptly for the security model to hold. So the idea of posting first to the forum seems inappropriate.
Yep - I get that. I pasted in the section from Informal’s blog above that outlined why this needed to go on chain ASAP. Appreciate the prompt action and understand why.
Seems like a clear NO for us. This is not a malicious behaviour, this is just a validators’ error, and we have quite a lot of cases of tombstones on other chains during the upgrades, and all of these times such tombstones were either discarded (like on Evmos), or reverted via a separate chain upgrade. We do not see why we should treat it any different here. NWV seems like a bad option here for us, as it’s not a harmful proposal, it’s a proposal we do not agree with.
Additionally, we should also keep in mind that ICS is still in the adoption phase, so there are not that many validators who has a lot of experience running consumer chains and knowing all the internals, so mistakes may happen.
I agree with this sentiment to the degree that I think it might make a lot of sense to automate these governance proposals. Instead of passing a/a consumer chain could pass a governance proposal to the hub.
Thank you for spending the time to make the governance proposal by the way.
my big question
Do we want to add a layer of meat to the machines?
I remember saying in 792 that I felt that Cosmos excels with social consensus, and I most certainly still think that.
Right now humans are most likely to be able to make more accurate decisions about these slashes.
The trouble is, it’s really a rather unpleasant choice to be making.
@aidan seems to be saying that he’s basically okay with these changes (and it matters because he is a team member of a consumer chain) but if her reading his commentary correctly he does have some concern about degraded security, and I think that should be all of our concerns.
Quokka Stake here.