[PROPOSAL 885][PASSED] Gaia v15 Software Upgrade

Hub Proposals Software Upgrade
1

Change log

  • 2024-2-14 Created initial post
  • 2024-3-1 Revised title for on-chain proposal
  • 2024-3-17 Revised title for vote passing

Important Notes for Node Operators

This is a large upgrade and we expect it to be fairly slow due to state migrations in upgrading the sdk to 0.47.x. There are revised hardware requirements just for this upgrade, but specs can return to current mainnet requirements afterwards.

  • Minimum: 64GB RAM + 32GB swap
  • Recommended: 128GB RAM
  • Bare minimum 32GB RAM + 64GB swap

Optimal CPU performance: 2.50GHz, 8 cores (eg Intel Xeon Gold 6248 or equivalent consumer grade processor).

It is paramount that the operators set enough SWAP to cover all cases. SWAP partitions can be used to supplement the RAM requirement but they will increase the upgrade time.

Further recommendations

  • If you are unable to meet the hardware requirements, consider skipping the upgrade and resyncing from snapshot once the upgrade goes through.
  • Logs will be sparse during this long upgrade. You can run your node with -log_level="*:debug" to get slightly more verbose logs during upgrade. Do not panic at the lack of info — just sit tight!
  • Do not restart your node mid-upgrade unless you are getting OOM errors. Restarting mid-upgrade may lead to a corrupted database.
  • Set min gas price to 0.005uatom or your node will fail on restart. Failing on restart just means that you’ll have to set the param at that point, but doing it ahead of time will be smoother.

Background

The Gaia v15 release is a major release that will follow the standard governance process by initially submitting this post on the Cosmos Hub forum. After collecting forum feedback (~ 1 week) and adapting the proposal as required, a governance proposal will be sent to the Cosmos Hub for voting. The on-chain voting period typically lasts 2 weeks.

On governance vote approval, validators will be required to update the Cosmos Hub binary at the halt-height specified in the on-chain proposal.

Proposed Release Contents

The relevant Github epic for this release is: Gaia v15 Release

The focus of this release is the upgrade of Cosmos SDK to v0.47 – this release uses v0.47.8-ics-lsm, a special Cosmos SDK branch with support for both ICS and LSM. Consequently, it also upgrades the following dependencies:

In addition, this release migrates the following state:

  • Sets the min commission rate staking parameter to 5% and updates the commission rate for all validators accordingly (according to governance proposal 826). Validators with a commission rate < 5% prior to upgrade will have a rate of 5% after upgrade is complete. This parameter can be changed via a param change proposal in the future.
  • Migrates the vesting funds from cosmos145hytrc49m0hn6fphp8d5h4xspwkawcuzmx498to the community pool (according to governance proposal 860).

Finally, this releases fixes a series of issues found during the Oak Security audit of SDK 0.47. As a result, this release introduces the following API changes:

  • Reject MsgVote messages from accounts with less than 1 ATOM staked. All validators should ensure that they are staking at least 1 ATOM from the wallet they use for sending MsgVote messages.
  • A new MinDepositRatio param is added (with a default value of 0.01) and now deposits are required to be at least MinDepositRatio*MinDeposit to be accepted.

Taken together, these changes upgrade Gaia from cosmos-sdk v0.45 and open the door for further integrations, such as adding Skip Block SDK required by Prop 842.

A release candidate can be found here.

Testing and Testnets

The v15 release has gone through rigorous testing, including e2e tests, integration tests, and differential tests. Differential tests are similar to integration tests, but they compare the system state to an expected state generated from a model implementation. In addition, v15 has been independently tested by the team at Hypha Co-op.

Validators and node operators have joined a public testnet to participate in a test upgrade to a release candidate before the Cosmos Hub upgrades to the final release. You can find the relevant information (genesis file, peers, etc.) to join the Release testnet (theta-testnet-001), or the Replicated Security testnet (provider).

Potential risk factors

Although very extensive testing and simulation will have taken place there always exists a risk that the Cosmos Hub might experience problems due to potential bugs or errors from the new features. In the case of serious problems, validators should stop operating the network immediately.

Coordination with validators will happen in the #cosmos-hub-validators-verified channel of the Cosmos Network Discord to create and execute a contingency plan. Likely this will be an emergency release with fixes or the recommendation to consider the upgrade aborted and revert back to the previous release of gaia (v14).

Governance votes

The following items summarize the voting options and what it means for this proposal:

YES - You agree that the Cosmos Hub should be updated with this release.

NO - You disagree that the Cosmos Hub should be updated with this release.

NO WITH VETO - A ‘NoWithVeto’ vote indicates a proposal either (1) is deemed to be spam, i.e., irrelevant to Cosmos Hub, (2) disproportionately infringes on minority interests, or (3) violates or encourages violation of the rules of engagement as currently set out by Cosmos Hub governance. If the number of ‘NoWithVeto’ votes is greater than a third of total votes, the proposal is rejected and the deposits are burned.

ABSTAIN - You wish to contribute to the quorum but you formally decline to vote either for or against the proposal.

12 Likes
2

For context, this upgrade took us about an hour when doing a stateful upgrade with a single whale validator. Usually our Gaia upgrades are very quick in this format (<10 min).

1 Like
3

Appreciate including the fix for missing addresses in /signing_info response: Missing address in signing_infos of rest api #1344 · Issue #1734 · cosmos/gaia · GitHub, that’ll optimise the queries I have to do in my tools.
Already waiting for this to land on Hub, LFG!

2 Likes
4

Are there plans for the future to abolish the need for custom releases/tags of cosmos-sdk with the changes specifically tailored for Cosmos Hub?

5

Look forward to it :saluting_face:

1 Like
6

The reason Gaia requires a special SDK release is LSM. SDK 0.51 should enable plug-ins for the staking module, which means that the LSM logic can be implemented as a plug-in and then Gaia can go back to mainline SDK.

2 Likes
7

Staking Facilities is looking forward to upgrading the nodes :muscle:

We always value the detailled information and instructions provided.

That being said - I just wanted to drop the link to the technical upgrade docs including possible rollback scenarios and backup recommendations:

https://github.com/cosmos/gaia/blob/release/v15.x/UPGRADING.md
1 Like
8

If i remember correctly a future change of the validator commission rate param will not trigger a change the comission rate of the current set?

9

It has evolved.

Validators with a commission rate < 5% prior to upgrade will have a rate of 5% after upgrade is complete.

10

Yes in this upgrade the rate will be changes to 5% but if i remember it correctly a gov param change without an upgrade will have no effect on the current val set but will have an effect on new validators/ validator commission changes.

11

ChainUp & NewHuo is excited to perform the upcoming node upgrade ! appreciate the detailed upgrade documentation previously provided. Does the upcoming upgrade will automatically set the rate to 5% for all validators?

12

This upgrade migrates the state to enforce the min 5% commission rate – gaia/app/upgrades/v15/upgrades.go at v15.0.0-rc1 · cosmos/gaia · GitHub

1 Like
13

The upcoming upgrade will automatically raise the rate to 5% for any validators under 5%. There will be no change in commission rate for any validators currently at 5% or higher.

4 Likes
14

Redundant question was removed as many people answered the same question.

16

This change will help small validators stay in the game, letting them keep pace with the big players. This way, the voting power within the network stays more balanced.

This change is a welcome development in combating spam and bot votes on the network, allowing for clearer visibility into authentic user data.

Appreciate the hard work :saluting_face:

2 Likes
17

Looking forward to it.

18

Yes thats the issue/open question i have about this upgrade.

If we change the paramater in the future (for example to 10%) we will have to migrate the state in an later update again. During the timeframe of the success of this hypothetical proposal and the upgrade, will new validators be forced to use the 10% min_commission and old validators will still be able to use the old min_commission of 5%?

19

The MinCommissionRate param is only used for CreateValidator and EditValidator messages. This means that once the param is updated to e.g., 10%,

  • new validators cannot be created with commission rates below 10%;
  • existing validators cannot update their commission rate to values below 10%.
    So yeah, to enforce this new min commission rate to every validator, migration will be required.
20

We have been notified of an issue with the v15 upgrade that may interfere with operations on the Asteroid Protocol. There is a potential workaround identified and our team is assessing implementation timing. We want to ensure that we are fully supporting all apps that are built on Gaia. At the moment, we will be delaying the final release until next week. I will update this forum post with further information on timing if required.

21

Thanks you for confirming my assumption.

I think MinCommissionRate should not be a governance paramter since it doesnt result in an immediate change of the commission rate of all validators but creates an unfair advantage over new validators.

I understand that the chances that this will happen are very small but inequalities like this should be avoided by design.