Introducing Testnet Wednesdays

Conversation
1

:ribbon: Testnet Wednesdays :ribbon:

With the successful launch of Neutron on Cosmos Hub :rocket: we at Hypha have learned a lot about administering the Hub’s testnet infrastructure, as well as what it means to rally 80+ validators together for many coordinated upgrades and new consumer chain launches. Together we’ve laughed, we’ve learned, and we’ve banged our heads on our desks after seeing the 17th AppHash error in a row.

Currently, testnet rehearsals for launches and upgrades are scheduled on an as-needed basis, with one or two rehearsals a week. Consumer chains represent a growth in complexity for not only The Hub, but also for the Testnets Program. Coupled with the regular cadence of coordinated Gaia upgrades, it’s clear that as more consumer chains onboard onto The Hub, the demand on the Testnets Program will also grow. Adding incentives to testnet participation has come up before, and Hypha is actively working on creative solutions in this space. In parallel, to address the short-term growing demand on validator attention and resources, we’re trialing a new experiment: Testnet Wednesdays.

All coordinated upgrade and launch activities will now take place on the Replicated Security testnet at 14:00 UTC each week. Some weeks, there might not be anything – we’ll let validators know by Monday if the agenda is empty, therefore no rehearsal is needed. In other weeks, there may be a provider chain upgrade and a new chain launch at the same time. We’ll do our best to space things out so that there aren’t more than two events in a single day. As we continue doing launches and upgrades together, we’ll hopefully get better and faster each week.

Emergency release tests for the Hub and for consumer chain teams will still be scheduled, as needed, outside of the normal rehearsal cadence. This will only be invoked in the event of critical security vulnerabilities that represent a major risk to the Hub.

For chain development teams, this means that we’ll be working more closely with you to make sure that your release cycles line up with testnet rehearsal days. If a binary is not ready with enough lead time to build confidence, the rehearsal will be pushed to the following Wednesday. Committing to a public timeline for release and testing should also create more predictability for validators who want to run their own tests ahead of time.

Here is a tentative schedule of events for each week:

Day What’s happening
Week prior
  • Development team makes target binary available for testing
  • Hypha runs the target binary on local testnets and runs some exploratory tests
    		•
    Monday By 16:00 UTC Hypha announces scope of the week’s tests
    Wednesday 15:00 UTC Rehearsal takes place on testnet
    Thursday
  • Optional retrospective with development teams
  • Hypha publishes rehearsal summary document
    		•

    If you have questions, comments, or feedback, we want to hear from you! Ping me, @lexa @dante or @uditvira.

    Are you a validator or node operator who is not yet part of the Replicated Security testnet? Join us!

    cc @ala.tusz.am

    Edited schedule to allow more lead time for testing.

    2 Likes
    2

    Hey. We are a validator, we took part in the RS testnet. But we still want to participate. How?

    2 Likes
    3

    Testnet Wednesdays sound like a great way to make coordination and testing more predictable. Looking forward to further collaboration with validators and development teams!

    1 Like
    4

    Hi! If you’re already a validator or node operator in the Replicated Security testnet, and in the #replicated-security channel on Discord (which it looks like you are) then you’re in all the right places. We’ll post an update on Discord soon about this week’s events and what you’ll need to do.

    2 Likes
    5

    This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.