Thanks for commenting here, @Rarma. Your response seems to have captured a lot of what everyone else was wondering too. To address your questions, and the questions of others, I’ll address it point by point:
Omnibus proposal
It’s important to have a core team that works on low level research and development of the Hub’s core features (right now this is ICS), while also integrating efforts from other teams. This is how most chains (and most software projects in general) function. Having a core team doing both maintenance and R&D builds expertise and continuity around the core features. If it’s not us, I would want to see another team filling this joint role.
While these items may seem related, they each have different scopes and deliverables, and teams work on them.
I’ll have to disagree on this a bit. Of the areas of focus we list, I think they are all quite related.
Testing & Testnets and Gaia Maintenance are obviously pretty core to the Cosmos Hub. These categories are also where the Hypha and Informal Hub teams work closely together. We could have submitted separate proposals for each team, but since we work so closely together, and our work would be greatly disrupted if one team’s proposal passed and the other didn’t, we decided that logically they should be combined.
A lot of work on ICS is maintenance, fixing issues in the backlog, improving performance, and upgrading to new SDK versions. There are also protocol improvements, as listed above. We anticipate that this ICS work will also encompass implementing and bringing to production variations on ICS developed through our work in the R&D section.
The tasks in the R&D section are less well-defined, since the nature of research is to find the best path forward. However, this section is also tightly focused on creating core infrastructure for the AEZ. Partial Set Security (an evolution of Opt-in Security) is obvious. The high cost of adding each consumer chain with replicated security slows the rate at which the AEZ grows. Partial Set Security provides a way for fewer validators to run each chain. Atomic IBC provides a way for chains in the AEZ to communicate instantly, while also reducing expenses by having them share blocks.
Mesh security and multihop routing are in the R&D section, but they might also be categorized as “go to market”. These are both technologies that are being developed by other teams (with some help from Informal in both cases), but where the Hub has the potential to be a market leader. Our goal with these is to make sure we are on the leading edge of their deployment, and do business development to get other chains to choose the Hub as their first choice for mesh security and multihop routing.
R&D accountability
This doesn’t instill confidence that this is worth funding via the community pool. The proposal should be specific in the items you will build and deliver for the Hub. You’re setting a roadmap of specific items the community would like to see implemented on the Hub, not what you’d like to pick from the list and implement. Another reason this should be funded by the ICF and specific items presented to the community to be implemented on the Hub/AEZ through governance approval.
It would be best to have the community vote on each as separate funding for the year, with their own schedules, deliverables and KPIs to be monitored against.
Researching and developing new technology is never a sure thing, and it’s impossible to predict with 100% certainty what the right path is. This uncertainty comes from the fact that specifications and prototypes are developed, information is gained which influences the course of subsequent research. We expect that either Atomic IBC or Partial Set Security will become the main focus of new feature development during 2024.
Uncertainty on Atomic IBC comes mostly from questions of whether the proposed underlying “megablocks” architecture will impose unacceptable costs on consumer chains in the form of resource contention, or unacceptable constraints on consumer chain’s use of the ABCI++ protocol. We will answer these questions by building a prototype of megablocks during early 2024, analyzing its performance, and developing solutions to enable the full use of ABCI++. Based on this information, we will either move on to hardening megablocks for production and making the modifications necessary to the IBC library, or we will shift focus to other R&D items.
Partial Set Security does not have as much uncertainty associated with it, since it is a relatively straightforward upgrade to ICS. However, the design space for the feature is quite large and there are still unresolved questions about it. We go over some of the questions in the CHIPs discussion phase post linked above.
Some of the concerns around Partial Set Security, ICS, and even Cosmos dPoS in general can be alleviated by proportional slashing, which slashes accidental double signs much more mildly while slashing real double spend attacks much more harshly. In some sense, research on Partial Set Security and proportional slashing could be seen as a more conservative research path as opposed to spending time on Atomic IBC. We’re open to any comments on where we should be focusing.
Requiring new proposals and votes for each phase of research would be a large amount of overhead, and would create a large amount of employment uncertainty for our engineers in a way that annual funding does not. Additionally, there would be perverse incentives at play. After putting a lot of work into specifying an area of research ahead of time, and hiring engineers for it, we would not be incentivized to shift our efforts to a more fruitful area, since that would require ending the grant and returning remaining money. This would result in dead ends being followed and a lack of flexibility. We do not think that this is a good way to do research.
That being said, we absolutely want to be accountable to the community. We are working on a process for Cosmos Hub feature development called CHIPs. We’ve posted it on the forum here to gather community input on the process. This process draws on Ethereum’s EIP process to ensure that features under development have sufficient community review during the process. The process has 9 phases. Phases 1, 2, and 3 will be taking place under the R&D budget in this proposal, while later phases move into the ICS, Gaia Maintenance, and Testing budgets. I’ll look at the first three here:
-
Discussion Phase: An idea is proposed and discussed on the Cosmos Hub forum. This proposal can be looser or higher level, but introduces and describes an idea, research direction, or feature. Depending on the outcome of this discussion, authors can evaluate if it is a good idea to put more time into the project.
-
Specification Phase: A design document is created providing detailed information on the proposed feature. The outcome should be a Cosmos Hub Improvement Proposal (CHIP) following the format defined in https://github.com/cosmos/cips. This document should be posted to the Cosmos Hub forum. At this point, the specification is a first draft. It will likely be changed during the Spike, Signaling, and Implementation phases.
-
Spike Phase: A prototype implementation of the feature is created that enables better estimation of the expected timeline. This implementation is not expected to be fully finished or hardened for production, but is intended to gain additional information about the feasibility and design of the feature.
(phases 4-10 are related to production implementation, hardening, and deployment, and not under the R&D budget)
Right now, Atomic IBC and Partial Set Security (an idea for a viable version of Opt-in without fraud proofs) are entering the discussion phase. Here are the forum posts for discussion of Atomic IBC Megablocks research and Partial Set Security. They should be in the specification stage in early 2024. We don’t have it ready yet, but we may also put proportional slashing (slash less if only one validator double signs by accident, like Polkadot and Ethereum) into the discussion phase for a 2024 implementation as well.
Once the discussion phase is complete, the finished specifications and plan for prototypes (phase 2) will also be posted to the forum. From this process, the community and oversight committee should be able to judge whether the R&D work we are doing is worth it. If we are not producing useful research, I’m sure we will get many responses to that effect on the forum. From here we can adjust and course-correct if this is necessary. If we are not course-correcting, or the course we are correcting to is crappy, the committee or community can put forward a proposal to cut our funding, possibly leaving funding for maintenance and testnets intact if that part of the mandate is going well.
Licensing
I think research into more restrictive licenses for Hub-funded R&D is a great idea, but we are not in the position to suggest a license, as we are not lawyers. However, we are open to working under any license that the community approves by a signaling proposal.
Headcount
The Informal Hub Team budgets for 14 team members. This includes:
- A project lead, product owner, and operations manager, who oversee and participate in every part of the team’s work.
- 5 generalist software engineers, who write most of the code.
- A protocol designer, who answers complicated questions about protocol correctness
- A verification engineer who builds and uses automated testing technology like CometMock, Quint, and Model-based testing
- A finance/economics researcher who answers quantitative questions about the financial impacts of technologies (like liquid staking)
- A devops engineer who helps Hypha with production operations and works on performance related aspects of the code and infrastructure
- 1 FTE equivalent for marketing and BD support, to help communicate about the Hub and our work, and find and guide prospective consumer chains.
Our per-head total rate averages around $325K per year, this is how with Informal’s 14 and Hypha’s 3.5 team members, we get to a cost of $5.7M. This total cost includes approximately $225K in salary (and relevant overhead such as employer taxes and benefits), the remainder includes standard overhead margin to pay for operational expenses like legal, HR, finance and other support as well as some profit margin to the companies, to ensure long term sustainable growth. Overall, this corresponds to an hourly rate of around $165, a mid-market rate for consultancies.
We will include details on the number of employees working on the Hub in our quarterly reports. In case we fall below the allocated number of employees during a quarter (due to turnover etc.), we will only withdraw the funding from the multisig needed for the number of employees actually employed.