[PROPOSAL #687][Passed]Replicated Security 3rd Party Audit

Hub Proposals Proposal Ideas
1

Interchain Security 3rd Party Audit

Update 16/02
Update 15/03 - Put Proposal On-Chain and changed the post below to be more in-line with what went live on the proposal.

TL;DR - Commissioning Oak Security to conduct a third-party audit of the Interchain Security (ICS) code with a similar scope as to the audit conducted by the Informal team.

This proposal aims to use community pool funds to commission a third-party audit for the Interchain Security code. This audit is to be conducted by Oak Security, one of the most-reputable auditors in the space.

The quote provided is $102K and has a timeline of around 2.5 - 3.5 Weeks for the audit. They will require 50% upfront to start the audit. Therefore, it is in the community’s best interest to get the on-chain proposal process going.

Since this is a community pool spend proposal, we want to ensure the community that the funds will arrive at the designated recipient by creating a multi-sig.

The multi-sig should comprise of:

  • Jehan (Informal Systems)
  • Zaki (Core Cosmos Contributor)
  • Jacob (Notional)
  • Kai (Neutron)

Breakdown of Fees:

With this proposal, We (Simply Staking) will be the main point of contact with Oak which means that we will handle all things related to answering their questions and queries. For this work, we seek a compensation fee of around 15% of the final quote. The fee will be calculated after the full invoice from Oak and any unused funds will be returned to the pool.

From that 15% a fee would be distributed to the multi-sig members as a form of compensation for undertaking this task with the utmost due diligence and care.

All payments are to be sent out to the recipients once the whole process is complete.

Funding:

OAK Quote: $102,000 + 15% price buffer to account for volatility of the ATOM token during voting period : $117,300

Simply Staking + Multi-Sig Fees: $15,300

Total ask 11,050 ATOM @ $12 per ATOM ~ $132,600

All leftover funds will be sent back to the community pool.

Initial Post:

TL;DR - With the proposal for Replicated Security (Interchain Security V1) due to go on-chain by the end of month, we believe there is a need for a Third-party audit of the code to supplement the in-house audit from Informal Systems to ensure that the code is sufficiently tested and supported by both internal and external audits.

Interchain Security being implemented on the Hub is right around the corner, with a Proposal projected to be submitted on-chain sometime in January.

Interchain Security, which has been developed by Informal Systems, is an important milestone for the Hub, introducing replicated security which allows multiple chains to inherit the same security of the Cosmos Hub validator set via IBC.

Since the announcement of its release, there have been rigorous tests on testnets (both private and public) to make sure there are no bugs or security issues that can lead to systemic risk.

There was also an audit conducted to ensure this code is fit for launch. The audit was conducted by a separate team within Informal Systems. While we have the utmost trust in Informal Systems and the work they have performed, we believe having an additional audit by a third-party auditor would provide further reassurances and gets us closer to the best practices within the traditional technology industry.

We (Simply Staking) are proposing the use of community pool funds to commission this third-party audit. This would ensure that Interchain Security (ICS) will have the backing of an internal and an external audit to give peace of mind to the consumer chains, ATOM holders, and the wider ecosystem.

Some things that need to be finalised during discussions before this proposal is put up on-chain include:

  • The form of the proposal. This includes how an auditor is to be chosen which could be done through mutual agreement in discussions, list and poll method or even through an ecosystem wide tender for work.

  • Whether ICS code should be enabled before the audit is completed or not

  • The funding mechanisms for the audit. How much the auditor should be paid and how the funds are managed and distributed should be clearly defined beforehand.

  • Criteria for the audit. The auditor should be given an idea of what they would need to audit. This can be done by a person/entity with the option of getting paid for it.
    Formation of an Audit Committee.

Please note that this is an early draft with no budget given and no auditor named. This has been done on purpose so that we can have an engaging discussion as to whether this is something the community wants before diving into the specifics of the matter.

We would love to hear your thoughts and opinions on the matter expressed above.

5 Likes
2

It is imo always good to have another set of eyes on code which is meant to be as big as a gamechanger as Replicated Security is projected to be.

I am not sure which auditor needs to do this though, I hear often mixxed signals on the quality of audits.

2 Likes
3

I agree that any ad all security audits can only be a good thing.

1 Like
4

Strongly support this idea.

1 Like
5

Agree on this proposal, but should not be a blocker for launch

4 Likes
6

I also strongly support this idea, and would be curious to know the perspective of the consumer chains set to launch about

As for the other points you raised:

Would it be appropriate to open up a public working group to hash those points out via synchronous calls, and then proceed to publish minutes to the forum for others to see and comment on the outcomes? It seems to me that synchronous comms about these items would be more effective and efficient than trying to do this asynch.

2 Likes
7

some months ago i tried to push that a part of the game of chains incentives’ funding should be used for external security checks,

then glad you made this proposal ! strong support.

2 Likes
8

Strong supprot for using community pool funds for additional third-pary audits!

1 Like
9

Agreed, it should not be a blocker for launch but as supplementary security.

2 Likes
10

Agree it should not block the launch. However, we may want to speed up the process!

1 Like
11

While I agree that it would be ideal to have this done before the launch, it is probably not realistic. We need to ensure everything is done proper with all the proper checks and balances in place. Also, we need to ensure that the auditor which will be chosen is up to the job to carry out this audit.

An audit of this scale can also take some time.

12

So here are some thoughts:

  • Informal is the best or second best auditor in cosmos, and my opinion is that they’re the best
    • Our team has worked with their audit reports before. Basically, on cleanup of their findings. This particular audit was not even done using their TLA plus techniques, it was a fully manual audit, which I believe greatly benefited the chain that was audited by making it more secure and also by paying attention to detail like code readability.
  • The osmosis team would be my second choice but I strongly believe that their workload is too high.

Remaining options

  • Entersoft - avoid at all costs
  • Oak - mixed feedback but generally positive
  • Halborn - seems to rely mainly on the codeql tooling built by crypto.org
  • Certik - seems to mainly focus on readability without deep focus on logic

So those are all of the options that I am aware of. Personally, I recommend informal to people who ask me about audits.

Notional is offering audits, but there are caveats:

  • We have not built out report tooling the same way that informal has, but frankly we do want for our tooling to mirror theirs. It’s really good.
  • We have massive experience with all layers of the hub stack. If there are SDK issues, we will very likely catch them.
  • Our audit experience mainly comes from developing a preflight checklist for joining new chains as a validator. We have the capabilities to do the kinds of in-depth logic reviews that informal does, but It would be our first time creating a formal report on a piece of software as large as this.

Thus:

  • I generally support this idea.
  • We are happy to work with the hub community on an audit, and we could explore that conversation here.
    • Notional is the team that discovered and fixed the issue that Quicksilver faced, in IBC-go, in september.
    • Anyone at the ICF and very sadly, at Quicksilver, will know that when we find a critical security issue, we do not disclose it, except to the chain team in question. (Note: we are working on modifications to this policy and believe that it was wrong not to disclose to Quicksilver, literally our team’s fault. But we had made commitments to the hub team / ICF around confidentiality) We would like to work with ICF on things like a structure for doing that kind of thing.
      • When we made a report to the hacker one, as is policy, we experienced an 18 day turnaround time. We no longer use hacker one. We just contact team members. That is what we did for the Quicksilver issue.
      • We have contributed numerous fixes to IBC. This has resulted in literally zero revenue.
  • We would likely veto proposals to engage with Halborn, Certik and Entersoft, based on past experience.
  • We would likely support a proposal to engage with oak
    • Osmosis grants program was able to line up a series of audits from oak and this is in my opinion a potential option for consumer chains.
  • We would abstain from any proposal naming Notional the auditor.
  • We would veto proposals that do not name a vendor and a price up front.

To generalize this issue, we simply lack auditors. Is my understanding that there is a well-deserved very long line for audits from informal.

The best auditors for Cosmos chains are almost certainly going to be engineers building cosmos chains. I imagine that this is why informal does so well with it.

The problem is that the best engineers building cosmos chains are incredibly incredibly busy.

So I’m not really trying to be a negative Nancy with any of this stuff, but instead lay out the set of issues so that we can try and solve them together.

Cheers!

4 Likes
13

Generally support the idea. Two heads are better than one. There can never be enough audit.

1 idea i had for years is to create some sort of national audit system. A system where each citizen (token or staked token holder) would be able to review upgrades in the lightest way possible. Of course for a reward in governace tokens. This process would be automatic and easy. I have actaully seen an ETH project work on this. But i lost who it was,

What about Notional and Jacob? The guys are well skilled to do this… Maybe we can hire them?

3 Likes
14

Notional is not really a 3d party, right ?

Maybe a dumb question but, what about something like code4arena ?

1 Like
15

We are definitely a third party in this case. Basically we would be auditing code from informal.

The other thing is just that there are very very few qualified auditors.

I don’t know the company that you just mentioned, maybe they’re a possibility and maybe not.

1 Like
16

Currently, we do this on an unpaid basis. Look at the volume of our code contributions to the cosmos hub, and compare it to literally any other validator that does not have a contract to work on the hub or SDK.

17

yeah i’m sure notional is qualified to review cosmos-kind-code. no doubt about your contribs as well.
i may have a definition of 3d party that is not the good one.

to me 3d party is some entity that have 0 tie with environment it has to review.

18

Hence my proposition to pay you :wink:

19

I would support probably any party to fulfill such an audit. A quote from a non-cosmos-specialized entity will probably cost a significant amount but willing to get a quote from various auditors (including and excluding the list Jacob posted above) as i have connections with some of them.

Example of pricing can be found here: StarShell Security Audit Funding Request - Secret Governance - Secret Network

This is a small wallet and their codebase audit quote was 20k from Certik and 100k+ from Least authority.
I am sure ICS would go into the 150k-400k ballpark.

2 Likes
20

Hey everyone,

Thanks for all the discussions that have been going on.

Now that we have seen that there is a need and want for this audit, we would like to shift the discussion towards finding the right auditor to carry out this task.

I can see that @jacobgadikian has shared some names with us including:

  • Oak

  • Halborn

  • Certik

  • And even his own team - Notional

There are also other auditors such as:

  • Least Authority

  • Trail of Bits

We are aware that these audits can carry a hefty price tag, especially with the scale of the audit needed. Thanks to @Ertemann for sending over that very informative post from a different forum.

There is also the need to outline and finalise a set of criteria regarding what we want from this audit and the auditor. This should be done before approaching any auditor for quotations.

With this in mind, we would like to gauge interest in having a Twitter Space (in around a week’s time) as a place to discuss this further as @ala.tusz.am mentioned regarding a working group. This will have a focus group type setting where everyone is allowed to discuss and after I will post the minutes/important points from the talks.

1 Like