PROPOSAL #90 [REJECTED] Community Health research, analytics and benchmarking

Twitter space for the proposal tomorrow:

1 Like

I would agree with @common_spelling that the framework is largely nice-sounding academic rhetoric at this point. I think for the community to fund this, some clarity would be good. I don’t see why the community should fund the actual development of the framework for you. But it would be different story if you already had a working service to offer, i.e. “we have this tested framework, here are our previous results/reports, would you benefit from quarterly reports on this?” Then I think this would be a lot more attractive.

3 Likes

For clarification, please. Have you seen the video of the dashboard prototype above?

We already have a working framework (and already did a report for Aragon and MetaGame), but that’s not the same as validated. Validated means we have really tested it, which requires data, which requires communities to partner with us… until we have at least 10 and can correlate patterns across, we’ll still be in draft form.

But even at this early stage Aragon had this to say:
“RnDAO’s Community Health Report helped us understand the overall feeling of our members towards our Community and the kinds of interactions happening within it, bringing to light hidden problems and providing us with the insights to take the necessary actions.”
Incandenza - Community Guild & Core Contributor at Aragon

MetaGame is sharing a tweet about it this week…

The issue is still that meaningfully assessing community health is a huge challenge, so there’s still a lot to do both conceptually and practically. The grant goes towards advancing that and getting the insights for Cosmos.

1 Like

sounds like you should be paying us for our data.

I might vote for this if the $ATOM from the community pool went to users in exchange for their survey responses.

We only collect the data that directly allows us to provide insights to you as a member of the Cosmos community. We have no use for your data other than helping the community (i.e. you). We don’t sell the data; we don’t use it to advertise anything to you. So it would make no sense for us to pay for it.

Now, if you want Cosmos to pay for the individual community members giving their data, that’s something we could explore moving forward and I’m curious to flesh it out. I’d suggest making an additional proposal to request incentives and we can discuss it there?

How will cherry-picking data points quantifying the level of group think pervading the cosmos help the community (i.e. me)?

That’s fair enough @danielo, then I stand corrected. Thanks for clarifying this and your prior collabs. To make myself perfectly clear: on the whole I think the idea is good, and pretty well-timed in the aftermath of ATOM 2.0. Could be some interesting synergies to explore with (if accepted) the other proposal coming out of RnDAO ([Proposal ##][DRAFT] {After the Big Bang: A constitutional process the Cosmos Hub}) and the works of Validator Commons. But again, would be cool to see some of the results from your work with Aragon and MetaGame beyond the quote you provided.

If anything, it is a good idea to introduce and leverage prior academic research in relevant social sciences etc. Blockchain governance needs more of that.

1 Like

Normally with proposals I try to remain as neutral as possible. But because this one directly affects discord, and because its part of my role to manage things there, and keep the community safe. I feel I must speak up about key issues surrounding this proposal, and why I don’t think its right for us at this current moment in time.

After the community call yesterday, I approached the team to ask some questions and raise some concerns. **You can read the discussion from the telegram group t. me/commhealth

From this discussion I got the impression that this was less about what was best for cosmos, and the community, and more about furthering their goals and their research. This was more evident by how many other projects they had reached out to for funding. Which is not necessarily a bad thing, but if they care less about where the funding is coming from, then do they have our best interests at heart?

I think these unique metrics look good on paper, but I also believe they could do more harm than good. e.g. Draw more attention to negative energy surrounding certain proposals that could damage our image as a whole. Every project goes through its rough patches, drama surrounding governance is going to happen, I think we need to get better at communicating with each other, but I don’t think we need a bot to tell us this has been a rough year, and that we need to work on some things. I think humans are much better equipped to identify bad patterns, and community sentiment.

As I mentioned in the discussion linked above, I don’t think the cosmos discord is the best testing ground for this bot. Its currently setup as a more developer focused space, with the majority of the server containing private channels that I would not feel comfortable giving a bot access to. (End-user support tickets, private testnets, and private developer channels) - These channels make up almost 95% of the whole server. So paying $30,000 to give a bot access to analyse the few public channels we do have (which have much lower traffic due to the bear market) Does not seem worth it to me right now. The data that could be extracted from these channels would be minimal.

  • I’d like to note that we do have plans to make the discord more community focused, and open up more public channels in the future. But again building a community in a bear market takes time. We are more in a “clean everything up, re-organise and re-design” stage before the next bull run, At which time there will be a lot more traffic, and data to analyse. - If its data you’re after, that would be a much more optimal time to get it.

I do think there is some merit here, metrics that go beyond just numbers and bar charts, is great. I think RnDAO has some good ideas here, and I am all for building a healthier space. But I’m unsure if this proposal was submitted with the right intentions, and I’m not entirely convinced by the execution of this proposal, and the urgency to get it on chain.

9 Likes

Thank you Alice for your comments.

Please allow me to clarify our intentions: our primary goal is to enable healthier communities. That’s indeed different from an exclusive focus on Cosmos but it also aligns with Cosmos. And we want to provide insights for Cosmos specifically, both from a theory and practice/strategy perspective. I hope you agree with us that collaboration across ecosystems is important.

Then, although indeed the discord data atm is limited and I know you’re doing great work to reorganize during a bear market, and I understand that us generating some form of KPIs can create some fear, let me share a bit more about the rationale we have: the insights will only make sense over time, meaning that a previous point for comparison is needed. So although building a community takes time, and especially during a bear market, the tool can actually help you identify areas to focus on. Any good assessment is not about bonuses or rewards/punishments (although that’s unfortunately common in web2) but about insight, learning, and strategy. So starting now would mean providing you with a tool to support your work of building community.

This will also create transparency for the community about how things are going, and enable the community to identify other proposals to make and grow stronger. As the saying goes, knowledge is power. We want to provide the whole community with said knowledge.

Also, I’d emphasize that the 30k is not just for discord insights but all the research that comes with defining and framing community health, and learning what contributes to it and what doesn’t. We’ll also have learnings from other communities that partner with us, which can also benefit Cosmos. As mentioned, our primary goal is healthier communities in Web3. And we strongly believe that in this market condition, a rising tide lifts all boats. So I hope you can see the merit on funding the public goods side of this proposal.

Finally, we do have plans to include Discord too. We could soon be making this available to Cosmos (assuming we get the funding for Discord so then funding from others can go directly to the Discourse data collection tool). But if we fail to get funding, the whole project goes in peril. Discord is just the starting point, and any funding goes to the workstream for community health.

We put this proposal first in September (see date at the top), so I could counter the “rush to get it on chain”. It took us a while to figure out how to get the attention of the community, but this is true of anyone who hasn’t been part of the ecosystem for a long time. I hope we can encourage ideas coming from others who’re not part of the core too :slight_smile:

1 Like

Thank you. Unfortunately, we’ve not been permitted to share Aragon and MetaGame results publicly. At least not yet.

We’re collaborating on drafting a twitter thread with MetaGame to share some of the insights, which I was told goes live tomorrow. If it helps, you’re welcome to follow them on twitter and only vote after seeing the tweet. I’ll also link it here once it’s out.

Group think leads to bad decisions and ultimately poor performance, so addressing it would lead to a healthier cosmos community and, in turn, to better atom price thanks to the success of the ecosystem.

If, say we identified a lot of group think, then an initiative could be created by the community team or community members to address that. The tool could also measure whether that group-think-reduction initiative is successful or needs iteration.

1 Like

Link to live proposal:

Thanks for your reply, Daniel

I agree that collaboration across ecosystems is important. This is cosmos, our goal is IBC adoption. I can understand the use cases for all of this in theory, and being able to compare data with other ecosystems is valuable.

I understand how insights only make sense over time. But I don’t think we need a bot to tell us that traffic is low, things are moving slowly, and everyone is depressed because of market conditions, and a very close result on a recent governance prop.

Furthermore, If its something the community desires I could utilize bots like dyno and Formeer ( top. gg/bot/992500626548334642 ) for less than $200 a year to send out automated commands and feedback forms every 2 weeks to survey our users opinions, and have them react with emoji’s to gauge how they are feeling, Formeer has a dashboard for this already, and for dyno I could spend a few hours manually translating this data into metrics, which I can later submit as a blogpost, and then every 3 to 6 months analyse and compare all the data of previous months. I think there are plenty of web2 solutions that I could work with to measure community health.

I’m all for empowering web3 projects and building a better future, but I’m not entirely convinced that this is something we need, especially with the limitations you would experience in the cosmos discord (channel access etc) when there are solutions out there already that we can utilise, and be in complete control of with no risk of compromising user privacy or safety.

From what I understand your tooling is a lot more advanced than that, and of course you have industry experts to properly analyse all of this data, and present it in a way that is much better than I could ever do on my own, but I still feel very strongly that the data you would collect in current low traffic conditions could be done by one person and a couple of cheap web2 bots.

Sorry if my “rush to get on chain” comment came across rudely, It was more to do with how quickly the proposal went live on chain after I had raised some major concerns. I felt more discussion could have happened the next day, and we would have ironed our some of these issues privately, but its not a big deal, and I’m sure you had likely planned to go on chain today.

I want to apologise for having such a strong opposing view to this proposal. I’m quite an empathetic person, and I don’t mean to harm you, or your teams progress. I had a read through the blog post you sent to me, and I do believe there is value in what you are proposing. But in its current form and at this moment in time, I am hesitant to support it.

3 Likes

I appreciate you sharing your thoughts, at least that way we can discuss :slight_smile:

I’m feeling frustration seeing this compared to dyno or statbot, as 1) those have their own database so the data is within their control. And more importantly 2) that’s reducing our project to a bot.

I understand the community is not fully developed yet, I understand the data in discord is limited. We’re also at an early stage and have a lot of work to do. What we’re proposing is embarking on a journey together.

The bot is a tiny part of that journey (it’s just a tool to collect the data). And although we agreed to start with public channels, we had already discussed in telegram the possibility to expand to other channels (after proving it’s safe, which we can easily do). And we discussed some of the applications like being able to identify which teams are likely to be high performing and which ones need support, identifying misalignments between teams, etc.

And the above is just from the passive data (discord messages). Because the richer dataset comes from the pulse survey, that’s where we can put in context what the community is doing. So even if there were 0 messages in the public channels, we could still generate a lot of insights that way, insights about wellbeing, burnout, alignment, etc. And the point of those insights is to help you build the community.

Morever, we also mentioned we’re operating as a workstream, we have plans to expand to include Discouse and other platforms too. Getting there is a matter of funding, where we’re splitting the costs across grantors but what every grantor provides goes to benefit them all as part of the coalition that gets formed to research community health.

Which brings me to the last point, about the output not only being metrics way beyond what most people are able to do (It has taken us months and multiple PhDs involved to figure it out, and we’re still refining and iterating) but also all the learnings about what works and what doesn’t for community health. Which can go to serve all the members of the coalition. Because when one tries say to include weekly AMAs and another introduced a tool for 1-1 coffee chats, how do you know what’s working and what’s not, without rigorous comparison across data sets?

1 Like

And the deadline is because that’s what we had agreed on the call we had with multiple community members. We converged on the 12th being a good date so we put it as last call and wanted to honour that. Then we struggled for 48h figuring out how to put the proposal on chain, and it was a developer colleague doing that, so they hadn’t event seen your comments.

As we had met your request to allow you to see a sample of the data collected, etc. to make sure it was safe, I couldn’t see any reason not to move ahead with what’s already been a very long process and prohibitively expensive time-wise, when other than @common_spelling, the community was showing us good support.

1 Like

Hi Daniel,

I’d like to state that I am not comparing you to dyno, statbot or any other discord bot. I am perfectly aware that what you’re proposing is much more than that. I’d like to think I understand your ideas for the future, and I 100% agree, some of those ideas are great. But I’m struggling to see your passion for us, and the community. I feel like we are just another one of many projects you’ve reached out to for funding, with no real second thought to who we are, and what we’re building.

Here’s how I understand our current community health: It’s a bear market, its not exciting, every project is getting hit hard, there is less engagement, and the majority of people in this space are either miserable or indifferent. Those that are indifferent have been through all of this before, they have a deep belief in the tech, and they know it will get better eventually. But right now, the core feelings are doubt, fear, anger, and uncertainty, and I think while its obvious to everyone, its not exactly something we want to put under a magnifying glass. I think this research would be amazing for your team, I just don’t understand how it would benefit us. You keep telling me that if we start now, we’ll be able to compare this data in the future, and see how we’ve grown. But the contrast between now, and the future is simply bear market versus bull market, one of which generally has a lot more happy emotions associated with it.

Don’t get me wrong, I think research into community health is very useful. I think there is a lot we could learn from it, and become better for it. But why not do this during a bull market? Surely being able to identify unhealthy behaviours during a time when everyone is typically more happy, would provide much better insights than figuring out what is painfully obvious during a bear market. Transitioning from a bear to a bull can be a matter of hours or days, its a very rapid change. But transitioning from a bull to a bear takes months. A lot of factors to examine, study, and pull data from.

The proposal you posted on the GnosisDAO forum, and other places is exactly the same minus the names, from the call you did, and the way you’ve communicated with me, I don’t get any impression that this proposal is about helping us. Some feedback would be to care a bit more about the projects you are requesting funding from, not just give them the same cut and paste.

Again, I emphasise with you, because I think what you and your team are building here is great, I just think your timing is off, and your intentions unclear. I would love to maybe see this proposal pass in the future, but right now I personally can’t support it. However If you can prove to me that you are passionate about us, and our vision for the future, maybe I would be more onboard to support it.

Remember I don’t speak for everyone, I’m just speaking for myself, my own beliefs, and what I think is best for the community at this time. If this proposal passes I will support the wishes of the community and work with you, and If it doesn’t pass, I hope you will consider sticking around, and giving it another shot in the future, because as I’ve said. I do think there is some value in this project, I just think we need to hear a little more from you, and maybe some other members of your team.

If I don’t hear from you before the new year, I wish you a happy holidays.

3 Likes

Let me push back a bit on the bear vs bull market, as I feel it’s missing the core ingredient: analyzing community. Not just sentiment (i.e. vibez, mood, etc.) but an actual sense of community.

Activity will increase in a bull, sentiment too. Burnout might go either way (and it’s good to check), need satisfaction can also go either way and it’s good to check. But most importantly, sense of community can go either way too.

Bull markets are not necessarily good for community. The noise and influx of people can actually weaken it. And it’s a lot harder to see because the change in vibez (general mood) can obfuscate this. As the bear starts to kick in, members who were consistently active can also get dragged elsewhere as it’s shiny new opportunities everywhere. And because community managers and mods are also dealing with the rapid increase in activity both in the community and generally in the industry, it’s really hard to notice until people have been gone for 3+ weeks and by then it’s likely too late.

Bulls drive the hyper-financialization of everything. We want to prevent that. We want to be ready to show what’s important, to show that hype is not everything. To be able to spot the first signs of change and enable the community to take action, having data to back their decisions so proposals can cut through the politics of opinions.

I’m all up for proving that this is not a selfish proposal. But how can I do that?

We’ve been having a lot of discussions with people talking about the multi-chain nature of Cosmos, but that doesn’t affect the current proposal. If anything, it works even better in this scenario, as network analysis is very suited to that. We’ll still have to start with the Hub and then expand, but that’s a matter of starting somewhere and then developing the initiative, not a matter of the initiative being unfit for purpose.

We’ve been talking a lot about the relatively low number of messages in Discord, and we’ve insisted that shouldn’t change the choice of platform as the other platforms - Discourse (the forum), telegram, etc- wouldn’t work well for the pulse survey which is key to measure sense of community. We also mentioned we have plans to include Discourse and twitter ASAP, and, still, from what we can see, the best starting place for this proposal is Discord.

We have put throughout the proposal that the idea is to work with the community in the research and refinement of the tool and metrics, so we have every intention to collaborate but also know that said collaboration has costs and hope we can cover ours as we work to deliver value and tailor to the cosmos community (as opposed to the current situation of having to do all the work upfront, without any pay nor guarantee).

We’re still offering to do what’s needed. Help me understand, what it is that we’re not getting that’s different about Cosmos so that the proposal doesn’t work? (or at least so we can make a V2 proposal)

Is this related to the 4 council scam DAO that balance each other out, posted in another thread?

are you planning to continue your community health funding to come up with a meaningful definition for community health by using the LSM to strip delegators voting power and install a governance cabal like govmos described in his model?

No idea what you’re referring to, so I’d say it’s not related at all.
Anyhow, the proposal was shut down

1 Like