Interesting thought to remove the voting power from validators. It has upsides and downsides.
The first question would for me would be; is there enough participation from individuals to do this? It gives a serious risk of gaming governance, because it would become way easier to sway a proposal in your direction with the right investment.
I asked the same question in the chat for the validator commons ([PROPOSAL][DRAFT] Validator Commons: improving validator governance and leadership - #14 by LeonoorsCryptoman); what is the direction we need to take? What is our dream and end-goal as desired by the community? At this point in time true decentralized governance is something which is certainly not in one of the top spots of being desired imo. Making changes to how we do governance can be an option, but they need to be directed to solve a problem and not create a new one imo