[ANN] Tendermint KMS v0.2: Validator Signing Support


#24

If you run both processes under supervisors (e.g. systemd), the ordering doesn’t matter as KMS will continue trying to connect, and gaiad will keep restarting waiting for a KMS connection.

That said, the timeouts right now on both sides are pretty bad and not presently adjustable. I think KMS should try to reconnect more frequently, and gaiad should wait longer for a KMS connection (and ideally start asynchronously while it waits for KMS to connect)

I suspect it was this bug, which should be fixed on the develop branch:

https://github.com/tendermint/tendermint/pull/2876


#25

Hello, i have a question:

Tendermint’s height increased with time, then i did three operations:
1、generate softsign.key, and corresponding amendments to tmkms.toml of tmkms.
2、use tests/support/secret_connection.key as secret_key in tmkms.toml of tmkms.
3、change priv_validator_laddr of tendermint and then they connect each other success.

Then i discovered the height of tendermint stop increasing, i do not know why?

Is that all right? Do i have anything else to do?


#26

I’m having trouble figuring out how to configure gaiad to use the KMS signing keys.

The KMS is connecting to my gaiad process. Using the gaiad tendermint show-validator command does not show the key that the KMS is serving up, it shows the key from the local priv_validator.json file. I’ve tried commenting out the priv_validator_file = line. I’ve tried deleting priv_validator.json. Not sure what else to try.

The KMS log indicates 01:02:58 [INFO] [genki-2000@tcp://127.0.0.1:12345] connected to validator successfully


#27

Matt, do you have errors after deleting the line or the file?


#28

There is no error when I delete the file, gaiad recreates a new one on startup.

I have it working now - I declared the validator using the public key from the kms. It works.


#29

Matt, is the height of tendermint still growing after connecting successed with kms?

My kms log also indicates [test-chain-CJ8kTB@tcp://127.0.0.1:26659] connected to validator successfully, but i don’t know if signing service is working correct or not, because the height of tendermint stopped growing.


#30

Great news! This is the only thing that worked for me too. I think there are parts of gaiad that still hardcoded to priv_validator.json.

To use tmkms on a testnet, I needed to take the Bech32 key that tmkms printed out (via tmkms yubihsm keys list) and pass that as the argument to gaiad tx stake create-validator:

$ gaiacli tx stake create-validator --amount=10000STAKE --pubkey=cosmosvalconspub1zcjduepqfgjuveq2raetnjt4xwpffm63kmguxv2chdhvhf5lhslmtgeunh8qmf7exk --moniker="iqlusion.io" --chain-id=gaia-9002 [...]

For this to work, however, you’ll need v0.2.2+ of tmkms (v0.2.3 includes some timeout fixes which you’ll probably want). Earlier versions printed out incorrect cosmosvalconspub addresses.

You’ll need to create a validator with a pubkey that matches one of the keys stored in tmkms for it to begin signing transactions using the command above.

As soon as you do, provided things are configured correctly it will begin signing transactions.


#31

Thank you for replying

1、The content of signing.key in tmkms is

GNx76GuJt8rWkruCYZKieE7P4eVYymPD4MkNS1OuWLc=

2、Then execute in tendermint

tendermint unsafe_reset_all

3、Editing the priv_validator.json in tendermint like below:

{
  ...

  "pub_key": {
    "type": "tendermint/PubKeyEd25519",
    "value": "GNx76GuJt8rWkruCYZKieE7P4eVYymPD4MkNS1OuWLc="
  }
}

4、Executing in tendermint

tendermint node --proxy_app=kvstore

But the height of tendermint still not growing. And it’s strangely enough when execute

tendermint show_validator

Will get the result:

{
  "type":"tendermint/PubKeyEd25519",
  "value":"gn5KbGzulPkvN2uExWqb9FR2M2ThlupP5zFh3Bxw6t4="
}

Is there a cache or did I correct it?


#32

Unfortunately, tendermint show_validator doesn’t query the kms for the public key but returns what it finds in $HOME/.tendermint/config/priv_validator.json (this is the default path). If you edited that file, you should see the same key as you’ve put in there.


#33

Yes, i means i have edited $HOME/.tendermint/config/priv_validator.json but the result of tendermint show_validator still not change.


#34

But if execute tendermint init,the result of tendermint show_validator will changed immediately.

But, when i change the value of pub key in the priv_validator.json, it would not changed.

I am very confused

Edit: if i remove priv_validator.json and restart tendermint, it will recreate priv_validator.json like matt said. And the result of tendermint show_validator will changed immediately too.

Seems the only thing that didn’t work was that i edited it. Strange :frowning:


#35

Looks like the pub-key in tendermint show_validator is derived from the private key in there and is not read from the pub_key field. That is confusing indeed.


#36

If you use KMS, you should read the pub key from the HSM but not using gaiad tendermint show-validator.


#37

Right, but i run show-validator just for testing. Because no effect after modify the pubkey’s value of priv_validator.json in tendermint


#38

Whether use this HSM or not, you have to change the pubkey of priv_validator.json in tendermint, right?


#39

How do you declared the validator using the public key from the kms ?

How to create a validator with this pubkey?
I just use tendermint instead gaia for testing, so what i did is modify the pubkey of priv_validator.json, but the value is

EgkEmDLdrPuhbQGWSKoHc9fLnvGbmpyDlQwHplbavHE=

Because the value like cosmosvalconspub1zcjduepqe9jexppzpyvg86unlu8d39xqa3xwwtn6allu5zdsp33kqc8tdnus7rmf78 is invalid, the error info after restart tendermint is:

Error reading PrivValidator from /root/.tendermint/config/priv_validator.json: illegal base64 data at input byte 80

then i encrypt it with Base64 encoding, the error info is:

Error reading PrivValidator from /root/.tendermint/config/priv_validator.json: decodeReflectJSONArray: byte-length mismatch, got 83 want 32

Maybe my understanding is wrong?

What should i do for using pubkey like cosmosvalconspub1zcjduepqe9jexppzpyvg86unlu8d39xqa3xwwtn6allu5zdsp33kqc8tdnus7rmf78
in tendermint instead of gaia?


#40

There are a couple methods to print out the public keys which are enrolled in tmkms:

  1. tmkms yubihsm keys list will list all accessible, valid Ed25519 cosmosvalconspub keys, but only for the YubiHSM. Note that the YubiHSM will be one of the only two supported validator key storage methods for the Cosmos mainnet launch, the other being the forthcoming Ledger app. The softsign API is for testing-only, and will not be supported for production validators. If you have already purchased a YubiHSM, I would suggest using it instead of attempting to use softsign.
  2. Running tmkms in verbose mode with tmkms start -v will print all public keys for all keys configured in the tmkms keyring including softsign keys. This is probably the easiest way to figure out which public key you should be using.

For testnet purposes including Game of Stakes, regardless of how the private key is stored in tmkms, you will need to use one of the two methods above to determine the cosmosvalconspub address for your validator’s signing key. Pass that public key as the --pubkey parameter when you run gaiacli tx stake create-validator (more info here).

As discussed above, there are some bugs/pitfalls with gaiad right now printing the wrong public keys (with e.g. tendermint show_validator). To avoid those, ensure you’re using tmkms v0.2.2+ and take the cosmosvalconspub straight from tmkms.


#41

Thanks for your patience.
Now the main problem with me is how to create validator with designated pubkey, or, how to using the command you gived in local single-node.

$ cd $HOME
$ gaiad init --chain-id testing --moniker testing
$ gaiacli keys add validator
$ gaiad add-genesis-account $(gaiacli keys show validator -a) 1000STAKE,1000validatorToken
$ gaiad gentx --name validator
$ gaiad collect-gentxs

Then error “ERROR: ABCIQuery: Post http://localhost:26657: dial tcp 127.0.0.1:26657: connect: connection refused” occured when execute

$ gaiacli tx stake create-validator --pubkey=cosmosvalconspub1zcjduepqe9jexppzpyvg86unlu8d39xqa3xwwtn6allu5zdsp33kqc8tdnus7rmf78 […]

But if execute “gaiad start” first, error “ERROR: Msg 0 failed: {“codespace”:“STAKE”,“code”:101,“message”:“validator already exist for this operator address, must use new validator operator address”}” occured when execute

$ gaiacli tx stake create-validator --pubkey=cosmosvalconspub1zcjduepqe9jexppzpyvg86unlu8d39xqa3xwwtn6allu5zdsp33kqc8tdnus7rmf78 […]

Which of the above commands create validator by using my operator address? What is the right time to create my validator with designated pubkey?

It will be better if you can show me the shell command.

Thanks for your patience again and sorry for my stupidity :slight_smile:


#42

Solved the problem now! with these command:

$ gaiad init --chain-id testing --moniker testing
$ gaiad add-genesis-account $(gaiacli keys show validator -a) 1000STAKE,1000validatorToken
$ gaiad gentx \
$   --amount 100STAKE \
$   --commission-rate="0.10" \
$   --commission-max-rate="0.20" \
$   --commission-max-change-rate="0.01" \
$   --pubkey cosmosvalconspub1zcjduepqe9jexppzpyvg86unlu8d39xqa3xwwtn6allu5zdsp33kqc8tdnus7rmf78 \
$   --name validator
$ gaiad collect-gentxs
$ gaiad start

then modify priv_validator_laddr of config.toml for connecting kms, and signing service works good!

Thanks!!!

Edit:
So i think the tendermint/tmkms is only suit for gaia, not for tendermint with kvstore, is it right?


#43

The long-term plan for tmkms is for it to be generally useful for any Tendermint application, not just Cosmos, which is why it’s called “Tendermint KMS” and located under the tendermint GitHub organization.

However, for practical reasons we’re focusing on Cosmos right now, which is why the description says “Key Management System for Cosmos Validators”. It’s a bit confusing, I understand.

If you’re interested in using tmkms in a non-Cosmos Tendermint application, that’s a great thing to open a GitHub issue about.