Gaia-8001 Hotfix

Conversation Validation
1

There was an issue with the v0.24.1 release where the wal (write ahead log) was not being rotated properly which lead to runaway disk usage. We have a fix on the v0.24.2-rc0 tag that will fix the issue with just an instance restart. To upgrade:

$ cd $GOPATH/src/github.com/cosmos/cosmos-sdk

# if there are changes locally (gopkg.lock commonly) you should stash them
$ git stash 
$ git fetch
$ git checkout v0.24.2-rc0
$ make install
$ gaiad version
0.24.2-0bf061b

Note: Running with systemctl I was unable to get the service to shut down with a systemctl stop. I had to stop the service then kill -9 the gaiad process that was running. After that I was able to bring the service back up with systemctl start gaiad

1 Like
2

It looks like this fix might not have gotten to the underlying issue, but the wal can be cleared by instance restart: https://github.com/tendermint/tendermint/issues/2259#issuecomment-415191032

Follow the github issue for updates.

1 Like
4

Expected behavior is for the wal file to grow ~1GB on v0.24.2 and then stabilize

1 Like
5

yes, the hotfix is working well in a new install

6

should be
0.24.2-0bf061b7 right?

7

For me is only working in a new clean install in a validator node.

after the update and made some test, in other nodes, I can’t have this working good.

don’t follow the steps for now imo don’t help. maybe try a new clean install

steps for the hotfix work

For Update the Hotfix upgrade

Stop gaiad

cd go/src/github.com/cosmos/cosmos-sdk
git fetch --tags
git checkout v0.24.2-rc0
make get_vendor_deps
make install
start gaiad

then for the cs.wal problem need …

du -h ~/.gaiad/
24G /home/cosmosvlondon/.gaiad/data/cs.wal
429M /home/cosmosvlondon/.gaiad/data/blockstore.db

Stop gaiad

cd .gaiad/data

rm -r cs.wal/*

du -h ~/.gaiad/
4,0K /home/cosmosvlondon/.gaiad/data/cs.wal
429M /home/cosmosvlondon/.gaiad/data/blockstore.db

cd ../

~/.gaiad$ cp -r data/ $HOME

gaiad unsafe_reset_all

Gaiad start

Wait for sync some blocks almost 11 or 1

Stop gaiad

sudo rm -r data/*

cd

cp -r data/* .gaiad/data

rm -rf data (or not) 

Start gaiad

have to sync since last block you backup and no cs.wal problems. Like

du -h ~/.gaiad/
11M /home/cosmosvlondon/.gaiad/data/cs.wal
436M /home/cosmosvlondon/.gaiad/data/blockstore.db

gaiacli status
{"node_info":{"id":"54e87a7baa9829b9579e1cf4f1d782800715ef5c","listen_addr":"51.11.1111:26656","network":"gaia-8001","version":"0.23.0","channels":"4020212223303800","moniker":"melea-trust-London","other":["amino_version=0.12.0","p2p_version=0.5.0","consensus_version=v1/0.2.2","rpc_version=0.7.0/3","tx_index=on","rpc_addr=tcp://0.0.0.0:26657"]},"sync_info":{"latest_block_hash":"206B91C8A6BE52580F20210CBEF8C59C8B17BF98","latest_app_hash":"A81C95B07B56E63C5D71D7E4E73527163634704D","latest_block_height":"15674","latest_block_time":"2018-08-23T14:07:18.185675383Z","catching_up":false},"validator_info":{"address":"4593BFD2D764244E2DE54C33ADE4F39D67C4BFBE","pub_key":{"type":"tendermint/PubKeyEd25519","value":"D8/X3Njt9xBS629sZkpzxWQgJEk7OVXVjRB5pyCdei0="},"voting_power":"0"}}

Now when you see cs.wal files growing up to 1gb, stop and start gaiad again.
Test in validator and two sentrys and works only in validator new clean install.

remember solution: start and stop

19:08meanwhile

19:09but have to be the hotfix

LAST UPDATE
works a new install, not an update
like

go get github.com/cosmos/cosmos-sdk

cd go/src/github.com/cosmos/cosmos-sdk

git fetch --tags

git checkout v0.24.2-rc0

make get_tools

make get_vendor_deps


make install

gaiad init --name melea


vi .gaiad/config/config.toml (add seeds and make your setup)

start gaiad
1 Like
8

Yes its the same.that is for ubuntu

9 
30G	/root/.gaiad/data/cs.wal
31G	/root/.gaiad/data
31G	/root/.gaiad
[root@cosmos-us ~]# gaiad version
0.24.2-0bf061b

runs on Centos 7.2. This bug is not fixed .

1 Like