Monitoring/Alerting for your Validator


#21

You mean the Grafana dashboard? It should work with any network your node is running, as long as the connection to Prometheus doesn’t get changed. So no need to update it :slight_smile: I haven’t tried it in 7004 yet, though.


#22

We have updated it and I’m testing with it now. I keep receiving absent validator notification if our validator node didn’t send vote to a certain height. You can add the bot and subscribe to your validator address to try.


#23

Where is prometheus.yml?


#24

Thanks for your work on this! I’m trying to configure the dashboard to monitor one of my validators. Could you please provide some guidance on the required settings for the Prometheus data source, show in the screenshot?


#25

The URL in the HTTP section needs to be configured to the HTTP API URL of the Prometheus server that is scraping your validator input.

The Grafana documentation on this topic might be helpful as well.


#26

Can you share your prometheus.yml so that we can understand the setup better


#27

hey, try switchin in data sources in grafana from server to browser.


#28

Because I’ve been asked this a lot, I provided a smat step-by-step instruction to setup Grafana with my dashboard. Hope this works and will help people to get started.

  1. Step: Install Grafana (http://docs.grafana.org/installation/debian/) & start it

  2. Step: In .gaiad/config.toml set prometheus=true

  3. Step: Restart gaiad to apply config changes

  4. Step: Download prometheus (https://prometheus.io/docs/introduction/first_steps/), edit prometheus.yml
    Add the following:

       # COSMOS MONITORING
       # The job name is added as a label `job=<job_name>` to any timeseries scraped$
       - job_name: 'cosmops'
    
       	# metrics_path defaults to '/metrics'
       	# scheme defaults to 'http'.
    
     	static_configs:
     	- targets: ['localhost:26660']
     		labels:
     			group: 'cosmops'
    
  5. Step: start prometheus with: ./prometheus --config.file=prometheus.yml

  6. Step: Open Grafana in Browser & Do initial Setup

  7. Step: Under Configuration -> Data Source -> Add a new Data Source

Name: CosmosDataSource
Type: Prometheus
URL: http://localhost:9090
Scrape Interval: 5s
Rest is Default

-> Save&Test should add DataSource

  1. Step: In Grafana goto Dashboard -> Import
  2. Step: Paste 7044 (this is my Dashboard template for Grafana), Choose “CosmosDataSource” as Data Source
  3. Step: You should now have a working Dashboard :slight_smile: