Grafana: Feature request: Accept variables in the grid thresholds levels

Created on 10 Oct 2014  ·  66Comments  ·  Source: grafana/grafana

Hi @torkelo , I'm missing variable thresholds levels per dashboard template

Suppose I'm designing a product dashboard template (apache web server by example) , and I need watch when response time is greater than a threshold.

While the dashboard is the same , we can switch to other instances with variables , but response time ( if it is good or not ) it depends on the backend which real process the ( can be an application server, can be static html or perhaps a LAMP server) , in each case the thresholds will differ a lot.

Also would be a good idea to map this thresholds ( 2 digits) with one of the variables in the template (instance name ) by example.

aredashboartemplating arepanegraph typfeature-request

Most helpful comment

@kmoczydlowski @bogatuadrian @sofixa @golddiga @cattt84
Please stop bumping with +1, you are causing every subscriber a notification (and possibly an e-mail)

Use the reactions at the right of each comment to express your vote.
Thanks 😄

All 66 comments

:+1:

Would love to see this, too :+1:

not sure if this works at all, but maybe if we could create a linked $threshold variable that updates the threshold based on the variable setting, and then the threshold just uses $threshold

+1 that will open a lot of new possibilities in terms of dynamic dashboards

I'd love to have an ability to map a metric to a threshold. I use Grafana to draw link usage graphs for network hardware and it's nice to have a threshold mapped to interface type. At the moment I use IfSpeed SNMP source to get active interface speed and draw it as a non-filled metric, but this way breaks dynamic max/min feature of Grafana.

+1, would be very cool to have this feature. I need to set the threshold by influxdb tags.

Is there any news on this topic?

I also badly need this. Ideally the template variable's value could be the result of a query (not the path name but the value returned by this query)

+1

Looking to set threshold value dynamically based on elasticsearch query.

For using Grafana as PNP4Nagios replacement it would be very helpful to have the possibillity to set warn/crit thresholds and min/max y-axis values based on field values!

Need this too.
For exmaple, i have memory graph and cant swtich $server variable. Servers have different amount of memory and i want to know, how much in GiB (not %) is used, but i don't remember total amount memory on them. So i need a threshold (or, maybe, Y-Max) with $total_mem variable.

+1 here need accept $variable

+1

I'm also eager for this feature, so that I can use a single graph template for multiple devices that have different thresholds.

I would like to have this feature too. What's current status of this?

+1

+1

+1

+1

@kmoczydlowski @bogatuadrian @sofixa @golddiga @cattt84
Please stop bumping with +1, you are causing every subscriber a notification (and possibly an e-mail)

Use the reactions at the right of each comment to express your vote.
Thanks 😄

why can not use templating?

Does dynamic threshold can be implemented to simpleJson data source .

+1

+1

+1

So now we have 2018. What about this feature?

+1

+1 on this.

How has this been open for 4 years? One of our use cases is in monitoring file storage quotas. The threshold is defined as a soft limit and a hard limit, both of which are defined per file share, so setting this manually is impossible as the charts are dynamically created. It seems trivial to let us chose a metric value to pass in as the value to the threshold. These thresholds should also be able to be used for alerting as well.

I was also after this (for the singlestat panel), and have implemented a version that'll hopefully get merged into the third party singlestat-math plugin (first attempt at making a plugin - so may have rough edges).
The PR is here: https://github.com/black-mirror-1/singlestat-math/pull/14
You can pull from my branch if you want to use it now.
All you need to do is add an extra data source, give it a name, and use that name in the thresholds. This plugin also supports n thresholds, which maybe of use to some.

I need this for checking if any job is lagging. As a workaround for now I use a template variable query max(my_prometheus_metric) and set that to a hidden variable, and then I have an extra query for my table view that does abs(my_prometheus_metric - $variable), and add threhold on that column. It's not possible to hide a column that's used for setting row color unfortunately so I too would be happy for this feature.

Any update on this feature?

@debmalyapan53 Sorry, I dropped the ball on this - but I have just put in another PR against the latest release. Hopefully it'll be merged in not too long. The current PR is at: https://github.com/black-mirror-1/singlestat-math/pull/30
Cheers.

Actually I want to implement a text box where user will be able to set threshold value, so that the same could be reflected in the graph.

+1

using the template, we can workaround this, this is my setup:

image

image

Notice that i disabled one server in the last few minutes and the threshold changed

Sadly this do not solve everything, we can't "fill above" and can't have alerts based on this. So while the workaround works for many cases, it doesn't solve all the problems, grafana still needed to be able to use variables/queries to setup the thresholds/alerts

5 years and no news about this feat. request?

I'd like to alert on the Elastic ML results with Grafana. Example dashboard which I'd like to alert on. I have the lower/above threshold in the graph which are the model plot.

image

+1
this is a very important feature. Curios why is it lagging.

This is really a top required feature. Can we have an update on this please?

This is really a top required feature. Can we have an update on this please?

I second this. Please please please....

So basically this is never going to happen despite the fact that lots of people are asking for it here, and in a multitude of other requests, bug reports, and around the web in general over the course of 5 years. Way to take your users input and better your application.

I need this too.

Check https://github.com/grafana/grafana/pull/19102 -- If you can use your variable to generate a value from a query, this could work. "SELECT ${var} FROM xyz"

As a workaround, I do the following. It assumes the second Y-axis is not used (or it has fixed min and max) value.

Exemple:
image

There's how to create it:

  • Add a metric named "down" (or whatever)

    • Use a Prometheus query like this (multiple lines!). see notes below.

      1 - (day_of_week(vector(time())) > bool 0)
      * (day_of_week(vector(time())) < bool 6)
      * (((hour(vector(time()))  >= bool 7 - 2)))
      * (hour(vector(time())) < bool 19 - 2)
  • Add a visualization override for this metric named "down":

    • Line fill: 1

    • Line width: 0

    • Staircase line: true

    • Color: _a light gray, like_ "rgb(167, 167, 167)"

    • Y-axis: 2

    • Z-index: 1 _(in front of other metrics)_

    • Legend: false

    • Hide in tooltip: false

  • Configure the 2nd Y-Axis visualization:

    • min: 0

    • max: 1 (or whatever you want... multiply the metric above accordingly)

Note regarding the query:

  • In the exemple query above, "7-2" is 7h00 in the morning, the "-2" is my time zone and daylight saving offset!
  • The exemple query also handles week-end
  • You are welcome to reply to this comment if you adapt the query for other datasources

Ugh... really need this and it's 2019... this is an old thread

same here..probably never going to happen?

Is it hard to implement or just no one want to do it? If so I would like to try

@EggTronic I had a look for what it would involve, I think it comes down to the Threshold object:
https://github.com/grafana/grafana/blob/4a25ec0d36edcb551e6efd60e665df0fa93cb954/packages/grafana-data/src/types/threshold.ts#L1-L4

I was looking at implementing this for the bar-guage specifically. The threshold(s) are used here:
https://github.com/grafana/grafana/blob/4a25ec0d36edcb551e6efd60e665df0fa93cb954/public/app/plugins/panel/bargauge/BarGaugePanel.tsx#L37

And further down in the same file, there's a replaceVariables function being used, I think its used to evaluate grafana variables:
https://github.com/grafana/grafana/blob/4a25ec0d36edcb551e6efd60e665df0fa93cb954/public/app/plugins/panel/bargauge/BarGaugePanel.tsx#L53-L61

The way I can see it being done is

  1. Modify the Threshold type so value can be a variable name (string) instead.
  2. Add logic to each panel type to call replaceVariables on any Threshold.value that are strings.
  3. Make threshold setting box accept strings (variable names).

I'm not sure if the design is appropriate for the codebase though, Its a bit of a hack.

+1 for the feature request

+1

I could really use this for the Gauge panel. I want to show a gauge for each taglogger writing into my database. Each logger reports its ingest rate and theoretical optimal ingest rate. But each logger needs different thresholds for their respective ingest rates.

+1

Is there a workaround for this kind of functionality (for the SingleStat panel)? Or can the thresholds be updated with a MySQL query?

What the status of these issue? Is this going to be implemented? Or is it to difficult to implement?

I have already asked this but whit no response: #25313

+1

I'm not sure if the design is appropriate for the codebase though, Its a bit of a hack.

@liamdiprose IMHO is appropriate, did you mind to propose a pull request?
A theoretical reason to avoid dynamic legends is that a panel template have to be referred to a uniform data set. Having dynamic legend open to have generic template that can manage different data stream and this could be not correct. IMHO I'm facing a use case where multiple indexes are sharing similar panel layout with different scales and color mapping.
Having dynamic thresholds allow me to manage a single dashboard instead of replicating if for different indexes.

Any feedback @torkelo or any other in grafana lab or core contributors ? some of the feature are managed in https://github.com/grafana/grafana/pull/24763 ?

@luipir Not yet, we need this feature badly at work but it's still on my backlog. :cry:

Grafana 7 seems to be moving in the data-driven direction, so I believe this feature will be added eventually. @ryantxu of GrafanaLabs mentioned that using variables this way were upcoming in a similar issue-thread: https://github.com/grafana/grafana/issues/4929#issuecomment-601958742

I'd love to see every setting box accept variables. Maybe every user-supplied value should be "evaluated": Variable names evaluate to their value, Values evaluate to themselves. Variable evaluation will need the correct replaceVariables function (supplied to the panel as a prop) to work.

It would be nice if replaceVariables was be replaced with eval(varname) -> value, and then replaceVariables can become a utility function that takes eval as a parameter.

+1

+1

+1

+1

This would be an extremely useful feature when charting out stocks. For example, I can have thresholds based on a stock symbol's open price without charting it as a metric. The issue that I have ran into charting it as a metric is that it will flatten/normalize the graph.

Without charting open price:
image

After charting open price:
image

Setting the open price as a threshold (blue line):
image

+1

+1, this has been a long requested feature now and seems fairly 'standard'. Looking forward to a future update with this functionality!

+1

Instead of the boring +1, I am putting a (giant) 🍺 as a bounty for the one bringing this feature to reality.

Usecase example: Trying to build a generic SLO dashboard for all the endpoints of an API, I would like the dashboard to receive SLO conditions via the URL of the dashboard.

I had a similar problem, I have a mysql query, and a dashboard with variables for min and max. Not being able to use the min/max variables in the thresholds was disappointing. But I managed to plot something similar, which at least allowed me to visualise the outliers.

  1. In the query, reference your variables as additional columns.
    select time, data, ${minimum} as data_min, ${maximum} as data_max from table
  2. Add a series override, with regex match on the min and max (e.g. /data_(min|max)/)
  3. Set the override to:

    1. Enable lines (assuming the graph is otherwise points)

    2. Disable points

    3. Use a different colour (or use individual overrides for min/max)

    4. Disable legend

    5. Disable tool tips

If you have a metric column, you will probably want to name your data value as '' (data as '') which leaves the legend/tooltip unadorned.

Of course this is just a visual workaround, you don't get the other benefits of thresholds, so this feature request is still much needed.

Was this page helpful?
0 / 5 - 0 ratings