Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add limes_project_commitment_min_expires_at metric #659

Merged
merged 1 commit into from
Feb 11, 2025
+51 −9
Merged

add limes_project_commitment_min_expires_at metric #659

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 17 additions & 4 deletions docs/operators/metrics.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ The API service exposes the standard HTTP metrics generated by the standard Go c

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One question regarding markdown: My IDE automatically formatted the tables with whitespaces so that the | borders align. I disabled this now, as I saw this only sometimes being done - e.g. not in this file. Would it be desirable to somehow auto-style the markdown?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer this style on the basis of my editor not having automatic alignment like this, so I would just break it when I edit the files again the next time. But not a strong preference from my side.

| Type | Metric | Labels |
| --- | --- | --- |
| Summary | `http_request_duration_microseconds` ||
| Summary | `http_request_size_bytes` ||
| Summary | `http_request_duration_microseconds` |
| Summary | `http_request_size_bytes` |
| Counter | `http_requests_total` | `code`, `method` |
| Summary | `http_response_size_bytes` ||
| Summary | `http_response_size_bytes` |

## Collector service

Expand Down Expand Up @@ -46,11 +46,24 @@ additional metrics:

| Type | Metric | Labels |
| --- | --- | --- |
| Gauge | `limes_autogrow_growth_multiplier` | `service`, `resource` |
| Gauge | `limes_autogrow_quota_overcommit_threshold_percent` | `service`, `resource` |
| Gauge | `limes_available_commitment_duration` | `service`, `resource` |
| Gauge | `limes_cluster_capacity` | `service`, `resource` |
| Gauge | `limes_cluster_capacity_per_az` | `service`, `resource`, `availabilityZone` |
| Gauge | `limes_cluster_usage_per_az` | `service`, `resource`, `availabilityZone` |
| Gauge | `limes_domain_quota` | `service`, `resource`, `domain`, `domain_id` |
| Gauge | `limes_project_backendquota` | `service`, `resource`, `domain`, `domain_id`, `project`, `project_id` |
| Gauge | `limes_project_quota` | `service`, `resource`, `domain`, `domain_id`, `project`, `project_id` |
| Gauge | `limes_project_commitment_min_expires_at` | `service`, `resource`, `domain`, `domain_id`, `project`, `project_id` |
| Gauge | `limes_project_committed_per_az` | `service`, `resource`, `domain`, `domain_id`, `project`, `project_id`, `availabilityZone`, `state` |
| Gauge | `limes_project_override_quota_from_config` | `service`, `resource`, `domain`, `domain_id`, `project`, `project_id` |
| Gauge | `limes_project_physical_usage` | `service`, `resource`, `domain`, `domain_id`, `project`, `project_id` |
| Gauge | `limes_project_quota` | `service`, `resource`, `domain`, `domain_id`, `project`, `project_id`, |
| Gauge | `limes_project_rate_usage` | `service`, `rate`, `domain`, `domain_id`, `project`, `project_id` |
| Gauge | `limes_project_usage` | `service`, `resource`, `domain`, `domain_id`, `project`, `project_id` |
| Gauge | `limes_project_usage_per_az` | `service`, `resource`, `domain`, `domain_id`, `project`, `project_id`, `availabilityZone` |
| Gauge | `limes_project_used_and_or_committed_per_az` | `service`, `resource`, `domain`, `domain_id`, `project`, `project_id`, `availabilityZone` |
| Gauge | `limes_unit_multiplier` | `service`, `resource` |

### Quota/capacity plugins

Expand Down
8 changes: 8 additions & 0 deletions internal/collector/fixtures/scrape_data_metrics.prom
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,14 @@ limes_project_backendquota{domain="germany",domain_id="uuid-for-germany",project
limes_project_backendquota{domain="germany",domain_id="uuid-for-germany",project="berlin",project_id="uuid-for-berlin",resource="things",service="unittest",service_name="generic-unittest"} 13
limes_project_backendquota{domain="germany",domain_id="uuid-for-germany",project="dresden",project_id="uuid-for-dresden",resource="capacity",service="unittest",service_name="generic-unittest"} 20
limes_project_backendquota{domain="germany",domain_id="uuid-for-germany",project="dresden",project_id="uuid-for-dresden",resource="things",service="unittest",service_name="generic-unittest"} 13
# HELP limes_project_commitment_min_expires_at Minimum expiredAt timestamp of all commitments for an Openstack project, grouped by resource and service.
# TYPE limes_project_commitment_min_expires_at gauge
limes_project_commitment_min_expires_at{domain="germany",domain_id="uuid-for-germany",project="berlin",project_id="uuid-for-berlin",resource="capacity",service="unittest",service_name="generic-unittest"} 3.154507e+07
limes_project_commitment_min_expires_at{domain="germany",domain_id="uuid-for-germany",project="berlin",project_id="uuid-for-berlin",resource="capacity_portion",service="unittest",service_name="generic-unittest"} 0
limes_project_commitment_min_expires_at{domain="germany",domain_id="uuid-for-germany",project="berlin",project_id="uuid-for-berlin",resource="things",service="unittest",service_name="generic-unittest"} 0
limes_project_commitment_min_expires_at{domain="germany",domain_id="uuid-for-germany",project="dresden",project_id="uuid-for-dresden",resource="capacity",service="unittest",service_name="generic-unittest"} 3.154507e+07
limes_project_commitment_min_expires_at{domain="germany",domain_id="uuid-for-germany",project="dresden",project_id="uuid-for-dresden",resource="capacity_portion",service="unittest",service_name="generic-unittest"} 0
limes_project_commitment_min_expires_at{domain="germany",domain_id="uuid-for-germany",project="dresden",project_id="uuid-for-dresden",resource="things",service="unittest",service_name="generic-unittest"} 0
# HELP limes_project_committed_per_az Sum of all active commitments of a Limes resource for an OpenStack project, grouped by availability zone and state.
# TYPE limes_project_committed_per_az gauge
limes_project_committed_per_az{availability_zone="az-one",domain="germany",domain_id="uuid-for-germany",project="berlin",project_id="uuid-for-berlin",resource="capacity",service="unittest",service_name="generic-unittest",state="active"} 15
Expand Down
4 changes: 4 additions & 0 deletions internal/collector/fixtures/scrape_data_metrics_skipzero.prom
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,10 @@ limes_project_backendquota{domain="germany",domain_id="uuid-for-germany",project
limes_project_backendquota{domain="germany",domain_id="uuid-for-germany",project="berlin",project_id="uuid-for-berlin",resource="things",service="unittest",service_name="generic-unittest"} 13
limes_project_backendquota{domain="germany",domain_id="uuid-for-germany",project="dresden",project_id="uuid-for-dresden",resource="capacity",service="unittest",service_name="generic-unittest"} 20
limes_project_backendquota{domain="germany",domain_id="uuid-for-germany",project="dresden",project_id="uuid-for-dresden",resource="things",service="unittest",service_name="generic-unittest"} 13
# HELP limes_project_commitment_min_expires_at Minimum expiredAt timestamp of all commitments for an Openstack project, grouped by resource and service.
# TYPE limes_project_commitment_min_expires_at gauge
limes_project_commitment_min_expires_at{domain="germany",domain_id="uuid-for-germany",project="berlin",project_id="uuid-for-berlin",resource="capacity",service="unittest",service_name="generic-unittest"} 3.154507e+07
limes_project_commitment_min_expires_at{domain="germany",domain_id="uuid-for-germany",project="dresden",project_id="uuid-for-dresden",resource="capacity",service="unittest",service_name="generic-unittest"} 3.154507e+07
# HELP limes_project_committed_per_az Sum of all active commitments of a Limes resource for an OpenStack project, grouped by availability zone and state.
# TYPE limes_project_committed_per_az gauge
limes_project_committed_per_az{availability_zone="az-one",domain="germany",domain_id="uuid-for-germany",project="berlin",project_id="uuid-for-berlin",resource="capacity",service="unittest",service_name="generic-unittest",state="active"} 15
Expand Down
27 changes: 22 additions & 5 deletions internal/collector/metrics.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -411,6 +411,7 @@ func (d *DataMetricsReporter) ServeHTTP(w http.ResponseWriter, r *http.Request)
printDataMetrics(bw, metricsBySeries, "limes_cluster_usage_per_az", "Actual usage of a Limes resource for an OpenStack cluster in a specific availability zone.")
printDataMetrics(bw, metricsBySeries, "limes_domain_quota", `Assigned quota of a Limes resource for an OpenStack domain.`)
printDataMetrics(bw, metricsBySeries, "limes_project_backendquota", `Actual quota of a Limes resource for an OpenStack project.`)
printDataMetrics(bw, metricsBySeries, "limes_project_commitment_min_expires_at", `Minimum expiredAt timestamp of all commitments for an Openstack project, grouped by resource and service.`)
printDataMetrics(bw, metricsBySeries, "limes_project_committed_per_az", `Sum of all active commitments of a Limes resource for an OpenStack project, grouped by availability zone and state.`)
printDataMetrics(bw, metricsBySeries, "limes_project_override_quota_from_config", `Quota override for a Limes resource for an OpenStack project, if any. (Value comes from cluster configuration.)`)
printDataMetrics(bw, metricsBySeries, "limes_project_physical_usage", `Actual (physical) usage of a Limes resource for an OpenStack project.`)
Expand Down Expand Up @@ -465,22 +466,33 @@ var domainMetricsQuery = sqlext.SimplifyWhitespace(`
`)

var projectMetricsQuery = sqlext.SimplifyWhitespace(`
WITH project_az_sums AS (
WITH project_sums AS (
SELECT resource_id,
SUM(usage) AS usage,
SUM(COALESCE(physical_usage, usage)) AS physical_usage,
COUNT(physical_usage) > 0 AS has_physical_usage
FROM project_az_resources
GROUP BY resource_id
),
project_commitment_minExpiresAt AS (
SELECT p.domain_id, p.id AS project_id, ps.type, pr.name, MIN(expires_at) AS project_commitment_min_expires_at
FROM projects p
JOIN project_services ps ON ps.project_id = p.id
JOIN project_resources pr ON pr.service_id = ps.id
JOIN project_az_resources par ON par.resource_id = pr.id
JOIN project_commitments pc ON pc.az_resource_id = par.id AND pc.state = 'active'
GROUP BY p.domain_id, p.id, ps.type, pr.name
)
SELECT d.name, d.uuid, p.name, p.uuid, ps.type, pr.name,
pr.quota, pr.backend_quota, pr.override_quota_from_config,
pas.usage, pas.physical_usage, pas.has_physical_usage
psums.usage, psums.physical_usage, psums.has_physical_usage,
pcmea.project_commitment_min_expires_at
FROM domains d
JOIN projects p ON p.domain_id = d.id
JOIN project_services ps ON ps.project_id = p.id
JOIN project_resources pr ON pr.service_id = ps.id
JOIN project_az_sums pas ON pas.resource_id = pr.id
JOIN project_sums psums ON psums.resource_id = pr.id
LEFT JOIN project_commitment_minExpiresAt pcmea ON d.id = pcmea.domain_id AND p.id = pcmea.project_id AND ps.type= pcmea.TYPE AND pr.name = pcmea.name
`)

var projectAZMetricsQuery = sqlext.SimplifyWhitespace(`
Expand Down Expand Up @@ -633,7 +645,7 @@ func (d *DataMetricsReporter) collectMetricsBySeries() (map[string][]dataMetric,
return nil
})
if err != nil {
return nil, fmt.Errorf("during projectMetricsQuery: %w", err)
return nil, fmt.Errorf("during domainMetricsQuery: %w", err)
}

// fetch values for project level (quota/usage)
Expand All @@ -651,9 +663,10 @@ func (d *DataMetricsReporter) collectMetricsBySeries() (map[string][]dataMetric,
usage uint64
physicalUsage uint64
hasPhysicalUsage bool
minExpiresAt *time.Time
)
err := rows.Scan(&domainName, &domainUUID, &projectName, &projectUUID, &dbServiceType, &dbResourceName,
&quota, &backendQuota, &overrideQuotaFromConfig, &usage, &physicalUsage, &hasPhysicalUsage)
&quota, &backendQuota, &overrideQuotaFromConfig, &usage, &physicalUsage, &hasPhysicalUsage, &minExpiresAt)
if err != nil {
return err
}
Expand Down Expand Up @@ -691,6 +704,10 @@ func (d *DataMetricsReporter) collectMetricsBySeries() (map[string][]dataMetric,
result["limes_project_physical_usage"] = append(result["limes_project_physical_usage"], metric)
}
}
if minExpiresAt != nil || d.ReportZeroes {
metric := dataMetric{Labels: labels, Value: timeAsUnixOrZero(minExpiresAt)}
result["limes_project_commitment_min_expires_at"] = append(result["limes_project_commitment_min_expires_at"], metric)
}
return nil
})
if err != nil {
Expand Down
Loading