i have nginx ingress configured into a k8s pod, with the following configuration:
location = /alert-cluster1 {
proxy_pass http://cma-cortex-alertmanager.cortex-aggregator.svc.cluster.local:8080/alertmanager;
proxy_set_header X-Scope-OrgID cluster1;
proxy_pass_request_headers on;
}
location = /alert-cluster2 {
proxy_pass http://cma-cortex-alertmanager.cortex-aggregator.svc.cluster.local:8080/alertmanager;
proxy_set_header X-Scope-OrgID cluster2;
proxy_pass_request_headers on;
}
location ~ /alertmanager {
proxy_pass http://cma-cortex-alertmanager.cortex-aggregator.svc.cluster.local:8080$request_uri;
}
Basically what i need is:
- when calling http://mydns/alert-cluster1, nginx should rewrite to /alertmanager with the header X-Scope-OrgID set to cluster1.
- when calling http://mydns/alert-cluster2, nginx should rewrite to /alertmanager with the header X-Scope-OrgID set to cluster2.
The proxy pass directive point to a k8s service.
When performing a cURL, the header is not set and during the forward to /alertmanager, X-Scope-OrgID header is not set and alertmanager response with no org id.
curl http://mydns/multitenant-cluster1 -vL.
* Connected to ..... port 80 (#0)
> GET /multitenant-cluster1 HTTP/1.1
> Host: mydns
> User-Agent: curl/7.79.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 301 Moved Permanently
< Content-Type: text/html; charset=utf-8
< Content-Length: 49
< Connection: keep-alive
< Server: nginx/1.22.0
< Date: Tue, 19 Jul 2022 10:08:57 GMT
< Location: /alertmanager/
< Vary: Accept-Encoding
< X-Kong-Upstream-Latency: 2
< X-Kong-Proxy-Latency: 0
< Via: kong/2.0.4
<
* Ignoring the response-body
* Connection #0 to host mydns left intact
* Issue another request to this URL: 'http://mydns/alertmanager/'
* Found bundle for host mydns: 0x6000014e40c0 [serially]
* Can not multiplex, even if we wanted to!
* Re-using existing connection! (#0) with host mydns
* Connected to mydns (10.228.41.23) port 80 (#0)
> GET /alertmanager/ HTTP/1.1
> Host: mydns
> User-Agent: curl/7.79.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 401 Unauthorized
< Content-Type: text/plain; charset=utf-8
< Content-Length: 10
< Connection: keep-alive
< Server: nginx/1.22.0
< Date: Tue, 19 Jul 2022 10:08:57 GMT
< Vary: Accept-Encoding
< X-Content-Type-Options: nosniff
< X-Kong-Upstream-Latency: 2
< X-Kong-Proxy-Latency: 0
< Via: kong/2.0.4
<
no org id
* Connection #0 to host mydns left intact
but if i call directly /alertmanager with the header hardcoded into the cURL curl http://mydns/alertmanager --header 'X-Scope-OrgID:cluster1' -L
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<link rel="icon" type="image/x-icon" href="favicon.ico" />
<title>Alertmanager</title>
</head>
<body>
<script>
// If there is no trailing slash at the end of the path in the url,
// add one. This ensures assets like script.js are loaded properly
if (location.pathname.substr(-1) != '/') {
location.pathname = location.pathname + '/';
console.log('added slash');
}
</script>
<script src="script.js"></script>
<script>
var app = Elm.Main.init({
flags: {
production: true,
defaultCreator: localStorage.getItem('defaultCreator'),
groupExpandAll: JSON.parse(localStorage.getItem('groupExpandAll'))
}
});
app.ports.persistDefaultCreator.subscribe(function(name) {
localStorage.setItem('defaultCreator', name);
});
app.ports.persistGroupExpandAll.subscribe(function(expanded) {
localStorage.setItem('groupExpandAll', JSON.stringify(expanded));
});
</script>
</body>
</html>
Am i missing something?
EDIT after some test i found out this
location ~ /alertmanager {
proxy_pass http://cma-cortex-alertmanager.cortex-aggregator.svc.cluster.local:8080$request_uri;
proxy_set_header X-Scope-OrgID cluster2;
}
If i add the header directly into the location alertmanager, the header is set and the system is working fine. It's like that proxy_pass do not pass header during the redirect
/. For example:proxy_pass http://cma-cortex-alertmanager.cortex-aggregator.svc.cluster.local:8080/alertmanager/;