blackvoid NGINX proxy manager

Guess this article was one long overdue considering how many Docker containers I run and a decent amount of those are running via reverse proxy.

Considering that I run Docker on Synology NAS, I used the built-in reverse proxy feature for years for several reasons. It was built in, it offered enough options out of the box, and I was lazy to run my separate one.

Well as it turns out, there are some solutions out there that require several features inside your reverse proxy to make them run correctly. This was the main reason why I made the switch to NGINX proxy manager.

Let's see how to run your version in parallel to your Synology NAS one.

Continue reading...

 

[one-eyed-king]

So the container to container communication itself is not the issue. That's good.

As the Nginx log reported before, "connection refused" this Unauthorized is the same error just from the front end app standpoint.

Hmm, this one requires further inspection. Connection refused typicaly takes place before the application layer. Since the general communication works, it might be a missconfiguration in the rp itself? You still might want to share the included snippets.

If it provides no further insights, you can still migrate back to the syno rp.

 

[BobW]

So the container to container communication itself is not the issue. That's good.

Hmm, this one requires further inspection. Connection refused typicaly takes place before the application layer. Since the general communication works, it might be a missconfiguration in the rp itself? You still might want to share the included snippets.

If it provides no further insights, you can still migrate back to the syno rp.

Thanks for helping out!
For now I'll stick to the default RP server of Synology that works fine and serves my needs for now.
You know what they say "don't try to fix what isn't broken"

 

[Shoop]

I tried to find a way to have NPM running on "host" network and changing its default listening pors from 80/443 to anything else but didn't find a way to do it ... It might be possible since I found a version in which the ports which need to be mapped are not the default HTTP/HTTPS ports => jlesage/docker-nginx-proxy-manager. In this image somehow, the ports are 8181 GUI, 8080 HTTP and 4443 HTTPS. After searching for a few hours on the internet I still did not find a way to change NPM default ports...
If I could change the default ports in jc21/nginx-proxy-manager then I could run the container with "host" network mode and not conflict with DSM for ports 80/443 ... and be able to get client real IP for fail2ban.
If anyone know how to achieve this ... I'll be more than pleased to know

 

[one-eyed-king]

check the *.conf files in the /etc/nginx folder and it's subdirectories. The "listen" directive is responsible to bind ports to nginx. Though, there is no guaranty that the files are not replaced by rendered files from npm itself.

 

[Shoop]

check the *.conf files in the /etc/nginx folder and it's subdirectories. The "listen" directive is responsible to bind ports to nginx. Though, there is no guaranty that the files are not replaced by rendered files from npm itself.

Tried all the .conf files I could find but doesn't seem to work...
The one I thought would be most promissing is "/etc/nginx/conf.d/default.conf" which I mapped to a volume but even with this modified ... no luck, nginx still tries to bind ports 80 and 443 on startup.
After modifying this file I checked the 1.conf file genrerated by NPM for my proxy host and I still found in it reference to listen on ports 80 and 443 ...
I can't seem to find a file or an option in the GUI to define these ports that are automagically written in the configuration of my proxy host :/

Edit: also checked all the include paths set in /etc/nginx/nginx.conf and can't find anything relevant...

 

[Shadow]

this tool doesn't support requesting wildcard LE certs itself out-of-the-box it seems..

 

[one-eyed-king]

On a second though, none of the conf's should bind port 80/443 anymore. Otherweise nginx will always bind itself to these ports.

If only Synology's docker wouldn't have broken the Swarm mode. Swarm scoped overlay networks actualy allows to bind single host ports to a container without performing SNAT like port publishing for bridge networks d. With plain docker it is an all or nothing approach: you either use the host interface or you don't.

 

[Rusty]

this tool doesn't support requesting wildcard LE certs itself out-of-the-box it seems..

It should work just fine. What gave you that conclusion?

 

[Shadow]

It should work just fine. What gave you that conclusion?

[5/16/2021] [7:59:41 PM] [Express ] › ⚠ warning Command failed: /opt/certbot/bin/certbot certonly --non-interactive --cert-name "npm-1" --agree-tos --email "[email protected]" --domains "*.vlnet.nl" --authenticator certbot-dns-transip:dns-transip --certbot-dns-transip:dns-transip-credentials "/etc/letsencrypt/credentials/credentials-1"

Saving debug log to /var/log/letsencrypt/letsencrypt.log

Plugin legacy name certbot-dns-transip:dns-transip may be removed in a future version. Please use dns-transip instead.

Plugins selected: Authenticator certbot-dns-transip:dns-transip, Installer None

Performing the following challenges:

dns-01 challenge for vlnet.nl

Cleaning up challenges

Encountered exception during recovery: ValueError: Could not deserialize key data. The data may be in an incorrect format or it may be encrypted with an unsupported algorithm.

An unexpected error occurred:

ValueError: Could not deserialize key data. The data may be in an incorrect format or it may be encrypted with an unsupported algorithm.

Please see the logfiles in /var/log/letsencrypt for more details.

 

[Rusty]

View attachment 3644

This could be a LE DNS configuration problem. Do you have the TXT record in your domain with the name _acme-challenge and value vlnet.nl? Certbot needs to have that in order to use the DNS validation method.

Give that a go.

 

[ilbarone87]

Tried all the .conf files I could find but doesn't seem to work...
The one I thought would be most promissing is "/etc/nginx/conf.d/default.conf" which I mapped to a volume but even with this modified ... no luck, nginx still tries to bind ports 80 and 443 on startup.
After modifying this file I checked the 1.conf file genrerated by NPM for my proxy host and I still found in it reference to listen on ports 80 and 443 ...
I can't seem to find a file or an option in the GUI to define these ports that are automagically written in the configuration of my proxy host :/

Edit: also checked all the include paths set in /etc/nginx/nginx.conf and can't find anything relevant...

Have you tried this?

Forward real remote IP to docker containers · Issue #674 · NginxProxyManager/nginx-proxy-manager

Hi, I am running nginx-proxy-manager version v2.6.1 on Ubuntu with Docker. I use nginx-proxy-manager to forward all incoming web traffic to the corresponding docker container on the host machine. N...

github.com

[automerge]1621253518[/automerge]

So the container to container communication itself is not the issue. That's good.

Hmm, this one requires further inspection. Connection refused typicaly takes place before the application layer. Since the general communication works, it might be a missconfiguration in the rp itself? You still might want to share the included snippets.

If it provides no further insights, you can still migrate back to the syno rp.

Same situation here. I keep trying to understand why the RP it's not working with NPM. Is it possible that the syno certs are not happy to work with NPM? (I'm not that into certs, sorry if it's a stupid assumption).

If anyone has been able to make the RP in NPM working with syno certs could let me know its configuration, please?

 

[Shoop]

If anyone has been able to make the RP in NPM working with syno certs could let me know its configuration, please?

You may try the following trick which consists in adding to your advanced Host proxy config the following :

ssl_certificate /usr/syno/etc/certificate/_archive/<active cert path>/fullchain.pem;
ssl_certificate_key /usr/syno/etc/certificate/_archive/<active cert path>/privkey.pem;

Your active certs should be in the directories with modification dates = creation or renewal date
This should allow to use the certs managed in DSM with NPM

 

[Rusty]

You may try the following trick which consists in adding to your advanced Host proxy config the following :

ssl_certificate /usr/syno/etc/certificate/_archive/<active cert path>/fullchain.pem;
ssl_certificate_key /usr/syno/etc/certificate/_archive/<active cert path>/privkey.pem;

Your active certs should be in the directories with modification dates = creation or renewal date
This should allow to use the certs managed in DSM with NPM

Think this will not work. Npm imports certs to its DB

 

[one-eyed-king]

lets assume for a minute that you created a volume that makes the content of the path available inside the npm container, shouldn't npm inject the proper ssl configuration in the indidual server blocks?

It's worth trying, though I wouldn't be surprised if the rendered configuration items would override the configuration or worst case colide with the manual settings?

 

[Shoop]

lets assume for a minute that you created a volume that makes the content of the path available inside the npm container

Good point ... I was forgetting that the container will not have access to the cert path used in DSM.
One thing I'm not sure of is if you can set a https redirect for a proxy host without specifying a certificate in the "SSL certificate" tab in NPM GUI and then add the ssl_certificate and ssl_certificate_key lines in advanced options
[automerge]1621274283[/automerge]

Have you tried this?

Forward real remote IP to docker containers · Issue #674 · NginxProxyManager/nginx-proxy-manager

Hi, I am running nginx-proxy-manager version v2.6.1 on Ubuntu with Docker. I use nginx-proxy-manager to forward all incoming web traffic to the corresponding docker container on the host machine. N...

github.com

I think it's not the same situation for me.
In the situation described in this issue it's never stated that the docker container can't be running on host network because ports 80 and 443 are already used by something else in the host side. So maybe for him the (easy?) way would have been to run NPM app and db in host network mode and be able to get real IP whitout bothering too much with networking with docker... To many unknown variables
The other thing bothering me in this thread is the fact that one post is asking to modify the code in the config file of the "application" you've proxied itself ... I don't want to have to check and modify anything in Vaultwarden or some other app to make it work in NPM and get the real IPs

 

[one-eyed-king]

Good point ... I was forgetting that the container will not have access to the cert path used in DSM.
One thing I'm not sure of is if you can set a https redirect for a proxy host without specifying a certificate in the "SSL certificate" tab in NPM GUI and then add the ssl_certificate and ssl_certificate_key lines in advanced options

I will leave the second part of the post for someone else who actualy uses npm. Though, you might want to take that idea to a spin drive Best case: it succeeds, worst case: you lost a couple of minutes time.

 

[BobW]

So the container to container communication itself is not the issue. That's good.

Hmm, this one requires further inspection. Connection refused typicaly takes place before the application layer. Since the general communication works, it might be a missconfiguration in the rp itself? You still might want to share the included snippets.

If it provides no further insights, you can still migrate back to the syno rp.

I finally figured out what was causing NPM to not work. It was my DNS server from Synology.

I have split DNS (views) set up for my VPN server. This is done for my users so they can login through the apps with (sub)domains when using the VPN server.
Apparently NPM can't handle spit DNS and only needs outside access to work.

Thus, Synology's Reverse Proxy can handle split DNS and works both internally and externally. So that led me astray, which is why I didn't think of it directly when looking for the problem. Split DNS I set up years ago and haven't looked back because everything just works as it should.

If something doesn't work and it seems too "simple" to me, then eventually I want to know why it doesn't work. So I made another attempt to find out and discovered it when I looked at my DNS server and temporarily disabled the views. It appeared to be working!

But since NPM cannot handle spilt DNS and does not work in an intranet it is of little use to me (at the moment). Too bad because it has some nice features that are missing in Synology's RP.

 

[Rusty]

finally figured out what was causing NPM to not work. It was my DNS server from Synology.

Great find!

This would probably work just fine if NPM was running on the host level, but in bridge it complicates things. Syno Nginx is bare metal, so if you would bring NPM on the same level, I'm sure it would work just as well.

As you said, compromises

 

[BobW]

Great find!

This would probably work just fine if NPM was running on the host level, but in bridge it complicates things. Syno Nginx is bare metal, so if you would bring NPM on the same level, I'm sure it would work just as well.

As you said, compromises

If that’s the case then I could try to install NPM bare metal on a debian VM and see if it works the same-way as syno’s RP. Eventually it doesn’t make a difference where the Reverse Proxy Server is installed (I think) to serve it purpose.

 

[Rusty]

If that’s the case then I could try to install NPM bare metal on a debian VM and see if it works the same-way as syno’s RP. Eventually it doesn’t make a difference where the Reverse Proxy Server is installed (I think) to serve it purpose.

Correct