Existing DNS challenge Websupport not migrated to new naming in v2.11.2 or latest

[sd92-sk]

Checklist

• Have you pulled and found the error with jc21/nginx-proxy-manager:latest docker image?
  • Yes (if upgrading)
• Are you sure you're not using someone else's docker image?
  • Yes
• Have you searched for similar issues (both open and closed)?
  • Yes

Describe the bug

There was version update of DNS challenge provider Websupport.
This has changed the name of the plugin and variables holding the API keys and secret. #3513
But it seems that NPM doesn't adjust existing working configuration so the names match, therefore I cannot move forward with upgrade through 2.11.2 up towards latest version - because Admin UI doesn't work properly. Note that proxy hosts still somehow work - NPM forwards traffic.
I also installed clean 2.11.1 -> setup DNS websupport to get cert -> upgrade 2.11.2 - doesn't work -> upgrade to :latest tag -> still doesn't work.

Accessing admin page directly gives 404 error.
Through set-up proxy host (already done in advance) I get login page, clicking Sign in does nothing - in dev tools/network there's 502 for https://npm.xxxxx.sk/api/tokens.

Websupport DNS challenge was working fine for me in 2.9.19, it still renewed wildcard certificate 26th of November (few days ago).

Nginx Proxy Manager Version

Issue is happening at minimum from 2.11.2 and 2.11.3, also in :latest / 2.12.1

To Reproduce
Steps to reproduce the behavior:

1. Have working DNS challenge via certbot plugin websupport in 2.11.1 or older version
2. Upgrade to 2.11.2 or later
3. In console there should be errors that plugin cannot be found, adminUI won't work properly.

Expected behavior
Upgrading towards :latest or :2.11.2 will still find websupport DNS certbot plugin, admin UI works.

Screenshots

Screenshot shows working DNS challenge from Websupport in 2.11.1.
2nd half shows updated to 2.12.1 (current :latest) and login to admin UI doesn't work.
Forwarding (proxy hosts) are working.

Operating System

Debian 12 amd64 in a VM/VPS in some hosting provider. Using built-in SQlite, no external database.

Additional context

Docker 27.3.1, Portainer for container management/orchestration. Using docker compose/stack in Portainer to run NPM.
Stack was created around May 2022 when 2.9.19 was new. I've found this issue as I planned to upgrade in steps, checking if things still work.

For the upgrade testing I've created 2nd stack with different listening ports, same version 2.9.19, copied everything from "prod" to my testing env.
Then upgraded, started NPM, waited a minute till console is quiet to be sure all processes (perhaps migrations?) are done, and checked if proxy hosts and admin UI are working.
First smaller steps as I've seen some warnings in release notes.
2.9.19 -> 2.9.20 -> 2.9.21 -> 2.9.22 (latest 2.9.x)
Then decided to take latest from 2.10 as probaby more bugs are fixed, so
2.10.4
And then went to 2.11.3 which was too much and websupport plugin cannot be found and NPM has troubles.

What I also tried:
Stop container, go to docker volumes
In npm-letsencrypt/renewals/npm-10.conf i've changed:
-authenticator = dns-websupportsk
+authenticator = dns-websupport
In npm-letsencrypt/credentials/credentials-10 i've changed the plugin prefixes:
-dns_websupportsk_api_key = xxxxxx
+dns_websupport_identifier = xxxxxx
-dns_websupportsk_secret = xxxxxx
+dns_websupport_secret_key = xxxxxxx

But it didn't help, still same error.
I suspect that this data is in the SQlite database, however I didn't tried editing it.

I can change the version in docker compose back to 2.11.1 and NPM works, moving forward to 2.11.2 has these issues again. So I suspect old configuration is not migrated to new naming.

I've just tried also installing fresh 2.11.1 install in a vm, setup DNS challenge with websupport, successfully got certificate, served by NPM. Now stopped container, changed version to 2.11.2.
Now issue appears:

❯ Starting nginx ...
❯ Starting backend ...
[12/1/2024] [11:14:30 PM] [Global   ] › ℹ  info      Using Sqlite: /data/database.sqlite
[12/1/2024] [11:14:32 PM] [Migrate  ] › ℹ  info      Current database version: none
[12/1/2024] [11:14:32 PM] [Global   ] › ⬤  debug     CMD: [ -f '/etc/letsencrypt/credentials/credentials-4' ] || { mkdir -p /etc/letsencrypt/credentials 2> /dev/null; echo 'dns_websupportsk_api_key = xxxxxx
dns_websupportsk_secret = xxxxxxxx
dns_websupportsk_domain = xxxxx.sk' > '/etc/letsencrypt/credentials/credentials-4' && chmod 600 '/etc/letsencrypt/credentials/credentials-4'; }
[12/1/2024] [11:14:32 PM] [Certbot  ] › ✖  error     Item Not Found - websupportsk
[12/1/2024] [11:14:32 PM] [Global   ] › ✖  error     Some plugins failed to install. Please check the logs above
[12/1/2024] [11:14:33 PM] [Migrate  ] › ℹ  info      Current database version: none
[12/1/2024] [11:14:33 PM] [Global   ] › ⬤  debug     CMD: [ -f '/etc/letsencrypt/credentials/credentials-4' ] || { mkdir -p /etc/letsencrypt/credentials 2> /dev/null; echo 'dns_websupportsk_api_key = xxxxxx
dns_websupportsk_secret = xxxxxx
dns_websupportsk_domain = xxxxxxx.sk' > '/etc/letsencrypt/credentials/credentials-4' && chmod 600 '/etc/letsencrypt/credentials/credentials-4'; }
[12/1/2024] [11:14:33 PM] [Certbot  ] › ✖  error     Item Not Found - websupportsk
[12/1/2024] [11:14:33 PM] [Global   ] › ✖  error     Some plugins failed to install. Please check the logs above

Relevant logs from console, the certbot part is repeated every second or so:

===== NPM 2.11.2
❯ Starting nginx ...
❯ Starting backend ...
[12/1/2024] [9:35:50 PM] [Global   ] › ℹ  info      Using Sqlite: /data/database.sqlite
[12/1/2024] [9:35:51 PM] [Migrate  ] › ℹ  info      Current database version: none
[12/1/2024] [9:35:51 PM] [Global   ] › ⬤  debug     CMD: [ -f '/etc/letsencrypt/credentials/credentials-10' ] || { mkdir -p /etc/letsencrypt/credentials 2> /dev/null; echo 'dns_websupportsk_api_key = xxxxxxxxxx
dns_websupportsk_secret = xxxxxxxx
dns_websupportsk_domain = xxxxxx.sk' > '/etc/letsencrypt/credentials/credentials-10' && chmod 600 '/etc/letsencrypt/credentials/credentials-10'; }
[12/1/2024] [9:35:51 PM] [Certbot  ] › ✖  error     Item Not Found - websupportsk
[12/1/2024] [9:35:51 PM] [Global   ] › ✖  error     Some plugins failed to install. Please check the logs above

===== NPM :latest / 2.12.1
nginx: [warn] the "listen ... http2" directive is deprecated, use the "http2" directive instead in /data/nginx/proxy_host/27.conf:14
nginx: [warn] the "listen ... http2" directive is deprecated, use the "http2" directive instead in /data/nginx/proxy_host/26.conf:15
nginx: [warn] protocol options redefined for [::]:443 in /data/nginx/proxy_host/27.conf:15
nginx: [warn] the "listen ... http2" directive is deprecated, use the "http2" directive instead in /data/nginx/proxy_host/8.conf:14
nginx: [warn] the "listen ... http2" directive is deprecated, use the "http2" directive instead in /data/nginx/proxy_host/9.conf:14
[12/1/2024] [9:38:23 PM] [Global   ] › ℹ  info      Using Sqlite: /data/database.sqlite
[12/1/2024] [9:38:24 PM] [Migrate  ] › ℹ  info      Current database version: none
[12/1/2024] [9:38:24 PM] [Global   ] › ⬤  debug     CMD: [ -f '/etc/letsencrypt/credentials/credentials-10' ] || { mkdir -p /etc/letsencrypt/credentials 2> /dev/null; echo 'dns_websupportsk_api_key = xxxxxxxxxx
dns_websupportsk_secret = xxxxxxxx
dns_websupportsk_domain = xxxxxxx.sk' > '/etc/letsencrypt/credentials/credentials-10' && chmod 600 '/etc/letsencrypt/credentials/credentials-10'; }
[12/1/2024] [9:38:24 PM] [Certbot  ] › ✖  error     Item Not Found - websupportsk
[12/1/2024] [9:38:24 PM] [Global   ] › ✖  error     Some plugins failed to install. Please check the logs above CommandError: Some plugins failed to install. Please check the logs above
    at /app/lib/certbot.js:39:14
    at Immediate.<anonymous> (/app/node_modules/batchflow/lib/batchflow.js:80:9)
    at process.processImmediate (node:internal/timers:483:21) {
  previous: undefined,
  code: 1,
  public: false
}
[12/1/2024] [9:38:25 PM] [Migrate  ] › ℹ  info      Current database version: none

[mash2k3]

had same issue, downgraded to v2.11.3. working now

[sd92-sk]

I've managed to workaround this problem in my test environment by editing the database.sqlite and npm-letsencrypt/_data/renewal/npm-4.conf.
Then, v2.12.1 started working.

For anyone curious what I did:
I used linuxserver/sqlitebrowser as i'm just running this docker env in a closed VM and wanted something simple.
Stop NPM, make a backup of the database, copy the database file to the sqlite browser, and open it. Make sure perms+owner are correct on the file, otherwise you won't be able to edit (no apparent error message).
In DB, table "certificate", edit last field in column "meta". It should contain some JSON mentioning websupport and your credentials.
Adjust:

-"dns_provider":"websupportsk"
+"dns_provider":"websupport"
-dns_websupportsk_api_key
+dns_websupport_identifier
-dns_websupportsk_secret
+dns_websupport_secret_key
You may remove this at the end - couldn't find it in plugin's documentation, seems unnecessary now:
\ndns_websupportsk_domain = xxxxxxxx.sk

After changes are done, write the changes to the file, close, copy it back to npm-data volume, start NPM.
This will make NPM work, and install correctly the new dns provider.

[12/10/2024] [12:40:51 AM] [Global   ] › ⬤  debug     CMD: [ -f '/etc/letsencrypt/credentials/credentials-4' ] || { mkdir -p /etc/letsencrypt/credentials 2> /dev/null; echo 'dns_websupport_identifier = xxxxxxxxxx
dns_websupport_secret_key = xxxxxxxx' > '/etc/letsencrypt/credentials/credentials-4' && chmod 600 '/etc/letsencrypt/credentials/credentials-4'; }
[12/10/2024] [12:40:51 AM] [Certbot  ] › ▶  start     Installing websupport...
[12/10/2024] [12:40:51 AM] [Global   ] › ⬤  debug     CMD: . /opt/certbot/bin/activate && pip install --no-cache-dir  certbot-dns-websupport~=2.0.1  && deactivate
[12/10/2024] [12:40:54 AM] [Certbot  ] › ☒  complete  Installed websupport
[12/10/2024] [12:40:54 AM] [Setup    ] › ℹ  info      Added Certbot plugins websupport
...
[12/10/2024] [12:40:55 AM] [SSL      ] › ℹ  info      Let's Encrypt Renewal Timer initialized
[12/10/2024] [12:40:55 AM] [SSL      ] › ℹ  info      Renewing SSL certs expiring within 30 days ...
[12/10/2024] [12:40:55 AM] [IP Ranges] › ℹ  info      IP Ranges Renewal Timer initialized
[12/10/2024] [12:40:55 AM] [Global   ] › ℹ  info      Backend PID 155 listening on port 3000 ...
[12/10/2024] [12:40:55 AM] [SSL      ] › ℹ  info      Completed SSL cert renew process

IDK about auto-renew, but manually renewing in NPM UI ended with internal error, old value is still somewhere:

[12/10/2024] [12:46:43 AM] [Global   ] › ⬤  debug     CMD: certbot renew --force-renewal --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name 'npm-4' --disable-hook-validation --no-random-sleep-on-renew 
[12/10/2024] [12:46:44 AM] [Express  ] › ⚠  warning   Saving debug log to /tmp/letsencrypt-log/letsencrypt.log
Failed to renew certificate npm-4 with error: The requested dns-websupportsk plugin does not appear to be installed

In letsencrypt error log this line confirms which file is problematic:

2024-12-10 00:46:44,389:DEBUG:certbot._internal.display.obj:Notifying user: Processing /etc/letsencrypt/renewal/npm-4.conf

So, in your renewal directory of npm-letsencrypt volume, find the file and adjust the lines in [renewalparams] like so:

-authenticator = dns-websupportsk
+authenticator = dns-websupport
-dns_websupportsk_credentials = 
+dns_websupport_credentials = 
-dns_websupportsk_propagation_seconds
+dns_websupport_propagation_seconds

Afterwards, renewal also worked; and after NPM restart (perhaps my browser was caching?), i'm presented with new certificate from today.

Btw, npm-letsencrypt/_data/credentials/credentials-4
I thought this will get corrected/overwritten after NPM start or renew, but it didn't change.

[Swedie]

Having the same issue as described above. Not really comfortable with editing the database as sd92-sk mentioned. Has there been any update on this issue?

[akavkl]

nope, even I'm looking for it.