Error message: "unable to verify the first certificate"
Due to changes at LetsEncrypt the intermediate certificate must be replaced and updated on each node that is using LetsEncrypt certificates (default if installed from the official guide) when it is renewed.
All nodes that have NOT received this error yet, the procedure is quite simple:
Update package cache and install the ca-certificates package
sudo apt-get update && sudo apt-get install ca-certificates -y
USE THE FOLLOWING FOR UBUNTU >=20.04. SKIP THIS, TO THE ENTRY AFTER FOR UBUNTU 18.04.x AND DEBIAN 10/11 NODES USING SNAP CERTBOT.
echo \ "[Unit] Description=zenupdate.service [Service] Type=oneshot ExecStart=/usr/bin/certbot -q renew --deploy-hook \"cat /etc/letsencrypt/live/$FQDN/chain.pem|sudo awk 'split_after==1{n++;split_after=0} /-----END CERTIFICATE-----/ {split_after=1} {print > \"/usr/local/share/ca-certificates/intermediate-cert\" n \".crt\"}' && update-ca-certificates --fresh && systemctl restart zend\" PrivateTmp=true" | sudo tee /lib/systemd/system/zenupdate.service
USE THE FOLLOWING FOR UBUNTU 18.04.x AND DEBIAN 10/11 NODES USING SNAP CERTBOT. DO NOT USE FOR UBUNTU 20.04 NODES.
echo \ "[Unit] Description=zenupdate.service [Service] Type=oneshot ExecStart=/usr/bin/certbot -q renew --preferred-chain \"ISRG Root X1\" --deploy-hook \"cat /etc/letsencrypt/live/$FQDN/chain.pem|sudo awk 'split_after==1{n++;split_after=0} /-----END CERTIFICATE-----/ {split_after=1} {print > \"/usr/local/share/ca-certificates/intermediate-cert\" n \".crt\"}' && update-ca-certificates --fresh && systemctl restart zend\" PrivateTmp=true" | sudo tee /lib/systemd/system/zenupdate.service
Reload certbot certificate renewal service:
sudo systemctl daemon-reload
This means that the next time your certificates renew the intermediate certificate will automatically be also updated and you should have no problems. For the vast majority of node operators this will be the solution.
You can stop here unless you are already receiving the error message "unable to verify the first certificate".
All nodes that are ALREADY receiving the error message "unable to verify the first certificate", please use the following procedure:
Add the new intermediate certificate to the certificate store:
echo \ "-----BEGIN CERTIFICATE----- MIIFFjCCAv6gAwIBAgIRAJErCErPDBinU/bWLiWnX1owDQYJKoZIhvcNAQELBQAw TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMjAwOTA0MDAwMDAw WhcNMjUwOTE1MTYwMDAwWjAyMQswCQYDVQQGEwJVUzEWMBQGA1UEChMNTGV0J3Mg RW5jcnlwdDELMAkGA1UEAxMCUjMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK AoIBAQC7AhUozPaglNMPEuyNVZLD+ILxmaZ6QoinXSaqtSu5xUyxr45r+XXIo9cP R5QUVTVXjJ6oojkZ9YI8QqlObvU7wy7bjcCwXPNZOOftz2nwWgsbvsCUJCWH+jdx sxPnHKzhm+/b5DtFUkWWqcFTzjTIUu61ru2P3mBw4qVUq7ZtDpelQDRrK9O8Zutm NHz6a4uPVymZ+DAXXbpyb/uBxa3Shlg9F8fnCbvxK/eG3MHacV3URuPMrSXBiLxg Z3Vms/EY96Jc5lP/Ooi2R6X/ExjqmAl3P51T+c8B5fWmcBcUr2Ok/5mzk53cU6cG /kiFHaFpriV1uxPMUgP17VGhi9sVAgMBAAGjggEIMIIBBDAOBgNVHQ8BAf8EBAMC AYYwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMBIGA1UdEwEB/wQIMAYB Af8CAQAwHQYDVR0OBBYEFBQusxe3WFbLrlAJQOYfr52LFMLGMB8GA1UdIwQYMBaA FHm0WeZ7tuXkAXOACIjIGlj26ZtuMDIGCCsGAQUFBwEBBCYwJDAiBggrBgEFBQcw AoYWaHR0cDovL3gxLmkubGVuY3Iub3JnLzAnBgNVHR8EIDAeMBygGqAYhhZodHRw Oi8veDEuYy5sZW5jci5vcmcvMCIGA1UdIAQbMBkwCAYGZ4EMAQIBMA0GCysGAQQB gt8TAQEBMA0GCSqGSIb3DQEBCwUAA4ICAQCFyk5HPqP3hUSFvNVneLKYY611TR6W PTNlclQtgaDqw+34IL9fzLdwALduO/ZelN7kIJ+m74uyA+eitRY8kc607TkC53wl ikfmZW4/RvTZ8M6UK+5UzhK8jCdLuMGYL6KvzXGRSgi3yLgjewQtCPkIVz6D2QQz CkcheAmCJ8MqyJu5zlzyZMjAvnnAT45tRAxekrsu94sQ4egdRCnbWSDtY7kh+BIm lJNXoB1lBMEKIq4QDUOXoRgffuDghje1WrG9ML+Hbisq/yFOGwXD9RiX8F6sw6W4 avAuvDszue5L3sz85K+EC4Y/wFVDNvZo4TYXao6Z0f+lQKc0t8DQYzk1OXVu8rp2 yJMC6alLbBfODALZvYH7n7do1AZls4I9d1P4jnkDrQoxB3UqQ9hVl3LEKQ73xF1O yK5GhDDX8oVfGKF5u+decIsH4YaTw7mP3GFxJSqv3+0lUFJoi5Lc5da149p90Ids hCExroL1+7mryIkXPeFM5TgO9r0rvZaBFOvV2z0gp35Z0+L4WPlbuEjN/lxPFin+ HlUjr8gRsI3qfJOQFy/9rKIJR0Y/8Omwt/8oTWgy1mdeHmmjk7j1nYsvC9JSQ6Zv MldlTTKB3zhThV1+XWYp6rjd5JW1zbVWEkLNxE7GJThEUG3szgBVGP7pSWTUTsqX nLRbwHOoq7hHwg== -----END CERTIFICATE-----" | sudo tee /usr/local/share/ca-certificates/New_LE_Int_R3.crt
Update the certificate store:
sudo update-ca-certificates --fresh
Restart zend to update the certificate that it is using:
sudo systemctl restart zend
USE THE FOLLOWING FOR UBUNTU >=20.04. SKIP THIS, TO THE ENTRY AFTER FOR UBUNTU 18.04.x AND DEBIAN 10/11 NODES USING SNAP CERTBOT.
echo \ "[Unit] Description=zenupdate.service [Service] Type=oneshot ExecStart=/usr/bin/certbot -q renew --deploy-hook \"cat /etc/letsencrypt/live/$FQDN/chain.pem|sudo awk 'split_after==1{n++;split_after=0} /-----END CERTIFICATE-----/ {split_after=1} {print > \"/usr/local/share/ca-certificates/intermediate-cert\" n \".crt\"}' && update-ca-certificates --fresh && systemctl restart zend\" PrivateTmp=true" | sudo tee /lib/systemd/system/zenupdate.service
USE THE FOLLOWING FOR UBUNTU 18.04.x AND DEBIAN 10/11 NODES USING SNAP CERTBOT. DO NOT USE FOR UBUNTU 20.04 NODES.
echo \ "[Unit] Description=zenupdate.service [Service] Type=oneshot ExecStart=/usr/bin/certbot -q renew --preferred-chain \"ISRG Root X1\" --deploy-hook \"cat /etc/letsencrypt/live/$FQDN/chain.pem|sudo awk 'split_after==1{n++;split_after=0} /-----END CERTIFICATE-----/ {split_after=1} {print > \"/usr/local/share/ca-certificates/intermediate-cert\" n \".crt\"}' && update-ca-certificates --fresh && systemctl restart zend\" PrivateTmp=true" | sudo tee /lib/systemd/system/zenupdate.service
Reload certbot certificate renewal service:
sudo systemctl daemon-reload
That's it, you're done