访问https时为何会出现x509 certificate signed by unknown authority

今天排查了一个HTTPS证书的问题, 虽然很快的就解决了, 但里面涉及到的东西学是蛮多啊的，学习一下

问题是这样, 一个运行在容器中的服务给一个https地址发送POST请求时提示x509: certificate signed by unknown authority

大家都知道, 一般https都需要通过ca认证，问题很显然, 证书认证不过, 但是为何会出这个问题呢?

要回答这个问题, 自然就引出另一个问题:当我们访问https的问题, 一般不会带上跟证书相关的参数，那又是如何验证网站身份的呢

这里使用curl来模拟，效果一样

curl

在ubuntu上16.04， 内核4.4.0-130-generic上运行curl

curl https://baidu.com -v
# 输出
* Rebuilt URL to: https://baidu.com/
*   Trying 39.156.69.79...
* Connected to baidu.com (39.156.69.79) port 443 (#0)
* found 148 certificates in /etc/ssl/certs/ca-certificates.crt
* found 592 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* 	 server certificate verification OK
* 	 server certificate status verification SKIPPED
* 	 common name: www.baidu.cn (matched)
* 	 server certificate expiration date OK
* 	 server certificate activation date OK
* 	 certificate public key: RSA
* 	 certificate version: #3
* 	 subject: C=CN,ST=Beijing,O=BeiJing Baidu Netcom Science Technology Co.\, Ltd,OU=service operation department,CN=www.baidu.cn
* 	 start date: Thu, 27 Feb 2020 00:00:00 GMT
* 	 expire date: Fri, 26 Feb 2021 12:00:00 GMT
* 	 issuer: C=US,O=DigiCert Inc,CN=DigiCert SHA2 Secure Server CA
* 	 compression: NULL
* ALPN, server accepted to use http/1.1
# ....

关注重点信息:

* found 148 certificates in /etc/ssl/certs/ca-certificates.crt
* found 592 certificates in /etc/ssl/certs

/etc/ssl

从上面可以看出, 在/etc/ssl中发现大量的certificates, 可以来看看/etc/ssl,

这个目录在使用命令apt install ca-certificates后生成

/etc/ssl 该目录下只有certs openssl.cnf private

这几个文件,有用的为certs目录，这个目录下有大量跟证书相关的pem文件, 其中就包含ca-certificates.crt文件.

pem跟crt都是证书相关的文件，不同的格式罢了，这个不是重点

重点在于,在使用curl的时候，如果不带证书相关的参数,则会引用默认的证书路径(依操作系统不同而不同)

这个默认值怎么来确定呢? 可以确认是curl底层的代码根据环境因素定义的默认值，可以通过strace方式来查看

strace curl https://www.baidu.com |& grep open

从上面可以看到, curl如果不指定ca参数的话,则会到/etc/ssl目录下查找

同时, curl也是支持传递参数来实现https的访问

我们可以通过 man curl来查看几个重要的跟参数相关的参数,写的很详细，这里就不翻译了

--cacert <CA certificate>
              (SSL)  Tells  curl  to use the specified certificate file to verify the peer. The file may contain multiple CA certificates. The cer‐
              tificate(s) must be in PEM format. Normally curl is built to use a default file for this, so this option is typically used  to  alter
              that default file.
              curl  recognizes the environment variable named 'CURL_CA_BUNDLE' if it is set, and uses the given path as a path to a CA cert bundle.
              This option overrides that variable.
              The windows version of curl will automatically look for a CA certs file named ´curl-ca-bundle.crt´, either in the same  directory  as
              curl.exe, or in the Current Working Directory, or in any folder along your PATH.
              If curl is built against the NSS SSL library, the NSS PEM PKCS#11 module (libnsspem.so) needs to be available for this option to work
              properly.
              If this option is used several times, the last one will be used.

--capath <CA certificate directory>
              (SSL) Tells curl to use the specified certificate directory to verify the peer. Multiple paths can be  provided  by  separating  them
              with  ":"  (e.g.   "path1:path2:path3").  The certificates must be in PEM format, and if curl is built against OpenSSL, the directory
              must have been processed using the c_rehash utility supplied with OpenSSL. Using --capath can allow OpenSSL-powered curl to make SSL-
              connections much more efficiently than using --cacert if the --cacert file contains many CA certificates.
              If this option is set, the default capath value will be ignored, and if it is used several times, the last one will be used.

ca-certificates

ca-certificates则是一个包, 用于维护根证书库, 所有的 CA 根证书实际上是由 Mozilla 维护的

可以通过dpkg -L ca-certificates来查看证书相关信息

也可以通过apt-cache show ca-certificates 查看相关信息，当然太多，看不出什么来,就是一堆证书

那么可能有人会问, 一般情况下, 系统安装好之后基本就不会再做操作了，那如何更新根证书呢?

有一个工具update-ca-certificates可以手动更新根证书信息, 可以使用man update-ca-certificates`, 说的非常清楚

因此可以手工执行以下命令来更新根证书列表

update-ca-certificates
# 全输出
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.

Resolve

通过curl可以知道, 访问https的时候默认会到/etc/ssl目录下查找根证书，通过根证书验证对端网站的身份,因为对端的证书一般也是由这些根证书签名我, 因此可以难通过

那么出现开头的问题的原因在于, 使用的镜像本身没有包含/etc/ssl目录，同时在Dockerfile中也没有使用apt install ca-certificates来安装, 因此在所有请求https时都会出现问题.

这里提一下， 在安装curl的时候，默认会安装ca-certificates，实际上这个包由 OpenSSL 安装的

Curl 是通过 OpenSSL 实现客户端 HTTPS 协议的，就是说在 Curl/OpenSSL 平台下，Curl 使用的根证书库都是由 ca-certificates 包处理

curl也是一种上层应用, 对于其它的比如golang程序出现这个问题，原理都是一样的.

修改镜像，问题解决

参考文章:

• http://manpages.ubuntu.com/manpages/bionic/man8/update-ca-certificates.8.html
• https://www.jianshu.com/p/abcee3270e9a
• https://curl.haxx.se/docs/sslcerts.html

转载请注明原作者: 周淑科(https://izsk.me)