• Egress Gateway 的双向 TLS。
    • 生成客户端和服务器证书和密钥
    • 部署 NGINX 服务器
    • 部署容器以测试 NGINX 部署
    • 使用客户端证书重新部署 Egress 网关
    • 为出口流量配置双向 TLS
    • 清理

    Egress Gateway 的双向 TLS。

    配置出口网关示例描述了如何配置 Istio 以通过名为 egress gateway 的专用服务引导出口流量。此示例展示如何配置出口网关以启用到外部服务的流量的双向 TLS。

    要模拟支持 mutual TLS 协议的实际外部服务，首先在 Kubernetes 集群中部署 NGINX 服务器，但在 Istio 服务网格之外运行，即在命名空间中运行没有启用 Istio 的代理注入 sidecar 。接下来，配置出口网关以与外部 NGINX 服务器执行双向 TLS。最后，通过出口网关将流量从网格内的应用程序 pod 引导到网格外的 NGINX 服务器。

    生成客户端和服务器证书和密钥

    • 克隆 https://github.com/nicholasjackson/mtls-go-example 存储库：
    1. $ git clone https://github.com/nicholasjackson/mtls-go-example
    • 进入克隆存储库的目录：
    1. $ cd mtls-go-example
    • 为 nginx.example.com 生成证书。使用以下命令（密码任意指定）：
    1. $ ./generate.sh nginx.example.com <password>

    为所有出现的提示选择 y。

    • 将证书移动到 nginx.example.com 目录：
    1. $ mkdir ../nginx.example.com && mv 1_root 2_intermediate 3_application 4_client ../nginx.example.com
    • 返回上级目录：
    1. $ cd ..

    部署 NGINX 服务器

    • 创建一个命名空间来表示 Istio 网格之外的服务，即 mesh-external。请注意，由于自动注入 sidecar 没有启用，因此 sidecar 代理不会自动注入此命名空间中的 pod。
    1. $ kubectl create namespace mesh-external
    • 创建 Kubernetes Secret 用来保存服务器端证书和 CA 证书。
    1. $ kubectl create -n mesh-external secret tls nginx-server-certs --key nginx.example.com/3_application/private/nginx.example.com.key.pem --cert nginx.example.com/3_application/certs/nginx.example.com.cert.pem
    2. $ kubectl create -n mesh-external secret generic nginx-ca-certs --from-file=nginx.example.com/2_intermediate/certs/ca-chain.cert.pem
    • 为 NGINX 服务器创建配置文件：
    1. $ cat <<EOF > ./nginx.conf
    2. events {
    3. }
    5. http {
    6.   log_format main '$remote_addr - $remote_user [$time_local]  $status '
    7.   '"$request" $body_bytes_sent "$http_referer" '
    8.   '"$http_user_agent" "$http_x_forwarded_for"';
    9.   access_log /var/log/nginx/access.log main;
    10.   error_log  /var/log/nginx/error.log;
    12.   server {
    13.     listen 443 ssl;
    15.     root /usr/share/nginx/html;
    16.     index index.html;
    18.     server_name nginx.example.com;
    19.     ssl_certificate /etc/nginx-server-certs/tls.crt;
    20.     ssl_certificate_key /etc/nginx-server-certs/tls.key;
    21.     ssl_client_certificate /etc/nginx-ca-certs/ca-chain.cert.pem;
    22.     ssl_verify_client on;
    23.   }
    24. }
    25. EOF
    • 创建一个 Kubernetes ConfigMap 用来保存 NGINX 服务器的配置：
    1. $ kubectl create configmap nginx-configmap -n mesh-external --from-file=nginx.conf=./nginx.conf
    • 部署 NGINX 服务器：
    1. $ kubectl apply -f - <<EOF
    2. apiVersion: v1
    3. kind: Service
    4. metadata:
    5.   name: my-nginx
    6.   namespace: mesh-external
    7.   labels:
    8.     run: my-nginx
    9. spec:
    10.   ports:
    11.   - port: 443
    12.     protocol: TCP
    13.   selector:
    14.     run: my-nginx
    15. ---
    16. apiVersion: apps/v1
    17. kind: Deployment
    18. metadata:
    19.   name: my-nginx
    20.   namespace: mesh-external
    21. spec:
    22.   selector:
    23.     matchLabels:
    24.       run: my-nginx
    25.   replicas: 1
    26.   template:
    27.     metadata:
    28.       labels:
    29.         run: my-nginx
    30.     spec:
    31.       containers:
    32.       - name: my-nginx
    33.         image: nginx
    34.         ports:
    35.         - containerPort: 443
    36.         volumeMounts:
    37.         - name: nginx-config
    38.           mountPath: /etc/nginx
    39.           readOnly: true
    40.         - name: nginx-server-certs
    41.           mountPath: /etc/nginx-server-certs
    42.           readOnly: true
    43.         - name: nginx-ca-certs
    44.           mountPath: /etc/nginx-ca-certs
    45.           readOnly: true
    46.       volumes:
    47.       - name: nginx-config
    48.         configMap:
    49.           name: nginx-configmap
    50.       - name: nginx-server-certs
    51.         secret:
    52.           secretName: nginx-server-certs
    53.       - name: nginx-ca-certs
    54.         secret:
    55.           secretName: nginx-ca-certs
    56. EOF
    • 为 nginx.example.com 定义一个 ServiceEntry 和 VirtualService，用来指示 Istio 将指向 nginx.example.com 的流量定向到你的 NGINX 服务器：
    1. $ kubectl apply -f - <<EOF
    2. apiVersion: networking.istio.io/v1alpha3
    3. kind: ServiceEntry
    4. metadata:
    5.   name: nginx
    6. spec:
    7.   hosts:
    8.   - nginx.example.com
    9.   ports:
    10.   - number: 80
    11.     name: http
    12.     protocol: HTTP
    13.   - number: 443
    14.     name: https
    15.     protocol: HTTPS
    16.   resolution: DNS
    17.   endpoints:
    18.   - address: my-nginx.mesh-external.svc.cluster.local
    19.     ports:
    20.       https: 443
    21. ---
    22. apiVersion: networking.istio.io/v1alpha3
    23. kind: VirtualService
    24. metadata:
    25.   name: nginx
    26. spec:
    27.   hosts:
    28.   - nginx.example.com
    29.   tls:
    30.   - match:
    31.     - port: 443
    32.       sni_hosts:
    33.       - nginx.example.com
    34.     route:
    35.     - destination:
    36.         host: nginx.example.com
    37.         port:
    38.           number: 443
    39.       weight: 100
    40. EOF

    部署容器以测试 NGINX 部署

    • 创建 Kubernetes Secret 来保存客户端证书和 CA 证书：
    1. $ kubectl create secret tls nginx-client-certs --key nginx.example.com/4_client/private/nginx.example.com.key.pem --cert nginx.example.com/4_client/certs/nginx.example.com.cert.pem
    2. $ kubectl create secret generic nginx-ca-certs --from-file=nginx.example.com/2_intermediate/certs/ca-chain.cert.pem
    • 使用已安装的客户端和 CA 证书部署 sleep 示例，以测试向 NGINX 服务器发送请求：
    1. $ kubectl apply -f - <<EOF
    2. # Copyright 2017 Istio Authors
    3. #
    4. #   Licensed under the Apache License, Version 2.0 (the "License");
    5. #   you may not use this file except in compliance with the License.
    6. #   You may obtain a copy of the License at
    7. #
    8. #       http://www.apache.org/licenses/LICENSE-2.0
    9. #
    10. #   Unless required by applicable law or agreed to in writing, software
    11. #   distributed under the License is distributed on an "AS IS" BASIS,
    12. #   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    13. #   See the License for the specific language governing permissions and
    14. #   limitations under the License.
    16. ##################################################################################################
    17. # Sleep service
    18. ##################################################################################################
    19. apiVersion: v1
    20. kind: Service
    21. metadata:
    22.   name: sleep
    23.   labels:
    24.     app: sleep
    25. spec:
    26.   ports:
    27.   - port: 80
    28.     name: http
    29.   selector:
    30.     app: sleep
    31. ---
    32. apiVersion: extensions/v1beta1
    33. kind: Deployment
    34. metadata:
    35.   name: sleep
    36. spec:
    37.   replicas: 1
    38.   template:
    39.     metadata:
    40.       labels:
    41.         app: sleep
    42.     spec:
    43.       containers:
    44.       - name: sleep
    45.         image: tutum/curl
    46.         command: ["/bin/sleep","infinity"]
    47.         imagePullPolicy: IfNotPresent
    48.         volumeMounts:
    49.         - name: nginx-client-certs
    50.           mountPath: /etc/nginx-client-certs
    51.           readOnly: true
    52.         - name: nginx-ca-certs
    53.           mountPath: /etc/nginx-ca-certs
    54.           readOnly: true
    55.       volumes:
    56.       - name: nginx-client-certs
    57.         secret:
    58.           secretName: nginx-client-certs
    59.       - name: nginx-ca-certs
    60.         secret:
    61.           secretName: nginx-ca-certs
    62. EOF
    • 定义一个环境变量来保存 sleep pod 的名称：
    1. $ export SOURCE_POD=$(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name})
    • 使用部署的 sleep pod 将请求发送到 NGINX 服务器。由于 nginx.example.com 实际上并不存在，因此 DNS 无法解析它，以下 curl 命令使用 —resolve 选项手动解析主机名。在 –resolve 选项（下面的1.1.1.1）中传递的 IP 值并不重要。可以使用 127.0.0.1 以外的任何值。通常，目标主机名存在 DNS 条目，您不会使用 curl 的 —resolve 选项。
    1. $ kubectl exec -it $SOURCE_POD -c sleep -- curl -v --resolve nginx.example.com:443:1.1.1.1 --cacert /etc/nginx-ca-certs/ca-chain.cert.pem --cert /etc/nginx-client-certs/tls.crt --key /etc/nginx-client-certs/tls.key https://nginx.example.com
    2. ...
    3. Server certificate:
    4.   subject: C=US; ST=Denial; L=Springfield; O=Dis; CN=nginx.example.com
    5.   start date: 2018-08-16 04:31:20 GMT
    6.   expire date: 2019-08-26 04:31:20 GMT
    7.   common name: nginx.example.com (matched)
    8.   issuer: C=US; ST=Denial; O=Dis; CN=nginx.example.com
    9.   SSL certificate verify ok.
    10. > GET / HTTP/1.1
    11. > User-Agent: curl/7.35.0
    12. > Host: nginx.example.com
    13. ...
    14. < HTTP/1.1 200 OK
    16. < Server: nginx/1.15.2
    17. ...
    18. <!DOCTYPE html>
    19. <html>
    20. <head>
    21. <title>Welcome to nginx!</title>
    22. ...
    • 验证服务器是否需要客户端证书：
    1. $ kubectl exec -it $(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name}) -c sleep -- curl -k --resolve nginx.example.com:443:1.1.1.1 https://nginx.example.com
    2. <html>
    3. <head><title>400 No required SSL certificate was sent</title></head>
    4. <body bgcolor="white">
    5. <center><h1>400 Bad Request</h1></center>
    6. <center>No required SSL certificate was sent</center>
    7. <hr><center>nginx/1.15.2</center>
    8. </body>
    9. </html>

    使用客户端证书重新部署 Egress 网关

    • 创建 Kubernetes Secret 以保存客户端证书和 CA 证书。
    1. $ kubectl create -n istio-system secret tls nginx-client-certs --key nginx.example.com/4_client/private/nginx.example.com.key.pem --cert nginx.example.com/4_client/certs/nginx.example.com.cert.pem
    2. $ kubectl create -n istio-system secret generic nginx-ca-certs --from-file=nginx.example.com/2_intermediate/certs/ca-chain.cert.pem
    • 生成 istio-egressgateway deployment，其中包含要从新 Secret 安装的 volume。用生成 istio.yaml 相同的选项：
    1. $ helm template install/kubernetes/helm/istio/ --name istio-egressgateway --namespace istio-system -x charts/gateways/templates/deployment.yaml --set gateways.istio-ingressgateway.enabled=false \
    2. --set gateways.istio-egressgateway.secretVolumes[0].name=egressgateway-certs \
    3. --set gateways.istio-egressgateway.secretVolumes[0].secretName=istio-egressgateway-certs \
    4. --set gateways.istio-egressgateway.secretVolumes[0].mountPath=/etc/istio/egressgateway-certs \
    5. --set gateways.istio-egressgateway.secretVolumes[1].name=egressgateway-ca-certs \
    6. --set gateways.istio-egressgateway.secretVolumes[1].secretName=istio-egressgateway-ca-certs \
    7. --set gateways.istio-egressgateway.secretVolumes[1].mountPath=/etc/istio/egressgateway-ca-certs \
    8. --set gateways.istio-egressgateway.secretVolumes[2].name=nginx-client-certs \
    9. --set gateways.istio-egressgateway.secretVolumes[2].secretName=nginx-client-certs \
    10. --set gateways.istio-egressgateway.secretVolumes[2].mountPath=/etc/nginx-client-certs \
    11. --set gateways.istio-egressgateway.secretVolumes[3].name=nginx-ca-certs \
    12. --set gateways.istio-egressgateway.secretVolumes[3].secretName=nginx-ca-certs \
    13. --set gateways.istio-egressgateway.secretVolumes[3].mountPath=/etc/nginx-ca-certs > \
    14. ./istio-egressgateway.yaml
    • 重新部署 istio-egressgateway：
    1. $ kubectl apply -f ./istio-egressgateway.yaml
    2. deployment "istio-egressgateway" configured
    • 验证密钥和证书是否已成功加载到 istio-egressgateway pod 中：
    1. $ kubectl exec -it -n istio-system $(kubectl -n istio-system get pods -l istio=egressgateway -o jsonpath='{.items[0].metadata.name}') -- ls -al /etc/nginx-client-certs /etc/nginx-ca-certs

    /etc/istio/nginx-client-certs 中应该存在 tls.crt 和 tls.key，而 /etc/istio/nginx-ca-certs 中应该存在 ca-chain.cert.pem 。

    为出口流量配置双向 TLS

    • 给 nginx.example.com 在 443 端口上创建出口 Gateway，以及目的地规则和虚拟服务，以通过出口网关和出口网关将流量引导到外部服务。
    1. $ kubectl apply -f - <<EOF
    2. apiVersion: networking.istio.io/v1alpha3
    3. kind: Gateway
    4. metadata:
    5.   name: istio-egressgateway
    6. spec:
    7.   selector:
    8.     istio: egressgateway
    9.   servers:
    10.   - port:
    11.       number: 443
    12.       name: https
    13.       protocol: HTTPS
    14.     hosts:
    15.     - nginx.example.com
    16.     tls:
    17.       mode: MUTUAL
    18.       serverCertificate: /etc/certs/cert-chain.pem
    19.       privateKey: /etc/certs/key.pem
    20.       caCertificates: /etc/certs/root-cert.pem
    21. ---
    22. apiVersion: networking.istio.io/v1alpha3
    23. kind: DestinationRule
    24. metadata:
    25.   name: egressgateway-for-nginx
    26. spec:
    27.   host: istio-egressgateway.istio-system.svc.cluster.local
    28.   subsets:
    29.   - name: nginx
    30.     trafficPolicy:
    31.       loadBalancer:
    32.         simple: ROUND_ROBIN
    33.       portLevelSettings:
    34.       - port:
    35.           number: 443
    36.         tls:
    37.           mode: ISTIO_MUTUAL
    38.           sni: nginx.example.com
    39. EOF
    • 定义一个 VirtualService 来引导流量通过出口网关，一个 DestinationRule 来执行双向的 TLS 的发起：
    1. $ kubectl apply -f - <<EOF
    2. apiVersion: networking.istio.io/v1alpha3
    3. kind: VirtualService
    4. metadata:
    5.   name: direct-nginx-through-egress-gateway
    6. spec:
    7.   hosts:
    8.   - nginx.example.com
    9.   gateways:
    10.   - istio-egressgateway
    11.   - mesh
    12.   http:
    13.   - match:
    14.     - gateways:
    15.       - mesh
    16.       port: 80
    17.     route:
    18.     - destination:
    19.         host: istio-egressgateway.istio-system.svc.cluster.local
    20.         subset: nginx
    21.         port:
    22.           number: 443
    23.       weight: 100
    24.   - match:
    25.     - gateways:
    26.       - istio-egressgateway
    27.       port: 443
    28.     route:
    29.     - destination:
    30.         host: nginx.example.com
    31.         port:
    32.           number: 443
    33.       weight: 100
    34. ---
    35. apiVersion: networking.istio.io/v1alpha3
    36. kind: DestinationRule
    37. metadata:
    38.   name: originate-mtls-for-nginx
    39. spec:
    40.   host: nginx.example.com
    41.   trafficPolicy:
    42.     loadBalancer:
    43.       simple: ROUND_ROBIN
    44.     portLevelSettings:
    45.     - port:
    46.         number: 443
    47.       tls:
    48.         mode: MUTUAL
    49.         clientCertificate: /etc/nginx-client-certs/tls.crt
    50.         privateKey: /etc/nginx-client-certs/tls.key
    51.         caCertificates: /etc/nginx-ca-certs/ca-chain.cert.pem
    52.         sni: nginx.example.com
    53. EOF
    • 发送 HTTP 请求到 http://nginx.example.com：
    1. $ kubectl exec -it $SOURCE_POD -c sleep -- curl -s --resolve nginx.example.com:80:1.1.1.1 http://nginx.example.com
    2. <!DOCTYPE html>
    3. <html>
    4. <head>
    5. <title>Welcome to nginx!</title>
    6. ...
    • 检查 istio-egressgateway pod 的日志，看看与我们的请求相对应的行。如果 Istio 部署在 istio-system 命名空间中，则打印日志的命令是：
    1. $ kubectl logs -l istio=egressgateway -n istio-system | grep 'nginx.example.com' | grep HTTP

    您应该看到与您的请求相关的行，类似于以下内容：

    1. [2018-08-19T18:20:40.096Z] "GET / HTTP/1.1" 200 - 0 612 7 5 "172.30.146.114" "curl/7.35.0" "b942b587-fac2-9756-8ec6-303561356204" "nginx.example.com" "172.21.72.197:443"

    清理

    • 按照配置出口网关示例的清理部分中的说明执行。

    • 删除创建的 Kubernetes 资源：

    1. $ kubectl delete secret nginx-server-certs nginx-ca-certs -n mesh-external
    2. $ kubectl delete secret nginx-client-certs nginx-ca-certs
    3. $ kubectl delete secret nginx-client-certs nginx-ca-certs -n istio-system
    4. $ kubectl delete configmap nginx-configmap -n mesh-external
    5. $ kubectl delete service my-nginx -n mesh-external
    6. $ kubectl delete deployment my-nginx -n mesh-external
    7. $ kubectl delete namespace mesh-external
    8. $ kubectl delete gateway istio-egressgateway
    9. $ kubectl delete serviceentry nginx
    10. $ kubectl delete virtualservice direct-nginx-through-egress-gateway
    11. $ kubectl delete destinationrule originate-mtls-for-nginx
    12. $ kubectl delete destinationrule egressgateway-for-nginx
    • 删除证书的目录和用于生成它们的存储库：
    1. $ rm -rf nginx.example.com mtls-go-example
    • 删除此示例中使用的生成的配置文件：
    1. $ rm -f ./nginx.conf ./istio-egressgateway.yaml
    • 删除 sleep service 和 deployment：
    1. $ kubectl delete service sleep
    2. $ kubectl delete deployment sleep