Kubernetes

Kubernetes 使用Canary佈署策略, 以Web App為例

Posted by Kubeguts on 2020-10-19

假設目前有一項服務叫做 Public Service，這時若要進行新版本的佈署，可以透過藍綠佈署的方式來進行

Step 1. 建立Blue-Green測試所需的服務

透過docker-compose先建立 nginx-blue 與 nginx-green 這兩個images

docker-compose.yml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
version: '3.7'

services:

  blue:
    container_name: nginx-blue
    image: nginx-blue
    build:
      context: ./blue
      dockerfile: blue.dockerfile
    ports:
     - 8080:80

  green:
    container_name: nginx-green
    image: nginx-green
    build:
      context: ./green
      dockerfile: green.dockerfile
    ports:
     - 8081:80

在有這份docker-compose.yml的資料夾底下，使用 docker-compose build 建立blue app與green app的images

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
$ docker-compose build

Building blue
Step 1/6 : FROM        nginx:alpine
 ---> 6f715d38cfe0
Step 2/6 : LABEL       author="Dan Wahlin"
 ---> Running in f14a42375dd7
Removing intermediate container f14a42375dd7
 ---> c76261eb53e5
Step 3/6 : WORKDIR     /usr/share/nginx/html
 ---> Running in 8108e1b74f33
Removing intermediate container 8108e1b74f33
 ---> 5be492e69a58
Step 4/6 : COPY        . .
 ---> 04a01adb6ad0
Step 5/6 : EXPOSE      80
 ---> Running in 01d9b4572d23
Removing intermediate container 01d9b4572d23
 ---> 73ab0e7b93c2
Step 6/6 : CMD         ["nginx", "-g", "daemon off;"]
 ---> Running in 71b9417682ff
Removing intermediate container 71b9417682ff
 ---> e2d289a6c692

Successfully built e2d289a6c692
Successfully tagged nginx-blue:latest


Building green
Step 1/6 : FROM        nginx:alpine
 ---> 6f715d38cfe0
Step 2/6 : LABEL       author="Dan Wahlin"
 ---> Using cache
 ---> c76261eb53e5
Step 3/6 : WORKDIR     /usr/share/nginx/html
 ---> Using cache
 ---> 5be492e69a58
Step 4/6 : COPY        . .
 ---> f1daf269ddf7
Step 5/6 : EXPOSE      80
 ---> Running in 93bdaa313166
Removing intermediate container 93bdaa313166
 ---> 586a5814cf36
Step 6/6 : CMD         ["nginx", "-g", "daemon off;"]
 ---> Running in ce2847a75363
Removing intermediate container ce2847a75363
 ---> 282afb56ea75

Step 2. 定義Public服務要作切換的Blue-Green 設定檔

Public主要會使用分別定義Service與Deployment的yaml檔案的模板，其中 metadata與 spec/selector/role 分別都設置了 $TARGET_ROLE 供指定是要部署藍色版本或綠色版的服務

nginx.service.yml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
kind: Service
apiVersion: v1
metadata:
  name: nginx-service
  labels:
    app: nginx
    role: $TARGET_ROLE
    env: prod
spec:
  type: LoadBalancer
  selector:
    app: nginx
    role: $TARGET_ROLE
  ports:
    - port: 80
      targetPort: 80

其中 metadata、 spec/selector/role、spec/template/metadata/labels/role 分別都設置了 $TARGET_ROLE 供抽換

在containers中，亦定義了 $IMAGE_VERSION 供指定是要用blue版的image, 或是green版的image

nginx.deployment.yml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment-$TARGET_ROLE
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
      role: $TARGET_ROLE
  template:
    metadata:
      labels:
        app: nginx
        role: $TARGET_ROLE
    spec:
      containers:
      - name: nginx-$TARGET_ROLE
        image: $IMAGE_VERSION
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 80
        readinessProbe:
          httpGet:
            path: /
            port: 80
        livenessProbe:
          httpGet:
            path: /
            port: 80
        resources:
          limits:
            memory: "128Mi" #128 MB
            cpu: "200m" #200 millicpu (.2 cpu or 20% of the cpu)

Blue與Green的測試設定部署檔案

以下定義blue與green的Service設定檔

nginx-blue-test.service

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
kind: Service
apiVersion: v1
metadata:
  name: nginx-blue-test
  labels:
    app: nginx
    role: blue-test
    env: test
spec:
  type: LoadBalancer
  selector:
    app: nginx
    role: blue
  ports:
    - port: 9000
      targetPort: 80

nginx-green-test.service

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
kind: Service
apiVersion: v1
metadata:
  name: nginx-green-test
  labels:
    app: nginx
    role: green-test
    env: test
spec:
  type: LoadBalancer
  selector:
    app: nginx
    role: green
  ports:
    - port: 9001
      targetPort: 80

Step 3. 進行Blue-Green佈署

這份腳本config.sh 會負責解析yaml檔內，若有 $TARGET_ROLE 更換成環境變數 $TARGET_ROLE的值， $IMAGE_VERSION也是一樣

config.sh

1
2
3
sed 's/\$TARGET_ROLE'"/$TARGET_ROLE/g" |
sed 's/\$IMAGE_VERSION'"/$IMAGE_VERSION/g" |
tee

佈署Public版本

先在環境變數下設置 $TARGET_ROLE為blue, 與 $IMAGE_VERSION 為nginx-blue

1
2
$ export TARGET_ROLE=blue
$ export IMAGE_VERSION=nginx-blue

使用 nginx.deployment.yml模板，建立出blue版本的deployment

1
2
3
$ cat nginx.deployment.yml | sh config.sh | kubectl create --save-config -f -

deployment.apps/nginx-deployment-blue created

接著再使用 nginx.service.yml模板，建立出 Public版本的service (使用blue template)

nginx.service.yml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
kind: Service
apiVersion: v1
metadata:
  name: nginx-service
  labels:
    app: nginx
    role: $TARGET_ROLE  -> 置換成blue
    env: prod
spec:
  type: LoadBalancer
  selector:
    app: nginx
    role: $TARGET_ROLE -> 置換成nginx-blue
  ports:
    - port: 80
      targetPort: 80
1
2
3
$ cat nginx.service.yml | sh config.sh | kubectl create --save-config -f -

service/nginx-service created

打開 http://localhost 可看到Public App的畫面

public_app.png

部署Blue版本

創建Blue版本的service

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
kind: Service
apiVersion: v1
metadata:
  name: nginx-blue-test
  labels:
    app: nginx
    role: blue-test
    env: test
spec:
  type: LoadBalancer
  selector:
    app: nginx
    role: blue
  ports:
    - port: 9000     
      targetPort: 80

創建blue-test的service

1
2
3
$ kubectl create -f nginx-blue-test.service.yml

service/nginx-blue-test created

最後在http://localhost:9000 可看到部署Blue App的畫面

blue_app.png

Green版本的佈署

調整環境變數為green

1
2
$ export TARGET_ROLE=green
$ export IMAGE_VERSION=nginx-green

使用deployment模板，部署Green的Deployment環境

1
2
3
$ cat nginx.deployment.yml | sh config.sh | kubectl create --save-config -f -

deployment.apps/nginx-deployment-green created

接著創建Green的Service

1
2
3
$ kubectl create -f nginx-green-test.service.yml --save-config

service/nginx-green-test created

這時就可以在 http://localhost:9001 看到Green App的畫面

green_app.png

Step 4. 執行移轉

若Green App版本若測試完畢，可以將Public環境，移轉至Green APP上

透過更改Public原先設定的 nginx.service.yml檔案，改變Service內 role為green即可

nginx.service.yml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
kind: Service
apiVersion: v1
metadata:
  name: nginx-service
  labels:
    app: nginx
    role: $TARGET_ROLE  -> 置換成green
    env: prod
spec:
  type: LoadBalancer
  selector:
    app: nginx
    role: $TARGET_ROLE -> 置換成nginx-green
  ports:
    - port: 80
      targetPort: 80

使用 kubectl apply -f 套用Green的設定

1
2
3
$ cat nginx.service.yml | sh config.sh | kubectl apply -f -

service/nginx-service configured

這時就可以看到 http:localhost 成功移轉為 Green App了

Step 5. 將舊環境Blue App 移除掉

透過 kubectl delete -f [yaml] 指定deployment 的yaml檔案，即可以將blue app環境下的depoloyment和service給移除掉

1
2
$ kubectl delete -f nginx-blue-test.service.yml 
service "nginx-blue-test" deleted

流程總結

  1. 準備好要移轉的images.
  2. 假設原本已經有運作的環境，叫做Public，先建立Blue Test版本的Deployment，創建出與Public一樣的環境 (Service都是指到原始版本的Pods)
  • 建立Blue Test的版本的用意是，先拷貝出Production環境，以此作為移植，若後續Production移轉過去Green後，若有問題的話還可以在倒回到Blue Test上。
  1. 創建Green Test版本的Deployment，創建新版的環境
  2. Green Test環境測試完畢後，調整Public的service，導引到Green deployment環境下的
    pods
  3. 最後若沒問題的話，可以將Blue Test的deployment環境給移除掉

參考

Pluralsight: Kubernetes for Developers: Deploying Your Code
https://app.pluralsight.com/library/courses/kubernetes-developers-deploying-code/table-of-contents

FEATURED TAGS
Kubernetes
FRIENDS