不过作为我们DevOps, 不是所有语言都能拿得起放得下(自黑), 在写自动化脚本过程中势必要跟我们的Jenkins本家groovy脚本语言打交道, 这玩意可是直接能够调用Jenkins API的敲门砖, 但是前提你能玩得转他的API doc, 本铲屎官给Jenkins铲屎已经够意思了, 还让我去研究如果优雅的铲屎, 不留一丝痕迹? 

这里多亏了Docker swam和JCasC, 他们组合起来就能把我们可爱的Jenkins小宠宠瞬间打回一只可以被随时替代的小牛, 我们可以不必一天在给Jenkins遛弯的过程, 怕他走丢啦, 怕他肚子吃坏啦, 怕他被人欺负了, 给他打一针JCasC, 扔进Docker Swarm牛圈, 瞬间变成一只可被随时打入冷宫的小牛, 想怎么玩就怎么玩, 玩坏了分分钟扔掉并直接给他再克隆一只, 是不是很神奇?

闲话不多说, 其实宠物和牛对应的就是可变基础设施与不可变基础设施, 具体的基础设施我们可以拿vm与docker做比较.

可变基础设施vs不可变基础设施:

可变基础设施: 服务器会不断更新和修改。使用此类基础架构的工程师和管理员可以通过SSH连接到他们的服务器，手动升级或降级软件包，逐个服务器地调整配置文件，以及将新代码直接部署到现有服务器上。换句话说，这些服务器是可变的; 它们可以在创建后进行更改。

不可变基础设施: 他们部署了服务器之后决不会被修改。如果需要以任何方式更新，修复或修改某些内容，则会根据具有相应更改的公共映像构建新服务器以替换旧服务器。经过验证后，它们就会投入使用，而旧的则会退役。

比起传统的虚拟机基础环境, 使用docker本身就能让我们节省资源的同时快速构建我们的应用, 他的持续集成、版本控制、可移植性、隔离性和安全性能够让我们摆脱传统的可变基础设施(宠物), 让我们不必去在对他创建后进行任何配置更改, 只需要把他当做牛一样, 所有的配置都在牛(镜像)使用前进行版本配置变更, 使用时直接扔到docker swarm, 做到召之即来挥之即去, 降低我们的运维成本.

这里的Jenkins CasC plugin对我们的Jenkins在不可变基础设施中又做了一次二次优化, 他让我们原本需要去在我们的自动化脚本内嵌入groovy语言变成了历史, 我们只需要按照插件的语法格式编写一个human readable的yaml文件并嵌入Jenkins docker启动, 他就能自动化帮我们完成装插件, 配slave节点, 配环境变量, 配CI工具, 配后台管理权限等工作, 真正做到快速版本迭代更新, 节省了我们大量版本更新时花费的人力物力.

OK, Let's rack and roll...

安装环境

system: CentOS 7.4 x64

Docker: 17.12.0-ce

Jenkins docker image: 2.164.1

Docker manager node:  192.168.0.102

Docker worker1 node:   192.168.0.102

Docker worker2 node:   192.168.0.104

一. 系统环境配置

1.关闭iptables和selinux

# su - root

# systemctl stop firewalld

# setenforce 0

# vi /etc/sysconfig/selinux

修改

SELINUX=disabled

2.在三台节点分别添加本地host DNS

# vi /etc/hosts

192.168.0.102  manage
192.168.0.103  worker1
192.168.0.104  worker2

3.更改对应系统主机名
Manager:
# hostnamectl set-hostname manager
Worker1:
# hostnamectl set-hostname worker1
Worker2
# hostnamectl set-hostname worker2

二. 安装Docker Engine

1. 下载yum镜像源

# wget https://download.docker.com/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker.repo

2. 安装docker engine

# yum install docker-ce –y

3.启动docker service 

# systemctl start docker
# systemctl enable docker

4.创建docker manager节点(manager)

# docker swarm init --advertise-addr 192.168.0.102

Swarm initialized: current node (viwovkb0bk0kxlk98r78apopo) is now a manager.

To add a worker to this swarm, run the following command:

    docker swarm join --token SWMTKN-1-3793hvb71g0a6ubkgq8zgk9w99hlusajtmj5aqr3n2wrhzzf8z-    1s38lymnir13hhso1qxt5pqru 192.168.0.102:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

5.加入worker节点到manager节点(worker1, worker2)

# docker swarm join --token SWMTKN-1-3793hvb71g0a6ubkgq8zgk9w99hlusajtmj5aqr3n2wrhzzf8z-1s38lymnir13hhso1qxt5pqru 192.168.0.102:2377

6.查看节点连接信息

# docker node ls

ID                            HOSTNAME           STATUS              AVAILABILITY        MANAGER STATUS
viwovkb0bk0kxlk98r78apopo *   manager            Ready               Active              Leader
yf6nb2er69pydlp6drijdfmwd     worker1            Ready               Active              
yyavdslji7ovmw5fd3v7l62g8     worker2            Ready               Active         

7.在docker swarm下创建http服务

# docker service create -p 80:80 --name webservice --replicas 5 httpd

ID                  NAME                IMAGE               NODE               DESIRED STATE       CURRENT STATE       ERROR           PORTS
xsa5wb0eg2ln        webservice.1        httpd:latest        manager            Running             Running about 10 minutes ago                                       
3sbscs7m9lnh        webservice.2        httpd:latest        worker2            Running             Running about 10 minutes ago                                      
yao2m5wi54kz        webservice.3        httpd:latest        worker2            Running             Running about 10 minutes ago                                 
dfg2mswa52sf        webservice.4        httpd:latest        worker1            Running             Running about 15 seconds ago                                 
kah1j5hs14as        webservice.5        httpd:latest        worker1            Running             Running about 15 seconds ago                                 

8.测试http站点

# curl 192.168.0.102

It works...

三.在docker swarm下构建Jenkins cluster with configuration as code

详见作者的github仓库

https://github.com/showerlee/docker-compose-with-JCasC

大功告成...

traefik 是一款开源的反向代理与负载均衡工具。它最大的优点是能够与常见的微服务系统直接整合，可以实现自动化动态配置。目前支持 Docker, Swarm, Mesos/Marathon, Mesos, Kubernetes, Consul, Etcd, Zookeeper, BoltDB, Rest API 等等后端模型。

由于微服务架构以及 Docker 技术和 kubernetes 编排工具最近几年才开始逐渐流行，所以一开始的反向代理服务器比如 nginx、apache 并未提供其支持，所以才会出现 Ingress Controller 来做 kubernetes 和前端负载均衡器如 nginx 之间做衔接；即 Ingress Controller 的存在就是为了能跟 kubernetes 交互，又能写 nginx 配置，还能 reload 它，这是一种折中方案；而 traefik 天生就是提供了对 kubernetes 的支持，也就是说 traefik 本身就能跟 kubernetes API 交互，感知后端变化，因此可以得知:  traefik 完全已经取代了 Ingress Controller 作为与k8s交互的代理，在此给大家这里整体架构如下:

为什么选择 traefik？

Golang 编写，单文件部署，与系统无关，同时也提供小尺寸 Docker 镜像。
支持 Docker/Etcd 后端，天然连接我们的微服务集群。
内置 Web UI，管理相对方便。
自动配置 ACME(Let’s Encrypt) 证书功能。
性能尚可，我们也没有到压榨 LB 性能的阶段，易用性更重要。

除了这些以外，traefik 还有以下特点：
Restful API 支持。
支持后端健康状态检查，根据状态自动配置。
支持动态加载配置文件和 graceful 重启。
支持 WebSocket 和 HTTP/2。

目前网上很多资料虽然能够成功在k8s下搭建ingress+traefik实现cluster内网的代理功能, 但如何将这个反向代理IP暴露在我们外网, 也就是不依赖于AWS等公有云, 在我们on-prem(本地k8s网络环境)通过定义hostnetwork去实现将我们的node ip直接作为我们在cluster下创建的反向代理ip,  将是我们本次需要给大家介绍的内容.

我们接着上一篇Kubernates1.9+Docker17离线安装部署, 给大家展开如何实现k8s通过使用Ingress+Traefik实现集群反向代理功能.

1.获取Traefik支持K8s仓库脚本

# cd /kube/

# git clone https://github.com/containous/traefik.git

# cd traefik/examples/k8s/

# ll

total 36
-rw-r--r-- 1 root root  140 Aug 26 04:23 cheese-default-ingress.yaml
-rw-r--r-- 1 root root 1805 Aug 26 04:23 cheese-deployments.yaml
-rw-r--r-- 1 root root  519 Aug 26 04:23 cheese-ingress.yaml
-rw-r--r-- 1 root root  509 Aug 26 04:23 cheese-services.yaml
-rw-r--r-- 1 root root  504 Aug 26 04:23 cheeses-ingress.yaml
-rw-r--r-- 1 root root 1144 Aug 26 06:40 traefik-deployment.yaml
-rw-r--r-- 1 root root 1206 Aug 26 04:23 traefik-ds.yaml
-rw-r--r-- 1 root root  694 Aug 26 04:23 traefik-rbac.yaml
-rw-r--r-- 1 root root  466 Aug 26 04:40 ui.yaml

这个目录下就是示例 Traefik 启动所需要的 yaml 文件，Traefik 提供了适配各个类型服务编排的部署方式，kubernetes 启动方式支持 Deployment 和 DaemonSet，二选一都可以, 这里笔者选择Deployment

2. Traefik脚本配置

还记得我之前提到的hostnetwork吗? 

这里我们需要在部署之前将我们的traefik-deployment.yaml下添加一行 hostNetwork: true

这样子就能保证我们的Traefik反向代理的会将自己Endpoints IP配置成我们的NodeIP, 供我们与K8S同一网段的设备访问.

我们可以这么配置:

# vi traefik-deployment.yaml

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
  labels:
    k8s-app: traefik-ingress-lb
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: traefik-ingress-lb
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      hostNetwork: true
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      containers:
      - image: traefik
        name: traefik-ingress-lb
        ports:
        - name: http
          containerPort: 80
        - name: admin
          containerPort: 8080
        args:
        - --api
        - --kubernetes
        - --logLevel=INFO
---
kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
    - protocol: TCP
      port: 80
      name: web
    - protocol: TCP
      port: 8080
      name: admin
  type: NodePort

# vi traefik-rbac.yaml

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
  name: traefik-ingress-controller
  namespace: kube-system

这样子我们就完成了K8S脚本的初始修改, 接下来我们就可以直接开始我们的Traefik安装

3. Traefik安装

# kubectl apply -f traefik-rbac.yaml

clusterrole "traefik-ingress-controller" created
clusterrolebinding "traefik-ingress-controller" created

# kubectl apply -f traefik-deployment.yaml

serviceaccount "traefik-ingress-controller" created
deployment "traefik-ingress-controller" created
service "traefik-ingress-service" created

# kubectl apply -f ui.yaml

service "traefik-web-ui" created
ingress "traefik-web-ui" created

# kubectl get pods --all-namespaces -o wide

NAME                                          READY     STATUS    RESTARTS   AGE       IP             NODE
…
traefik-ingress-controller-795ffb7d78-ppw94   1/1       Running   0          5m        10.110.16.15   kube-node1

可以看到我们的traefik-ingress pods成功的打开了我们的hostnetwork, 这里显示的是我们的node ip而不是pod内网 ip, 也就是我们可以直接通过访问node ip 10.110.16.15来连接traefik反向代理, 利用traefik来分配我们的K8S的容器连接.

# kubectl get service --all-namespaces

NAMESPACE      NAME                        TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                       AGE
default        frontend                    ClusterIP   10.110.87.1      <none>        80/TCP                        4h
default        kubernetes                  ClusterIP   10.96.0.1        <none>        443/TCP                       1d
default        my-nginx                    ClusterIP   10.102.200.83    <none>        80/TCP                        4h
kube-system    kube-dns                    ClusterIP   10.96.0.10       <none>        53/UDP,53/TCP                 1d
kube-system    kubernetes-dashboard        NodePort    10.108.187.244   <none>        443:32666/TCP                 1d
kube-system    tiller-deploy               ClusterIP   10.111.251.3     <none>        44134/TCP                     1d
kube-system    traefik-ingress-service     NodePort    10.100.70.138    <none>        80:31087/TCP,8080:31017/TCP   13m
kube-system    traefik-web-ui              ClusterIP   10.96.54.167     <none>        80/TCP                        13m
newegg-nginx   newegg-nginx-newegg-nginx   NodePort    10.97.200.124    <none>        80:32239/TCP                  8h

我们可以通过两种方式去访问我们的traefik后台管理员界面.

1).通过Node Port方式

我们可以在与K8S node同网段的机器去访问其Node暴露的端口31017

2).直接通过ui.yaml配置traefik+ingress反向代理, 这样我们就可以通过host域名直接访问后台

# vi ui.yaml

---
apiVersion: v1
kind: Service
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - name: web
    port: 80
    targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  rules:
  - host: traefik-ui.k8s
    http:
      paths:
      - path: /
        backend:
          serviceName: traefik-web-ui
          servicePort: web

在本地MacOS系统添加traefik-ui.k8s解析到10.110.16.15

# echo "10.110.16.15 traefik-ui.k8s" >> /etc/hosts

我们可以同样利用traefik反向代理, 通过http域名的方式访问admin后台.

4.部署两个服务nginx1-7和nginx1-8，配置Traefik去负载这两个服务：

配置一个deployment类型的1.7版本nginx实例, 并利用service开启其内部80端口监听.

# vi nginx1-7.yaml

apiVersion: v1
kind: Service
metadata:
  name: frontend
spec:
  ports:
    - port: 80
      targetPort: 80
  selector:
    app: nginx1-7
---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: nginx1-7-deployment
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: nginx1-7
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80

配置一个deployment类型的1.8版本nginx实例, 并利用service开启其内部80端口监听.

# vi nginx1-8.yaml

apiVersion: v1
kind: Service
metadata:
  name: my-nginx
spec:
  ports:
    - port: 80
      targetPort: 80
  selector:
    app: nginx1-8
---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: nginx1-8-deployment
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: nginx1-8
    spec:
      containers:
      - name: nginx
        image: nginx:1.8
        ports:
        - containerPort: 80

通过利用Ingress绑定1.7与1.8nginx实例下的serviceName, 给其定义不同的域名, 实现traefik反向代理.

# vi traefik.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-ingress
  namespace: default
spec:
  rules:
  - host: traefik.nginx.io
    http:
      paths:
      - path: /
        backend:
          serviceName: my-nginx
          servicePort: 80
  - host: traefik.frontend.io
    http:
      paths:
      - path: /
        backend:
          serviceName: frontend
          servicePort: 80

我们这里通过配置MacOS本地Host DNS, 将所有的DNS记录指向我们暴露在外网的traefik IP, 从而让traefik作为代理接管所有访问K8S内部实例的连接, 实际的原理和我们之前用过的apache, nginx基本一致.

# echo "10.110.16.15 traefik.nginx.io traefik.frontend.io" >> /etc/hosts

这样子我们就可以直接在MacOS浏览器下访问traefik.nginx.io与traefik.frontend.io从而实现traefik反向代理.

5.配置HTTPS访问traefik管理员后台

1).创建并配置https证书并保存到k8s secret下.

# mkdir -p /opt/k8s/ssl

# cd /opt/k8s/ssl

# openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=traefik-ui.k8s"

# kubectl create secret generic traefik-cert --from-file=tls.crt --from-file=tls.key -n kube-system

2).配置traefik反向代理并保存到k8s configmap下.

# mkdir -p /opt/k8s/conf/

# cd /opt/k8s/ssl/

# vi traefik-ui.toml

defaultEntryPoints = ["http","https"]
[entryPoints]
  [entryPoints.http]
  address = ":80"
    [entryPoints.http.redirect]
    regex = "^http://traefik-ui.k8s/(.*)"
    replacement = "https://traefik-ui.k8s/$1"
  [entryPoints.https]
  address = ":443"
    [entryPoints.https.tls]
      [[entryPoints.https.tls.certificates]]
      certFile = "/opt/k8s/ssl/tls.crt"
      keyFile = "/opt/k8s/ssl/tls.key"

# kubectl create configmap traefik-ui-conf --from-file=traefik-ui.toml -n kube-system

这样子我们就成功配置traefik反向代理, 仅使http://traefik-ui.k8s会重定向到https://traefik-ui.k8s, 实现https加密访问.

3).创建traefik deployment实例并支持https反向代理.

# vi traefik-deployment-ssl.yaml 

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: traefik-ingress-lb
  namespace: kube-system
  labels:
    k8s-app: traefik-ingress-lb
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: traefik-ingress-lb
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      terminationGracePeriodSeconds: 60
      hostNetwork: true
      restartPolicy: Always
      serviceAccountName: traefik-ingress-controller
      volumes:
      - name: ssl
        secret:
          secretName: traefik-cert
      - name: config
        configMap:
          name: traefik-ui-conf
      containers:
      - image: traefik
        name: traefik-ingress-lb
        volumeMounts:
        - mountPath: "/opt/k8s/ssl"
          name: "ssl"
        - mountPath: "/opt/k8s/conf"
          name: "config"
        ports:
        - name: http
          containerPort: 80
          hostPort: 80
        - name: https
          containerPort: 443
        - name: admin
          containerPort: 8080
          hostPort: 8080
        args:
        - --configFile=/opt/k8s/conf/traefik-ui.toml
        - --web
        - --web.address=:8080
        - --kubernetes
---
kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
    - protocol: TCP
      port: 80
      name: web
    - protocol: TCP
      port: 443
      name: https
    - protocol: TCP
      port: 8080
      name: admin
  type: NodePort

这里我们在原来的traefik deployment实例基础上将之前配置的secret与configmap导入该deployment下, 并mount ssl与conf目录用来让traefik获取我们在k8s node目录下保存的相应SSL证书与traefik反向代理配置.

最后我们通过service类型打开我们的deployment 443端口.

4).创建traefik管理员界面的ingress, 从而实现https域名访问.

# vi ui-ssl.yaml

---
apiVersion: v1
kind: Service
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - name: web
    port: 80
    targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: kube-system
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  tls:
    - secretName: traefik-cert
  rules:
  - host: traefik-ui.k8s
    http:
      paths:
      - path: /
        backend:
          serviceName: traefik-web-ui
          servicePort: web

最后我们访问https://traefik-ui.k8s 验证结果.

这样子我们就成功的开启了traefik后台admin界面的https加密连接...

更多traefik+ingress相关内容: https://docs.traefik.io/user-guide/kubernetes/

这里简单的介绍一下我们这个自动化流水线所使用到的工具:

Jenkins Pipeline: 目前国内外DevOps, CI/CD比较主流的一种将我们软件开发周期所涉及到的环节通过pipeline流水线完美的串联在一起的自动化部署框架. 它提出了一种pipeline as a code的概念, 意在将我们的所有软件开发周期中涉及到的所有步骤(版本控制 - 代码检查 - 编译 - 单元测试 - 打包 - 测试环境部署 - 集成测试 - 功能测试 - 生产环境部署)通过代码的方式推送给我们的Jenkins进行pipeline自动化部署, 实现将我们的自动化流水线的配置代码化, 并同样实现版本控制. 

目前Jenkins官方有两种pipeline的写法, 一种叫做Declarative Pipeline, 另外一种叫做Scripted Pipeline. 前者较为方便阅读, 适合初学者入门, 但相对对集成模块的兼容性以及pipeline逻辑编写的功能性较后者相对弱一些, 后者较为专业, 可以实现逻辑编写, 并支持很多主流集成模块, 推荐具有groovy脚本编写经验的人员使用.

具体内容详见: http://www.showerlee.com/archives/1972

Kubernetes: 目前这几年流行的一种容器化管理系统, 用于自动部署、扩展和管理容器化（containerized）应用程序的开源系统。它旨在提供“跨主机集群的自动部署、扩展以及运行应用程序容器的平台”。它支持一系列容器工具, 目前主流会使用Docker作为他的主流配置容器.使用它的原因也在于它将是目前DevOps领域的一个主流的Docker容器管理系统, 目前国内外很多公司都即将或者已经把自己的传统的虚拟机架构转向为Docker微服务架构, 这里我们非常有必要去学习如何基于k8s下去做自动化部署.

具体内容详见: http://www.showerlee.com/archives/2200

Helm: 这个应该算作k8s的一个功能性的工具, 它作为k8s的包管理工具, 非常方便的帮助我们整合k8s下零散的部署脚本为一个具体的项目包, 最终简化了Kubernetes部署应用的版本控制、打包、发布、删除、更新等操作.

具体内容详见: http://www.showerlee.com/archives/2455

这里本次自动化流水线实现的部署内容就是通过编写pipeline脚本, 将一个从Docker官方拿到的centos docker镜像容器, 进行二次安装配置, 打包为一个nignx静态网站容器, 发布到我们k8s下, 期间我们会对这个容器进行一些常规的功能测试, 从而模拟我们在k8s下实现自动化流水线交付的完整架构.

本次内容涉及到的代码可以参考我的github仓库:

https://github.com/showerlee/Jenkins-Pipeline-CI-CD-with-Helm-on-Kubernetes

Okay, Let's roll out...

安装环境

Local Desktop: MacOS

Virtual Machine: Virtual Box

Virtual System: CentOS 7.4

Jenkins: Jenkins 2.138

Kubernetes: Kubernetes 1.9

Docker: 17.03.2-ce

Helm: helm-v2.7.0

kube-master 10.110.16.14

kube-node-1 10.110.16.15

一. 系统环境配置

1.关闭SELINUX和firewall

# vi /etc/sysconfig/selinux

...
SELINUX=disabled 
...

# setenforce 0

# systemctl stop firewalld  && systemctl disable firewalld

2.安装k8s环境.

http://www.showerlee.com/archives/2200

3.安装helm环境.

http://www.showerlee.com/archives/2455

4.安装Jenkins环境.

http://www.showerlee.com/archives/1880

二. Jenkins Pipeline配置

1.将jenkins用户添加到默认docker用户组下, 从而保证jenkins可以直接访问/var/run/docker.sock

# usermod -a -G docker jenkins

2.更改全局安全配置, 保证pipeline可以直接调用helm

进入Jenkins --> Manage Jenkins --> Configure Global Security
在Authorization下选择Project-based Matrix Authorization Strategy

配置Anonymous User具有Read Jenkins Jobs的权限

如图:

3.配置jenkins用户加载kubectl环境变量.

# mkdir -p /home/jenkins/.kube

# cp -i /etc/kubernetes/admin.conf /home/jenkins/.kube/config

# chown jenkins:jenkins /home/jenkins/.kube/config

4.配置github与dockerhub的Jenkins账户凭证, 后面pipeline对应模块需要调用.

5.创建pipeline任务

#!groovy

def kubectlTest() {
    // Test that kubectl can correctly communication with the Kubernetes API
    echo "running kubectl test"
    sh "kubectl get nodes"

}

def helmLint(String chart_dir) {
    // lint helm chart
    sh "/usr/local/bin/helm lint ${chart_dir}"

}

def helmDeploy(Map args) {
    //configure helm client and confirm tiller process is installed

    if (args.dry_run) {
        println "Running dry-run deployment"

        sh "/usr/local/bin/helm upgrade --dry-run --debug --install ${args.name} ${args.chart_dir} --set ImageTag=${args.tag},Replicas=${args.replicas},Cpu=${args.cpu},Memory=${args.memory},DomainName=${args.name} --namespace=${args.name}"
    } else {
        println "Running deployment"
        sh "/usr/local/bin/helm upgrade --install ${args.name} ${args.chart_dir} --set ImageTag=${args.tag},Replicas=${args.replicas},Cpu=${args.cpu},Memory=${args.memory},DomainName=${args.name} --namespace=${args.name}"

        echo "Application ${args.name} successfully deployed. Use helm status ${args.name} to check"
    }
}



timeout(time: 2000, unit: 'SECONDS') {
    node {
        println "----------------------------------------------------------------------------"
        stage 'Check out pipeline from GitHub Repo'
        //git url: 'https://github.com/showerlee/Jenkins-Pipeline-CI-CD-with-Helm-on-Kubernetes.git'
        git branch: 'master',
            credentialsId: 'showerlee-github',
            url: 'https://github.com/showerlee/Jenkins-Pipeline-CI-CD-with-Helm-on-Kubernetes.git'

        // Setup the Docker Registry (Docker Hub) + Credentials 
        registry_url = "https://index.docker.io/v1/" // Docker Hub
        docker_creds_id = "showerlee-dockerhub" // name of the Jenkins Credentials ID

        def pwd = pwd()
        def chart_dir = "${pwd}/charts/newegg-nginx"

        // Add build tag version
        Properties props = new Properties()
        File propsFile = new File("${pwd}/promote.properties")
        props.load(propsFile.newDataInputStream())
        def build_tag_raw = props.getProperty('BUILD_TAG')
        float build_tag = Float.parseFloat(build_tag_raw)+0.1;
        println("Set current build_tag="+build_tag+" temporarily")

        //Set build_tag to index.html
        sh """
        echo "<h1>Welcome Newegg Nginx Test Version: ${build_tag}</h1>" > index.html
        """

        def inputFile = readFile('config.json')
        def config = new groovy.json.JsonSlurperClassic().parseText(inputFile)
        println "pipeline config ==> ${config}"
        println "----------------------------------------------------------------------------"
        
        stage 'Register DockerHub'
        echo "[INFO] Register Dockerhub"
        docker.withRegistry("${registry_url}", "${docker_creds_id}") {
        
            // Set up the container to build 
            maintainer_name = "showerlee"
            container_name = "nginx-test"
            println "----------------------------------------------------------------------------"

            stage "Build Nginx Container"
            echo "[INFO] Building Nginx with docker.build(${maintainer_name}/${container_name}:${build_tag})"
            container = docker.build("${maintainer_name}/${container_name}:${build_tag}", '.')
            println "----------------------------------------------------------------------------"
            try {
                
                // Start Testing
                stage "Spin up Nginx Container"
                echo "[INFO] Spin up Nginx Container"
                
                // Run the container with the env file, mounted volumes and the ports:
                docker.image("${maintainer_name}/${container_name}:${build_tag}").withRun("--name=${container_name}  -p 80:80 ")  { c ->
                       
                    // wait for the django server to be ready for testing
                    // the 'waitUntil' block needs to return true to stop waiting
                    // in the future this will be handy to specify waiting for a max interval: 
                    // https://issues.jenkins-ci.org/browse/JENKINS-29037
                    //
                    waitUntil {
                        sh """
                        set +x
                        ss -antup | grep :::80[^0-9] | grep LISTEN | wc -l | tr -d '\n' > /tmp/wait_results
                        set -x
                        """
                        wait_results = readFile '/tmp/wait_results'

                        echo "[INFO] Wait Results(${wait_results})"
                        if ("${wait_results}" == "1")
                        {
                            echo "[INFO] Nginx is listening on port 80"
                            sh "rm -f /tmp/wait_results"
                            return true
                        }
                        else
                        {
                            echo "[INFO] Nginx is not listening on port 80 yet"
                            return false
                        }
                    } // end of waitUntil
                    
                    // At this point Nginx is running
                    echo "[INFO] Docker Container is running"
                    input 'You can check the running container on docker build server now! Click Proceed to next stage...'    
                    // this pipeline is using 3 tests 
                    // by setting it to more than 3 you can test the error handling and see the pipeline Stage View error message
                    MAX_TESTS = 3
                    for (test_num = 1; test_num <= MAX_TESTS; test_num++) {     
                        println "----------------------------------------------------------------------------"   
                        echo "Running Test(${test_num})"
                    
                        expected_results = 0
                        if (test_num == 1 ) 
                        {
                            // Test we can download the home page from the running docker container
                            echo "[INFO] Check validation of home page"
                            sh """
                            set +x
                            docker exec -t ${container_name} curl -s http://localhost | grep Welcome | wc -l | tr -d '\n' > /tmp/test_results
                            set -x
                            """
                            expected_results = 1
                        }
                        else if (test_num == 2)
                        {
                            // Test if port 80 is exposed
                            echo "[INFO] Check if port 80 is exposed"
                            sh """
                            set +x
                            docker inspect --format '{{ (.NetworkSettings.Ports) }}' ${container_name}
                            docker inspect --format '{{ (.NetworkSettings.Ports) }}' ${container_name} | grep map | grep '80/tcp:' | wc -l | tr -d '\n' > /tmp/test_results
                            set -x
                            """
                            expected_results = 1
                        }
                        else if (test_num == 3)
                        {
                            // Test there's nothing established on the port since nginx is not running:
                            echo "[INFO] Check if nothing established from nginx container"
                            sh """
                            set +x
                            docker exec -t ${container_name} ss -apn | grep 80 | grep ESTABLISHED | wc -l | tr -d '\n' > /tmp/test_results
                            set -x
                            """
                            expected_results = 0
                        }
                        else
                        {
                            err_msg = "Missing Test(${test_num})"
                            echo "[ERROR] ${err_msg}"
                            currentBuild.result = 'FAILURE'
                            error "Failed to finish container testing with Message(${err_msg})"
                        }
                        
                        // Now validate the results match the expected results
                        stage "Test(${test_num}) - Validate Results"
                        test_results = readFile '/tmp/test_results'
                        echo "[INFO] Test(${test_num}) Results($test_results) == Expected(${expected_results})"
                        sh """
                        set +x
                        if [ \"${test_results}\" != \"${expected_results}\" ]; 
                        then 
                            echo \" --------------------- Test(${test_num}) Failed--------------------\"
                            echo \" - Test(${test_num}) Failed\"
                            exit 1
                        else 
                            echo \" - Test(${test_num}) Passed\"
                            exit 0
                        fi
                        set -x
                        """

                        echo "[INFO] Finished Running Test(${test_num})"
                    
                        // cleanup after the test run
                        sh "rm -f /tmp/test_results"
                        currentBuild.result = 'SUCCESS'
                    }
                }
                
            } catch (Exception err) {
                err_msg = "Test had Exception(${err})"
                currentBuild.result = 'FAILURE'
                error "FAILED - Stopping build for Error(${err_msg})"
            }
            println "----------------------------------------------------------------------------"
            stage "Push to DockerHub"
            input 'Do you approve to push?'
            container.push()
            currentBuild.result = 'SUCCESS'
            println "----------------------------------------------------------------------------"
            stage "Push properties to git repo"
            echo "Push current build_tag="+build_tag+" to git repo"
            withCredentials([usernamePassword(credentialsId: 'showerlee-github', passwordVariable: 'GIT_PASSWORD', usernameVariable: 'GIT_USERNAME')]) {
                sh """
                set +x
                git --version
                echo 'BUILD_TAG=${build_tag}' > ${pwd}/promote.properties
                git config --global user.email "showerlee@vip.qq.com"
                git config --global user.name "showerlee"
                git add ${pwd}/promote.properties ${pwd}/index.html
                git commit -m"Update docker tag to ${build_tag}"
                git push --set-upstream https://${GIT_USERNAME}:${GIT_PASSWORD}@github.com/showerlee/Jenkins-Pipeline-CI-CD-with-Helm-on-Kubernetes.git master
                set -x
                """
            }
            println "----------------------------------------------------------------------------"
            
        }
        
        stage ('helm test') { 
            echo "[INFO] Start helm test"   
            // run helm chart linter
            echo "[INFO] Run helm chart linter"
            helmLint(chart_dir)

            // dry-run helm chart installation
            echo "[INFO] Dry-run helm chart installation"
            helmDeploy(
                dry_run       : true,
                name          : config.app.name,
                chart_dir     : chart_dir,
                tag           : build_tag,
                replicas      : config.app.replicas,
                cpu           : config.app.cpu,
                memory        : config.app.memory
            )
            println "----------------------------------------------------------------------------"
        }
        
        stage ('helm deploy') {
            input 'Do you approve to deploy?'
            echo "[INFO] Start helm deployment"
            // Deploy using Helm chart
            helmDeploy(
                dry_run       : false,
                name          : config.app.name,
                chart_dir     : chart_dir,
                tag           : build_tag,
                replicas      : config.app.replicas,
                cpu           : config.app.cpu,
                memory        : config.app.memory
            )
            echo "[INFO] Deployment Finished..."
        }
        
        ///////////////////////////////////////
        //
        // Coming Soon Feature Enhancements
        //
        // 1. Add Docker Compose testing as a new Pipeline item that is initiated after this one for "Integration" testing
        // 2. Make sure to set the Pipeline's "Throttle builds" to 1 because the docker containers will collide on resources like ports and names
        // 3. Should be able to parallelize the docker.withRegistry() methods to ensure the container is running on the slave
        // 4. After the tests finish (and before they start), clean up container images to prevent stale docker image builds from affecting the current test run
    }
}

6.执行pipeline任务构建

Started by user admin
Obtained Jenkinsfile from git https://github.com/showerlee/Jenkins-Pipeline-CI-CD-with-Helm-on-Kubernetes.git
[Pipeline] timeout
Timeout set to expire in 33 min
[Pipeline] {
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/kube-helm-pipeline@2
[Pipeline] {
[Pipeline] echo
----------------------------------------------------------------------------
[Pipeline] stage (Check out pipeline from GitHub Repo)
Using the ‘stage’ step without a block argument is deprecated
Entering stage Check out pipeline from GitHub Repo
Proceeding
[Pipeline] git
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/showerlee/Jenkins-Pipeline-CI-CD-with-Helm-on-Kubernetes.git # timeout=10
Fetching upstream changes from https://github.com/showerlee/Jenkins-Pipeline-CI-CD-with-Helm-on-Kubernetes.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials github credential
 > git fetch --tags --progress https://github.com/showerlee/Jenkins-Pipeline-CI-CD-with-Helm-on-Kubernetes.git +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision ce4fc2593333f3ed89cd5654966919b0aa97628d (refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f ce4fc2593333f3ed89cd5654966919b0aa97628d
 > git branch -a -v --no-abbrev # timeout=10
 > git branch -D master # timeout=10
 > git checkout -b master ce4fc2593333f3ed89cd5654966919b0aa97628d
Commit message: "Update docker tag to 1.5"
 > git rev-list --no-walk ce4fc2593333f3ed89cd5654966919b0aa97628d # timeout=10
[Pipeline] pwd
[Pipeline] echo
Set current build_tag=1.6 temporarily
[Pipeline] sh
[kube-helm-pipeline@2] Running shell script
+ echo '<h1>Welcome Newegg Nginx Test Version: 1.6</h1>'
[Pipeline] readFile
[Pipeline] echo
pipeline config ==> [app:[memory:128Mi, replicas:3, name:newegg-nginx, cpu:10m], pipeline:[library:[branch:master], enabled:true]]
[Pipeline] echo
----------------------------------------------------------------------------
[Pipeline] stage (Register DockerHub)
Using the ‘stage’ step without a block argument is deprecated
Entering stage Register DockerHub
Proceeding
[Pipeline] echo
[INFO] Register Dockerhub
[Pipeline] withEnv
[Pipeline] {
[Pipeline] withDockerRegistry
$ docker login -u showerlee -p ******** https://index.docker.io/v1/
Login Succeeded
[Pipeline] {
[Pipeline] echo
----------------------------------------------------------------------------
[Pipeline] stage (Build Nginx Container)
Using the ‘stage’ step without a block argument is deprecated
Entering stage Build Nginx Container
Proceeding
[Pipeline] echo
[INFO] Building Nginx with docker.build(showerlee/nginx-test:1.6)
[Pipeline] sh
[kube-helm-pipeline@2] Running shell script
+ docker build -t showerlee/nginx-test:1.6 .
Sending build context to Docker daemon 2.164 MB

Step 1/6 : FROM centos:centos7
 ---> 5182e96772bf
Step 2/6 : MAINTAINER showerlee
 ---> Using cache
 ---> cc77ae5c175a
Step 3/6 : RUN yum -y update         && yum clean all         && yum install -y epel-release         && yum install -y nginx iproute
 ---> Using cache
 ---> 2c718c146ba5
Step 4/6 : EXPOSE 80
 ---> Using cache
 ---> 829b29a719e6
Step 5/6 : COPY index.html /usr/share/nginx/html/
 ---> Using cache
 ---> 22e805b84cee
Step 6/6 : CMD nginx -g daemon off;
 ---> Using cache
 ---> de02105d9033
Successfully built de02105d9033
[Pipeline] dockerFingerprintFrom
[Pipeline] echo
----------------------------------------------------------------------------
[Pipeline] stage (Spin up Nginx Container)
Using the ‘stage’ step without a block argument is deprecated
Entering stage Spin up Nginx Container
Proceeding
[Pipeline] echo
[INFO] Spin up Nginx Container
[Pipeline] sh
[kube-helm-pipeline@2] Running shell script
+ docker run -d --name=nginx-test -p 80:80 showerlee/nginx-test:1.6
[Pipeline] dockerFingerprintRun
[Pipeline] waitUntil
[Pipeline] {
[Pipeline] sh
[kube-helm-pipeline@2] Running shell script
+ set +x
[Pipeline] readFile
[Pipeline] echo
[INFO] Wait Results(1)
[Pipeline] echo
[INFO] Nginx is listening on port 80
[Pipeline] sh
[kube-helm-pipeline@2] Running shell script
+ rm -f /tmp/wait_results
[Pipeline] }
[Pipeline] // waitUntil
[Pipeline] echo
[INFO] Docker Container is running
[Pipeline] input
You can check the running container on docker build server now! Click Proceed to next stage...
Proceed or Abort
Approved by admin
[Pipeline] echo
----------------------------------------------------------------------------
[Pipeline] echo
Running Test(1)
[Pipeline] echo
[INFO] Check validation of home page
[Pipeline] sh
[kube-helm-pipeline@2] Running shell script
+ set +x
[Pipeline] stage (Test(1) - Validate Results)
Using the ‘stage’ step without a block argument is deprecated
Entering stage Test(1) - Validate Results
Proceeding
[Pipeline] readFile
[Pipeline] echo
[INFO] Test(1) Results(1) == Expected(1)
[Pipeline] sh
[kube-helm-pipeline@2] Running shell script
+ set +x
 - Test(1) Passed
[Pipeline] echo
[INFO] Finished Running Test(1)
[Pipeline] sh
[kube-helm-pipeline@2] Running shell script
+ rm -f /tmp/test_results
[Pipeline] echo
----------------------------------------------------------------------------
[Pipeline] echo
Running Test(2)
[Pipeline] echo
[INFO] Check if port 80 is exposed
[Pipeline] sh
[kube-helm-pipeline@2] Running shell script
+ set +x
map[80/tcp:[{0.0.0.0 80}]]
[Pipeline] stage (Test(2) - Validate Results)
Using the ‘stage’ step without a block argument is deprecated
Entering stage Test(2) - Validate Results
Proceeding
[Pipeline] readFile
[Pipeline] echo
[INFO] Test(2) Results(1) == Expected(1)
[Pipeline] sh
[kube-helm-pipeline@2] Running shell script
+ set +x
 - Test(2) Passed
[Pipeline] echo
[INFO] Finished Running Test(2)
[Pipeline] sh
[kube-helm-pipeline@2] Running shell script
+ rm -f /tmp/test_results
[Pipeline] echo
----------------------------------------------------------------------------
[Pipeline] echo
Running Test(3)
[Pipeline] echo
[INFO] Check if nothing established from nginx container
[Pipeline] sh
[kube-helm-pipeline@2] Running shell script
+ set +x
[Pipeline] stage (Test(3) - Validate Results)
Using the ‘stage’ step without a block argument is deprecated
Entering stage Test(3) - Validate Results
Proceeding
[Pipeline] readFile
[Pipeline] echo
[INFO] Test(3) Results(0) == Expected(0)
[Pipeline] sh
[kube-helm-pipeline@2] Running shell script
+ set +x
 - Test(3) Passed
[Pipeline] echo
[INFO] Finished Running Test(3)
[Pipeline] sh
[kube-helm-pipeline@2] Running shell script
+ rm -f /tmp/test_results
[Pipeline] sh
[kube-helm-pipeline@2] Running shell script
+ docker stop 05a19a7c8519a04cc03029d7273a8e2ae3363e74a03b6d4167dfbec0bf6ee978
05a19a7c8519a04cc03029d7273a8e2ae3363e74a03b6d4167dfbec0bf6ee978
+ docker rm -f 05a19a7c8519a04cc03029d7273a8e2ae3363e74a03b6d4167dfbec0bf6ee978
05a19a7c8519a04cc03029d7273a8e2ae3363e74a03b6d4167dfbec0bf6ee978
[Pipeline] echo
----------------------------------------------------------------------------
[Pipeline] stage (Push to DockerHub)
Using the ‘stage’ step without a block argument is deprecated
Entering stage Push to DockerHub
Proceeding
[Pipeline] input
Do you approve to push?
Proceed or Abort
Approved by admin
[Pipeline] sh
[kube-helm-pipeline@2] Running shell script
+ docker tag showerlee/nginx-test:1.6 index.docker.io/showerlee/nginx-test:1.6
[Pipeline] sh
[kube-helm-pipeline@2] Running shell script
+ docker push index.docker.io/showerlee/nginx-test:1.6
The push refers to a repository [docker.io/showerlee/nginx-test]
aedd55edeb74: Preparing
85e9d4859663: Preparing
1d31b5806ba4: Preparing
85e9d4859663: Layer already exists
1d31b5806ba4: Layer already exists
aedd55edeb74: Retrying in 5 seconds
aedd55edeb74: Retrying in 4 seconds
aedd55edeb74: Retrying in 3 seconds
aedd55edeb74: Retrying in 2 seconds
aedd55edeb74: Retrying in 1 second
aedd55edeb74: Pushed
1.6: digest: sha256:8ee214129c880a0d759837daf30c31772797fa6709f11edd9e30db6891710de4 size: 948
[Pipeline] echo
----------------------------------------------------------------------------
[Pipeline] stage (Push properties to git repo)
Using the ‘stage’ step without a block argument is deprecated
Entering stage Push properties to git repo
Proceeding
[Pipeline] echo
Push current build_tag=1.6 to git repo
[Pipeline] withCredentials
[Pipeline] {
[Pipeline] sh
[kube-helm-pipeline@2] Running shell script
+ set +x
git version 1.8.3.1
[master cdbb62c] Update docker tag to 1.6
 2 files changed, 2 insertions(+), 2 deletions(-)
To https://****:****@github.com/****/Jenkins-Pipeline-CI-CD-with-Helm-on-Kubernetes.git
   ce4fc25..cdbb62c  master -> master
Branch master set up to track remote branch master from https://****:****@github.com/****/Jenkins-Pipeline-CI-CD-with-Helm-on-Kubernetes.git.
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] echo
----------------------------------------------------------------------------
[Pipeline] }
[Pipeline] // withDockerRegistry
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] stage
[Pipeline] { (helm test)
[Pipeline] echo
[INFO] Start helm test
[Pipeline] echo
[INFO] Run helm chart linter
[Pipeline] sh
[kube-helm-pipeline@2] Running shell script
+ /usr/local/bin/helm lint /var/jenkins_home/workspace/kube-helm-pipeline@2/charts/newegg-nginx
==> Linting /var/jenkins_home/workspace/kube-helm-pipeline@2/charts/newegg-nginx
[INFO] Chart.yaml: icon is recommended

1 chart(s) linted, no failures
[Pipeline] echo
[INFO] Dry-run helm chart installation
[Pipeline] echo
Running dry-run deployment
[Pipeline] sh
[kube-helm-pipeline@2] Running shell script
+ /usr/local/bin/helm upgrade --dry-run --debug --install newegg-nginx /var/jenkins_home/workspace/kube-helm-pipeline@2/charts/newegg-nginx --set ImageTag=1.6,Replicas=3,Cpu=10m,Memory=128Mi,DomainName=newegg-nginx --namespace=newegg-nginx
[debug] Created tunnel using local port: '39460'

[debug] SERVER: "localhost:39460"

Release "newegg-nginx" does not exist. Installing it now.
[debug] CHART PATH: /var/jenkins_home/workspace/kube-helm-pipeline@2/charts/newegg-nginx

NAME:   newegg-nginx
REVISION: 1
RELEASED: Sun Aug 26 01:38:02 2018
CHART: newegg-nginx-1.0.0
USER-SUPPLIED VALUES:
Cpu: 10m
DomainName: newegg-nginx
ImageTag: "1.6"
Memory: 128Mi
Replicas: 3

COMPUTED VALUES:
ContainerPort: 80
Cpu: 10m
DomainName: newegg-nginx
Image: showerlee/nginx-test
ImagePullPolicy: Always
ImageTag: "1.6"
Imagetag: latest
Memory: 128Mi
Replicas: 3
ServicePort: 80
ServiceType: NodePort

HOOKS:
MANIFEST:

---
# Source: newegg-nginx/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: newegg-nginx-newegg-nginx
  labels:    
    app: newegg-nginx-newegg-nginx
    version: 1.0.0
    release: newegg-nginx
spec:
  type: "NodePort"
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
  selector:
    app: newegg-nginx-newegg-nginx
---
# Source: newegg-nginx/templates/deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: newegg-nginx-newegg-nginx
  labels:    
    app: newegg-nginx-newegg-nginx
    version: 1.0.0
    release: newegg-nginx
spec:
  replicas: 3
  template:
    metadata:
      labels:        
        app: newegg-nginx-newegg-nginx
        version: 1.0.0
        release: newegg-nginx
    spec:
      containers:
        - name: newegg-nginx
          image: "showerlee/nginx-test:1.6"
          imagePullPolicy: "Always"
          ports:
            - containerPort: 80
              protocol: TCP
          resources:
            requests:
              cpu: "10m"
              memory: "128Mi"
---
# Source: newegg-nginx/templates/ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: newegg-nginx-newegg-nginx
  labels:    
    app: newegg-nginx-newegg-nginx
    version: 1.0.0
    release: newegg-nginx
spec:
  rules:
  - host: newegg-nginx.buyabs.corp
    http:
      paths:
      - path: /
        backend:
          serviceName: newegg-nginx-newegg-nginx
          servicePort: 80
[Pipeline] echo
----------------------------------------------------------------------------
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (helm deploy)
[Pipeline] input
Do you approve to deploy?
Proceed or Abort
Approved by admin
[Pipeline] echo
[INFO] Start helm deployment
[Pipeline] echo
Running deployment
[Pipeline] sh
[kube-helm-pipeline@2] Running shell script
+ /usr/local/bin/helm upgrade --install newegg-nginx /var/jenkins_home/workspace/kube-helm-pipeline@2/charts/newegg-nginx --set ImageTag=1.6,Replicas=3,Cpu=10m,Memory=128Mi,DomainName=newegg-nginx --namespace=newegg-nginx
Release "newegg-nginx" does not exist. Installing it now.
NAME:   newegg-nginx
LAST DEPLOYED: Sun Aug 26 01:38:24 2018
NAMESPACE: newegg-nginx
STATUS: DEPLOYED

RESOURCES:
==> v1beta1/Ingress
NAME                       HOSTS                     ADDRESS  PORTS  AGE
newegg-nginx-newegg-nginx  newegg-nginx.buyabs.corp  80       1m

==> v1/Pod(related)
NAME                                        READY  STATUS             RESTARTS  AGE
newegg-nginx-newegg-nginx-56c478c888-6n6rt  0/1    ContainerCreating  0         1m
newegg-nginx-newegg-nginx-56c478c888-hsbcp  0/1    ContainerCreating  0         1m
newegg-nginx-newegg-nginx-56c478c888-vdfgb  0/1    ContainerCreating  0         1m

==> v1/Service
NAME                       TYPE      CLUSTER-IP     EXTERNAL-IP  PORT(S)       AGE
newegg-nginx-newegg-nginx  NodePort  10.97.200.124  <none>       80:32239/TCP  1m

==> v1beta1/Deployment
NAME                       DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE
newegg-nginx-newegg-nginx  3        3        3           0          1m


[Pipeline] echo
Application newegg-nginx successfully deployed. Use helm status newegg-nginx to check
[Pipeline] echo
[INFO] Deployment Finished...
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // timeout
[Pipeline] End of Pipeline
Finished: SUCCESS

7.验证结果

这样我们就成功的利用Jenkins集成k8s+helm实现将我们的静态网站部署到我们的kubernetes集群中.

大功告成...

Docker就不多介绍了，大家可以理解为是他是一个虚拟集装箱，将原来以服务器为基础单元的集群，细化到以每个服务进程为单元。这样的优点在于：

1. 隔离开发环境和具体可执行进程，可跨平台跨主机使用，统一开发环境

2. 节约时间成本，硬件成本，秒级别快速部署启动

3. 支持持续集成，可利用相关部署工具(Ansible, saltstack)进行快速部署

4. 可作为轻量级别主机或节点部署应用

5. 未来可能代替虚拟机的虚拟容器

最后利用Ansible去对Docker容器进行具体服务的部署，最大限度的体现轻量级，统一开发部署环境的好处。

在部署之前首先给大家介绍一下我的部署结构:

dockerfiles
├── mariadb
│   ├── Dockerfile
│   ├── ansible
│   │   ├── mariadb-install-container.yml
│   │   ├── mariadb.sh
│   │   ├── mariadb.sql
│   │   └── my.cnf
│   └── run.sh
├── nginx
│   ├── Dockerfile
│   ├── ansible
│   │   ├── default.conf
│   │   ├── nginx-container.yml
│   │   └── nginx.conf
│   └── run.sh
└── php
    ├── Dockerfile
    ├── ansible
    │   ├── info.php
    │   ├── php-container.yml
    │   ├── test-db-conn.php
    │   ├── wp-config.php
    │   └── http://www.conf
    └── run.sh

Dockerfiles主目录下面分别是3个服务所对应的目录，每个目录大致分为Dockerfile配置文件, ansible目录，以及run.sh

Dockerfile这个大家都清楚，用来Build容器镜像的配置脚本。

ansible目录下保存需要在容器内使用到的ansible playbook相应配置文件。

run.sh则是创建和运行容器的启动脚本，我们分别去执行对应目录的执行脚本用来构建Docker容器所对应的服务。

这里给大家提供一个我的gitlab私人仓库的链接，方便大家去下载这个部署脚本：

https://git.showerlee.com/showerlee/ansible-docker-wp

大家如果想对Docker和Ansible有不太明白的地方可以先阅读我之前写的文章。

Docker: http://www.showerlee.com/archives/1758

Ansible: http://www.showerlee.com/archives/1649

一.环境部署

Local system:  MAC OS X 10.12.1
Docker:   Docker 1.12.3

Tip: 这里说明一下，较早版本的Docker若在非Linux内核下去安装需要去配合安装virtualbox虚拟机才能正常使用，这里我们使用的最新版本Docker配合MAC OS 10.12.1最新开放出来的内核接口，可无需调用额外虚拟机直接安装使用即可。

二.Docker部署

Mariadb container

Mariadb是Mysql在CentOS7上的一个开源分支, 大家可以理解就是一个mysql. 这里首先我们为什么先去部署并启动Mariadb, 是因为我们后面启动Docker时会用到它的一个Link参数，这个参数不仅保证两台容器会安全的利用这个Link去传输数据，而且还会在被Link主机内创建一个host记录，方便我们直接用FQDN名去访问相应主机。

Dockerfile

FROM ansible/centos7-ansible/:stable

ADD ansible /data/ansible

WORKDIR /data/ansible

RUN ansible-playbook mariadb-install-container.yml -c local

# Initialize and start mariadb
RUN chmod +x mariadb.sh

CMD ["./mariadb.sh"]

EXPOSE 3306

这里简单来说就是下载集成Ansible的CentOS7容器镜像，并将ansible playbook及相应配置文件复制到容器内去部署MariaDB初始环境，最终启动服务，并开启3306DB端口.

ansible/mariadb-install-container.yml

- name: Install Mariadb container
  hosts: local
  vars:
    conf_path: /etc/my.cnf
    src_socket: /data/mariadb_data/mysql.sock
    link_socket: /var/lib/mysql/mysql.sock
    user: mysql
  tasks:
  - name: Install epel-release package
    yum: name=epel-release state=latest

  - name: Install required packages
    yum: name={{ item }} state=latest
    with_items:
      - net-tools
      - lsof 
      - nmap
      - mariadb-server
      - MySQL-python

  - name: Copy mariadb conf
    copy: src=my.cnf dest={{ conf_path }} mode=0644

  - name: Link mysql socket
    file: 
      src={{ src_socket }} dest={{ link_socket }} state=link force=yes

这里是ansible playbook, 基本上就是利用ansible本地安装Epel源以及Mariadb相应的安装包，然后进行DB的相关配置。

ansible/mariadb.sh

#!/bin/sh
 chown -R mysql:mysql /var/lib/mysql

 mysql_install_db --user=mysql --datadir=/data/mariadb_data > /dev/null

 mysqld_safe &

 sleep 5s

 mysql -v < mariadb.sql

 sleep 5s

 ps -wef | grep mysql | grep -v grep | awk '{print $2}' | xargs kill -9

 mysqld_safe --user mysql

这里是对Mariadb进行初始化安装，并使用sql脚本更改root密码及相应权限和创建Wordpress数据库，最终重启DB服务。

ansible/Mariadb.sql

USE mysql;
 GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' WITH GRANT OPTION;
 FLUSH PRIVILEGES;
 UPDATE user SET password=PASSWORD("123456") WHERE user='root';
 FLUSH PRIVILEGES;
 CREATE DATABASE wordpress;
 FLUSH PRIVILEGES;

ansible/my.cnf

[mysqld]
datadir=/data/mariadb_data
socket=/data/mariadb_data/mysql.sock
port=3306
bind-address = 0.0.0.0

# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
# Settings user and group are ignored when systemd is used.
# If you need to run mysqld under a different user or group,
# customize your systemd unit file for mariadb according to the
# instructions in http://fedoraproject.org/wiki/Systemd

[mysqld_safe]
log-error=/var/log/mariadb/mariadb.log
pid-file=/var/run/mariadb/mariadb.pid

#
# include all files from the config directory
#
!includedir /etc/my.cnf.d

Mariadb的DB配置.

run.sh

#/bin/bash

IMAGE="leon/mariadb"
Container="mariadb"

docker build -t ${IMAGE} .

docker run -d -p 3306:3306 --name ${Container} ${IMAGE}

最终我们使用docker build去创建一个基于Dockerfile配置的MariaDB容器镜像，并用docker run去启动这个容器镜像。

Tip: -d为后台启动容器，-p为打开端口，--name 为创建容器名，-t为打一个镜像标签。

PHP container

接下来我们会部署启动php-fpm服务，作为解析Wordpress网站的php服务, 它会开启9000端口，方便后面的nginx container去进行反向连接。

Dockerfile

FROM ansible/centos7-ansible:stable

ADD ansible /data/ansible

WORKDIR /data/ansible

RUN ansible-playbook php-container.yml -c local

# Add the volume to php application
VOLUME ["/data/app"]

# CMD ["php-fpm", "--nodaemonize"]
ENTRYPOINT ["/usr/sbin/php-fpm", "-F"]

EXPOSE 9000

这里的配置基本上和MariaDB类似，区别在于我们会在这里去创建一个与nginx共同使用的共享网站目录，里面会保存Wordpress源代码和相关php测试文件。

ansible/php-container.yml

- name: Create PHP container
  hosts: local
  vars:
    app_dir: /data/app
    src_dir: /data/src
    phpfpm_path: /etc/php-fpm.d/www.conf
    info_path: /data/app/info.php
    test_path: /data/app/test-db-conn.php
    wp_url: https://cn.wordpress.org/wordpress-4.5.3-zh_CN.tar.gz
    wp_config: /data/app/wp-config.php
  tasks:
  - name: Install epel-release packages
    yum: name=epel-release state=latest 

  - name: Install required packages
    yum: name={{ item }} state=latest
    with_items:
      - net-tools
      - lsof
      - nmap
      - unzip
      - mariadb

  - name: Install php packages
    yum: name={{ item }} state=latest
    with_items:
      - php
      - php-mysql
      - php-fpm
      - php-gd
      - php-mbstring

  - name: Create dir
    file: path={{ item }} state=directory mode=0755
    with_items:
      - "{{ app_dir }}"
      - "{{ src_dir }}"

  - name: Unpack WP package
    unarchive: src={{ wp_url }} dest={{ src_dir }} remote_src=yes

  - name: Move WP source code to app_dir
    shell: "shopt -s dotglob && mv {{ src_dir }}/wordpress/* {{ app_dir }}" 

  - name: Copy wp configuration
    copy: src=wp-config.php dest={{ wp_config }}

  - name: Copy php info page
    copy: src=info.php dest={{ info_path }} mode=0755

  - name: Copy test-db-conn page
    copy: src=test-db-conn.php dest={{ test_path }} mode=0755

  - name: Copy nginx configuration
    copy: src=www.conf dest={{ phpfpm_path }} mode=0755

这里playbook配置内容较多，不过基本上就是安装php-fpm以及相应安装包，配置php-fpm配置项，并下载Wordpress源代码并将DB信息写入WP配置文件，复制php测试脚本等。

ansible/info.php

<?php
phpinfo();
?>

ansible/test-db-conn.php

<html><body><h1>The page to test Mariadb connection.</h1></body></html>
<?php
 $conn=mysql_connect('mariadb','root','123456');
 if ($conn)
   echo "<h2>Success...</h2>";
 else
   echo "<h2>Failure...</h2>";
 
?>

ansible/wp-config.php

<?php
/**
 * WordPress基础配置文件。
 *
 * 这个文件被安装程序用于自动生成wp-config.php配置文件，
 * 您可以不使用网站，您需要手动复制这个文件，
 * 并重命名为“wp-config.php”，然后填入相关信息。
 *
 * 本文件包含以下配置选项：
 *
 * * MySQL设置
 * * 密钥
 * * 数据库表名前缀
 * * ABSPATH
 *
 * @link https://codex.wordpress.org/zh-cn:%E7%BC%96%E8%BE%91_wp-config.php
 *
 * @package WordPress
 */

// ** MySQL 设置 - 具体信息来自您正在使用的主机 ** //
/** WordPress数据库的名称 */
define('DB_NAME', 'wordpress');

/** MySQL数据库用户名 */
define('DB_USER', 'root');

/** MySQL数据库密码 */
define('DB_PASSWORD', '123456');

/** MySQL主机 */
define('DB_HOST', 'mariadb');

/** 创建数据表时默认的文字编码 */
define('DB_CHARSET', 'utf8');

/** 数据库整理类型。如不确定请勿更改 */
define('DB_COLLATE', '');

/**#@+
 * 身份认证密钥与盐。
 *
 * 修改为任意独一无二的字串！
 * 或者直接访问{@link https://api.wordpress.org/secret-key/1.1/salt/
 * WordPress.org密钥生成服务}
 * 任何修改都会导致所有cookies失效，所有用户将必须重新登录。
 *
 * @since 2.6.0
 */
define('AUTH_KEY',         'put your unique phrase here');
define('SECURE_AUTH_KEY',  'put your unique phrase here');
define('LOGGED_IN_KEY',    'put your unique phrase here');
define('NONCE_KEY',        'put your unique phrase here');
define('AUTH_SALT',        'put your unique phrase here');
define('SECURE_AUTH_SALT', 'put your unique phrase here');
define('LOGGED_IN_SALT',   'put your unique phrase here');
define('NONCE_SALT',       'put your unique phrase here');

/**#@-*/

/**
 * WordPress数据表前缀。
 *
 * 如果您有在同一数据库内安装多个WordPress的需求，请为每个WordPress设置
 * 不同的数据表前缀。前缀名只能为数字、字母加下划线。
 */
$table_prefix  = 'wp_';

/**
 * 开发者专用：WordPress调试模式。
 *
 * 将这个值改为true，WordPress将显示所有用于开发的提示。
 * 强烈建议插件开发者在开发环境中启用WP_DEBUG。
 *
 * 要获取其他能用于调试的信息，请访问Codex。
 *
 * @link https://codex.wordpress.org/Debugging_in_WordPress
 */
define('WP_DEBUG', false);

/**
 * zh_CN本地化设置：启用ICP备案号显示
 *
 * 可在设置→常规中修改。
 * 如需禁用，请移除或注释掉本行。
 */
define('WP_ZH_CN_ICP_NUM', true);

/* 好了！请不要再继续编辑。请保存本文件。使用愉快！ */

/** WordPress目录的绝对路径。 */
if ( !defined('ABSPATH') )
	define('ABSPATH', dirname(__FILE__) . '/');

/** 设置WordPress变量和包含文件。 */
require_once(ABSPATH . 'wp-settings.php');

ansible/www.conf

; Start a new pool named 'www'.
[www]

; The address on which to accept FastCGI requests.
; Valid syntaxes are:
;   'ip.add.re.ss:port'    - to listen on a TCP socket to a specific address on
;                            a specific port;
;   'port'                 - to listen on a TCP socket to all addresses on a
;                            specific port;
;   '/path/to/unix/socket' - to listen on a unix socket.
; Note: This value is mandatory.
listen = 0.0.0.0:9000

; Set listen(2) backlog. A value of '-1' means unlimited.
; Default Value: -1
;listen.backlog = -1
 
; List of ipv4 addresses of FastCGI clients which are allowed to connect.
; Equivalent to the FCGI_WEB_SERVER_ADDRS environment variable in the original
; PHP FCGI (5.2.2+). Makes sense only with a tcp listening socket. Each address
; must be separated by a comma. If this value is left blank, connections will be
; accepted from any ip address.
; Default Value: any
;listen.allowed_clients = any

; Set permissions for unix socket, if one is used. In Linux, read/write
; permissions must be set in order to allow connections from a web server. Many
; BSD-derived systems allow connections regardless of permissions. 
; Default Values: user and group are set as the running user
;                 mode is set to 0666
listen.owner = nobody
listen.group = nobody
listen.mode = 0666

; Unix user/group of processes
; Note: The user is mandatory. If the group is not set, the default user's group
;       will be used.
; RPM: apache Choosed to be able to access some dir as httpd
user = nobody
; RPM: Keep a group allowed to write in log dir.
group = nobody

; Choose how the process manager will control the number of child processes.
; Possible Values:
;   static  - a fixed number (pm.max_children) of child processes;
;   dynamic - the number of child processes are set dynamically based on the
;             following directives:
;             pm.max_children      - the maximum number of children that can
;                                    be alive at the same time.
;             pm.start_servers     - the number of children created on startup.
;             pm.min_spare_servers - the minimum number of children in 'idle'
;                                    state (waiting to process). If the number
;                                    of 'idle' processes is less than this
;                                    number then some children will be created.
;             pm.max_spare_servers - the maximum number of children in 'idle'
;                                    state (waiting to process). If the number
;                                    of 'idle' processes is greater than this
;                                    number then some children will be killed.
; Note: This value is mandatory.
pm = dynamic

; The number of child processes to be created when pm is set to 'static' and the
; maximum number of child processes to be created when pm is set to 'dynamic'.
; This value sets the limit on the number of simultaneous requests that will be
; served. Equivalent to the ApacheMaxClients directive with mpm_prefork.
; Equivalent to the PHP_FCGI_CHILDREN environment variable in the original PHP
; CGI.
; Note: Used when pm is set to either 'static' or 'dynamic'
; Note: This value is mandatory.
pm.max_children = 5

; The number of child processes created on startup.
; Note: Used only when pm is set to 'dynamic'
; Default Value: min_spare_servers + (max_spare_servers - min_spare_servers) / 2
pm.start_servers = 2

; The desired minimum number of idle server processes.
; Note: Used only when pm is set to 'dynamic'
; Note: Mandatory when pm is set to 'dynamic'
pm.min_spare_servers = 1

; The desired maximum number of idle server processes.
; Note: Used only when pm is set to 'dynamic'
; Note: Mandatory when pm is set to 'dynamic'
pm.max_spare_servers = 3
 
; The number of requests each child process should execute before respawning.
; This can be useful to work around memory leaks in 3rd party libraries. For
; endless request processing specify '0'. Equivalent to PHP_FCGI_MAX_REQUESTS.
; Default Value: 0
;pm.max_requests = 500

; The URI to view the FPM status page. If this value is not set, no URI will be
; recognized as a status page. By default, the status page shows the following
; information:
;   accepted conn    - the number of request accepted by the pool;
;   pool             - the name of the pool;
;   process manager  - static or dynamic;
;   idle processes   - the number of idle processes;
;   active processes - the number of active processes;
;   total processes  - the number of idle + active processes.
; The values of 'idle processes', 'active processes' and 'total processes' are
; updated each second. The value of 'accepted conn' is updated in real time.
; Example output:
;   accepted conn:   12073
;   pool:             www
;   process manager:  static
;   idle processes:   35
;   active processes: 65
;   total processes:  100
; By default the status page output is formatted as text/plain. Passing either
; 'html' or 'json' as a query string will return the corresponding output
; syntax. Example:
;   http://www.foo.bar/status
;   http://www.foo.bar/status?json
;   http://www.foo.bar/status?html
; Note: The value must start with a leading slash (/). The value can be
;       anything, but it may not be a good idea to use the .php extension or it
;       may conflict with a real PHP file.
; Default Value: not set 
;pm.status_path = /status
 
; The ping URI to call the monitoring page of FPM. If this value is not set, no
; URI will be recognized as a ping page. This could be used to test from outside
; that FPM is alive and responding, or to
; - create a graph of FPM availability (rrd or such);
; - remove a server from a group if it is not responding (load balancing);
; - trigger alerts for the operating team (24/7).
; Note: The value must start with a leading slash (/). The value can be
;       anything, but it may not be a good idea to use the .php extension or it
;       may conflict with a real PHP file.
; Default Value: not set
;ping.path = /ping

; This directive may be used to customize the response of a ping request. The
; response is formatted as text/plain with a 200 response code.
; Default Value: pong
;ping.response = pong
 
; The timeout for serving a single request after which the worker process will
; be killed. This option should be used when the 'max_execution_time' ini option
; does not stop script execution for some reason. A value of '0' means 'off'.
; Available units: s(econds)(default), m(inutes), h(ours), or d(ays)
; Default Value: 0
;request_terminate_timeout = 0
 
; The timeout for serving a single request after which a PHP backtrace will be
; dumped to the 'slowlog' file. A value of '0s' means 'off'.
; Available units: s(econds)(default), m(inutes), h(ours), or d(ays)
; Default Value: 0
;request_slowlog_timeout = 0
 
; The log file for slow requests
; Default Value: not set
; Note: slowlog is mandatory if request_slowlog_timeout is set
slowlog = /var/log/php-fpm/www-slow.log
 
; Set open file descriptor rlimit.
; Default Value: system defined value
;rlimit_files = 1024
 
; Set max core size rlimit.
; Possible Values: 'unlimited' or an integer greater or equal to 0
; Default Value: system defined value
;rlimit_core = 0
 
; Chroot to this directory at the start. This value must be defined as an
; absolute path. When this value is not set, chroot is not used.
; Note: chrooting is a great security feature and should be used whenever 
;       possible. However, all PHP paths will be relative to the chroot
;       (error_log, sessions.save_path, ...).
; Default Value: not set
;chroot = 
 
; Chdir to this directory at the start. This value must be an absolute path.
; Default Value: current directory or / when chroot
;chdir = /var/www
 
; Redirect worker stdout and stderr into main error log. If not set, stdout and
; stderr will be redirected to /dev/null according to FastCGI specs.
; Default Value: no
;catch_workers_output = yes
 
; Limits the extensions of the main script FPM will allow to parse. This can
; prevent configuration mistakes on the web server side. You should only limit
; FPM to .php extensions to prevent malicious users to use other extensions to
; exectute php code.
; Note: set an empty value to allow all extensions.
; Default Value: .php
;security.limit_extensions = .php .php3 .php4 .php5

; Pass environment variables like LD_LIBRARY_PATH. All $VARIABLEs are taken from
; the current environment.
; Default Value: clean env
;env[HOSTNAME] = $HOSTNAME
;env[PATH] = /usr/local/bin:/usr/bin:/bin
;env[TMP] = /tmp
;env[TMPDIR] = /tmp
;env[TEMP] = /tmp

; Additional php.ini defines, specific to this pool of workers. These settings
; overwrite the values previously defined in the php.ini. The directives are the
; same as the PHP SAPI:
;   php_value/php_flag             - you can set classic ini defines which can
;                                    be overwritten from PHP call 'ini_set'. 
;   php_admin_value/php_admin_flag - these directives won't be overwritten by
;                                     PHP call 'ini_set'
; For php_*flag, valid values are on, off, 1, 0, true, false, yes or no.

; Defining 'extension' will load the corresponding shared extension from
; extension_dir. Defining 'disable_functions' or 'disable_classes' will not
; overwrite previously defined php.ini values, but will append the new value
; instead.

; Default Value: nothing is defined by default except the values in php.ini and
;                specified at startup with the -d argument
;php_admin_value[sendmail_path] = /usr/sbin/sendmail -t -i -f www@my.domain.com
;php_flag[display_errors] = off
php_admin_value[error_log] = /var/log/php-fpm/www-error.log
php_admin_flag[log_errors] = on
;php_admin_value[memory_limit] = 128M

; Set session path to a directory owned by process user
php_value[session.save_handler] = files
php_value[session.save_path] = /var/lib/php/session

run.sh

#/bin/bash

IMAGE="leon/php"
Container="php"
Link="mariadb"

docker build -t ${IMAGE} .

docker run -d -p 9000:9000 --name ${Container} --link ${Link}:${Link} ${IMAGE}

这里去创建PHP并启动镜像，需要提到这里的link就是文章开头说到的需要与DB进行安全传输，并在php容器内创建一个DB的host记录。

我们可以使用这个命令去进入容器，查看该记录。

# docker exec -it php bash
[root@7e6188415cb2 ansible]# cat /etc/hosts
127.0.0.1      	localhost
::1    	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
ff00::0	ip6-mcastprefix
ff02::1	ip6-allnodes
ff02::2	ip6-allrouters
172.17.0.2     mariadb 29ae9593fbfc
172.17.0.3     7e6188415cb2

Tip: 这里可以看到mariadb的内网ip为172.17.0.2, php-fpm的为172.17.0.3, 我们MAC本机作为Docker主机与容器间会进行DNAT通信，容器因为在同一内网，可自由通信，前提是需要在创建和启动镜像前打开相应服务端口。

Nginx container

这里最终我们会利用Nginx作为一个反向代理，配置www.example.com去访问我们的PHP容器下的Wordpress页面.

Dockerfile

FROM ansible/centos7-ansible/epel:stable

ADD ansible /data/ansible

WORKDIR /data/ansible

RUN ansible-playbook nginx-container.yml -c local

VOLUME ["/data/app"]

# CMD ["nginx", "-g", "daemon off;"]
ENTRYPOINT [ "/usr/sbin/nginx" ]

EXPOSE 80
EXPOSE 443

这里基本与PHP的容器配置类似，执行playbook，共享网站目录，启动nginx服务并打开80，443端口。

ansible/nginx-container.yml

- name: Create Nginx container
  hosts: local
  vars:
    global_conf_path: /etc/nginx/nginx.conf
    server_conf_path: /etc/nginx/conf.d/default.conf
  tasks:
  - name: Install epel-release package
    yum: name=epel-release state=latest

  - name: Install required packages
    yum: name={{ item }} state=latest
    with_items:
      - net-tools
      - lsof
      - nmap
      - nginx

  - name: Copy nginx global conf
    copy: src=nginx.conf dest={{ global_conf_path }} mode=0644

  - name: Copy nginx server conf
    copy: src=default.conf dest={{ server_conf_path }} mode=0644

这里安装nginx，并将配置文件复制到对应nginx目录.

ansible/defalut.conf

# The default server
#

upstream backend {
    server php:9000;
}

server {
    listen 80;
    server_name http://www.example.com;
    root /data/app;
    index index.php index.html index.htm;

    location / {
          try_files $uri $uri/ =404;
    }

    location ~ \.php$ {
        fastcgi_pass backend;
        fastcgi_index index.php;
        include fastcgi.conf;
    }

}

ansible/nginx.conf

# For more information on configuration, see:
#   * Official English Documentation: http://nginx.org/en/docs/
#   * Official Russian Documentation: http://nginx.org/ru/docs/

user nobody;
worker_processes 1;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
daemon off;

# Load dynamic modules. See /usr/share/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;

events {
    worker_connections 768;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    # Load modular configuration files from the /etc/nginx/conf.d directory.
    # See http://nginx.org/en/docs/ngx_core_module.html#include
    # for more information.
    include /etc/nginx/conf.d/*.conf;

    server {
        listen       80 default_server;
        listen       [::]:80 default_server;
        server_name  _;
        root         /usr/share/nginx/html;

        # Load configuration files for the default server block.
        include /etc/nginx/default.d/*.conf;

        location / {
        }

        error_page 404 /404.html;
            location = /40x.html {
        }

        error_page 500 502 503 504 /50x.html;
            location = /50x.html {
        }
    }

# Settings for a TLS enabled server.
#
#    server {
#        listen       443 ssl http2 default_server;
#        listen       [::]:443 ssl http2 default_server;
#        server_name  _;
#        root         /usr/share/nginx/html;
#
#        ssl_certificate "/etc/pki/nginx/server.crt";
#        ssl_certificate_key "/etc/pki/nginx/private/server.key";
#        ssl_session_cache shared:SSL:1m;
#        ssl_session_timeout  10m;
#        ssl_ciphers HIGH:!aNULL:!MD5;
#        ssl_prefer_server_ciphers on;
#
#        # Load configuration files for the default server block.
#        include /etc/nginx/default.d/*.conf;
#
#        location / {
#        }
#
#        error_page 404 /404.html;
#            location = /40x.html {
#        }
#
#        error_page 500 502 503 504 /50x.html;
#            location = /50x.html {
#        }
#    }

}

run.sh

#/bin/bash

IMAGE="leon/nginx"
Container="nginx"
Link="php"
Volume="php"

docker build -t ${IMAGE} .

docker run -d -p 80:80 -p 443:443 --name ${Container} --link ${Link}:${Link} --volumes-from ${Volume} ${IMAGE}

这里需要注意的是我们添加了一个--volumes-from参数用来调用PHP share出来的网站目录。

这里我们分别执行3个目录下的执行脚本(run.sh)后，效果如下:

# docker ps

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                      NAMES
7e6188415cb2        leon/php            "/usr/sbin/php-fpm -F"   2 hours ago         Up About an hour    0.0.0.0:9000->9000/tcp                     php
29ae9593fbfc        leon/mariadb        "./mariadb.sh"           2 hours ago         Up About an hour    0.0.0.0:3306->3306/tcp                     mariadb
d07bb4aea5c1        leon/nginx          "/usr/sbin/nginx"        3 days ago          Up 4 hours          0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp   nginx

# docker images

REPOSITORY                     TAG                 IMAGE ID            CREATED             SIZE
leon/php                       latest              f243f43c99a2        3 hours ago         883 MB
leon/mariadb                   latest              7046c76b434a        3 hours ago         1.694 GB
leon/nginx                     latest              d588d0a73eee        7 days ago          737.6 MB
ansible/centos7-ansible        stable              5108f665e079        3 weeks ago         433.3 MB

到这里我们就成功创建并启动了我们Wordpress所需要的三个基础服务。

我们可以利用Docker start "容器名"或者Docker stop "容器名" 去启动或者关闭这个包含具体服务的容器，可以作为我们在Docker服务器下重启服务的命令。

如果需要进入容器进行调测，我们也可以使用Docker exec -it "容器名" bash

测试结果

我们给MAC主机添加一个host记录

# cat /etc/hosts

127.0.0.1 http://www.example.com

然后使用PHP测试文件分别测试PHP是否部署成功以及PHP与MariaDB是否联通

最终我们访问www.example.com查看wordpress是否部署成功。

至此使用Docker+Ansible部署Wordpress大功告成。。。 

这里我们先给大家介绍一些常用Dockerfile编写规范.

Docker配置传送门: http://www.showerlee.com/archives/1758

1.FROM 

FROM <image>:<tag>

FROM会使用当前本地或者远程Docker仓库的image, 这个要首先写到该脚本的第一行.

例:

FROM centos67base/apache:apache_base

这里代表我们使用本地的centos67base/apache镜像.

查看本地缓存镜像可以使用

# docker images

2.MAINTAINER

MAINTAINER <name>

这个会给生成的镜像创建一个作者名

3.RUN

RUN <command> (命令行格式, 执行一段shell命令)

RUN ["executable", "param1", "param2"] (列表格式, "executable"可以更改不同shell格式, 例如/bin/sh或/bin/bash, "param1"代表shell参数)

这个可以理解为执行shell命令到image, 一般来说用来给镜像安装package, 或者更改系统配置等.

例:

RUN /bin/bash -c echo 123
RUN ["/bin/bash" "-c" "echo 123"]

4.CMD

CMD ["executable","param1","param2"] (同RUN)

CMD ["param1","param2"] (使用系统默认shell)

CMD command param1 param2 (命令行格式)

这个CMD基本使用格式与RUN类似, 区别在于如果存在多个CMD行, 则只有最后一行生效, 这个我们常用执行默认的shell命令给该容器.

5.LABEL

LABEL <key>=<value> <key>=<value> <key>=<value> ...

例:

LABEL "com.example.vendor"="ACME Incorporated"
LABEL com.example.label-with-value="foo"
LABEL version="1.0"
LABEL description="This text illustrates \
that label-values can span multiple lines."

这个可以理解为给image传送一个键值对.

可以使用如下命令查看已Build好的image LABEL

# docker inspect

6.EXPOSE

EXPOSE <port> [<port>...]

这个用来通知容器在某一具体端口进行监听, 但不会打开该端口, 如需打开要在创建容器容器的时候配合-P参数使用.

7.ENV

ENV <key> <value>

ENV <key>=<value> ...

例如:

ENV myName="John Doe" myDog=Rex\ The\ Dog \
       myCat=fluffy

相当于给镜像添加环境变量.

8.ADD

ADD <src>... <dest>

ADD ["<src>",... "<dest>"] (这种写法会避免路径名含有空格而报错)

ADD命令会copy本地目录, 文件或远程URL到容器目的路径, <src>, <dest>支持通配符和绝对相对路径.

例如:

ADD hom* /mydir/        # adds all files starting with "hom"
ADD hom?.txt /mydir/    # ? is replaced with any single character, e.g., "home.txt"
ADD test relativeDir/   # adds "test" to `WORKDIR`/relativeDir/
ADD test /absoluteDir   # adds "test" to /absoluteDir

9.COPY

COPY <src>... <dest>

COPY ["<src>",... "<dest>"] (这种写法会避免路径名含有空格而报错)

本身用法和ADD类似, 唯独不能使用远程URL

10.VOLUME

VOLUME ["/data"]
VOLUME会给该image创建一个挂载点.

11.WORKDIR

WORKDIR /path/to/workdir

顾名思义, 指定一个当前的工作目录.

12.Dockerfile实例

# Nginx
#
# VERSION               0.0.1

FROM      ubuntu
MAINTAINER Victor Vieux <victor@docker.com>

LABEL Description="This image is used to start the foobar executable" Vendor="ACME Products" Version="1.0"
RUN apt-get update && apt-get install -y inotify-tools nginx apache2 openssh-server


# Firefox over VNC
#
# VERSION               0.3

FROM ubuntu

# Install vnc, xvfb in order to create a 'fake' display and firefox
RUN apt-get update && apt-get install -y x11vnc xvfb firefox
RUN mkdir ~/.vnc
# Setup a password
RUN x11vnc -storepasswd 1234 ~/.vnc/passwd
# Autostart firefox (might not be the best way, but it does the trick)
RUN bash -c 'echo "firefox" >> /.bashrc'

EXPOSE 5900
CMD    ["x11vnc", "-forever", "-usepw", "-create"]

更多详细配置请参考: http://www.showerlee.com/archives/1758 第10项

Docker源意码头工人, 是一款更轻量级的虚拟化快速部署工具, 他的优点在于非常傻瓜的配置和管理, CPU/内存的低消耗, 快速开/关机, 可以非常方便的运行和释放容器, 便捷的连接宿主机器和容器以及0成本的commit and export到其他任意环境, 绝对是一款优于vagrant等同类型工具的SA居家旅行必备利器.

通俗来讲Docker其实就是将我们在虚拟机上定制的系统打包成一个私有Docker镜像, 然后通过将镜像传到本地, Docker本身会利用Vbox创建一个虚拟机实例(该虚拟机内核支持Docker)并与本地共享同一个目录结构, 最终利用docker内嵌命令使用我们的私有镜像创建若干子虚拟容器, 从而实现本地虚拟化部署.

Windows与MAC平台内核因为本身与Docker不兼容, 所以我们需要本地安装VirtualBox+Docker, 利用VirtualBox创建一个可兼容Docker的虚拟机.

Linux内核为3.0以上的发行版可直接安装Docker, 无需安装第三方虚拟机.

本文将详细介绍如何创建一个私有的Docker容器, 并部署到不同环境中, 使用Windows和Linux平台的同学也可以参考本文的配置.

话说一开始我是拒绝的, 但经过2天的研究和部署, 我震精了, 这他喵的绝对是神一般的利器, 他的跨平台的一些功能(有待挖掘)绝对可以成为未来的运维部署发展趋势.

还有不得不佩服老外的想象力, 这官网的动物代表不同的系统环境, Docker代表码头工人的对不同系统环境的迁移整合, 整得都成动物世界了, 不过还真别说, 这码头工人的效率还真不错, 动物园的动物们对这小哥的发货效率绝对满意, 必须五星好评.

不说了, 我去啃老外的面包去了.... 

官方文档: https://docs.docker.com

一.环境部署

Local system:  MAC OS X 10.10.5
VirtualBox:    VirtualBox-5.0.10
docker:   DockerToolbox-1.9.1b.pkg
Docker sample system: CentOS6.7 x64 minimal (docker.example.com)

本机需安装Virtualbox和Docker,  安装过程(略).

TIP:本机为ubuntu或其他Linux发行版内核为3.0以上的朋友因为本地系统内嵌KVM, Docker可直接调用KVM, 所以无需安装Virtualbox作为Docker虚拟机.

二.Docker配置

1.在Virtualbox新建虚拟机并安装CentOS 6.7 x64(略)

2.安装openssh-clients (VirtualBox虚拟机)

# yum install openssh-clients -y

3.登陆CentOS虚拟机并安装EPEL YUM源 (VirtualBox虚拟机)

# rpm -Uvh http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

4.配置yum (VirtualBox虚拟机)

# cd /etc/yum.repos.d
# vi CentOS-Base.repo
找到[centosplus]源下,修改enabled=1
# sed -i 's/$releasever/6/g' CentOS-Base.repo
# sed -i 's/$basearch/x86_64/g' CentOS-Base.repo

5.安装Docker并开启Docker服务 (VirtualBox虚拟机)

# yum install docker-io -y
# service docker start

6.部署docker

1).常用命令(本地)

# docker images

REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE

# docker ps -a

CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

可以看到目前docker并未有可用的镜像.

2).配置脚本目录 (VirtualBox虚拟机)

# cd /root

# mkdir scripts

# cd scripts

# vi createimg.sh

源自:https://github.com/docker/docker/blob/master/contrib/mkimage-yum.sh

#!/usr/bin/env bash
#
# Create a base CentOS Docker image.
#
# This script is useful on systems with yum installed (e.g., building
# a CentOS image on CentOS).  See contrib/mkimage-rinse.sh for a way
# to build CentOS images on other systems.

usage() {
    cat <<EOOPTS
$(basename $0) [OPTIONS] <name>
OPTIONS:
  -y <yumconf>  The path to the yum config to install packages from. The
                default is /etc/yum.conf for Centos/RHEL and /etc/dnf/dnf.conf for Fedora
EOOPTS
    exit 1
}

# option defaults
yum_config=/etc/yum.conf
if [ -f /etc/dnf/dnf.conf ] && command -v dnf &> /dev/null; then
yum_config=/etc/dnf/dnf.conf
alias yum=dnf
fi 
while getopts ":y:h" opt; do
    case $opt in
        y)
            yum_config=$OPTARG
            ;;
        h)
            usage
            ;;
        \?)
            echo "Invalid option: -$OPTARG"
            usage
            ;;
    esac
done
shift $((OPTIND - 1))
name=$1

if [[ -z $name ]]; then
    usage
fi

target=$(mktemp -d --tmpdir $(basename $0).XXXXXX)

set -x

mkdir -m 755 "$target"/dev
mknod -m 600 "$target"/dev/console c 5 1
mknod -m 600 "$target"/dev/initctl p
mknod -m 666 "$target"/dev/full c 1 7
mknod -m 666 "$target"/dev/null c 1 3
mknod -m 666 "$target"/dev/ptmx c 5 2
mknod -m 666 "$target"/dev/random c 1 8
mknod -m 666 "$target"/dev/tty c 5 0
mknod -m 666 "$target"/dev/tty0 c 4 0
mknod -m 666 "$target"/dev/urandom c 1 9
mknod -m 666 "$target"/dev/zero c 1 5

# amazon linux yum will fail without vars set
if [ -d /etc/yum/vars ]; then
mkdir -p -m 755 "$target"/etc/yum
cp -a /etc/yum/vars "$target"/etc/yum/
fi

yum -c "$yum_config" --installroot="$target" --releasever=/ --setopt=tsflags=nodocs \
    --setopt=group_package_types=mandatory -y groupinstall Core
yum -c "$yum_config" --installroot="$target" -y clean all

cat > "$target"/etc/sysconfig/network <<EOF
NETWORKING=yes
HOSTNAME=localhost.localdomain
EOF

# effectively: febootstrap-minimize --keep-zoneinfo --keep-rpmdb --keep-services "$target".
#  locales
rm -rf "$target"/usr/{{lib,share}/locale,{lib,lib64}/gconv,bin/localedef,sbin/build-locale-archive}
#  docs and man pages
rm -rf "$target"/usr/share/{man,doc,info,gnome/help}
#  cracklib
rm -rf "$target"/usr/share/cracklib
#  i18n
rm -rf "$target"/usr/share/i18n
#  yum cache
rm -rf "$target"/var/cache/yum
mkdir -p --mode=0755 "$target"/var/cache/yum
#  sln
rm -rf "$target"/sbin/sln
#  ldconfig
rm -rf "$target"/etc/ld.so.cache "$target"/var/cache/ldconfig
mkdir -p --mode=0755 "$target"/var/cache/ldconfig

version=
for file in "$target"/etc/{redhat,system}-release
do
    if [ -r "$file" ]; then
        version="$(sed 's/^[^0-9\]*\([0-9.]\+\).*$/\1/' "$file")"
        break
    fi
done

if [ -z "$version" ]; then
    echo >&2 "warning: cannot autodetect OS version, using '$name' as tag"
    version=$name
fi

tar --numeric-owner -c -C "$target" . | docker import - $name:$version

docker run -i -t $name:$version echo success

rm -rf "$target"

# chmod +x createimg.sh

3).执行脚本(VirtualBox虚拟机)

该脚本将自动生成docker镜像,centos6.7base为镜像名,脚本执行log保存在/tmp下

# ./createimg.sh centos6.7base

4).查看生成的docker镜像(VirtualBox虚拟机)

# docker images

REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
centos6.7base       6.7                 8355bdcdcde0        5 minutes ago       166.3 MB

5).查看生成的docker容器(VirtualBox虚拟机)

# docker ps -a

CONTAINER ID        IMAGE               COMMAND             CREATED              STATUS                          PORTS               NAMES
a7392bd18606        centos6.7base:6.7   "echo success"      About a minute ago   Exited (0) About a minute ago                       sleepy_swartz 

6).删除容器(VirtualBox虚拟机)

# docker rm a7392bd18606

7).导出Docker缓存镜像到磁盘(VirtualBox虚拟机)

# docker save 8355bdcdcde0 > /tmp/centos67base.tar
# ll /tmp

-rw-r--r--  1 root root 174095360 Dec 14 04:23 centos67base.tar
-rw-r--r--  1 root root     20284 Oct 16 03:21 vboxguest-Module.symvers
-rw-------. 1 root root         0 Oct 16 00:21 yum.log

8).拷贝该镜像到本地(本地)

Tip: 因为笔者是MAC系统, 所以系统自带CLI scp命令, windows用户可以使用winscp进行ssh传输.

# cd ~/Work/Docker/centos67base

# scp root@docker.example.com:/tmp/centos67base.tar .

7.使用MAC下的docker加载该镜像

在MAC Application/Docker目录下运行Docker Quickstart Terminal, 会弹出Terminal并运行docker的相应初始化配置, 完成后会进入CLI界面.

TIP: 这里实际上是Docker调用了本地VirtualBox, 并在其创建了一个新的Docker虚拟机instance(可以通过打开VBOX GUI查看), 所以我们实际上已经进入了这个instance CLI界面, 但笔者惊奇的发现这里的目录结构和MAC本地是一致的, 可以得出来的结论是Docker和MAC共同share一个CLI. 所以大家可能会有错觉自己还在本地, 实际情况是我们已经登录到Docker母虚拟机, 随后我们会在这个母机上创建多个容器(子虚拟机).

Windows平台下该执行文件可在开始菜单获取

1).加载刚才制作的镜像(Docker CLI)

# docker load < centos67base.tar
# docker images

REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
<none>              <none>              8355bdcdcde0        21 hours ago        166.3 MB

2).添加Repository和tag(Docker CLI)

默认这两项都为空,所以可以根据IMAGE ID去给这个image添加相关的标注
# docker tag 8355bdcdcde0 centos67base:6.7
# docker images

REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
centos67base        6.7                 8355bdcdcde0        21 hours ago        166.3 MB

3).创建docker容器并加载该镜像(Docker CLI)

# docker run -i -t --privileged 8355bdcdcde0 /bin/bash

Tip: -i和-t配合使用实际上是为了保持一个TTY标准输入处于一个LISTEN状态无论是否被连接, 简单来说就是创建一个CLI终端连接.

       --privileged用来开启扩展权限给容器, 主要开启容器iptables等类似内核集成模块, 默认无此参数使用iptables会报错.

       8355bdcdcde0为IMAGE ID, /bin/bash表示使用bash shell格式进入终端.

已成功登录子虚拟容器

# uname -a    

Linux 33e93edb56e3 4.1.13-boot2docker #1 SMP Fri Nov 20 19:05:50 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

# cat /etc/system-release

CentOS release 6.7 (Final)

可以看到已经登陆到子虚拟容器, 至此我们就成功制作并加载了我们私有的docker镜像.

4).关闭子虚拟容器

# exit
# docker ps -a

CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                     PORTS               NAMES
33e93edb56e3        8355bdcdcde0        "/bin/bash"         16 minutes ago      Exited (0) 8 seconds ago                       awesome_joliot

这里可以看到这个容器在16分钟前生成, 现已经关闭了8秒钟, 简单来说这里去退出Docker容器可以理解为虚拟机的保存状态(挂起), 在我们创建这个容器后的所有的change都不会丢失, 除非你删除这个容器, 但之前开启的服务socket会丢失, 所以重新打开这个容器需重启该服务.

5).重新开启并连接该容器(Docker CLI)

# docker start 33e93edb56e3

# docker attach 33e93edb56e3

Tip: 这里执行attach后不会立即进入容器命令行,需要再回车才能显示.

6).断开该容器(Docker CLI)

这里可以在CLI下利用快捷键ctrl+P, 然后Ctrl+Q去快速切回本地环境.

# docker ps -a

CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
33e93edb56e3        8355bdcdcde0        "/bin/bash"         33 minutes ago      Up 7 minutes                            awesome_joliot

这里可以看到容器并没有退出, 其状态显示为已运行7分钟.

8).重命名该容器(Docker CLI)

# docker rename awesome_joliot centos6.7_base

CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
33e93edb56e3        8355bdcdcde0        "/bin/bash"         35 minutes ago      Up 9 minutes                            centos6.7_base

9).重新连接该容器(Docker CLI)

# docker attach 33e93edb56e3

这里也可以直接使用刚才修改后的容器名登陆 

# docker attach centos6.7_base

所以实际上整个Docker的运行步骤:
加载tar包到docker缓存镜像 => 利用该镜像创建容器 => 启动容器 => 连接容器

8).删除并重新创建容器(Docker CLI)

# docker rm 33e93edb56e3

# docker run -i -t --privileged 8355bdcdcde0 /bin/bash

# exit

$ docker ps -a

CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                     PORTS               NAMES
861dd99563ef        8355bdcdcde0        "/bin/bash"         18 seconds ago      Exited (0) 2 seconds ago                       gloomy_lovelace

我们可以看到容器ID变成861dd99563ef, 之前的容器和容器的所有change已经彻底删除.

到这里我们介绍了Docker的使用, 基本上公司sa会将一份基本的Docker系统镜像分发给每位开发人员, 开发人员会利用这个统一的系统环境去coding, 如果在开发过程中遇到了一个BUG或者问题无法修复, 如何将当前的状态备份并分发给其他同事去处理? 这里就可以利用Dock commit去保存我们的系统状态到镜像中, 利用之前的命令export and transfer该镜像给相应Programmer进行进一步的处理, 当然我们也可以利用该功能去保存不同开发产品的版本状态.

8.Commit Docker镜像(Docker CLI)

我们可以安装一个apache并commit镜像来模拟上述情况.

# docker start 861dd99563ef

e8b478f41b5c

# docker attach 861dd99563ef

# exit

# docker commit 861dd99563ef centos67base/apache:apache_base

d9aef85ea282439634c36b3e8d51356396bb116c6ea51f11f8b128bf1fdb89a6

# docker images

REPOSITORY            TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
centos67base/apache   apache_base         d9aef85ea282        7 seconds ago       233.6 MB
centos67base          6.7                 8355bdcdcde0        23 hours ago        166.3 MB

我们可以看到我们生成了一个基于centos67base的apache版本镜像

9.导出该新版本镜像(Docker CLI)

# docker save centos67base/apache > centos67_apache.tar

# ll

-rw-r--r-- 1 XXX staff 242053632 Dec 15 16:34 centos67_apache.tar
-rw-r--r-- 1 XXX staff 174095360 Dec 15 13:26 centos67base.tar

TIP: 由于Docker虚拟机与本地MAC share一个CLI, 这里相当于这个镜像已经保存在本地.

10.通过Dockerfile Build虚拟镜像(Docker CLI)

1).这里我们通过Docker脚本在原有centos67base/apache镜像基础上安装vim, 实现一个简单的自动化部署.

# cd ~/Work/Docker/Build

# vi Dockerfile

FROM centos67base/apache:apache_base
RUN yum install vim -y

TIP: 这里centos67base/apache:apache_base对应REPOSITORY:TAG

2).开始Build

# docker build -t centos67base/apache/vim .

Sending build context to Docker daemon 416.2 MB
Step 1 : FROM centos67base/apache:apache_base
 ---> d9aef85ea282
Step 2 : RUN yum install vim -y
 ---> Running in 721c0cd73025
Loaded plugins: fastestmirror
Setting up Install Process
Determining fastest mirrors
 * base: mirrors.yun-idc.com
 * extras: mirrors.yun-idc.com
 * updates: mirrors.yun-idc.com
Resolving Dependencies
--> Running transaction check
---> Package vim-enhanced.x86_64 2:7.4.629-5.el6 will be installed
--> Processing Dependency: vim-common = 2:7.4.629-5.el6 for package: 2:vim-enhanced-7.4.629-5.el6.x86_64
--> Processing Dependency: which for package: 2:vim-enhanced-7.4.629-5.el6.x86_64
--> Processing Dependency: perl(:MODULE_COMPAT_5.10.1) for package: 2:vim-enhanced-7.4.629-5.el6.x86_64
--> Processing Dependency: libperl.so()(64bit) for package: 2:vim-enhanced-7.4.629-5.el6.x86_64
--> Processing Dependency: libgpm.so.2()(64bit) for package: 2:vim-enhanced-7.4.629-5.el6.x86_64
--> Running transaction check
---> Package gpm-libs.x86_64 0:1.20.6-12.el6 will be installed
---> Package perl.x86_64 4:5.10.1-141.el6_7.1 will be installed
--> Processing Dependency: perl(version) for package: 4:perl-5.10.1-141.el6_7.1.x86_64
--> Processing Dependency: perl(Pod::Simple) for package: 4:perl-5.10.1-141.el6_7.1.x86_64
--> Processing Dependency: perl(Module::Pluggable) for package: 4:perl-5.10.1-141.el6_7.1.x86_64
---> Package perl-libs.x86_64 4:5.10.1-141.el6_7.1 will be installed
---> Package vim-common.x86_64 2:7.4.629-5.el6 will be installed
--> Processing Dependency: vim-filesystem for package: 2:vim-common-7.4.629-5.el6.x86_64
---> Package which.x86_64 0:2.19-6.el6 will be installed
--> Running transaction check
---> Package perl-Module-Pluggable.x86_64 1:3.90-141.el6_7.1 will be installed
---> Package perl-Pod-Simple.x86_64 1:3.13-141.el6_7.1 will be installed
--> Processing Dependency: perl(Pod::Escapes) >= 1.04 for package: 1:perl-Pod-Simple-3.13-141.el6_7.1.x86_64
---> Package perl-version.x86_64 3:0.77-141.el6_7.1 will be installed
---> Package vim-filesystem.x86_64 2:7.4.629-5.el6 will be installed
--> Running transaction check
---> Package perl-Pod-Escapes.x86_64 1:1.04-141.el6_7.1 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package                  Arch      Version                    Repository  Size
================================================================================
Installing:
 vim-enhanced             x86_64    2:7.4.629-5.el6            base       1.0 M
Installing for dependencies:
 gpm-libs                 x86_64    1.20.6-12.el6              base        28 k
 perl                     x86_64    4:5.10.1-141.el6_7.1       updates     10 M
 perl-Module-Pluggable    x86_64    1:3.90-141.el6_7.1         updates     40 k
 perl-Pod-Escapes         x86_64    1:1.04-141.el6_7.1         updates     33 k
 perl-Pod-Simple          x86_64    1:3.13-141.el6_7.1         updates    213 k
 perl-libs                x86_64    4:5.10.1-141.el6_7.1       updates    579 k
 perl-version             x86_64    3:0.77-141.el6_7.1         updates     52 k
 vim-common               x86_64    2:7.4.629-5.el6            base       6.7 M
 vim-filesystem           x86_64    2:7.4.629-5.el6            base        15 k
 which                    x86_64    2.19-6.el6                 base        38 k

Transaction Summary
================================================================================
Install      11 Package(s)

Total download size: 19 M
Installed size: 59 M
Downloading Packages:
--------------------------------------------------------------------------------
Total                                           7.0 MB/s |  19 MB     00:02     
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : 1:perl-Pod-Escapes-1.04-141.el6_7.1.x86_64                  1/11 
  Installing : 1:perl-Module-Pluggable-3.90-141.el6_7.1.x86_64             2/11 
  Installing : 3:perl-version-0.77-141.el6_7.1.x86_64                      3/11 
  Installing : 4:perl-libs-5.10.1-141.el6_7.1.x86_64                       4/11 
  Installing : 1:perl-Pod-Simple-3.13-141.el6_7.1.x86_64                   5/11 
  Installing : 4:perl-5.10.1-141.el6_7.1.x86_64                            6/11 
  Installing : 2:vim-filesystem-7.4.629-5.el6.x86_64                       7/11 
  Installing : 2:vim-common-7.4.629-5.el6.x86_64                           8/11 
  Installing : which-2.19-6.el6.x86_64                                     9/11 
  Installing : gpm-libs-1.20.6-12.el6.x86_64                              10/11 
  Installing : 2:vim-enhanced-7.4.629-5.el6.x86_64                        11/11 
  Verifying  : 1:perl-Pod-Simple-3.13-141.el6_7.1.x86_64                   1/11 
  Verifying  : 1:perl-Pod-Escapes-1.04-141.el6_7.1.x86_64                  2/11 
  Verifying  : gpm-libs-1.20.6-12.el6.x86_64                               3/11 
  Verifying  : which-2.19-6.el6.x86_64                                     4/11 
  Verifying  : 2:vim-enhanced-7.4.629-5.el6.x86_64                         5/11 
  Verifying  : 2:vim-filesystem-7.4.629-5.el6.x86_64                       6/11 
  Verifying  : 1:perl-Module-Pluggable-3.90-141.el6_7.1.x86_64             7/11 
  Verifying  : 2:vim-common-7.4.629-5.el6.x86_64                           8/11 
  Verifying  : 3:perl-version-0.77-141.el6_7.1.x86_64                      9/11 
  Verifying  : 4:perl-libs-5.10.1-141.el6_7.1.x86_64                      10/11 
  Verifying  : 4:perl-5.10.1-141.el6_7.1.x86_64                           11/11 

Installed:
  vim-enhanced.x86_64 2:7.4.629-5.el6                                           

Dependency Installed:
  gpm-libs.x86_64 0:1.20.6-12.el6                                               
  perl.x86_64 4:5.10.1-141.el6_7.1                                              
  perl-Module-Pluggable.x86_64 1:3.90-141.el6_7.1                               
  perl-Pod-Escapes.x86_64 1:1.04-141.el6_7.1                                    
  perl-Pod-Simple.x86_64 1:3.13-141.el6_7.1                                     
  perl-libs.x86_64 4:5.10.1-141.el6_7.1                                         
  perl-version.x86_64 3:0.77-141.el6_7.1                                        
  vim-common.x86_64 2:7.4.629-5.el6                                             
  vim-filesystem.x86_64 2:7.4.629-5.el6                                         
  which.x86_64 0:2.19-6.el6                                                     

Complete!
 ---> dfd4d0e4442c
Removing intermediate container 721c0cd73025
Successfully built dfd4d0e4442c

# docker images

REPOSITORY                TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
centos67base/apache/vim   latest              dfd4d0e4442c        10 seconds ago      327 MB
centos67base/apache       apache_base         d9aef85ea282        2 weeks ago         233.6 MB

这里我们就在之前镜像的基础上Build出一个新的image, 有特殊需求的同学可以在此脚本的基础上编写更加详细的部署步骤.

更多配置可以参考官方文档: https://docs.docker.com/engine/reference/builder/ 

11. 本地与容器间传输文件

我们可以通过docker内建命令docker cp对已创建好的容器来进行文件传输.

# docker ps -a

CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
3d9136b53a7f        8355bdcdcde0        "/bin/bash"         6 minutes ago       Up 6 minutes                            centos67base

# ls

Dockerfile 

# docker cp Dockerfile centos67base:/Dockerfile

# docker attach centos67base

# ls

Dockerfile  boot  etc   lib    media  opt   root  selinux  sys  usr
bin         dev   home  lib64  mnt    proc  sbin  srv      tmp  var

12. 访问容器端口

如果我们需要在本地MAC访问容器的具体服务, 我们可以通过DOCKER的NAT Forwarding来实现访问该容器端口服务.

这里我们可以利用之前安装的apache实现MAC本地访问容器的apache web服务.

1). 创建apache容器, 并打开其80 NAT Forwarding (Docker CLI)

# docker run -i -t --privileged -p 80:80 d9aef85ea282 /bin/bash

Tip: 80:80 前者为母虚拟机端口, 后者为子虚拟容器端口

2). 登陆进容器, 开启apache服务并登出(子虚拟容器)

# service httpd start

# echo "This is a test web page" > /var/www/html/index.html

Ctrl+P, 然后Ctrl+Q登出

3).访问该服务(Docker CLI)

查看Docker母虚拟机IP信息

# docker-machine ls

NAME      ACTIVE   DRIVER       STATE     URL                         SWARM   ERRORS
default   *        virtualbox   Running   tcp://192.168.99.101:2376

查看子容器NAT配置

# docker ps -a

# docker port cce502d8a21a

80/tcp -> 0.0.0.0:80

这里可以看到NAT已配置成功

成功访问web服务

# curl http://192.168.99.101/

This is a test web page!!!!

TIP: 这里由于我们实际上是在VirtualBox下的一个Docker instance虚拟机下, 所以访问的IP为192.168.99.101, 而不是本地127.0.0.1.

这里再次给同学们强调虽然我们的目录结构感觉是在本地, 但实际上我们处在Docker虚拟机下, 只是这个虚拟机嵌入到本地目录中.

至此我们大致将SA的Docker原理, 制作过程和配置使用详细讲解给大家, 其中借鉴了很多老外的部署经验, 希望有英语基础的同学可以去国外的技术网站去获取相关的知识. 随后我会陆续更新Docker自动化部署等文档.

Hope you enjoy…