本文介绍在k8s环境中进行jenkins server的部署和配置。Jenkins是一个开源的、功能强大的持续集成和持续构建工具，采用master和salve架构，我们通过将jenkins集成环境部署在k8s集群中，可以实现jenkins slave按需创建、动态的伸缩。同时也提供了在k8s环境中应用的持续部署解决方案。

一、准备docker镜像文件

1、编译jenkins server docker镜像，默认的jenkis镜像已包含jdk，版本为1.8.0_171

# cat dockerfileFROM jenkinsMAINTAINER ylw@fjhb.cnENV MAVEN_HOME /usr/local/mavenENV JAVA_HOME  /usr/local/javaENV CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jarENV PATH ${JAVA_HOME}/bin:${MAVEN_HOME}/bin:${PATH}COPY apache-maven-3.5.4 /usr/local/mavenUSER rootRUN mkdir -p /usr/local/maven/repository  &&  ln -s /usr/java/jdk1.8.0_171 /usr/local/java

# docker build -t harbor.59iedu.com/fjhb/jenkins:2018-08-12-v1 . # docker push harbor.59iedu.com/fjhb/jenkins:2018-08-12-v1

2、编译jenkins slave镜像可以根据实际情况配置maven内网私服nexus，私服可以避免编译过程中通过公网下载依赖的jar包，配置私服需要把对应的setting.xml文件打包到apache-maven-3.5.4/conf目录下；libltdl.so.7文件的获取路径为操作系统路径/usr/lib64/libltdl.so.7（实际上是个软链接，需要copy出来重命名）slave.jar文件的获取路径为http://jenkins-server/jnlpJars/slave.jar

# cat DockerfileFROM openshift/base-centos7MAINTAINER ylw@fjhb.cnCOPY apache-maven-3.5.4 /usr/local/mavenCOPY jdk1.8.0_171       /usr/local/javaCOPY kubectl            /usr/local/bin/kubectlCOPY libltdl.so.7 /usr/lib64/libltdl.so.7COPY slave.jar /usr/share/jenkins/slave.jar COPY jenkins-slave /usr/local/bin/jenkins-slaveENV HOME /home/jenkinsENV AGENT_WORKDIR=/home/jenkins/agentENV JAVA_HOME /usr/local/javaENV MAVEN_HOME /usr/local/maven/ENV CLASSPATH .:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jarENV PATH ${JAVA_HOME}/bin:${MAVEN_HOME}/bin:${PATH} ENV MAVEN_CONFIG "$USER_HOME_DIR/.m2"RUN  chmod 755 /usr/share/jenkins \  && chmod 644 /usr/share/jenkins/slave.jar RUN mkdir -p /home/jenkins/.jenkins \ && mkdir -p ${AGENT_WORKDIR} \ && yum -y install git subversion sshpassVOLUME /home/jenkins/.jenkinsVOLUME ${AGENT_WORKDIR}WORKDIR /home/jenkinsUSER rootENTRYPOINT ["jenkins-slave"]

# docker build -t harbor.59iedu.com/fjhb/jenkins-slave-toolkit:2018-08-10-v1 . # docker push harbor.59iedu.com/fjhb/jenkins-slave-toolkit:2018-08-10-v1

二、创建jenkins server

1、创建pv和pvc

# cat pv.yaml ---apiVersion: v1kind: PersistentVolumemetadata:  name: jenkins-master-volspec:  capacity:    storage: 5Gi   accessModes:  - ReadWriteMany   nfs:     path: /home/jenkins    server: 192.168.115.6  persistentVolumeReclaimPolicy: Recycle ---apiVersion: v1kind: PersistentVolumemetadata:  name: maven-repositoryspec:  capacity:    storage: 5Gi  accessModes:  - ReadWriteMany  nfs:    path: /home/maven    server: 192.168.115.6  persistentVolumeReclaimPolicy: Recycle---kind: PersistentVolumeClaimapiVersion: v1metadata:  name: jenkins-master-claimspec:  accessModes:    - ReadWriteMany  resources:    requests:      storage: 5Gi---kind: PersistentVolumeClaimapiVersion: v1metadata:  name: maven-repository-claimspec:  accessModes:    - ReadWriteMany  resources:    requests:      storage: 5Gi

2、创建deployment和service

# cat deploy.yaml ---apiVersion: extensions/v1beta1kind: Deploymentmetadata:  name: jenkins-masterspec:  template:    metadata:      labels:        name: jenkins-master    spec:      securityContext:        fsGroup: 1000      containers:        - name: jenkins-master          image: harbor.59iedu.com/fjhb/jenkins:2018-08-12-v1          imagePullPolicy: Always          ports:            - containerPort: 8080              name: http            - containerPort: 50000              name: agent          volumeMounts:            - name: jenkins-master-vol              mountPath: /var/jenkins_home            - name: maven-repository              mountPath: /opt/maven/repository            - name: docker              mountPath: /usr/bin/docker            - name: docker-sock              mountPath: /var/run/docker.sock      volumes:        - name: jenkins-master-vol          persistentVolumeClaim:            claimName: jenkins-master-claim        - name: maven-repository          persistentVolumeClaim:            claimName: maven-repository-claim        - name: docker          hostPath:            path: /usr/bin/docker        - name: docker-sock          hostPath:            path: /var/run/docker.sock      serviceAccount: "jenkins-master"      imagePullSecrets:        - name: harborsecret---apiVersion: v1kind: Servicemetadata:  name: jenkins-masterspec:  type: NodePort  ports:    - port: 8080      name: http      targetPort: 8080      nodePort: 8452    - port: 50000      name: agent      nodePort: 50000      targetPort: 50000   selector:    name: jenkins-master

3、rbac授权

# cat sa.yaml ---apiVersion: v1kind: ServiceAccountmetadata:  name: jenkins-master  namespace: default---kind: RoleapiVersion: rbac.authorization.k8s.io/v1beta1metadata:  name: jenkins-masterrules:- apiGroups: [""]  resources: ["pods"]  verbs: ["create","delete","get","list","patch","update","watch"]- apiGroups: [""]  resources: ["pods/exec"]  verbs: ["create","delete","get","list","patch","update","watch"]- apiGroups: [""]  resources: ["pods/log"]  verbs: ["get","list","watch"]- apiGroups: [""]  resources: ["secrets"]  verbs: ["get"]---apiVersion: rbac.authorization.k8s.io/v1beta1kind: RoleBindingmetadata:  name: jenkins-masterroleRef:  apiGroup: rbac.authorization.k8s.io  kind: Role  name: jenkins-mastersubjects:- kind: ServiceAccount  name: jenkins-master  namespace: default

4、default sa的rbac授权

# cat default-sa.yaml ---kind: RoleapiVersion: rbac.authorization.k8s.io/v1beta1metadata:  name: default-rolerules:- apiGroups: [""]  resources: ["pods"]  verbs: ["create","delete","get","list","patch","update","watch"]- apiGroups: [""]  resources: ["pods/exec"]  verbs: ["create","delete","get","list","patch","update","watch"]- apiGroups: [""]  resources: ["pods/log"]  verbs: ["get","list","watch"]- apiGroups: [""]  resources: ["secrets"]  verbs: ["get"]---apiVersion: rbac.authorization.k8s.io/v1beta1kind: RoleBindingmetadata:  name: default-rolebindingroleRef:  apiGroup: rbac.authorization.k8s.io  kind: Role  name: default-role subjects:- kind: ServiceAccount  name: default  namespace: default

三、初始化jenkins server

1、通过秘钥解锁jenkins（本文为了方便局域网其他主机访问，在vmware上配置了nat规则）

2、配置代理3、安装插件4、创建管理员账号

四、配置jenkins server

1、系统管理 —— 系统设置 —— 新增一个云”kubernetes”

Kubernetes URL: 输入api-server的地址Jenkins URL: 输入jenkins server的服务名,端口8080Jenkins Tunnel: 指的是slave连接master的端口，默认是50000上图pod的模板名称为jenkins-slave，Container的模板名称为jnlp。这里有非常重要的两点要注意：当Container的模板名称为jnlp的时候，jenkins-slave才会使用下面配置的docker镜像来启动pod，如果不为jnlp，则会使用默认的镜像jenkins/jnlp-slave:alpine

当使用自定义的docker镜像来启动jenkins slave pod的时候，下面的command to run（默认值是 sh -c）和arguments to pass to the command(默认值是cat)两个值需要清空。否则会出现jenkins slave jnlp连接不上master的情况，尝试100次连接之后销毁pod，然后再创建一个pod继续尝试连接，无限循环。

2、系统管理 —— Configure Global Security

确认jnlp agent的端口默认为50000，如果有修改，要保障这里的配置及前面部署deployment、service的端口配置、前文的云环境Jenkins Tunnel设置保持一致

3、系统管理 —— Global Tool Configuration

在这里设置对应的工具及环境变量，为了避免不必要的问题，前面通过dockerfile把jenkins server 和jenkins slave的环境变量调整成一致, java目录通过软连接的方式实现。

4、系统管理 —— 管理插件

推荐安装的几个插件：maven、 gitlab 、subversion、pipeline、Kubernetes Continuous Deploy、Publish Over SSH完成插件安装后需要对jenkins server进行重启操作，可以点击“系统管理 ”——“准备关机”来完成重启操作，至此我们就完成了jenkins server在k8s环境中的部署和配置工作，下文开始介绍使用jenkins完成项目构建和发布。