使用Kaniko构建Docker镜像

由 SuKai August 1, 2021

在Kubernetes集群中构建容器镜像，有两种方式：1，Docker in Docker方式，将Kubernetes的主机节点的docker socket文件共享给Pod中的容器，在Pod容器中运行docker命令构建镜像。2，使用Kaniko来构建容器镜像，Kaniko不依赖Docker daemon，有用户态空间就能执行dockerfile的全部指令。这使得在Kubernetes中构建容器镜像变得简单又安全。

下面介绍两种在k8s中使用Kaniko构建容器镜像的方式，1，Kubeflow fairing构建容器镜像。2，GitLab流水线构建容器镜像。因为最近在接触kubeflow，所以才有了kubeflow fairing构建容器镜像的玩法，kubeflow fairing主要用于将开发人员完成的模型打包，用于在Kubernetes中创建训练任务。

| Kubeflow Fairing构建容器镜像

!pip install msrestazure -i https://pypi.tuna.tsinghua.edu.cn/simple --user

import logging
from kubeflow.fairing import constants
from kubeflow.fairing.preprocessors import base as base_preprocessor
DOCKER_REGISTRY = '192.168.0.93/ai'
constants.constants.KANIKO_IMAGE = "aiotceo/kaniko-executor:v1.6.0"

from kubeflow.fairing.builders import cluster

# output_map is a map of extra files to add to the notebook.
# It is a map from source location to the location inside the context.
output_map =  {
    "Dockerfile.model": "Dockerfile",
    "model.py": "model.py"
}

preprocessor = base_preprocessor.BasePreProcessor(
    command=["python"], # The base class will set this.
    input_files=[],
    path_prefix="/app", # irrelevant since we aren't preprocessing any files
    output_map=output_map)

preprocessor.preprocess()


from kubeflow.fairing.cloud.k8s import MinioUploader
from kubeflow.fairing.builders.cluster.minio_context import MinioContextSource

minio_endpoint = "http://10.244.34.231:9000/"
minio_username = "minio"
minio_key = "minio123"
minio_region = "us-east-1"

minio_uploader = MinioUploader(endpoint_url=minio_endpoint, minio_secret=minio_username, minio_secret_key=minio_key, region_name=minio_region)
minio_context_source = MinioContextSource(endpoint_url=minio_endpoint, minio_secret=minio_username, minio_secret_key=minio_key, region_name=minio_region)

cluster_builder = cluster.cluster.ClusterBuilder(registry=DOCKER_REGISTRY,
                                                 base_image="", # base_image is set in the Dockerfile
                                                 preprocessor=preprocessor,
                                                 image_name="mnist",
                                                 dockerfile_path="Dockerfile",
                                                 context_source=minio_context_source)
cluster_builder.build()
logging.info(f"Built image {cluster_builder.image_tag}")

image-20211101200119152

image-20211101200147906

image-20211101200205197

在Harbor中创建机器人帐号，提供给Kaniko使用

sukai@sukai:/mnt/d/03.workspace/k8s$ kubectl create --namespace kubeflow-user-example-com configmap docker-config --from-file=/tmp/config.json

sukai@sukai:/mnt/d/03.workspace/k8s$ cat /tmp/config.json
{"auths":{"192.168.0.93":{"username":"robot$gitlab-robot","password":"eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE2MzU3NDYwMTgsImlzcyI6ImhhcmJvci10b2tlbi1kZWZhdWx0SXNzdWVyIiwiaWQiOjEsInBpZCI6NywiYWNjZXNzIjpbeyJSZXNvdXJjZSI6Ii9wcm9qZWN0LzcvcmVwb3NpdG9yeSIsIkFjdGlvbiI6InB1c2giLCJFZmZlY3QiOiIifSx7IlJlc291cmNlIjoiL3Byb2plY3QvNy9oZWxtLWNoYXJ0IiwiQWN0aW9uIjoicmVhZCIsIkVmZmVjdCI6IiJ9LHsiUmVzb3VyY2UiOiIvcHJvamVjdC83L2hlbG0tY2hhcnQtdmVyc2lvbiIsIkFjdGlvbiI6ImNyZWF0ZSIsIkVmZmVjdCI6IiJ9XX0.tM7Tagi9W7OnKAwQYXUrulNRq1p-1DBCwSl5aIEEqTcqoBEinRDRdW6BP5rnOU3GVct826naFTDKn9osrEdhxEba8PyOA0F7uQiSR_XI_3xz8_XJlrhaZuEkKBN4tO_WymoHjzCcCdlay8lq-5k2tz6iUwLf3RNzG2ET7etZ2q3PQr3u_hx0m4zzKsZXVuhBNFA9Zph0mlniND7z7Uua5eNmdC0f25ZZ1_wBVwdScPCvgEDqd6p9Jfa42uk47aaCnxDK8-NOfbt1cz6AHSQtdOrczy30F6ffxpJsqhGOIW-OGnuKi52X3qjOluyREjpkec3fN4zqQcfa_wg4ULFLxUlAULJqyUuf87hufNIRcEvJQZBG9KQEv57h3aO3s9fSKlQLc7bAIBaXIjhNht3J4GCl6tRST5MV79WDQooz4rxswP5SWxsrq2O-bSkUjQBsUZ3nuPi28ppR7cpresF4JekBi8j46fAMe0MGQ1hwAoZFkUaUntzfcfiP7Bz9szPf5AGEcmKsiNpE-A0yuTG9sZ7g8d8qMEXa5hN-4PALEtHE41MMaC9bGi2DCD56ksXJTIA6ZvFI9mITpfFTqeDW1P3dRDaQdtYUn11CTog9E1jDE51cGn0h1LdVUlK-1aMGQQxwAdj-lvSGdNQ3CLYVtiKOna4__gw-3UAhXzB8RB8"}}}

在Minio中查看fairing构建存放的文件

image-20211101200924193

修改Fairing的代码，解决Harbor自签名证书的问题

cd /home/jovyan/.local/lib/python3.8/site-packages/kubeflow/fairing/builders/cluster/

vi minio_context.py

image-20211101200636042

在Harbor中查看构建的镜像

image-20211101200825797

| GitLab流水线构建容器镜像

在.gitlab-ci.yml添加构建阶段

build:
  stage: build
  image:
    name: daocloud.io/gcr-mirror/kaniko-project-executor:debug
    entrypoint: [""]
  script:
    - mkdir -p /kaniko/.docker
    - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"robot\$gitlab-robot\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
    - cat /kaniko/.docker/config.json
    - /kaniko/executor
      --context "${CI_PROJECT_DIR}"
      --dockerfile "${CI_PROJECT_DIR}/Dockerfile"
      --skip-tls-verify "true"
      --destination "${CI_REGISTRY}/ai/ai-demo:${CI_COMMIT_TAG}"

查看流水线

image-20211101201218801

image-20211101201241215

搜索

分类

标签