-
Choerodon平台版本: 0.12.0
-
遇到问题的执行步骤:
-
文档地址:http://choerodon.io/zh/docs/installation-configuration/steps/install/parts/gitlab-runner/
-
报错日志:
error: SchemaError(io.k8s.api.autoscaling.v2beta2.ExternalMetricSource): invalid object doesn’t have additional properties
-
原因分析:
k8s版本1.15.1，gitlab版本12，能否麻烦大佬尽快发布一个适配1.15.1的的charts，急用，感谢
你好，你可以尝试直接更换: runner 镜像为 gitlab/gitlab-runner:alpine-v12.1.0
以及更换configmap中 runner-helper镜像为gitlab/gitlab-runner-helper:x86_64-de7731dd
感谢，修改之后，又是另外一个错误。
error: SchemaError(io.k8s.api.extensions.v1beta1.NetworkPolicyPort): invalid object doesn’t have additional properties
你好，请在集群中执行 kubectl api-versions并反馈一下结果
admissionregistration.k8s.io/v1beta1
apiextensions.k8s.io/v1beta1
apiregistration.k8s.io/v1
apiregistration.k8s.io/v1beta1
apps/v1
apps/v1beta1
apps/v1beta2
authentication.k8s.io/v1
authentication.k8s.io/v1beta1
authorization.k8s.io/v1
authorization.k8s.io/v1beta1
autoscaling/v1
autoscaling/v2beta1
autoscaling/v2beta2
batch/v1
batch/v1beta1
certificates.k8s.io/v1beta1
coordination.k8s.io/v1
coordination.k8s.io/v1beta1
crd.projectcalico.org/v1
events.k8s.io/v1beta1
extensions/v1beta1
management.cattle.io/v3
metrics.k8s.io/v1beta1
monitoring.coreos.com/v1
networking.k8s.io/v1
networking.k8s.io/v1beta1
node.k8s.io/v1beta1
policy/v1beta1
project.cattle.io/v3
rbac.authorization.k8s.io/v1
rbac.authorization.k8s.io/v1beta1
scheduling.k8s.io/v1
scheduling.k8s.io/v1beta1
storage.k8s.io/v1
storage.k8s.io/v1beta1
v1
当前 runner 使用的是 client-go 7.0
而 client-go 7.0 适配的 k8s 集群为 1.10，建议runner搭建在低版本 k8s 集群中
意思是高版本的完全没办法用还是？要重新降级k8s还挺麻烦
建议再搭建一个低版本 k8s 集群专门处理 runner 所生成的 job
这个具体应该咋做呢。比较懵。因为现在是使用的Rancher2，整个集群版本就比较高
您好，您可以参照我们官网教程搭建 1.10.12 版本的 k8s 集群，然后在该集群上部署 runner
https://choerodon.io/zh/docs/installation-configuration/steps/kubernetes/
你好，请提供一下charts所生成的 configmap 文件
是楼上截图的官方的0.7.0的对吧，还是猪齿鱼的呢
请执行以下命令，反馈一下结果
kubectl get cm -n c7nn-system runner-cm -o yaml
gitlab-runner-0.7.0-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "gitlab-runner.fullname" . }}
labels:
app: {{ include "gitlab-runner.fullname" . }}
chart: {{ include "gitlab-runner.chart" . }}
release: "{{ .Release.Name }}"
heritage: "{{ .Release.Service }}"
data:
entrypoint: |
#!/bin/bash
set -e
mkdir -p /home/gitlab-runner/.gitlab-runner/
cp /scripts/config.toml /home/gitlab-runner/.gitlab-runner/
# Register the runner
if [[ -f /secrets/accesskey && -f /secrets/secretkey ]]; then
export CACHE_S3_ACCESS_KEY=$(cat /secrets/accesskey)
export CACHE_S3_SECRET_KEY=$(cat /secrets/secretkey)
fi
if [[ -f /secrets/gcs-applicaton-credentials-file ]]; then
export GOOGLE_APPLICATION_CREDENTIALS="/secrets/gcs-applicaton-credentials-file"
else
if [[ -f /secrets/gcs-access-id && -f /secrets/gcs-private-key ]]; then
export CACHE_GCS_ACCESS_ID=$(cat /secrets/gcs-access-id)
# echo -e used to make private key multiline (in google json auth key private key is oneline with \n)
export CACHE_GCS_PRIVATE_KEY=$(echo -e $(cat /secrets/gcs-private-key))
fi
fi
if [[ -f /secrets/runner-registration-token ]]; then
export REGISTRATION_TOKEN=$(cat /secrets/runner-registration-token)
fi
if [[ -f /secrets/runner-token ]]; then
export CI_SERVER_TOKEN=$(cat /secrets/runner-token)
fi
if ! sh /scripts/register-the-runner; then
exit 1
fi
# Start the runner
/entrypoint run --user=gitlab-runner \
--working-directory=/home/gitlab-runner
config.toml: |
concurrent = {{ .Values.concurrent }}
check_interval = {{ .Values.checkInterval }}
log_level = {{ default "info" .Values.logLevel | quote }}
{{- if .Values.metrics.enabled }}
listen_address = '[::]:9252'
{{- end }}
configure: |
set -e
cp /init-secrets/* /secrets
register-the-runner: |
#!/bin/bash
MAX_REGISTER_ATTEMPTS=30
for i in $(seq 1 "${MAX_REGISTER_ATTEMPTS}"); do
echo "Registration attempt ${i} of ${MAX_REGISTER_ATTEMPTS}"
/entrypoint register \
{{- range .Values.runners.imagePullSecrets }}
--kubernetes-image-pull-secrets {{ . | quote }} \
{{- end }}
{{- range $key, $val := .Values.runners.nodeSelector }}
--kubernetes-node-selector {{ $key | quote }}:{{ $val | quote }} \
{{- end }}
{{- range $key, $value := .Values.runners.podLabels }}
--kubernetes-pod-labels {{ $key | quote }}:{{ $value | quote }} \
{{- end }}
{{- range $key, $val := .Values.runners.podAnnotations }}
--kubernetes-pod-annotations {{ $key | quote }}:{{ $val | quote }} \
{{- end }}
{{- range $key, $value := .Values.runners.env }}
--env {{ $key | quote -}} = {{- $value | quote }} \
{{- end }}
--non-interactive
retval=$?
if [ ${retval} = 0 ]; then
break
elif [ ${i} = ${MAX_REGISTER_ATTEMPTS} ]; then
exit 1
fi
sleep 5
done
exit 0
check-live: |
#!/bin/bash
if /usr/bin/pgrep -f .*register-the-runner; then
exit 0
elif /usr/bin/pgrep gitlab.*runner; then
exit 0
else
exit 1
fi
你好，你的 runner 并不是按照我们这边的教程大家的是吗？
刚才的是我能跑在我高版本k8s上的runner。
现在猪齿鱼的cm是这样的。我根据大佬讲的试了不行之后，我又换了其他的版本
apiVersion: v1
data:
config.toml: |
concurrent = 5
check_interval = 0
listen_address = '[::]:9252'
[session_server]
session_timeout = 1800
[[runners]]
name = "RUNER_NAME"
url = "http://10.211.55.5"
token = "CI_SERVER_TOKEN"
executor = "kubernetes"
environment = [
"CHOERODON_URL=http://api.example.choerodon.io",
"DOCKER_PWD=Harbor12345",
"DOCKER_REGISTRY=10.211.55.5:9000",
"DOCKER_USER=admin",
]
[runners.cache]
[runners.kubernetes]
image = "registry.cn-hangzhou.aliyuncs.com/choerodon-tools/cibase:0.7.0"
namespace = "xxx"
namespace_overwrite_allowed = ""
privileged = true
service_account_overwrite_allowed = ""
pull_policy = "always"
cpu_limit = "1"
cpu_request = "500m"
memory_request = "1Gi"
memory_limit = "2Gi"
service_cpu_limit = "500m"
service_cpu_request = "100m"
service_memory_limit = "800Mi"
service_memory_request = "500Mi"
helper_cpu_limit = "500m"
helper_cpu_request = "100m"
helper_memory_limit = "200Mi"
helper_memory_request = "100Mi"
helper_image = "gitlab/gitlab-runner-helper:x86_64-fc6f2571"
[runners.kubernetes.volumes]
[[runners.kubernetes.volumes.host_path]]
name = "docker"
mount_path = "/var/run/docker.sock"
host_path = "/var/run/docker.sock"
[[runners.kubernetes.volumes.pvc]]
name = "runner-cache-pvc"
mount_path = "/cache"
readonly = false
[[runners.kubernetes.volumes.pvc]]
name = "runner-maven-pvc"
mount_path = "/root/.m2"
readonly = false
entrypoint: |
#!/bin/bash
set -e
mkdir -p /home/gitlab-runner/.gitlab-runner/
cp /scripts/config.toml /home/gitlab-runner/.gitlab-runner/config.toml.template
# Got environment
if [[ -f /secrets/runner-registration-token ]]; then
export REGISTRATION_TOKEN=$(cat /secrets/runner-registration-token)
fi
if [[ -f /secrets/runner-token ]]; then
export CI_SERVER_TOKEN=$(cat /secrets/runner-token)
fi
# Register the runner and Rendering
/entrypoint register --non-interactive
export RUNER_NAME=$(grep 'name =' /home/gitlab-runner/.gitlab-runner/config.toml | cut -d'"' -f2)
sed -i "s RUNER_NAME $RUNER_NAME g" /home/gitlab-runner/.gitlab-runner/config.toml.template
if [ x$CI_SERVER_TOKEN == x ] ;then
export CI_SERVER_TOKEN=$(grep 'token =' /home/gitlab-runner/.gitlab-runner/config.toml | cut -d'"' -f2)
# Save CI_SERVER_TOKEN
/data/kubectl get secret -n xxx runner-secret -o json | \
/data/jq '.data["runner-token"]|='\"$(echo $CI_SERVER_TOKEN | base64)\"'' | \
/data/kubectl replace -f -
fi
sed -i "s CI_SERVER_TOKEN $CI_SERVER_TOKEN g" /home/gitlab-runner/.gitlab-runner/config.toml.template
cp -f /home/gitlab-runner/.gitlab-runner/config.toml.template /home/gitlab-runner/.gitlab-runner/config.toml
# Start the runner
/entrypoint run --user=gitlab-runner \
--working-directory=/home/gitlab-runner
kind: ConfigMap
metadata:
creationTimestamp: "2019-08-12T02:41:01Z"
name: runner-cm
namespace: xxx
resourceVersion: "160716"
selfLink: /api/v1/namespaces/xxx/configmaps/runner-cm
uid: 9f9f4ed2-bcaa-11e9-9fc5-001c4262d0e6
你好，刚刚我在 1.15.1 版本 k8s 做实验，按我们这边提供的 runner 搭建操作，应该是直接 Error 状态哒，Pod都无法运行，你确认按我们提供的教程搭建runner pod可以运行并且有以上报错日志？
是的，没错，直接用猪齿鱼的charts版本的runner是直接报错的，pods异常，error: SchemaError(io.k8s.api.extensions.v1beta1.NetworkPolicyPort): invalid object doesn’t have additional properties。
我贴出来的截图是说我可以用gitlab-runner官方的charts跑的，跑的时候报的错
好的，大概了解了。
请按以下命令尝试搭建
# 删除 runner
helm delete --purge runner
# 安装runner，注意替换相应变量，与官网功能的命令不同之处已标出
helm install c7n/gitlab-runner \
--set rbac.create=true \
--set env.concurrent=3 \
+ --set image.init.tag=v1.14.3 \
--set env.gitlabUrl=http://gitlab.example.choerodon.io/ \
--set env.runnerRegistrationToken=xwxobLNoPQUzyMt_4RGF \
--set env.environment.DOCKER_REGISTRY=registry.example.choerodon.io \
--set env.environment.DOCKER_USER=admin \
--set env.environment.DOCKER_PWD=Harbor12345 \
--set env.environment.CHOERODON_URL=http://api.example.choerodon.io \
--set env.persistence.runner-maven-pvc="/root/.m2" \
--set env.persistence.runner-cache-pvc="/cache" \
--name runner \
--version 0.2.0 \
--namespace c7n-system