应用重新部署提示deployment存在

[Fruticicoli]

• Choerodon平台版本：0.17.0

• 运行环境：自主搭建

• 问题描述：

  请尽量详细的描述您遇到的问题，以便我们能更快速的提供解决办法。

  如：今天配合开发测试skyjacking的时候多次删除了应用然后再新建，现在重新执行应用部署的时候，猪齿鱼页面一直提示deployment.apps已经存在了，如下截图：
  

  

但kubernetes里确认是不存在这个deployment的，如下：

kubectl get all --all-namespaces | grep onecenter
dev           pod/onecenter-bff-b00ac-7fc6b8867d-dcpbq                        1/1     Running     0          34h

dev           service/onecenter-bff                           NodePort    10.70.184.200   <none>          8080:9030/TCP                                   19d


dev           deployment.apps/onecenter-bff-b00ac                        1/1     1            1           19d

dev           replicaset.apps/onecenter-bff-b00ac-54d7778c94                        0         0         0       12d
dev           replicaset.apps/onecenter-bff-b00ac-598845995d                        0         0         0       13d
dev           replicaset.apps/onecenter-bff-b00ac-5db7d65dd6                        0         0         0       13d
dev           replicaset.apps/onecenter-bff-b00ac-66dfdf46c8                        0         0         0       13d
dev           replicaset.apps/onecenter-bff-b00ac-6f44d9b566                        0         0         0       16d
dev           replicaset.apps/onecenter-bff-b00ac-74c4457444                        0         0         0       19d
dev           replicaset.apps/onecenter-bff-b00ac-7fc6b8867d                        1         1         1       9d
dev           replicaset.apps/onecenter-bff-b00ac-84c65d89d                         0         0         0       12d
dev           replicaset.apps/onecenter-bff-b00ac-865b4c4d58                        0         0         0       12d
dev           replicaset.apps/onecenter-bff-b00ac-d46fb98f4                         0         0         0       13d
dev           replicaset.apps/onecenter-bff-b00ac-d5d568c58                         0         0         0       14d

• 咨询如下：
  如: 页面上的这个报错是那个组件返回的？从那里获取的数据说明onecenter-service存在deployment？

• 报错信息(请尽量使用代码块的形式展现)：

• 建议：

  提出您认为不合理的地方，帮助我们优化用户操作

[younger]

你好，检查下你应用代码的chart目录里面的templates文件。有没有定义了2个同名的deployment对象

[Fruticicoli]

附件为agent的日志

I0718 13:57:10.144031       6 client.go:173] send response key inter:inter, type node_sync
I0718 13:57:18.203636       6 worker.go:143] get command: cluster:3.env:dev.envId:7/git_ops_sync
I0718 13:57:18.578358       6 client.go:173] send response key cluster:3.env:dev.envId:7, type git_ops_sync
I0718 13:57:18.580054       6 daemon.go:81] env: dev get refreshed event for git repository ssh://git@dev/operation-dongfang-gitops/dev.git, branch master, HEAD effc82e8429086f265973667e893552dafdea1d2, previous HEAD cdc3fbd31cb217f22dcf42a6527c70b0d1971af5
I0718 13:57:18.739821       6 sync.go:125] cmd: delete, args: , cout: 1
I0718 13:57:23.033974       6 sync.go:178] kubectl delete -n dev -f - , took 4.294077043s, err: <nil>, output: c7nhelmrelease.choerodon.io "onecenter-service-7f51d" deleted
I0718 13:57:23.034184       6 daemon.go:339] env:dev git_ops_sync_event:
{"id":0,"resourceIDs":[],"type":"sync","startedAt":"2019-07-18T05:57:18.580090151Z","endedAt":"2019-07-18T05:57:18.580090151Z","metadata":{"commit":"effc82e8429086f265973667e893552dafdea1d2"}}
I0718 13:57:23.034305       6 client.go:173] send response key env:dev, type git_ops_sync_event
E0718 13:57:23.035931       6 c7nhelmrelease_controller.go:265] C7NHelmReleases 'dev/onecenter-service-7f51d' in work queue no longer exists
I0718 13:57:23.035975       6 c7nhelmrelease_controller.go:267] release onecenter-service-7f51d delete
I0718 13:57:23.036042       6 worker.go:143] get command: env:dev.release:onecenter-service-7f51d/helm_release_delete
E0718 13:57:23.107929       6 error.go:10] delete release onecenter-service-7f51d: rpc error: code = Unknown desc = release: "onecenter-service-7f51d" not found
I0718 13:57:23.107995       6 client.go:173] send response key env:dev.release:onecenter-service-7f51d, type helm_release_delete_failed
I0718 13:57:24.167269       6 daemon.go:321] dev tag: agent-sync, old: cdc3fbd31cb217f22dcf42a6527c70b0d1971af5, new: effc82e8429086f265973667e893552dafdea1d2
I0718 13:57:24.577704       6 daemon.go:81] env: dev get refreshed event for git repository ssh://git@dev/operation-dongfang-gitops/dev.git, branch master, HEAD effc82e8429086f265973667e893552dafdea1d2, previous HEAD effc82e8429086f265973667e893552dafdea1d2
I0718 13:57:40.190986       6 client.go:173] send response key inter:inter, type node_sync
I0718 13:57:42.316687       6 worker.go:143] get command: cluster:3.env:dev.envId:7/git_ops_sync
I0718 13:57:42.683444       6 client.go:173] send response key cluster:3.env:dev.envId:7, type git_ops_sync
I0718 13:57:42.685128       6 daemon.go:81] env: dev get refreshed event for git repository ssh://git@dev/operation-dongfang-gitops/dev.git, branch master, HEAD 71a39ee15a0a865a4c0bc665a1b8a9843989dc54, previous HEAD effc82e8429086f265973667e893552dafdea1d2
W0718 13:57:42.813727       6 client.go:695] label object not matched: &{map[kind:C7NHelmRelease metadata:map[namespace:dev name:onecenter-service-ed3d5] spec:map[values: chartName:onecenter-service chartVersion:2019.7.18-112116-dev repoUrl:http://chart-his.fr-inc.cn/operation/dongfang/] apiVersion:choerodon.io/v1alpha1]}
I0718 13:57:42.850574       6 sync.go:125] cmd: apply, args: , cout: 1
I0718 13:57:43.257905       6 sync.go:178] kubectl apply -n dev -f - , took 407.262683ms, err: <nil>, output: c7nhelmrelease.choerodon.io/onecenter-service-ed3d5 created
I0718 13:57:43.258036       6 daemon.go:339] env:dev git_ops_sync_event:
{"id":0,"resourceIDs":["c7nhelmrelease/onecenter-service-ed3d5"],"type":"sync","startedAt":"2019-07-18T05:57:42.685162709Z","endedAt":"2019-07-18T05:57:42.685162709Z","metadata":{"commit":"71a39ee15a0a865a4c0bc665a1b8a9843989dc54","filesCommit":[{"file":"release-onecenter-service-ed3d5.yaml","commit":"71a39ee15a0a865a4c0bc665a1b8a9843989dc54"}],"resourceCommits":[{"resourceId":"c7nhelmrelease/onecenter-service-ed3d5","file":"release-onecenter-service-ed3d5.yaml","commit":"71a39ee15a0a865a4c0bc665a1b8a9843989dc54"}]}}
I0718 13:57:43.258188       6 client.go:173] send response key env:dev, type git_ops_sync_event
I0718 13:57:43.339326       6 c7nhelmrelease_controller.go:288] release onecenter-service-ed3d5 install
I0718 13:57:43.339552       6 worker.go:143] get command: env:dev.release:onecenter-service-ed3d5/helm_release_pre_install
I0718 13:57:43.852301       6 utils.go:117] info: manifest "onecenter-service/templates/ingress.yaml" is empty. Skipping.
I0718 13:57:43.854380       6 client.go:173] send response key env:dev.release:onecenter-service-ed3d5, type helm_release_pre_install
I0718 13:57:43.854430       6 worker.go:143] get command: env:dev.release:onecenter-service-ed3d5/helm_install_release
I0718 13:57:43.984046       6 daemon.go:321] dev tag: agent-sync, old: effc82e8429086f265973667e893552dafdea1d2, new: 71a39ee15a0a865a4c0bc665a1b8a9843989dc54
I0718 13:57:44.181601       6 utils.go:117] info: manifest "onecenter-service/templates/ingress.yaml" is empty. Skipping.
I0718 13:57:44.398905       6 daemon.go:81] env: dev get refreshed event for git repository ssh://git@dev/operation-dongfang-gitops/dev.git, branch master, HEAD 71a39ee15a0a865a4c0bc665a1b8a9843989dc54, previous HEAD 71a39ee15a0a865a4c0bc665a1b8a9843989dc54
I0718 13:57:45.848183       6 client.go:173] send response key env:dev.Service:onecenter-service, type resource_update
I0718 13:57:45.854929       6 client.go:173] send response key env:dev.release:onecenter-service-ed3d5.Deployment:onecenter-service-ed3d5, type resource_update
I0718 13:57:45.876995       6 client.go:173] send response key env:dev.release:onecenter-service-ed3d5.ReplicaSet:onecenter-service-ed3d5-67cf47b7bc, type resource_update
I0718 13:57:45.891016       6 client.go:173] send response key env:dev.release:onecenter-service-ed3d5.Deployment:onecenter-service-ed3d5, type resource_update
I0718 13:57:45.951573       6 client.go:173] send response key env:dev.release:onecenter-service-ed3d5.Pod:onecenter-service-ed3d5-67cf47b7bc-mjzkl, type resource_update
I0718 13:57:45.980544       6 client.go:173] send response key env:dev.release:onecenter-service-ed3d5.Deployment:onecenter-service-ed3d5, type resource_update
I0718 13:57:45.980687       6 client.go:173] send response key env:dev.release:onecenter-service-ed3d5.ReplicaSet:onecenter-service-ed3d5-67cf47b7bc, type resource_update
I0718 13:57:45.980854       6 client.go:173] send response key env:dev.release:onecenter-service-ed3d5.Pod:onecenter-service-ed3d5-67cf47b7bc-mjzkl, type resource_update
I0718 13:57:46.005645       6 client.go:173] send response key env:dev.release:onecenter-service-ed3d5.ReplicaSet:onecenter-service-ed3d5-67cf47b7bc, type resource_update
I0718 13:57:46.022335       6 client.go:173] send response key env:dev.release:onecenter-service-ed3d5.Deployment:onecenter-service-ed3d5, type resource_update
I0718 13:57:46.027425       6 client.go:173] send response key env:dev.release:onecenter-service-ed3d5.Pod:onecenter-service-ed3d5-67cf47b7bc-mjzkl, type resource_update
I0718 13:57:46.036018       6 client.go:173] send response key env:dev.release:onecenter-service-ed3d5.Event:onecenter-service-ed3d5-67cf47b7bc-mjzkl.15b26ad0dec74476, type release_pod_event
W0718 13:57:46.062462       6 service_controller.go:164] service 'dev/onecenter-service' in work queue no longer exists
I0718 13:57:46.062525       6 client.go:173] send response key env:dev.Service:onecenter-service, type resource_delete
W0718 13:57:46.100076       6 deployment_controller.go:192] deployment 'dev/onecenter-service-ed3d5' in work queue no longer exists
I0718 13:57:46.100130       6 client.go:173] send response key env:dev.Deployment:onecenter-service-ed3d5, type resource_delete
W0718 13:57:46.123000       6 replicaset_controller.go:167] Rs 'dev/onecenter-service-ed3d5-67cf47b7bc' in work queue no longer exists
I0718 13:57:46.123051       6 client.go:173] send response key env:dev.ReplicaSet:onecenter-service-ed3d5-67cf47b7bc, type resource_delete
I0718 13:57:46.133356       6 client.go:173] send response key env:dev.release:onecenter-service-ed3d5.Pod:onecenter-service-ed3d5-67cf47b7bc-mjzkl, type resource_update
I0718 13:57:46.156352       6 client.go:173] send response key env:dev.release:onecenter-service-ed3d5.Pod:onecenter-service-ed3d5-67cf47b7bc-mjzkl, type resource_update
E0718 13:57:46.184261       6 error.go:19] install release onecenter-service-ed3d5: rpc error: code = Unknown desc = release onecenter-service-ed3d5 failed: deployments.apps "onecenter-service-ed3d5" already exists
I0718 13:57:46.184357       6 client.go:173] send response key env:dev.release:onecenter-service-ed3d5.commit:71a39ee15a0a865a4c0bc665a1b8a9843989dc54, type helm_release_install_failed
W0718 13:57:56.861807       6 pod_controller.go:162] pod 'dev/onecenter-service-ed3d5-67cf47b7bc-mjzkl' in work queue no longer exists
I0718 13:57:56.861860       6 client.go:173] send response key env:dev.Pod:onecenter-service-ed3d5-67cf47b7bc-mjzkl, type resource_delete
I0718 13:58:10.240339       6 client.go:173] send response key inter:inter, type node_sync

[Fruticicoli]

我大概猜到是什么错误了，我在chart的template下把deoloyment文件备份了一个。

[younger]

嗯 chart里面的模板文件 Helm install的时候都会去部署相应资源的

[Fruticicoli]

提一个意见，猪齿鱼整个平台组件比较多，日志看起来实在是太费劲了，附带一个ELK吧，把所有的猪齿鱼的各个组件的日志都收集起来然后搜索查看吧。
现在每次出现问题，只能kubectl logs -f这样一个一个的去看，搜索太费劲了。

[Lano]

有个es+kibana的组件做日志管理，官网有提供，没放到推荐安装的流程里，这东西跟监控都比较耗资源，而且运维使用难度会很高，看个人需求，后续会集成

[Fruticicoli]

现在看日志实在是费劲，关键是按照过程里的微服务介绍的都很简单，每个页面和那个微服务的交互都只能靠猜测。或者是把猪齿鱼的日志持久化到nfs的一个目录里，能搜索也可以。