应用部署的时候提示Auth Fail,要如何解决?

  • Choerodon平台版本:0.16.0

  • 运行环境:自主搭建

  • 问题描述:

    请尽量详细的描述您遇到的问题,以便我们能更快速的提供解决办法。

部署流水线—>应用部署的时候提示Auth Failed

  • 执行的操作:
    如: 部署流水线—>应用部署的时候提示Auth Failed

  • 报错信息(请尽量使用代码块的形式展现):

如下为devops_service这个微服务的容器日志

2019-05-16 16:28:32.578  INFO 7 --- [nio-8061-exec-3] c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at : http://register-server.c7n-system:8000/
2019-05-16 16:28:32.592  INFO 7 --- [nio-8061-exec-3] c.c.c.ConfigServicePropertySourceLocator : Located environment: name=devops-service, profiles=[default], label=, version=0.16.4, state=
2019-05-16 16:28:40.245  INFO 7 --- [nio-8060-exec-6] i.c.websocket.websocket.SocketHandler    : receive node_sync msg of inter:inter,
2019-05-16 16:29:10.281  INFO 7 --- [nio-8060-exec-7] i.c.websocket.websocket.SocketHandler    : receive node_sync msg of inter:inter,
2019-05-16 16:29:40.330  INFO 7 --- [nio-8060-exec-5] i.c.websocket.websocket.SocketHandler    : receive node_sync msg of inter:inter,
2019-05-16 16:30:10.375  INFO 7 --- [nio-8060-exec-6] i.c.websocket.websocket.SocketHandler    : receive node_sync msg of inter:inter,
2019-05-16 16:30:25.706  INFO 7 --- [io-8060-exec-10] i.c.websocket.websocket.SocketHandler    : receive status_sync_event msg of env:develop,
2019-05-16 16:30:25.708  INFO 7 --- [io-8060-exec-10] i.c.d.a.s.i.DeployMsgHandlerServiceImpl  : sync command status!
2019-05-16 16:30:40.421  INFO 7 --- [nio-8060-exec-2] i.c.websocket.websocket.SocketHandler    : receive node_sync msg of inter:inter,
2019-05-16 16:30:54.574  INFO 7 --- [trap-executor-0] c.n.d.s.r.aws.ConfigClusterResolver      : Resolving eureka endpoints via configuration
2019-05-16 16:31:10.462  INFO 7 --- [nio-8060-exec-5] i.c.websocket.websocket.SocketHandler    : receive node_sync msg of inter:inter,
2019-05-16 16:31:30.683  INFO 7 --- [nio-8060-exec-2] i.c.r.h.ControllerExceptionHandler       : exception info io.choerodon.core.exception.CommonException: ssh://git@gitlab.example.cn:2289/operation-dongfang-gitops/develop.git: Auth fail
	at io.choerodon.devops.infra.common.util.GitUtil.cloneBySsh(GitUtil.java:182)
	at io.choerodon.devops.infra.common.util.EnvUtil.handDevopsEnvGitRepository(EnvUtil.java:150)
	at io.choerodon.devops.app.service.impl.ApplicationInstanceServiceImpl.createOrUpdate(ApplicationInstanceServiceImpl.java:856)
	at io.choerodon.devops.api.controller.v1.ApplicationInstanceController.deploy(ApplicationInstanceController.java:380)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:209)
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136)
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:891)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797)
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:991)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:925)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:974)
	at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:877)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:661)
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:851)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at io.choerodon.resource.filter.JwtTokenFilter.doFilter(JwtTokenFilter.java:90)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.boot.actuate.web.trace.servlet.HttpTraceFilter.doFilterInternal(HttpTraceFilter.java:90)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:209)
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:357)
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:270)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:155)
	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:123)
	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:108)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:800)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:813)
Caused by: org.eclipse.jgit.api.errors.TransportException: ssh://git@gitlab.example.cn:2289/operation-dongfang-gitops/develop.git: Auth fail
	at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:224)
	at org.eclipse.jgit.api.CloneCommand.fetch(CloneCommand.java:216)
	at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:137)
	at io.choerodon.devops.infra.common.util.GitUtil.cloneBySsh(GitUtil.java:180)
	... 76 more
Caused by: org.eclipse.jgit.errors.TransportException: ssh://git@gitlab.example.cn:2289/operation-dongfang-gitops/develop.git: Auth fail
	at org.eclipse.jgit.transport.JschConfigSessionFactory.getSession(JschConfigSessionFactory.java:160)
	at org.eclipse.jgit.transport.SshTransport.getSession(SshTransport.java:137)
	at org.eclipse.jgit.transport.TransportGitSsh$SshFetchConnection.<init>(TransportGitSsh.java:274)
	at org.eclipse.jgit.transport.TransportGitSsh.openFetch(TransportGitSsh.java:169)
	at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:136)
	at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:122)
	at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1236)
	at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:213)
	... 79 more
Caused by: com.jcraft.jsch.JSchException: Auth fail
	at com.jcraft.jsch.Session.connect(Session.java:519)
	at org.eclipse.jgit.transport.JschConfigSessionFactory.getSession(JschConfigSessionFactory.java:117)
	... 86 more

2019-05-16 16:31:30.684  WARN 7 --- [nio-8060-exec-2] .m.m.a.ExceptionHandlerExceptionResolver : Resolved [io.choerodon.core.exception.CommonException: ssh://git@gitlab.example.cn:2289/operation-dongfang-gitops/develop.git: Auth fail]
2019-05-16 16:31:40.508  INFO 7 --- [nio-8060-exec-8] i.c.websocket.websocket.SocketHandler    : receive node_sync msg of inter:inter,
2019-05-16 16:32:10.547  INFO 7 --- [nio-8060-exec-2] i.c.websocket.websocket.SocketHandler    : receive node_sync msg of inter:inter

  • 建议:

    提出您认为不合理的地方,帮助我们优化用户操作

我再次点击部署,提示主键冲突,然后去devops_service里将冲突的数据删除后,再次部署就通过了。
但是部署结果是失败的,如下:

点击红色的感叹号也没有任何提示。

点击部署之前的日志打印在devops_service这个微服务对应的容器日志中,这个将应用部署到kubernetes集群中是通过那个微服务来操作的,我需要看到详细的日志,到对是哪里出错了?

  1. 部署功能需要启用gitlab的ssh协议,devops-service里面会通过ssh方式克隆环境对应的gitops库
    http://choerodon.io/zh/docs/installation-configuration/steps/install/parts/base/gitlab/#启用ssh协议
  2. 校验是否部署成功可以查看环境总览界面,当3个同步的时候,证明部署步骤是正常的

这2个页面都没有相关的出错信息,部署出错是提示deploy timeout。要怎么继续定位?

配置库的commit都没有,说明gitops库里面的webhook没有发送成功,先检查下配置库的webhook

了解了,我这里的gitlab缺失存在webhook不生效的问题,我周末重新安装下gitlab,不能再使用数据恢复了,数据恢复下异常太多了。
我手动触发了下webhook,

然后配置库读取到了一个参数

这一块的流程是在部署应用的时候,会触发gitops库里面的webhook,然后将数据上报到配置库,然后再将应用发布到kubernetes里?

这一块的流程是,在界面上创建一个实例后,会在gitops环境库里面创建一个对象文件。然后devops-service进行对象文件的解析。解析完之后发送指令给agent,agent里面执行解析,然后agent解析完之后会在去k8s集群执行相关k8s对象的操作,大概流程图如下:


具体细节,这里官网上有一篇博客上有具体的逻辑设计介绍
http://choerodon.io/zh/blog/agent/

好的,我这里的问题是


gitops环境库里面创建一个对象文件已经创建了,但是webhook未触发,然后devops_service后续的操作就没有了。
我去看看文档,周末把gitlab重新搞搞,非常感谢!