gitlab-gitlab-core-backup-scheduled. ERROE

Choerodon Installation management
#1

  • Choerodon平台版本: 0.22

  • 报错日志:

gitlab-gitlab-core-backup-scheduled-1599930000-5rt62   0/1     Error       0          23h    172.20.4.106   192.168.168.106   <none>           <none>
gitlab-gitlab-core-backup-scheduled-1599930000-kjfwp   0/1     Error       0          23h    172.20.4.104   192.168.168.106   <none>           <none>
gitlab-gitlab-core-backup-scheduled-1599930000-pcj2t   0/1     Error       0          23h    172.20.4.103   192.168.168.106   <none>           <none>
gitlab-gitlab-core-backup-scheduled-1599930000-t99x5   0/1     Error       0          23h    172.20.4.107   192.168.168.106   <none>           <none>
gitlab-gitlab-core-backup-scheduled-1599930000-wnwkk   0/1     Error       0          23h    172.20.4.108   192.168.168.106   <none>           <none>
gitlab-gitlab-core-backup-scheduled-1599930000-zbm4t   0/1     Error       0          23h    172.20.4.105   192.168.168.106   <none>           <none>


/home/git/gitlab/lib/gitlab/repository_cache.rb:24:in `fetch'
/home/git/gitlab/lib/gitlab/repository_cache_adapter.rb:99:in `block in cache_method_output'
/home/git/gitlab/lib/gitlab/utils/strong_memoize.rb:30:in `strong_memoize'
/home/git/gitlab/lib/gitlab/repository_cache_adapter.rb:127:in `block in memoize_method_output'
/home/git/gitlab/lib/gitlab/repository_cache_adapter.rb:136:in `no_repository_fallback'
/home/git/gitlab/lib/gitlab/repository_cache_adapter.rb:126:in `memoize_method_output'
/home/git/gitlab/lib/gitlab/repository_cache_adapter.rb:98:in `cache_method_output'
/home/git/gitlab/lib/gitlab/repository_cache_adapter.rb:20:in `block in cache_method'
/home/git/gitlab/app/models/repository.rb:556:in `empty?'
/home/git/gitlab/app/models/repository.rb:385:in `expire_emptiness_caches'
/home/git/gitlab/lib/backup/repository.rb:157:in `empty_repo?'
/home/git/gitlab/lib/backup/repository.rb:26:in `block in dump'
/home/git/gitlab/vendor/bundle/ruby/2.5.0/gems/activerecord-5.1.7/lib/active_record/relation/batches.rb:63:in `block (2 levels) in find_each'
/home/git/gitlab/vendor/bundle/ruby/2.5.0/gems/activerecord-5.1.7/lib/active_record/relation/batches.rb:63:in `each'
/home/git/gitlab/vendor/bundle/ruby/2.5.0/gems/activerecord-5.1.7/lib/active_record/relation/batches.rb:63:in `block in find_each'
/home/git/gitlab/vendor/bundle/ruby/2.5.0/gems/activerecord-5.1.7/lib/active_record/relation/batches.rb:129:in `block in find_in_batches'
/home/git/gitlab/vendor/bundle/ruby/2.5.0/gems/activerecord-5.1.7/lib/active_record/relation/batches.rb:230:in `block in in_batches'
/home/git/gitlab/vendor/bundle/ruby/2.5.0/gems/activerecord-5.1.7/lib/active_record/relation/batches.rb:214:in `loop'
/home/git/gitlab/vendor/bundle/ruby/2.5.0/gems/activerecord-5.1.7/lib/active_record/relation/batches.rb:214:in `in_batches'
/home/git/gitlab/vendor/bundle/ruby/2.5.0/gems/activerecord-5.1.7/lib/active_record/relation/batches.rb:128:in `find_in_batches'
/home/git/gitlab/vendor/bundle/ruby/2.5.0/gems/activerecord-5.1.7/lib/active_record/relation/batches.rb:62:in `find_each'
/home/git/gitlab/vendor/bundle/ruby/2.5.0/gems/activerecord-5.1.7/lib/active_record/querying.rb:9:in `find_each'
/home/git/gitlab/lib/backup/repository.rb:17:in `dump'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:80:in `block (4 levels) in <top (required)>'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:11:in `block (3 levels) in <top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.5.0/gems/rake-12.3.2/exe/rake:27:in `<top (required)>'
Tasks: TOP => gitlab:backup:repo:create
(See full trace by running task with --trace)
Dumping PostgreSQL database gitlabhq_production ... [DONE]
2020-09-13 01:05:45 +0800 -- done
2020-09-13 01:05:45 +0800 -- Dumping repositories ...
 * operation-gaming/client-comm ...
#2

你好，请执行下面语句看是否可以备份成功

kubectl exec -it -n c7n-system gitlab-gitlab-core-0 bash
cd /home/git/gitlab && bundle exec rake gitlab:backup:create RAILS_ENV=production
#3

image
image1604×204 8.86 KB

#4

你好，请执行下面语句看是否可以备份成功

kubectl exec -it -n c7n-system gitlab-gitlab-core-0 bash
cd /home/git/gitlab && bundle exec rake gitlab:backup:create RAILS_ENV=production
#5

image
image2164×1256 168 KB

#6

你好，请问该Gitlab是否恢复过数据？且只恢复了数据库其他的并没有恢复？

#7

没有的
全新环境

#8

你好 ，经测试确实没有遇到与你相同的问题，可尝试使用下面语句备份后重新部署恢复Gitlab看是否可以正常。

# 备份Gitlab数据
kubectl exec -it -n c7n-system gitlab-gitlab-core-0 bash
cd /home/git/gitlab && bundle exec rake gitlab:backup:create RAILS_ENV=production

# 删除Gitlab
helm delete gitlab -n c7n-system
# 查看与Gitlab相关的pvc
kubectl get pvc  -n c7n-system | grep gitlab
# 查看Gitlab数据目录所在位置，将 backup 目录中的备份文件拷贝出来
kubectl describe pvc gitlab-gitlab-core -n c7n-system
# 删除所有Gitlab相关的pvc，
kubectl delete pvc -n c7n-system gitlab-gitlab-core data-gitlab-gitlab-redis-0 database-data-gitlab-gitlab-database-0

+++ 按官网步骤重新安装Gitlab +++

# 查看Gitlab数据目录所在位置，将备份文件拷贝至现有 backup 目录中
kubectl describe pvc gitlab-gitlab-core -n c7n-system
# 恢复Gitlab数据
kubectl exec -it -n c7n-system gitlab-gitlab-core-0 bash
cd /home/git/gitlab && bundle exec rake gitlab:backup:restore RAILS_ENV=production
#9

每天都会产生

gitlab-gitlab-core-0                                   1/1     Running     0          3d17h
gitlab-gitlab-core-backup-scheduled-1600102800-54qv2   0/1     Error       0          12h
gitlab-gitlab-core-backup-scheduled-1600102800-97s5d   0/1     Error       0          12h
gitlab-gitlab-core-backup-scheduled-1600102800-bv99w   0/1     Error       0          12h
gitlab-gitlab-core-backup-scheduled-1600102800-f8cth   0/1     Error       0          12h
gitlab-gitlab-core-backup-scheduled-1600102800-r49tk   0/1     Error       0          12h
gitlab-gitlab-core-backup-scheduled-1600102800-rczph   0/1     Error       0          12h
gitlab-gitlab-core-backup-scheduled-1600102800-zjvvs   0/1     Error       0          12h
#10

也可以删除该cronjob，以后手动对Gitlab进行备份即可。

kubectl delete cronjob -n c7n-system gitlab-gitlab-core-backup-scheduled
#11

你好，也可使用下面命令进行修复

helm repo update

helm upgrade gitlab c7n/gitlab-ha --reuse-values  --version 0.2.2-1 --namespace c7n-system