NAME="CentOS Linux"
VERSION="7 (Core)"
System information
System:
Proxy: no
Current User: git
Using RVM: no
Ruby Version: 2.6.3p62
Gem Version: 2.7.9
Bundler Version:1.17.3
Rake Version: 12.3.3
Redis Version: 3.2.12
Git Version: 2.22.2
Sidekiq Version:5.2.7
Go Version: unknown
GitLab information
Version: 12.5.4-ee
Revision: 2a57951c0ee
Directory: /opt/gitlab/embedded/service/gitlab-rails
DB Adapter: PostgreSQL
DB Version: 10.9
URL: http://gitlab.grtlab.com
HTTP Clone URL: http://gitlab.grtlab.com/some-group/some-project.git
SSH Clone URL: git@gitlab.grtlab.com:some-group/some-project.git
Elasticsearch: no
Geo: no
Using LDAP: no
Using Omniauth: yes
Omniauth Providers:
GitLab Shell
Version: 10.2.0
Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories
GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell
Git: /opt/gitlab/embedded/bin/git
相当古いですね
# バックアップの実行
sudo gitlab-rake gitlab:backup:create
Creating backup archive: 1786444658_2026_08_11_12.5.4-ee_gitlab_backup.tar ... done
Uploading backup archive to remote storage ... skipped
Deleting tmp directories ... done
done
Deleting old backups ... skipping
Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data
and are not included in this backup. You will need these files to restore a backup.
Please back them up manually.
Backup task is done.
ローカルに、gitlab.rbとsecrets.jsonもダウンロード!
cent os 7 リポジトリ設定
# 公式ミラーからVaultへの自動書き換え
sudo sed -i -e 's/mirrorlist=#mirrorlist/mirrorlist=/#mirrorlist/g' /etc/yum.repos.d/CentOS-*.repo
sudo sed -i -e 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*.repo
# yumキャッシュの更新確認
sudo yum clean all
sudo yum makecache
swapを8GBに変更しておく
sudo swapoff /swapfile && \
sudo dd if=/dev/zero of=/swapfile bs=1024 count=8388608 && \
sudo chmod 600 /swapfile && \
sudo mkswap /swapfile && \
sudo swapon /swapfile
8388608+0 records in
8388608+0 records out
8589934592 bytes (8.6 GB) copied, 79.5421 s, 108 MB/s
Setting up swapspace version 1, size = 8388604 KiB
no label, UUID=9e87a71d-6337-4b98-9ac8-491bbdcc7fce
# free -h
total used free shared buff/cache available
Mem: 1.8G 135M 83M 17M 1.6G 1.5G
Swap: 8.0G 0B 8.0G
バックグラウンド処理の競合を防ぐため一回設定反映
# sudo gitlab-ctl reconfigure
Running handlers:
Running handlers complete
Chef Client finished, 7/701 resources updated in 12 seconds
gitlab Reconfigured!
# gitlab-ctl stop
ok: down: crond: 497s, normally up
ok: down: gitaly: 497s, normally up
ok: down: gitlab-exporter: 496s, normally up
ok: down: gitlab-workhorse: 496s, normally up
ok: down: grafana: 495s, normally up
ok: down: logrotate: 495s, normally up
ok: down: node-exporter: 495s, normally up
ok: down: postgres-exporter: 494s, normally up
ok: down: postgresql: 494s, normally up
ok: down: prometheus: 493s, normally up
ok: down: redis: 493s, normally up
ok: down: redis-exporter: 492s, normally up
ok: down: sidekiq: 491s, normally up
ok: down: unicorn: 488s, normally up
# Puma(Web サーバー)のログを確認(Ctrl + C で抜けられます)
$ sudo gitlab-ctl tail unicorn ( 新しいものはpuma
==> /var/log/gitlab/unicorn/unicorn_stderr.log <==
I, [2026-08-11T11:23:04.707Z #9532] INFO -- : listening on addr=127.0.0.1:8080 fd=20
I, [2026-08-11T11:23:04.708Z #9532] INFO -- : unlinking existing socket=/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket
I, [2026-08-11T11:23:04.714Z #9532] INFO -- : listening on addr=/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket fd=21
I, [2026-08-11T11:23:05.146Z #9532] INFO -- : master process ready
Toggling services: OK
==== Upgrade has completed ====
Please verify everything is working and run the following if so
sudo rm -rf /var/opt/gitlab/postgresql/data.10
sudo rm -f /var/opt/gitlab/postgresql-version.old
Running handlers:
There was an error running gitlab-ctl reconfigure:
rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash[migrate gitlab-rails database] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/rails_migration.rb line 16) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of "bash" "/tmp/chef-script20260811-7035-5hnmp3" ----
STDOUT: rake aborted!
StandardError: An error has occurred, all later migrations canceled:
Expected batched background migration for the given configuration to be marked as 'finished', but it is 'active': {:job_class_name=>"CopyColumnUsingBackgroundMigrationJob", :table_name=>"events", :column_name=>"id", :job_arguments=>[["id"], ["id_convert_to_bigint"]]}
Finalize it manualy by running
sudo gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,events,id,'[["id"]\, ["id_convert_to_bigint"]]']
For more information, check the documentation
https://docs.gitlab.com/ee/user/admin_area/monitoring/background_migrations.html#database-migrations-failing-because-of-batched-background-migration-not-finished
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:1109:in `ensure_batched_background_migration_is_finished'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210622045705_finalize_events_bigint_conversion.rb:11:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:31:in `ddl_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:61:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
Expected batched background migration for the given configuration to be marked as 'finished', but it is 'active': {:job_class_name=>"CopyColumnUsingBackgroundMigrationJob", :table_name=>"events", :column_name=>"id", :job_arguments=>[["id"], ["id_convert_to_bigint"]]}
Finalize it manualy by running
sudo gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,events,id,'[["id"]\, ["id_convert_to_bigint"]]']
For more information, check the documentation
https://docs.gitlab.com/ee/user/admin_area/monitoring/background_migrations.html#database-migrations-failing-because-of-batched-background-migration-not-finished
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:1109:in `ensure_batched_background_migration_is_finished'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210622045705_finalize_events_bigint_conversion.rb:11:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:31:in `ddl_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:61:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
== 20210622045705 FinalizeEventsBigintConversion: migrating ===================
STDERR:
---- End output of "bash" "/tmp/chef-script20260811-7035-5hnmp3" ----
Ran "bash" "/tmp/chef-script20260811-7035-5hnmp3" returned 1
# GitLab のバックアップを実行
sudo gitlab-backup create
Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data
and are not included in this backup. You will need these files to restore a backup.
Please back them up manually.
2026-08-12 04:29:52 UTC -- Backup 1786508884_2026_08_12_16.3.7-ee is done.
2026-08-12 04:29:52 UTC -- Deleting backup and restore PID file ... done
# sudo gitlab-backup restore BACKUP=1786508884_2026_08_12_16.3.7-ee
2026-08-12 05:48:28 UTC -- Unpacking backup ...
2026-08-12 05:48:48 UTC -- Unpacking backup ... done
2026-08-12 05:48:48 UTC -- Restoring database ...
2026-08-12 05:48:48 UTC -- Be sure to stop Puma, Sidekiq, and any other process that
connects to the database before proceeding. For Omnibus
installs, see the following link for more information:
Before restoring the database, we will remove all existing
tables to avoid future upgrade problems. Be aware that if you have
custom tables in the GitLab database these tables and all data will be
removed.
Do you want to continue (yes/no)? yes
puma sidekiqなどは停めてから。と。
This task will now rebuild the authorized_keys file.
You will lose any data stored in the authorized_keys file.
Do you want to continue (yes/no)? yes
2026-08-12 05:52:18 UTC -- Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data
and are not included in this backup. You will need to restore these files manually.
2026-08-12 05:52:18 UTC -- Restore task is done.
2026-08-12 05:52:18 UTC -- Deleting backup and restore PID file ... done
詳細なログをありがとうございます!ログ全体を確認したところ、根本的な原因は「アップデートによって新しくなった Redis などのコンポーネントがメモリ上に古いまま残っており、その影響で Gitaly の再起動(sv restart)がタイムアウト(1351秒経過)してエラーになった」という現象です。 ログ内にある以下のメッセージが大きなヒントになっています。 Warnings: The version of the running redis service is different than what is installed.Please restart redis to start the new version.全サービスを一括でリフレッシュしてプロセスを掃除すれば、問題なく通過できます!
sudo gitlab-rails runner -e production 'puts GitLab::BackgroundMigration.remaining'
(結果が 0 になれば、次の 16.11.10 等へのアップグレードへ進んで大丈夫です)
16.11.10でrestart画面
Daemons using outdated libraries ├─────┐
│ Which services should be restarted? │
│ [*] gitlab-runsvdir.service │
│ <Ok> <Cancel>
こんな画面が出ました enterでrestartされました
Restarting previously running GitLab services
ok: run: gitaly: (pid 82382) 35s
ok: run: gitlab-kas: (pid 82884) 0s
ok: run: gitlab-workhorse: (pid 82894) 1s
ok: run: logrotate: (pid 82905) 0s
ok: run: nginx: (pid 82911) 1s
ok: run: postgresql: (pid 81836) 204s
ok: run: redis: (pid 54620) 1039s
ok: run: sidekiq: (pid 82921) 0s
_______ __ __ __
/ ____(_) /_/ / ____ _/ /_
/ / __/ / __/ / / __ `/ __ \
/ /_/ / / /_/ /___/ /_/ / /_/ /
\____/_/\__/_____/\__,_/_.___/
Upgrade complete! If your GitLab server is misbehaving try running
sudo gitlab-ctl restart
before anything else.
If you need to roll back to the previous version you can use the database
backup made during the upgrade (scroll up for the filename).
Scanning processes...
Scanning candidates...
Scanning linux images...
Running kernel seems to be up-to-date.
Restarting services...
systemctl restart gitlab-runsvdir.service
No containers need to be restarted.
No user sessions are running outdated binaries.
No VM guests are running outdated hypervisor (qemu) binaries on this host.
2026-08-12 10:16:44 UTC -- Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data
and are not included in this backup. You will need these files to restore a backup.
Please back them up manually.
コメント