Release notes - GitHub Enterprise Server 3.14 Docs

Home

- About enterprise accounts
- Create a README
- - Configuring rate limits
  - Verify or approve a domain
- Configuring backups
- - Restrict email notifications
  - Personal access token policies
- - Manage pre-receive hooks
- - Configure a cluster network
  - Monitor cluster health
  - Configure HA replication
  - Replacing the primary MySQL node, step 14 (running ghe-cluster-config-apply) might fail with errors. If this occurs, re-running ghe-cluster-config-apply is expected to succeed.
  - In the header bar displayed to site administrators, some icons are not available.
  - When enabling automatic update checks for the first time in the Management Console, the status is not dynamically reflected until the "Updates" page is reloaded.
  - When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.
  - After a restore, existing outside collaborators are unable to be added to repositories in a new organization. This issue can be resolved by running /usr/local/share/enterprise/ghe-es-search-repair on the appliance.
  - After a geo-replica is promoted to be a primary by running ghe-repl-promote, the actions workflow of a repository does not have any suggested workflows.
Enterprise Server 3.14.8
Download GitHub Enterprise Server 3.14.8
February 18, 2025
📣 This is not the latest patch release of this release series, and this is not the latest release of Enterprise Server. Please use the latest release for the latest security, performance, and bug fixes.
Warning: For instances installed on Google Cloud Platform (GCP), hotpatches to GitHub Enterprise Server version 3.14.8 will result in errors being reported in the upgrade log. We recommend hotpatching to a newer 3.14 version instead. [Updated: 2025-03-11]
3.14.8: Security fixes
- HIGH: An attacker could access environment variables in the debug artifacts uploaded by the CodeQL action after a failed code scanning workflow run. This includes any secrets that were exposed to the workflow as environment variables. The attacker requires read access to the repository to access the debug artifact. Users who do not have debug logging enabled are unaffected. The impact to GitHub Enterprise Server users is limited to internal actors. To mitigate this issue, GitHub no longer logs the complete environment by default. GitHub has requested CVE-2025-24362 for this vulnerability, which was reported via the GitHub Bug Bounty program.
- Packages have been updated to the latest security versions.
3.14.8: Bug fixes
- In some cluster configurations, it was not possible to enable GitHub Advanced Security in bulk.
- In certain cases, on an instance in a cluster configuration, secret scanning would fail to run due to misconfiguration of a Kafka service.
- In an instance in a high-availability or cluster configuration, administrators who updated the instance's license did not see the change reflected on the "Licenses" page in the UI.
- Audit log indices from 2018 could occasionally fail to be created when migrating to Elasticsearch 8.
- In some cases, a file in the code view would appear as JSON instead of HTML.
- Attachment records were not created when JWT tokens were included in user asset URLs on issues.
- When an administrator suspended a user from the site admin dashboard, the form required them to complete Digital Services Act (DSA) fields that are not relevant on GitHub Enterprise Server.
- Enterprise owners could not modify the "Outside collaborators" policy. Instead a 404 Not Found response was returned.
- In cluster environments, API rate limits were calculated using the cluster node IP address instead of the client IP address. This could lead to incorrect rate limiting and the wrong IP address being recorded in audit log entries.
- The relative date for commits was sometimes incorrectly displayed in the web UI.
- Users were unable to open issues where the events timeline contained references to projects that were not moved over during a migration. Instead, the 500 error page was displayed.
- Users who had authenticated to multiple accounts, then logged out of one account, were unable to switch to a different account on the platform.
- Certain search terms for repositories and wikis did not return all valid results.
- In some cluster configurations, secret scanning failed to run normally due to connection failures.
3.14.8: Changes
- Log files on the appliance root disk are compressed immediately upon rotation daily instead of after a 24 hour delay. You can revert to the previous delaycompress behavior by signing in as an SSH admin user, setting ghe-config logrotate.delaycompress true and then running ghe-config-apply.
- The CodeQL Action has been updated to v3.28.6 to enable uploading artifacts in debug mode without logging the complete environment when running CodeQL CLI v2.20.3+.
- The ghe-live-migrations --init-target command fails with a descriptive error message if the specified upgrade path is not supported.
3.14.8: Known issues
- Instances installed on Google Cloud Platform (GCP) could experience errors when the latest hotpatch was applied. We recommend waiting for the next patch release to hotpatch. [Updated: 2025-03-11]
- During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.
- If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see "Troubleshooting access to the Management Console."
- On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.
- In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.
- Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.
- Admin stats REST API endpoints may timeout on appliances with many users or repositories. Retrying the request until data is returned is advised.
- When following the steps for Replacing the primary MySQL node, step 14 (running ghe-cluster-config-apply) might fail with errors. If this occurs, re-running ghe-cluster-config-apply is expected to succeed.
- In the header bar displayed to site administrators, some icons are not available.
- When enabling automatic update checks for the first time in the Management Console, the status is not dynamically reflected until the "Updates" page is reloaded.
- When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.
- For appliances in a high availability configuration, Elasticsearch indices are deleted in two situations:
  
  On failover
  
  When running ghe-repl-teardown <REPLICA_HOSTNAME> from the primary instance
  
  All indices are recoverable, except for Audit Log indices. Since Elasticsearch itself is the source of truth for these logs, they may only be recoverable from a backup. If you need assistance, visit GitHub Enterprise Support.
  
  [Updated: 2025-03-12]
3.14.8: Errata
- The warning and known issues section have been updated to accurately reflect that instances installed on GCP will face issues while hotpatching to 3.14.8. Previously, the warning and known issue indicated that customers would face issues either while upgrading or hotpatching to version 3.14.8. [Updated: 2025-03-11]
Enterprise Server 3.14.7
https://github.com/dotnet/core/issues/9671.
To avoid unnecessary error messages when users attempt to create a ruleset in evaluate mode in a repository that is user owned, we removed the evaluate mode option on the ruleset.

3.14.7: Known issues

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.
If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see "Troubleshooting access to the Management Console."
On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.
In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.
Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.
Admin stats REST API endpoints may timeout on appliances with many users or repositories. Retrying the request until data is returned is advised.
When following the steps for Replacing the primary MySQL node, step 14 (running ghe-cluster-config-apply) might fail with errors. If this occurs, re-running ghe-cluster-config-apply is expected to succeed.
In the header bar displayed to site administrators, some icons are not available.
When enabling automatic update checks for the first time in the Management Console, the status is not dynamically reflected until the "Updates" page is reloaded.
When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.
For appliances in a high availability configuration, Elasticsearch indices are deleted in two situations:
- On failover
- When running ghe-repl-teardown <REPLICA_HOSTNAME> from the primary instance
All indices are recoverable, except for Audit Log indices. Since Elasticsearch itself is the source of truth for these logs, they may only be recoverable from a backup. If you need assistance, visit GitHub Enterprise Support.

[Updated: 2025-03-12]

Enterprise Server 3.14.6

Failure details.

The removal rate of issues from Git repositories was slower than necessary.

3.14.6: Changes

Log output for git maintenance now includes the time taken to complete the maintenance process.
When exporting repositories to blob storage using the migrations REST API endpoint to start an organization migration, the maximum compressed archive size is limited to 90 GB. This is an increase from 30 GB.
Removes the minimum date for the new commit filter bar.
When exporting repositories using the migrations REST API, prior to blob storage upload the tarball is staged in the root volume. For more disk capacity, the tarball will now be staged in the data volume.

3.14.6: Known issues

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.
If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.
On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.
In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.
Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.
Admin stats REST API endpoints may timeout on appliances with many users or repositories. Retrying the request until data is returned is advised.
When following the steps for Replacing the primary MySQL node, step 14 (running ghe-cluster-config-apply) might fail with errors. If this occurs, re-running ghe-cluster-config-apply is expected to succeed.
In the header bar displayed to site administrators, some icons are not available.
When enabling automatic update checks for the first time in the Management Console, the status is not dynamically reflected until the "Updates" page is reloaded.
When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.
For appliances in a high availability configuration, Elasticsearch indices are deleted in two situations:
- On failover
- When running ghe-repl-teardown <REPLICA_HOSTNAME> from the primary instance
All indices are recoverable, except for Audit Log indices. Since Elasticsearch itself is the source of truth for these logs, they may only be recoverable from a backup. If you need assistance, visit GitHub Enterprise Support.

[Updated: 2025-03-12]

Enterprise Server 3.14.5

Download GitHub Enterprise Server 3.14.5

December 03, 2024

📣 This is not the latest patch release of this release series, and this is not the latest release of Enterprise Server. Please use the latest release for the latest security, performance, and bug fixes.

3.14.5: Security fixes

LOW: Instance administrators could see tokens used to authenticate against gitauth in plaintext in/var/log/github-audit.log.
Packages have been updated to the latest security versions.

3.14.5: Bug fixes

Embedded images in wiki pages were broken.

3.14.5: Known issues

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.
If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.
On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.
In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.
Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.
Admin stats REST API endpoints may timeout on appliances with many users or repositories. Retrying the request until data is returned is advised.
When following the steps for Replacing the primary MySQL node, step 14 (running ghe-cluster-config-apply) might fail with errors. If this occurs, re-running ghe-cluster-config-apply is expected to succeed.
In the header bar displayed to site administrators, some icons are not available.
When enabling automatic update checks for the first time in the Management Console, the status is not dynamically reflected until the "Updates" page is reloaded.
When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.
Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.
Attempting to stop replications after stopping GitHub Actions on a GitHub Enterprise Server instance would fail, reporting that MSSQL was not responding. The can be avoided by start MSSQL prior to stopping replication /usr/local/share/enterprise/ghe-nomad-jobs queue /etc/nomad-jobs/mssql/mssql.hcl
For appliances in a high availability configuration, Elasticsearch indices are deleted in two situations:
- On failover
- When running ghe-repl-teardown <REPLICA_HOSTNAME> from the primary instance
All indices are recoverable, except for Audit Log indices. Since Elasticsearch itself is the source of truth for these logs, they may only be recoverable from a backup. If you need assistance, visit GitHub Enterprise Support.

[Updated: 2025-03-12]

Enterprise Server 3.14.4

Download GitHub Enterprise Server 3.14.4

November 12, 2024

3.14.4: Bug fixes

Customers performing a feature version upgrade to 3.13.6 or 3.14.3 may experience issues with database migrations due to data issues during database conversions.

3.14.4: Known issues

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.
If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.
On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.
In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.
Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.
Admin stats REST API endpoints may timeout on appliances with many users or repositories. Retrying the request until data is returned is advised.
When following the steps for Replacing the primary MySQL node, step 14 (running ghe-cluster-config-apply) might fail with errors. If this occurs, re-running ghe-cluster-config-apply is expected to succeed.
In the header bar displayed to site administrators, some icons are not available.
When enabling automatic update checks for the first time in the Management Console, the status is not dynamically reflected until the "Updates" page is reloaded.
When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.
Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.
Attempting to stop replications after stopping GitHub Actions on a GHES instanstance would fail, reporting that MSSQL was not responding. The can be avoided by start MSSQL prior to stopping replication /usr/local/share/enterprise/ghe-nomad-jobs queue /etc/nomad-jobs/mssql/mssql.hcl.
When operating in a high availability configuration, running ghe-repl-promote on a replica node will fail if the original primary cannot be reached by the replica node. This is because the ghe-repl-promote script attempts to decommission all Elasticsearch nodes other than the promoted node, however these requests are made to the original primary node which is no longer reachable. The error message written to the terminal will be similar to:
```
Maintenance mode has been enabled for active replica <REPLICA_HOSTNAME>
{"message": "No server is currently available to service your request. Sorry about that. Please try resubmitting your request and contact your local GitHub Enterprise site administrator if the problem persists."}
jq: error (at :3): Cannot index string with string "node"
```
If this occurs, workaround this issue by running the following command — this changes the ghe-repl-promote script in place:
```
sudo sed -i.bak -e '/for node_hostname in/i if ! $forced; then' -e '/^  done/a fi' /usr/local/bin/ghe-repl-promote
```
Then re-run the updated ghe-repl-promote script.

[Updated: 2024-11-29]
For appliances in a high availability configuration, Elasticsearch indices are deleted in two situations:
- On failover
- When running ghe-repl-teardown <REPLICA_HOSTNAME> from the primary instance
All indices are recoverable, except for Audit Log indices. Since Elasticsearch itself is the source of truth for these logs, they may only be recoverable from a backup. If you need assistance, visit GitHub Enterprise Support.

[Updated: 2025-03-12]

Enterprise Server 3.14.3

CVE-2024-10001 for this vulnerability, which was reported via the GitHub Bug Bounty program. [Updated: 2025-01-27]

3.14.3: Bug fixes

When saving settings in the Management Console, the configuration run would stop if the enterprise-manage process was restarted.
On an instance with GitHub Actions enabled, some maintenance tasks could fail due to incomplete upgrade steps during previous upgrades to new releases of GitHub Enterprise Server.
A repeated error message concerning connectivity to port 6002 was emitted to the system logs when GitHub Actions was enabled.
The initial setup certificate generation in AWS took longer than expected due to fallback to private IPs. The time for this fallback has been reduced.
The ghe-support-bundle generation would fail when the aqueduct-lite service is down.
If the primary instance was unreachable, running ghe-repl-stop --force on a replica would fail during the config apply run.
Administrators in the SCIM private beta (versions < 3.14) that decided to upgrade their private beta appliance see an incorrectly checked box in the "SCIM Configuration" section of the Enterprise settings authentication security page in 3.14.
Certain URLs may have caused a 500 error on instances that use the mandatory message feature logging.
When restoring from a backup, repositories that had been deleted in the last 90 days were not completely restored.
For instances that use secret scanning, custom messages for push protection set by the enterprise did not display to users.
Restoring Git repositories using backup-utils occasionally failed.
Enterprise installations experienced unpredictable repository search results due to the default 4,000 repository limit. A relaxed repository filter mode, which includes all single-tenant organization repositories and bypasses the limit, has been introduced. Administrators can enable this mode using ghe-config app.github.enterprise-repo-search-filter-enabled true && ghe-config-apply.
Running config-apply became stuck under certain circumstances due to a misconfiguration with Packages and Elasticsearch.
Audit log events for secret scanning alerts incorrectly displayed a blank secret type when generated for a custom pattern.
Some customers upgrading to 3.14 experienced issues with undecryptable records during the upgrade. This issue has now been resolved. A diagnostic script will run to assess impact, if no records are affected the message "SUCCESS: Encrypted records OK." will print to the console and can be ignored. If the error message "WARN: Error reading encrypted records!" is output, we recommend you read Undecryptable records. [Updated: 2024-01-22]

3.14.3: Changes

When connecting to an appliance via SSH, a notification about upcoming root disk changes displays.

3.14.3: Known issues

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.
If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. See Troubleshooting access to the Management Console.
On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.
In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.
Repositories originally imported using ghe-migrator will not correctly track GitHub Advanced Security contributions.
Admin stats REST API endpoints may timeout on appliances with many users or repositories. Retrying the request until data is returned is advised.
When following the steps for Replacing a node in an emergency may fail with errors if the node being replaced is still reachable. If this occurs, shutdown the node and repeat the steps.
If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.
When restoring data originally backed up from a 3.13 appliance onto a 3.13 appliance, the Elasticsearch indices need to be reindexed before some of the data will show up. This happens via a nightly scheduled job. It can also be forced by running /usr/local/share/enterprise/ghe-es-search-repair.
An organization-level code scanning configuration page is displayed on instances that do not use GitHub Advanced Security or code scanning.
In the header bar displayed to site administrators, some icons are not available.
When enabling automatic update checks for the first time in the Management Console, the status is not dynamically reflected until the "Updates" page is reloaded.
When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.
Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.
Customers doing feature version upgrade to 3.14.3 may experience issues with database migrations due to data issues during database conversions. [Added: 2024-11-08]
When operating in a high availability configuration, running ghe-repl-promote on a replica node will fail if the original primary cannot be reached by the replica node. This is because the ghe-repl-promote script attempts to decommission all Elasticsearch nodes other than the promoted node, however these requests are made to the original primary node which is no longer reachable. The error message written to the terminal will be similar to:
```
Maintenance mode has been enabled for active replica <REPLICA_HOSTNAME>
{"message": "No server is currently available to service your request. Sorry about that. Please try resubmitting your request and contact your local GitHub Enterprise site administrator if the problem persists."}
jq: error (at :3): Cannot index string with string "node"
```
If this occurs, workaround this issue by running the following command — this changes the ghe-repl-promote script in place:
```
sudo sed -i.bak -e '/for node_hostname in/i if ! $forced; then' -e '/^  done/a fi' /usr/local/bin/ghe-repl-promote
```
Then re-run the updated ghe-repl-promote script.

[Updated: 2024-11-29]
For appliances in a high availability configuration, Elasticsearch indices are deleted in two situations:
- On failover
- When running ghe-repl-teardown <REPLICA_HOSTNAME> from the primary instance
All indices are recoverable, except for Audit Log indices. Since Elasticsearch itself is the source of truth for these logs, they may only be recoverable from a backup. If you need assistance, visit GitHub Enterprise Support.

[Updated: 2025-03-12]

Enterprise Server 3.14.2

CVE-2024-9487. This vulnerability was reported via the GitHub Bug Bounty program.

3.14.2: Bug fixes

A missing configuration value would cause Dependabot to be unable to create group update pull requests.
HAProxy reloading was failure prone, which could lead to failed Git operations. This reloading process has been replaced with a more resilient Systemd process.
This error message mbind: Operation not permitted was repeatedly showing in the /var/log/mysql/mysql.err MySQL logs.
The backup of audit logs could take longer after upgrading to Elasticsearch 8.
An unhandled nil value when configuring Actions storage with AWS S3 via OIDC configuration in the terminal could cause an error.
Users were unable to sign out from gist pages.
On an instance with secret scanning enabled, the custom pattern page would not load because dry run results were tied to a deleted repository.
Suspended users were not always correctly routed to the correct "suspended" page.
The "List teams" API endpoint returned duplicate results when paginating.
When managing the organization permissions required for fine-grained personal access tokens, for custom properties or projects, the Admin access level could not be selected.
A model with no URL could cause a ghe-migrator import to fail.
The ghe-spokesctl status command showed repaired repositories as broken if their network ID changed during the repair (for example, when the repository was detached from it's original network).
Missing URLs on import could lead to migration failures without logging or explanation.
On the security overview dashboard, data initialization could fail when creating new organizations or changing GitHub Advanced Security licensing.
Restore could fail when restoring MySQL using backup-utils.
The help documentation for the Actions Workflow editor was not loading correctly. [Updated: 2025-02-18]

3.14.2: Changes

ghe-remove-node will display the log file location when running in quiet mode.
Pre-receive hook environments can use the clone3() system call.
The creation, deletion, or change in visibility of a gist has been added to the audit log.

3.14.2: Known issues

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.
If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.
On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.
In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.
Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.
The admin stats REST API endpoints may time out on appliances with many users or repositories. Retrying the request until data is returned is advised.
When following the steps for Replacing a node in an emergency may fail with errors if the node being replaced is still reachable. If this occurs, shutdown the node and repeat the steps.
If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.
When restoring data originally backed up from a 3.13 appliance onto a 3.13 appliance, the elasticsearch indices need to be reindexed before some of the data will show up. This happens via a nightly scheduled job. It can also be forced by running /usr/local/share/enterprise/ghe-es-search-repair.
An organization-level code scanning configuration page is displayed on instances that do not use GitHub Advanced Security or code scanning.
In the header bar displayed to site administrators, some icons are not available.
When enabling automatic update checks for the first time in the Management Console, the status is not dynamically reflected until the "Updates" page is reloaded.
When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.
Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.
Images embedded in wiki pages may stop rendering shortly after being published. [Updated: 2024-10-16]
When operating in a high availability configuration, running ghe-repl-promote on a replica node will fail if the original primary cannot be reached by the replica node. This is because the ghe-repl-promote script attempts to decommission all Elasticsearch nodes other than the promoted node, however these requests are made to the original primary node which is no longer reachable. The error message written to the terminal will be similar to:
```
Maintenance mode has been enabled for active replica <REPLICA_HOSTNAME>
{"message": "No server is currently available to service your request. Sorry about that. Please try resubmitting your request and contact your local GitHub Enterprise site administrator if the problem persists."}
jq: error (at :3): Cannot index string with string "node"
```
If this occurs, workaround this issue by running the following command — this changes the ghe-repl-promote script in place:
```
sudo sed -i.bak -e '/for node_hostname in/i if ! $forced; then' -e '/^  done/a fi' /usr/local/bin/ghe-repl-promote
```
Then re-run the updated ghe-repl-promote script.

[Updated: 2024-11-29]
For appliances in a high availability configuration, Elasticsearch indices are deleted in two situations:
- On failover
- When running ghe-repl-teardown <REPLICA_HOSTNAME> from the primary instance
All indices are recoverable, except for Audit Log indices. Since Elasticsearch itself is the source of truth for these logs, they may only be recoverable from a backup. If you need assistance, visit GitHub Enterprise Support.

[Updated: 2025-03-12]

3.14.2: Deprecations

The option to "copy Storage settings from Actions" in the Management Console ("GitHub Packages" > "Packages Storage Settings") has been removed. [Updated: 2024-11-20]

Enterprise Server 3.14.1

CVE ID CVE-2024-8810 for this vulnerability, which was reported via the GitHub Bug Bounty Program. [Updated: 2024-11-07]

3.14.1: Bug fixes

On an instance with GitHub Actions enabled, due to an insufficient wait time, MS SQL and MySQL replication could fail with the error message Failed to start nomad service!.
ghe-storage-find was sometimes unable to identify a data disk.
After upgrading the relevant GHES version, the resolvconf service failed to start due to a missing directory.
Some pre-receive hooks using the faccessat2 system call, such as those using Alpine Linux as the base, failed unexpectedly.
When configuring a high availability replica and during the database seeding of a MySQL replica node, restarting the nomad service could time out. Consequently, when MySQL replication attempted to start an error was reported, and setting up replication failed.
On an instance in a cluster configuration, the ghe-cluster-status command returned an error if a soft-deleted repository had a checksum mismatch.
Fixes and improvements for the git core module.
Some repositories could miss spokes information after restoring in a clustering topology due to unrescued exceptions.
In organizations with a large number of repositories, when an administrator used repository properties to target repositories in an organization ruleset, the ruleset index page timed out.
After a user created a Projects Insights chart with time as the X-axis, the chart became hidden and inaccessible.
Fixes a known issue where some links to GitHub Docs from GitHub Enterprise Server may lead to a “Page not found.” Previously, the links incorrectly added enterprise-cloud@latest to the URL.
A bug introduced in 3.12 which prevented the search input in the global navigation from displaying a dropdown of search suggestions has been fixed. The search input functionality prior to 3.12 has been restored, and users are once again able to see and submit suggested search queries, including scope suggestions.
Custom links to other repositories displayed incorrect breadcrumbs.
The Secret Scanning Push Protection custom resource link set at the Enterprise level was not being displayed to users being blocked when pushing secrets to a repository using git through the command line interface.
Following an upgrade, Elasticsearch search migrations are sometimes incorrectly reported as failing in the audit log, even though the migrations completed successfully. [Updated: 2024-09-27]

3.14.1: Changes

For instances deployed on Amazon Web Services (AWS), site administrators can configure regional AWS STS endpoints for OIDC from the Management Console.
Site administrators can now configure the instance with NUMA optimizations.

3.14.1: Known issues

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.
If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.
On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.
In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.
Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.
Admin stats REST API endpoints may timeout on appliances with many users or repositories. Retrying the request until data is returned is advised.
When following the steps for Replacing the primary MySQL node, step 14 (running ghe-cluster-config-apply) might fail with errors. If this occurs, re-running ghe-cluster-config-apply is expected to succeed.
In the header bar displayed to site administrators, some icons are not available.
When enabling automatic update checks for the first time in the Management Console, the status is not dynamically reflected until the "Updates" page is reloaded.
When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.
Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.
Images embedded in wiki pages may stop rendering shortly after being published. [Updated: 2024-10-16]
When operating in a high availability configuration, running ghe-repl-promote on a replica node will fail if the original primary cannot be reached by the replica node. This is because the ghe-repl-promote script attempts to decommission all Elasticsearch nodes other than the promoted node, however these requests are made to the original primary node which is no longer reachable. The error message written to the terminal will be similar to:
```
Maintenance mode has been enabled for active replica <REPLICA_HOSTNAME>
{"message": "No server is currently available to service your request. Sorry about that. Please try resubmitting your request and contact your local GitHub Enterprise site administrator if the problem persists."}
jq: error (at :3): Cannot index string with string "node"
```
If this occurs, workaround this issue by running the following command — this changes the ghe-repl-promote script in place:
```
sudo sed -i.bak -e '/for node_hostname in/i if ! $forced; then' -e '/^  done/a fi' /usr/local/bin/ghe-repl-promote
```
Then re-run the updated ghe-repl-promote script.

[Updated: 2024-11-29]
For appliances in a high availability configuration, Elasticsearch indices are deleted in two situations:
- On failover
- When running ghe-repl-teardown <REPLICA_HOSTNAME> from the primary instance
All indices are recoverable, except for Audit Log indices. Since Elasticsearch itself is the source of truth for these logs, they may only be recoverable from a backup. If you need assistance, visit GitHub Enterprise Support.

[Updated: 2025-03-12]

Enterprise Server 3.14.0

Download GitHub Enterprise Server 3.14.0

August 27, 2024

For upgrade instructions, see Upgrading GitHub Enterprise Server.

3.14.0: Features

Instance administration
- On an instance with multiple replica nodes, to start or stop replication for all nodes in a single configuration run, administrators can use the ghe-repl-start-all and ghe-repl-stop-all commands.
Instance services
- Administrators can scale the appliance using generation 2 virtual machines, with support for booting in UEFI mode. This requires deploying a new instance and restoring data onto it. See Using generation 2 virtual machines.
- Nomad has been upgraded to 1.5.17 and Consul has been upgraded to 1.17.4. These services are used in GitHub Enterprise Server to orchestrate containers and configuration.
Identity and access management
- Automated user provisioning via the System for Cross-domain Identity Management (SCIM) standard is available in public beta. Instances that use SAML authentication can enable SCIM to provision user accounts and manage their lifecycle from an identity provider (IdP). You can configure SCIM using an application for supported IdPs, or using the REST API endpoints for SCIM. See Configuring user provisioning with SCIM on GitHub Enterprise Server.
  - If your instance already uses SAML, you will need to configure a new IdP application that supports automated provisioning via SCIM.
  - Existing private beta customers should also reconfigure their implementation with an updated application.
  - During the public beta, we recommend testing SCIM support for your identity system in a non-production instance before adding SCIM to your current setup.
- Organization owners can create and assign custom organization roles, delegating administrative duties to trusted teams and users. See Managing custom organization roles.
- Users can use the account switcher to switch between multiple accounts. See Switching between accounts.
- On an instance that uses built-in authentication, users can use passkeys to sign in securely to GitHub, without needing to input their password. See Authenticating with a passkey.
- Enterprises that use an SSH certificate authority can allow SSH certificates to be used to access user-owned repositories. See Enforcing policies for security settings in your enterprise.
Audit logs
- Every 24 hours, a health check runs for each audit log stream. If a stream is set up incorrectly, an email will be sent to the enterprise owners as notification that their audit log stream is not properly configured.
Secret scanning
- Users can specify which teams or roles have the ability to bypass push protection. This feature is in public beta and subject to change. See Editing your configuration of default setup.
- Organizations that use default setup for code scanning can use organization-level model packs to extend the coverage of multiple repositories. This feature is in public beta and subject to change. See Editing your configuration of default setup.
- CodeQL can scan Java projects without a build. This feature is in public beta and subject to change.
- This release comes installed with version 2.17.6 of the CodeQL CLI, used in the CodeQL action for code scanning. Significant updates since the default version installed on GitHub Enterprise Server 3.13 include:
  - Support for Java 22, Swift 5.10, TS 5.4, and C# 12
  - New queries for C/C++, Go, Java, and Ruby:
    - cpp/type-confusion: Detects casts to invalid types
    - cpp/iterator-to-expired-container: Detects the creation of iterators owned by temporary objects that are about to be destroyed
    - go/uncontrolled-allocation-size: Detects slice memory allocation with excessive size value
    - java/unvalidated-url-forward: Prevents information disclosure caused by unsafe URL construction
    - rb/insecure-mass-assignment: Detects instances of mass assignment operations accepting arbitrary parameters
    - rb/csrf-protection-not-enabled: Detects cases where Cross-Site Request Forgery protection is not enabled in Ruby on Rails controllers
Dependabot
- Users can consolidate Dependabot pull requests by enabling grouped security updates for related dependencies in a package ecosystem. See Dependabot options reference. [Updated: 2024-10-07]
Code security
- The security overview dashboard, with the ability to view secret scanning metrics and trending data for the enablement of security features, is available at the enterprise level. See Viewing security insights.
- On the security overview dashboard, users can filter by security tool. This feature is in public beta and subject to change.
- In the dependency graph, a software bill of materials (SBOM) generated for a package now includes the package URL for more packages. Previously, the package URL was not included if the manifest file referenced a package with a version range.
GitHub Actions
- For self-hosted GitHub Actions runners on this GitHub Enterprise Server release, the minimum required version of the GitHub Actions Runner application is 2.317.0. See the release notes for this version in the okta-scim for more information.
- Custom firewall rules are removed during the upgrade process.
- When enabling automatic update checks for the first time in the Management Console, the status is not dynamically reflected until the "Updates" page is reloaded.
- During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.
- If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.
- On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.
- In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.
- Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.
- REST API endpoints for admin stats may time out on appliances with many users or repositories. Retrying the request until data is returned is advised.
- When following the steps for Replacing a node in an emergency may fail with errors if the node being replaced is still reachable. If this occurs, shut down the node and repeat the steps.
- If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.
- When restoring data originally backed up from a 3.13 appliance, the Elasticsearch indices need to be reindexed before some of the data will show up. This happens via a nightly scheduled job. It can also be forced by running /usr/local/share/enterprise/ghe-es-search-repair.
- The global search bar does not have suggestions enabled due to the redesigned navigation and pending new search experience.
- Upgrades include an error concerning Error deregistering job for consul-template. This message does not indicate any problems with your install and can be safely ignored.
- Some links to GitHub Docs from GitHub Enterprise Server may lead to a "Page not found," because an enterprise-cloud@latest portion is incorrectly added to the URL.
- An organization-level code scanning configuration page is displayed on instances that do not use GitHub Advanced Security or code scanning.
- In the header bar displayed to site administrators, some icons are not available.
- When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.
- On boot, the resolvconf service may fail to start because the /run/resolvconf directory does not exist when the service attempts to touch a file there, with the error:
```
/bin/touch: cannot touch '/run/resolvconf/postponed-update': No such file or directory
```
  If this occurs, workaround this issue with the following commands — this change will persist on reboots, but not upgrades:
```
sudo sed -i.bak \
'/\[Service\]/a ExecStartPre\=\/bin\/mkdir \-p \/run\/resolvconf' \
/etc/systemd/system/resolvconf.service.d/local.conf

sudo systemctl daemon-reload
sudo systemctl start resolvconf 
```
- Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.
- When enabling automatic update checks for the first time in the Management Console, the status is not dynamically reflected until the "Updates" page is reloaded.
- Following an upgrade, Elasticsearch search migrations are sometimes incorrectly reported as failing in the audit log, even though the migrations completed successfully. [Updated: 2024-09-27]
- Images embedded in wiki pages may stop rendering shortly after being published. [Updated: 2024-10-16]
- When operating in a high availability configuration, running ghe-repl-promote on a replica node will fail if the original primary cannot be reached by the replica node. This is because the ghe-repl-promote script attempts to decommission all Elasticsearch nodes other than the promoted node, however these requests are made to the original primary node which is no longer reachable. The error message written to the terminal will be similar to:
```
Maintenance mode has been enabled for active replica <REPLICA_HOSTNAME>
{"message": "No server is currently available to service your request. Sorry about that. Please try resubmitting your request and contact your local GitHub Enterprise site administrator if the problem persists."}
jq: error (at :3): Cannot index string with string "node"
```
  If this occurs, workaround this issue by running the following command — this changes the ghe-repl-promote script in place:
```
sudo sed -i.bak -e '/for node_hostname in/i if ! $forced; then' -e '/^  done/a fi' /usr/local/bin/ghe-repl-promote
```
  Then re-run the updated ghe-repl-promote script.
  
  [Updated: 2024-11-29]
- For appliances in a high availability configuration, Elasticsearch indices are deleted in two situations:
  - On failover
  - When running ghe-repl-teardown <REPLICA_HOSTNAME> from the primary instance
  All indices are recoverable, except for Audit Log indices. Since Elasticsearch itself is the source of truth for these logs, they may only be recoverable from a backup. If you need assistance, visit GitHub Enterprise Support.
  
  [Updated: 2025-03-12]

3.14.0: Deprecations

The Manage GHES API reached feature parity with the Management Console API in GHES 3.12. As a result, we will remove the Management Console API in GitHub Enterprise Server 3.15. For information about updating tooling that relies on the Management Console API, see REST API endpoints for Management Console.
Team discussions have been removed from GitHub Enterprise Server. The sunset of this feature was announced in 2023. See the GitHub Blog post. [Updated: 2025-02-13]

3.14.0: Errata

These release notes previously indicated as a known issue that on GitHub Enterprise Server 3.14.0 when log forwarding is enabled, some forwarded log entries may be duplicated. The fix for this problem was already included prior to the release of GitHub Enterprise Server 3.14.0. [Updated: 2024-09-16]
These release notes did not include a note for support of the directories key in dependabot.yml files. [Updated: 2024-10-07]
The "Changes" section indicated that "Pushes that update over 5,000 branches no longer trigger webhooks or GitHub Actions workflows." The change instead affects GitHub Enterprise Server version 3.15. [Updated: 2024-10-30]
These release notes previously did not include a note for the deprecation of team discussions.