Skip to content

Automated Test: feature-rolling-updates-implementation #319

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion common/src/main/java/org/keycloak/common/Profile.java
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,9 @@ public enum Feature {

USER_EVENT_METRICS("Collect metrics based on user events", Type.PREVIEW),

IPA_TUURA_FEDERATION("IPA-Tuura user federation provider", Type.EXPERIMENTAL)
IPA_TUURA_FEDERATION("IPA-Tuura user federation provider", Type.EXPERIMENTAL),

ROLLING_UPDATES("Rolling Updates", Type.PREVIEW),
;

private final Type type;
Expand Down
15 changes: 12 additions & 3 deletions docs/guides/operator/advanced-configuration.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -443,6 +443,12 @@ Check the https://kubernetes.io/docs/concepts/services-networking/network-polici

The Keycloak Operator offers updates strategies to control how the Operator handles changes to the Keycloak CR.

[CAUTION]
====
While on preview stage, the feature `rolling-updates` must be enabled.
Otherwise, the {project_name} Operator will fail.
====
Comment on lines +446 to +450

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

# First, let's examine the file context around lines 446-450
head -n 480 docs/guides/operator/advanced-configuration.adoc | tail -n 50

Repository: admin-coderabbit/coderabbit-keycloak-mirror

Length of output: 1973


🏁 Script executed:

# Search for rolling-updates feature mentions in the codebase
rg "rolling-updates" --type adoc --type md --type java --type yaml -B 2 -A 2

Repository: admin-coderabbit/coderabbit-keycloak-mirror

Length of output: 115


🏁 Script executed:

# Look for operator failure behavior related to rolling-updates
rg "rolling.?update" -i --type java -B 3 -A 3 | head -100

Repository: admin-coderabbit/coderabbit-keycloak-mirror

Length of output: 12841


🏁 Script executed:

# Check if there's operator code that validates rolling-updates enablement
rg "rolling" --type java -l | head -20

Repository: admin-coderabbit/coderabbit-keycloak-mirror

Length of output: 1464


🏁 Script executed:

# Search for validation of rolling-updates feature
rg "rolling.?update" -i --type java -B 5 -A 5 | grep -E "(fail|error|throw|require|validate)" -B 3 -A 3

Repository: admin-coderabbit/coderabbit-keycloak-mirror

Length of output: 5057


🏁 Script executed:

# Search for feature validation logic
rg "features.*enabled" --type java -B 3 -A 3 | head -80

Repository: admin-coderabbit/coderabbit-keycloak-mirror

Length of output: 9971


🏁 Script executed:

# Look for upgrade logic that might require rolling-updates
rg "UpgradeType|ROLLING|RECREATE" --type java | grep -i "strategy\|feature" | head -20

Repository: admin-coderabbit/coderabbit-keycloak-mirror

Length of output: 1150


🏁 Script executed:

# Check BaseUpgradeLogic which likely has upgrade handling
cat -n operator/src/main/java/org/keycloak/operator/upgrade/impl/BaseUpgradeLogic.java | head -150

Repository: admin-coderabbit/coderabbit-keycloak-mirror

Length of output: 7218


🏁 Script executed:

# Find and read UpdateCompatibilityCheck.java
find . -name "UpdateCompatibilityCheck.java" -type f -exec cat -n {} \;

Repository: admin-coderabbit/coderabbit-keycloak-mirror

Length of output: 3976


🏁 Script executed:

# Find and read UpdateCompatibilityMetadata.java
find . -name "UpdateCompatibilityMetadata.java" -type f -exec cat -n {} \;

Repository: admin-coderabbit/coderabbit-keycloak-mirror

Length of output: 4631


🏁 Script executed:

# Search for any error or failure behavior when rolling-updates feature is absent
rg "ROLLING_UPDATES" --type java -B 5 -A 5

Repository: admin-coderabbit/coderabbit-keycloak-mirror

Length of output: 5394


Clarify the specific failure mode and fix grammar.

The CAUTION states the Operator "will fail" if rolling-updates is not enabled, but the actual behavior is more limited: the update compatibility check commands (kc update check and kc update metadata) exit with a feature-disabled status when the feature is disabled. The Operator itself continues to function; it does not prevent general operation. Consider: "While the feature is in preview stage" (or "While in preview stage") instead of "While on preview stage" for correct grammar. Reword to clarify that this constraint applies to using rolling update compatibility checks, not to the Operator as a whole.

🤖 Prompt for AI Agents
In `@docs/guides/operator/advanced-configuration.adoc` around lines 446 - 450,
Reword the CAUTION block to fix grammar and clarify the actual failure mode:
state that while the rolling-updates feature is in preview, attempting to run
the update compatibility check commands (kc update check and kc update metadata)
will exit with a feature-disabled status; the {project_name} Operator itself
continues to operate normally. Replace "While on preview stage" with "While the
feature is in preview" (or "While in preview stage") and make clear this
limitation applies only to using the rolling-updates compatibility checks, not
general Operator functionality.


**Supported Updates Types:**

Rolling Updates:: Update the StatefulSet in a rolling fashion, minimizing downtime (requires multiple replicas).
Expand All @@ -466,11 +472,14 @@ kind: Keycloak
metadata:
name: example-kc
spec:
features:
enabled:
- rolling-updates # <1>
update:
strategy: Recreate|<not set> # <1>
strategy: Recreate|<not set> # <2>
----

<1> Set the desired update strategy here (Recreate in this example).
<1> Enable preview feature `rolling-updates`.
<2> Set the desired update strategy here (Recreate in this example).

[%autowidth]
.Possible field values
Expand Down
10 changes: 9 additions & 1 deletion docs/guides/server/update-compatibility.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,11 @@ preview="true"
previewDiscussionLink="https://github.com/keycloak/keycloak/discussions/36785"
>

// TODO Link to discussion?
[CAUTION]
====
While on preview stage, the feature `rolling-updates` must be enabled.
Otherwise, the commands will fail.
====

The goal of this tool is to assist with modifying a {project_name} deployment, whether upgrading to a new version, enabling/disabling features, or changing configuration.
The outcome will indicate whether a rolling upgrade is possible or if a recreate upgrade is required.
Expand Down Expand Up @@ -124,6 +128,10 @@ m|2
m|3
|Rolling Upgrade is not possible.
The deployment must be shut down before applying the new configuration.

m|4
|Rolling Upgrade is not possible.
The feature `rolling-updates` is disabled.
Comment on lines +132 to +134

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Reword exit-code 4 to reflect “feature disabled.”

Exit code 4 is returned before any compatibility evaluation, so describing it as “Rolling Upgrade is not possible” is misleading. Suggest wording like “Command cannot run because the preview feature rolling-updates is disabled.”

✏️ Suggested doc tweak
-m|4
-|Rolling Upgrade is not possible.
-The feature `rolling-updates` is disabled.
+m|4
+|Command cannot run because the preview feature `rolling-updates` is disabled.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
m|4
|Rolling Upgrade is not possible.
The feature `rolling-updates` is disabled.
m|4
|Command cannot run because the preview feature `rolling-updates` is disabled.
🤖 Prompt for AI Agents
In `@docs/guides/server/update-compatibility.adoc` around lines 132 - 134, Update
the documentation text that currently maps exit code 4 to "Rolling Upgrade is
not possible" so it accurately reflects that the command aborted because the
preview feature is disabled; change the wording for exit code 4 to something
like "Command cannot run because the preview feature `rolling-updates` is
disabled" and ensure the doc line referencing exit code 4 (the entry showing
"Rolling Upgrade is not possible" and mention of `rolling-updates`) is replaced
accordingly.

|===

</@tmpl.guide>
2 changes: 1 addition & 1 deletion docs/guides/templates/kc.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,6 @@ bin/kc.[sh|bat] bootstrap-admin ${parameters}
<#macro updatecompatibility parameters>
[source,bash]
----
bin/kc.[sh|bat] update-compatibility ${parameters}
bin/kc.[sh|bat] update-compatibility ${parameters} --features=rolling-updates
----
</#macro>
2 changes: 1 addition & 1 deletion operator/scripts/Dockerfile-custom-image
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ ARG IMAGE=keycloak
ARG VERSION=latest
FROM $IMAGE:$VERSION

RUN /opt/keycloak/bin/kc.sh build --db=postgres --health-enabled=true
RUN /opt/keycloak/bin/kc.sh build --db=postgres --health-enabled=true --features=rolling-updates
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@

package org.keycloak.operator.testsuite.integration;

import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
Expand All @@ -27,10 +28,11 @@
import org.awaitility.Awaitility;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
import org.keycloak.common.Profile;
import org.keycloak.operator.crds.v2alpha1.deployment.Keycloak;
import org.keycloak.operator.crds.v2alpha1.deployment.KeycloakStatusCondition;
import org.keycloak.operator.crds.v2alpha1.deployment.ValueOrSecret;
import org.keycloak.operator.crds.v2alpha1.deployment.spec.UnsupportedSpec;
import org.keycloak.operator.crds.v2alpha1.deployment.spec.FeatureSpec;
import org.keycloak.operator.crds.v2alpha1.deployment.spec.UpdateSpec;
import org.keycloak.operator.upgrade.UpdateStrategy;

Expand Down Expand Up @@ -105,11 +107,12 @@ private static Keycloak createInitialDeployment(UpdateStrategy updateStrategy) {
}
var updateSpec = new UpdateSpec();
updateSpec.setStrategy(updateStrategy);
kc.getSpec().setUpdateSpec(updateSpec);

if (kc.getSpec().getUnsupported() == null) {
kc.getSpec().setUnsupported(new UnsupportedSpec());
if (kc.getSpec().getFeatureSpec() == null) {
kc.getSpec().setFeatureSpec(new FeatureSpec());
}
kc.getSpec().setUpdateSpec(updateSpec);
kc.getSpec().getFeatureSpec().setEnabledFeatures(List.of(Profile.Feature.ROLLING_UPDATES.getKey()));
return kc;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -80,4 +80,8 @@ void printPreviewWarning() {
printError("Warning! This command is preview and is not recommended for use in production. It may change or be removed at a future release.");
}

void printFeatureDisabled() {
printError("Unable to use this command. The preview feature 'rolling-updates' is not enabled.");
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,9 @@
import java.io.File;
import java.io.IOException;

import org.keycloak.common.Profile;
import org.keycloak.quarkus.runtime.cli.PropertyException;
import org.keycloak.quarkus.runtime.compatibility.CompatibilityResult;
import org.keycloak.quarkus.runtime.compatibility.ServerInfo;
import org.keycloak.util.JsonSerialization;
import picocli.CommandLine;
Expand All @@ -41,6 +43,11 @@ public class UpdateCompatibilityCheck extends AbstractUpdatesCommand {

@Override
public void run() {
if (!Profile.isFeatureEnabled(Profile.Feature.ROLLING_UPDATES)) {
printFeatureDisabled();
picocli.exit(CompatibilityResult.FEATURE_DISABLED);
return;
}
printPreviewWarning();
validateConfig();
var info = readServerInfo();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,9 @@
import java.io.IOException;

import com.fasterxml.jackson.core.JsonProcessingException;
import org.keycloak.common.Profile;
import org.keycloak.quarkus.runtime.cli.PropertyException;
import org.keycloak.quarkus.runtime.compatibility.CompatibilityResult;
import org.keycloak.quarkus.runtime.compatibility.ServerInfo;
import org.keycloak.util.JsonSerialization;
import picocli.CommandLine;
Expand All @@ -41,6 +43,11 @@ public class UpdateCompatibilityMetadata extends AbstractUpdatesCommand {

@Override
public void run() {
if (!Profile.isFeatureEnabled(Profile.Feature.ROLLING_UPDATES)) {
printFeatureDisabled();
picocli.exit(CompatibilityResult.FEATURE_DISABLED);
return;
}
printPreviewWarning();
validateConfig();
var info = compatibilityManager.current();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,11 @@
public interface CompatibilityResult {

int ROLLING_UPGRADE_EXIT_CODE = 0;
int RECREATE_UPGRADE_EXIT_CODE = 4;
// see picocli.CommandLine.ExitCode
// 1 -> software error
// 2 -> usage error
int RECREATE_UPGRADE_EXIT_CODE = 3;
int FEATURE_DISABLED = 4;
Comment on lines +31 to +35

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

# Search for references to RECREATE_UPGRADE_EXIT_CODE and exit code values
rg -n "RECREATE_UPGRADE_EXIT_CODE|FEATURE_DISABLED" --type java

# Check for test files that might reference exit codes
rg -n "exitCode|exit.*4|exit.*3" --type java -A 2 -B 2 | head -100

# Search for documentation files mentioning exit codes
fd -e adoc -e md | xargs grep -l "exit.code\|exitCode" 2>/dev/null || true

Repository: admin-coderabbit/coderabbit-keycloak-mirror

Length of output: 15287


🏁 Script executed:

# More focused search: find where these constants are used
rg -n "return RECREATE_UPGRADE_EXIT_CODE|return.*4|System.exit" --type java -A 2 -B 2

# Check the IncompatibilityResult class mentioned in original scratchpad
fd -n "IncompatibilityResult.java" --type f

Repository: admin-coderabbit/coderabbit-keycloak-mirror

Length of output: 50648


Breaking change: Exit code 4 now has different semantics.

Previously, exit code 4 indicated that a recreate upgrade was required. After this change, exit code 4 now indicates that a feature is disabled, while exit code 3 indicates recreate upgrade is required. Any external scripts or automation tools checking for exit code 4 to detect the need for a recreate upgrade scenario will break and require updates.

🤖 Prompt for AI Agents
In
`@quarkus/runtime/src/main/java/org/keycloak/quarkus/runtime/compatibility/CompatibilityResult.java`
around lines 31 - 35, The change swapped semantics of exit codes so external
tooling expecting recreate-upgrade on code 4 will break; restore backward
compatibility by keeping RECREATE_UPGRADE_EXIT_CODE = 4 (or add a
LEGACY_RECREATE_UPGRADE_EXIT_CODE = 4) and assign FEATURE_DISABLED to a
different code (e.g., 5), or introduce both constants
(RECREATE_UPGRADE_EXIT_CODE = 4 and FEATURE_DISABLED = 5) and mark the old
mapping as deprecated; update the CompatibilityResult constants
(RECREATE_UPGRADE_EXIT_CODE, FEATURE_DISABLED or add
LEGACY_RECREATE_UPGRADE_EXIT_CODE) and add a short deprecation comment so
callers/tools can adapt.


/**
* The compatible {@link CompatibilityResult} implementation
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,36 +41,44 @@
@RawDistOnly(reason = "Requires creating JSON file to be available between containers")
public class UpdateCommandDistTest {

private static final String ENABLE_FEATURE = "--features=rolling-updates";

@Test
@Launch({UpdateCompatibility.NAME, UpdateCompatibilityMetadata.NAME})
public void testFeatureNotEnabled(CLIResult cliResult) {
cliResult.assertError("Unable to use this command. The preview feature 'rolling-updates' is not enabled.");
}

@Test
@Launch({UpdateCompatibility.NAME})
public void testMissingSubCommand(CLIResult cliResult) {
cliResult.assertError("Missing required subcommand");
}

@Test
@Launch({UpdateCompatibility.NAME, UpdateCompatibilityMetadata.NAME})
@Launch({UpdateCompatibility.NAME, UpdateCompatibilityMetadata.NAME, ENABLE_FEATURE})
public void testMissingOptionOnSave(CLIResult cliResult) {
cliResult.assertNoMessage("Missing required argument");
}

@Test
@Launch({UpdateCompatibility.NAME, UpdateCompatibilityCheck.NAME})
@Launch({UpdateCompatibility.NAME, UpdateCompatibilityCheck.NAME, ENABLE_FEATURE})
public void testMissingOptionOnCheck(CLIResult cliResult) {
cliResult.assertError("Missing required argument: " + UpdateCompatibilityCheck.INPUT_OPTION_NAME);
}

@Test
public void testCompatible(KeycloakDistribution distribution) throws IOException {
var jsonFile = createTempFile("compatible");
var result = distribution.run(UpdateCompatibility.NAME, UpdateCompatibilityMetadata.NAME, UpdateCompatibilityMetadata.OUTPUT_OPTION_NAME, jsonFile.getAbsolutePath());
var result = distribution.run(UpdateCompatibility.NAME, UpdateCompatibilityMetadata.NAME, UpdateCompatibilityMetadata.OUTPUT_OPTION_NAME, jsonFile.getAbsolutePath(), ENABLE_FEATURE);
result.assertMessage("Metadata:");
assertEquals(0, result.exitCode());

var info = JsonSerialization.mapper.readValue(jsonFile, ServerInfo.class);
assertEquals(Version.VERSION, info.getVersions().get(CompatibilityManagerImpl.KEYCLOAK_VERSION_KEY));
assertEquals(org.infinispan.commons.util.Version.getVersion(), info.getVersions().get(CompatibilityManagerImpl.INFINISPAN_VERSION_KEY));

result = distribution.run(UpdateCompatibility.NAME, UpdateCompatibilityCheck.NAME, UpdateCompatibilityCheck.INPUT_OPTION_NAME, jsonFile.getAbsolutePath());
result = distribution.run(UpdateCompatibility.NAME, UpdateCompatibilityCheck.NAME, UpdateCompatibilityCheck.INPUT_OPTION_NAME, jsonFile.getAbsolutePath(), ENABLE_FEATURE);
result.assertMessage("[OK] Rolling Upgrade is available.");
result.assertNoError("Rolling Upgrade is not available.");
}
Expand All @@ -85,7 +93,7 @@ public void testWrongVersions(KeycloakDistribution distribution) throws IOExcept
CompatibilityManagerImpl.INFINISPAN_VERSION_KEY, org.infinispan.commons.util.Version.getVersion()));
JsonSerialization.mapper.writeValue(jsonFile, info);

var result = distribution.run(UpdateCompatibility.NAME, UpdateCompatibilityCheck.NAME, UpdateCompatibilityCheck.INPUT_OPTION_NAME, jsonFile.getAbsolutePath());
var result = distribution.run(UpdateCompatibility.NAME, UpdateCompatibilityCheck.NAME, UpdateCompatibilityCheck.INPUT_OPTION_NAME, jsonFile.getAbsolutePath(), ENABLE_FEATURE);
result.assertError("[Versions] Rolling Upgrade is not available. 'keycloak' is incompatible: Old=0.0.0.Final, New=%s".formatted(Version.VERSION));

// incompatible infinispan version
Expand All @@ -94,7 +102,7 @@ public void testWrongVersions(KeycloakDistribution distribution) throws IOExcept
CompatibilityManagerImpl.INFINISPAN_VERSION_KEY, "0.0.0.Final"));
JsonSerialization.mapper.writeValue(jsonFile, info);

result = distribution.run(UpdateCompatibility.NAME, UpdateCompatibilityCheck.NAME, UpdateCompatibilityCheck.INPUT_OPTION_NAME, jsonFile.getAbsolutePath());
result = distribution.run(UpdateCompatibility.NAME, UpdateCompatibilityCheck.NAME, UpdateCompatibilityCheck.INPUT_OPTION_NAME, jsonFile.getAbsolutePath(), ENABLE_FEATURE);
result.assertError("[Versions] Rolling Upgrade is not available. 'infinispan' is incompatible: Old=0.0.0.Final, New=%s".formatted(org.infinispan.commons.util.Version.getVersion()));
}

Expand Down