Feature: Key Management Service (KMS) by vishesh92 · Pull Request #12711 · apache/cloudstack

vishesh92 · 2026-02-26T06:20:32Z

Key Management Service (KMS) with HSM Integration

Description

Introduces a Key Management Service (KMS) framework for CloudStack that provides envelope encryption for volume encryption. KEKs (Key Encryption Keys) stored in PKCS#11-compliant HSMs or the CloudStack database wrap per-volume DEKs (Data Encryption Keys), ensuring key material is never stored in plaintext.

Design Document: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Key+Management+Service+%28KMS%29+with+HSM+Integration
Docs PR: apache/cloudstack-documentation#634

New APIs

API	Auth	Description
`createKMSKey`	All users	Create a KMS key (KEK) bound to an HSM profile
`listKMSKeys`	All users	List KMS keys
`updateKMSKey`	All users	Update name, description, or enabled state
`deleteKMSKey`	All users	Delete a KMS key (if not in use)
`rotateKMSKey`	Admin	Rotate KEK with optional cross-HSM migration
`migrateVolumesToKMS`	Admin	Migrate passphrase-encrypted volumes to KMS
`addHSMProfile`	Admin	Add HSM profile configuration
`listHSMProfiles`	All users	List available HSM profiles
`updateHSMProfile`	Admin	Update HSM profile
`deleteHSMProfile`	Admin	Delete HSM profile

New Database Tables

kms_hsm_profiles, kms_hsm_profile_details, kms_keys, kms_kek_versions, kms_wrapped_key, kms_database_kek_objects

Modified: cloud.volumes — added kms_key_id and kms_wrapped_key_id columns.

New Global Settings

Setting	Default	Description
`kms.dek.size.bits`	`256`	DEK size in bits
`kms.retry.count`	`3`	Retry attempts for transient failures
`kms.retry.delay.ms`	`1000`	Delay between retries
`kms.operation.timeout.sec`	`30`	Per-operation timeout
`kms.rewrap.batch.size`	`50`	Keys rewrapped per background batch
`kms.rewrap.interval.ms`	`300000`	Background rewrap interval

UI Changes

New KMS top-level menu with KMS Keys and HSM Profiles sub-sections
KMS key selection in Deploy VM and Create Volume wizards
HSM profile management restricted to Admin users in UI

How to Test

Tested with:

# 1. Add an HSM profile (use SoftHSM2 for testing)

# 2. Create a KMS key with HSM profile

# 3. Create a disk offering with Encryption enabled

# 4. Deploy a VM/create a volume and specify the KMS key

codecov · 2026-02-26T07:12:27Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 3.51%. Comparing base (b744824) to head (b68661c).
⚠️ Report is 27 commits behind head on main.

❗ There is a different number of reports uploaded between BASE (b744824) and HEAD (b68661c). Click for more details.

HEAD has 1 upload less than BASE

Flag BASE (b744824) HEAD (b68661c)

unittests 1 0

Additional details and impacted files

@@              Coverage Diff              @@
##               main   #12711       +/-   ##
=============================================
- Coverage     18.02%    3.51%   -14.51%     
=============================================
  Files          5968      465     -5503     
  Lines        537086    40194   -496892     
  Branches      65961     7572    -58389     
=============================================
- Hits          96819     1414    -95405     
+ Misses       429347    38590   -390757     
+ Partials      10920      190    -10730

Flag	Coverage Δ
uitests	`3.51% <ø> (-0.02%)`	⬇️
unittests	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot

Pull request overview

This pull request introduces a comprehensive Key Management Service (KMS) framework for CloudStack that provides envelope encryption for volume encryption. KEKs (Key Encryption Keys) stored in PKCS#11-compliant HSMs or the CloudStack database wrap per-volume DEKs (Data Encryption Keys), ensuring key material is never stored in plaintext.

Changes:

Adds KMS framework with HSM integration for envelope encryption
Introduces 8 new admin/user APIs for KMS key and HSM profile management
Creates 6 new database tables for storing KMS metadata
Adds UI support for KMS key selection in volume and VM deployment workflows

Reviewed changes

Copilot reviewed 123 out of 123 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
`framework/kms/`	Core KMS framework interfaces and exceptions
`plugins/kms/`	Database and PKCS#11 KMS provider implementations
`engine/schema/`	Database entities and DAOs for KMS tables
`api/src/main/java/org/apache/cloudstack/api/command/`	New API commands for KMS operations
`ui/src/views/`	UI components for KMS key selection
`server/src/main/java/`	KMS manager implementation and integration
Test files	Unit tests for KMS retry logic and key creation

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

blueorangutan · 2026-02-26T13:44:04Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16962

Copilot

Pull request overview

Copilot reviewed 124 out of 124 changed files in this pull request and generated 8 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

tools/apidoc/gen_toc.py

engine/schema/src/main/java/com/cloud/storage/dao/VolumeDaoImpl.java

engine/storage/volume/src/main/java/org/apache/cloudstack/storage/volume/VolumeObject.java

server/src/main/java/com/cloud/api/query/dao/VolumeJoinDaoImpl.java

engine/schema/src/main/java/org/apache/cloudstack/kms/dao/KMSWrappedKeyDaoImpl.java

ui/src/views/compute/DeployVM.vue

server/src/main/java/com/cloud/api/ApiResponseHelper.java

vishesh92 · 2026-03-04T08:41:41Z

@blueorangutan package

blueorangutan · 2026-03-04T08:44:04Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-03-04T09:36:20Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16998

vishesh92 · 2026-03-05T06:59:04Z

@blueorangutan package

blueorangutan · 2026-03-05T07:00:04Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-03-05T07:52:47Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17008

sureshanaparti · 2026-03-23T09:02:07Z

api/src/main/java/com/cloud/event/EventTypes.java

+        entityEventDetails.put(EVENT_HSM_PROFILE_CREATE, HSMProfile.class);
+        entityEventDetails.put(EVENT_HSM_PROFILE_UPDATE, HSMProfile.class);
+        entityEventDetails.put(EVENT_HSM_PROFILE_DELETE, HSMProfile.class);
+


VOLUME.MIGRATE.TO.KMS event?

sureshanaparti · 2026-03-23T09:06:04Z

api/src/main/java/org/apache/cloudstack/api/command/admin/kms/MigrateVolumesToKMSCmd.java

+import java.util.List;
+
+@APICommand(name = "migrateVolumesToKMS",
+            description = "Migrates passphrase-based volumes to KMS (admin only)",


Suggested change

description = "Migrates passphrase-based volumes to KMS (admin only)",

description = "Migrates encrypted volumes to KMS",

not need of admin only, it's authorized, listed as admin's API

github-actions · 2026-03-26T20:00:55Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

sureshanaparti · 2026-04-02T07:29:28Z

api/src/main/java/org/apache/cloudstack/api/command/user/kms/hsm/AddHSMProfileCmd.java

+               description = "the ID of the project to add the HSM profile for")
+    private Long projectId;
+
+    @Parameter(name = "system", type = CommandType.BOOLEAN,


Suggested change

@Parameter(name = "system", type = CommandType.BOOLEAN,

@Parameter(name = "ApiConstants.IS_PUBLIC", type = CommandType.BOOLEAN,

sureshanaparti · 2026-04-02T07:30:47Z

api/src/main/java/org/apache/cloudstack/api/command/user/kms/hsm/AddHSMProfileCmd.java

+    @Parameter(name = "system", type = CommandType.BOOLEAN,
+               description = "whether this is a system HSM profile available to all users globally (root admin only). "
+                             + "Default is false")
+    private Boolean system;


Suggested change

private Boolean system;

private Boolean isPublic;

sureshanaparti · 2026-04-02T07:41:03Z

api/src/main/java/org/apache/cloudstack/api/response/VolumeResponse.java

    private String encryptionFormat;

+    @SerializedName(ApiConstants.KMS_KEY)
+    @Param(description = "KMS key id of the volume", since = "4.23.0")


Suggested change

@Param(description = "KMS key id of the volume", since = "4.23.0")

@Param(description = "KMS key of the volume", since = "4.23.0")

sureshanaparti · 2026-04-02T07:45:45Z

...chestration/src/main/java/org/apache/cloudstack/engine/orchestration/VolumeOrchestrator.java

+        return setPassphraseForVolumeEncryption(volume, null, null);
+    }
+
+    private VolumeVO setPassphraseForVolumeEncryption(VolumeVO volume, KMSKeyVO kmsKey, Long callerAccountId) {


Suggested change

private VolumeVO setPassphraseForVolumeEncryption(VolumeVO volume, KMSKeyVO kmsKey, Long callerAccountId) {

private VolumeVO setKmsKeyForVolumeEncryption(VolumeVO volume, KMSKeyVO kmsKey, Long callerAccountId) {

change if relavant.

sureshanaparti · 2026-04-02T07:47:37Z

engine/schema/src/main/resources/META-INF/db/schema-42210to42300.sql

+    CONSTRAINT `fk_kms_keys__account_id` FOREIGN KEY (`account_id`) REFERENCES `account`(`id`) ON DELETE CASCADE,
+    CONSTRAINT `fk_kms_keys__domain_id` FOREIGN KEY (`domain_id`) REFERENCES `domain`(`id`) ON DELETE CASCADE,
+    CONSTRAINT `fk_kms_keys__zone_id` FOREIGN KEY (`zone_id`) REFERENCES `data_center`(`id`) ON DELETE CASCADE,
+    CONSTRAINT `fk_kms_keys__hsm_profile_id` FOREIGN KEY (`hsm_profile_id`) REFERENCES `kms_hsm_profiles`(`id`)


Suggested change

CONSTRAINT `fk_kms_keys__hsm_profile_id` FOREIGN KEY (`hsm_profile_id`) REFERENCES `kms_hsm_profiles`(`id`)

CONSTRAINT `fk_kms_keys__hsm_profile_id` FOREIGN KEY (`hsm_profile_id`) REFERENCES `kms_hsm_profiles`(`id`) ON DELETE CASCADE

sureshanaparti · 2026-04-02T07:47:52Z

engine/schema/src/main/resources/META-INF/db/schema-42210to42300.sql

+    INDEX `idx_kms_key_status` (`kms_key_id`, `status`, `removed`),
+    INDEX `idx_kek_label` (`kek_label`),
+    CONSTRAINT `fk_kms_kek_versions__kms_key_id` FOREIGN KEY (`kms_key_id`) REFERENCES `kms_keys`(`id`) ON DELETE CASCADE,
+    CONSTRAINT `fk_kms_kek_versions__hsm_profile_id` FOREIGN KEY (`hsm_profile_id`) REFERENCES `kms_hsm_profiles`(`id`)


Suggested change

CONSTRAINT `fk_kms_kek_versions__hsm_profile_id` FOREIGN KEY (`hsm_profile_id`) REFERENCES `kms_hsm_profiles`(`id`)

CONSTRAINT `fk_kms_kek_versions__hsm_profile_id` FOREIGN KEY (`hsm_profile_id`) REFERENCES `kms_hsm_profiles`(`id`) ON DELETE CASCADE

sureshanaparti · 2026-04-02T07:52:59Z

api/src/main/java/com/cloud/event/EventTypes.java

+    public static final String EVENT_KMS_KEY_UPDATE = "KMS.KEY.UPDATE";
+    public static final String EVENT_KMS_KEY_ROTATE = "KMS.KEY.ROTATE";
+    public static final String EVENT_KMS_KEY_DELETE = "KMS.KEY.DELETE";
+    public static final String EVENT_VOLUME_MIGRATE_TO_KMS = "VOLUME.MIGRATE.TO.KMS";


Suggested change

public static final String EVENT_VOLUME_MIGRATE_TO_KMS = "VOLUME.MIGRATE.TO.KMS";

sureshanaparti · 2026-04-02T07:58:46Z

api/src/main/java/org/apache/cloudstack/api/command/user/kms/RotateKMSKeyCmd.java

+import javax.inject.Inject;
+
+@APICommand(name = "rotateKMSKey",
+            description = "Rotates KEK by creating new version and scheduling gradual re-encryption",


Suggested change

description = "Rotates KEK by creating new version and scheduling gradual re-encryption",

description = "Rotates KMS key (KEK) by creating new version and scheduling gradual re-encryption",

sureshanaparti · 2026-04-02T08:04:05Z

api/src/main/java/org/apache/cloudstack/api/command/user/kms/hsm/AddHSMProfileCmd.java

+import java.util.HashMap;
+import java.util.Map;
+
+@APICommand(name = "addHSMProfile", description = "Adds a new HSM profile", responseObject = HSMProfileResponse.class,


Suggested change

@APICommand(name = "addHSMProfile", description = "Adds a new HSM profile", responseObject = HSMProfileResponse.class,

@APICommand(name = "createHSMProfile", description = "Creates a new HSM profile", responseObject = HSMProfileResponse.class,

sureshanaparti · 2026-04-02T08:20:02Z

api/src/main/java/org/apache/cloudstack/api/response/HSMProfileResponse.java

+    @Param(description = "whether the HSM profile is enabled")
+    private Boolean enabled;
+
+    @SerializedName("system")


Suggested change

@SerializedName("system")

@SerializedName(ApiConstants.IS_PUBLIC)

sureshanaparti · 2026-04-02T08:27:39Z

...chestration/src/main/java/org/apache/cloudstack/engine/orchestration/VolumeOrchestrator.java

            return volume;
        }
+
+        if (kmsKey != null) {


refactor the kms key to a method

sureshanaparti · 2026-04-02T08:29:15Z

tools/apidoc/gen_toc.py

    'Domain': 'Domain',
    'Template': 'Template',
+    'KMS': 'KMS',
+    'HSM': 'KMS',


Suggested change

'HSM': 'KMS',

'HSM': 'Key Management System',

sureshanaparti · 2026-04-02T08:30:39Z

ui/src/config/section/kms.js

+        {
+          api: 'migrateVolumesToKMS',
+          icon: 'swap-outlined',
+          docHelp: 'adminguide/storage.html#lifecycle-operations',


update the doc links

boring-cyborg bot added component:api component:build labels Feb 26, 2026

vishesh92 force-pushed the feature-kms branch from 0a283c9 to 5112eea Compare February 26, 2026 06:57

vishesh92 force-pushed the feature-kms branch from ceb2913 to d629f87 Compare February 26, 2026 07:39

vishesh92 requested a review from Copilot February 26, 2026 07:39

Copilot started reviewing on behalf of vishesh92 February 26, 2026 07:40 View session

Copilot AI reviewed Feb 26, 2026

View reviewed changes

vishesh92 changed the title ~~Feature kms~~ Feature: Key Management Service (KMS) Feb 26, 2026

vishesh92 force-pushed the feature-kms branch from ba00a5d to 943f19a Compare February 26, 2026 12:39

apache deleted a comment from blueorangutan Feb 26, 2026

weizhouapache added this to the 4.23.0 milestone Feb 26, 2026

vishesh92 force-pushed the feature-kms branch 3 times, most recently from 8ea09bb to df2df4b Compare March 2, 2026 10:36

vishesh92 requested a review from Copilot March 2, 2026 11:40

Copilot started reviewing on behalf of vishesh92 March 2, 2026 11:41 View session

Copilot AI reviewed Mar 2, 2026

View reviewed changes

vishesh92 force-pushed the feature-kms branch 2 times, most recently from 75519c2 to 1fce0b2 Compare March 4, 2026 06:38

apache deleted a comment from blueorangutan Mar 5, 2026

vishesh92 requested a review from sureshanaparti March 5, 2026 12:23

vishesh92 added 10 commits March 23, 2026 14:12

fixups

529fd9d

fix list apis

6ba0af5

directly create the key in the store

bca1b25

Add missing license

d7c5e24

fixup

d8cdddb

Fix missing labels and label generation

949c00e

Fixup ui bug

1087503

fixups

0fe31fb

Add smoke test

30891ee

Fix test failures

2c49e92

vishesh92 force-pushed the feature-kms branch from fc55897 to 2c49e92 Compare March 23, 2026 08:44

sureshanaparti reviewed Mar 23, 2026

View reviewed changes

fixups

b68661c

github-actions bot added status:has-conflicts and removed status:has-conflicts labels Mar 23, 2026

sureshanaparti reviewed Apr 2, 2026

View reviewed changes

	description = "Migrates passphrase-based volumes to KMS (admin only)",
	description = "Migrates encrypted volumes to KMS",

	@Parameter(name = "system", type = CommandType.BOOLEAN,
	@Parameter(name = "ApiConstants.IS_PUBLIC", type = CommandType.BOOLEAN,

	@Param(description = "KMS key id of the volume", since = "4.23.0")
	@Param(description = "KMS key of the volume", since = "4.23.0")

	private VolumeVO setPassphraseForVolumeEncryption(VolumeVO volume, KMSKeyVO kmsKey, Long callerAccountId) {
	private VolumeVO setKmsKeyForVolumeEncryption(VolumeVO volume, KMSKeyVO kmsKey, Long callerAccountId) {

	CONSTRAINT `fk_kms_keys__hsm_profile_id` FOREIGN KEY (`hsm_profile_id`) REFERENCES `kms_hsm_profiles`(`id`)
	CONSTRAINT `fk_kms_keys__hsm_profile_id` FOREIGN KEY (`hsm_profile_id`) REFERENCES `kms_hsm_profiles`(`id`) ON DELETE CASCADE

	CONSTRAINT `fk_kms_kek_versions__hsm_profile_id` FOREIGN KEY (`hsm_profile_id`) REFERENCES `kms_hsm_profiles`(`id`)
	CONSTRAINT `fk_kms_kek_versions__hsm_profile_id` FOREIGN KEY (`hsm_profile_id`) REFERENCES `kms_hsm_profiles`(`id`) ON DELETE CASCADE

	description = "Rotates KEK by creating new version and scheduling gradual re-encryption",
	description = "Rotates KMS key (KEK) by creating new version and scheduling gradual re-encryption",

	@APICommand(name = "addHSMProfile", description = "Adds a new HSM profile", responseObject = HSMProfileResponse.class,
	@APICommand(name = "createHSMProfile", description = "Creates a new HSM profile", responseObject = HSMProfileResponse.class,

	@SerializedName("system")
	@SerializedName(ApiConstants.IS_PUBLIC)

Conversation

vishesh92 commented Feb 26, 2026 • edited by sureshanaparti Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Key Management Service (KMS) with HSM Integration

Description

New APIs

New Database Tables

New Global Settings

UI Changes

How to Test

Uh oh!

codecov bot commented Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

blueorangutan commented Feb 26, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vishesh92 commented Mar 4, 2026

Uh oh!

blueorangutan commented Mar 4, 2026

Uh oh!

blueorangutan commented Mar 4, 2026

Uh oh!

vishesh92 commented Mar 5, 2026

Uh oh!

blueorangutan commented Mar 5, 2026

Uh oh!

blueorangutan commented Mar 5, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Mar 26, 2026

Uh oh!

sureshanaparti Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sureshanaparti Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sureshanaparti Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vishesh92 commented Feb 26, 2026 •

edited by sureshanaparti

Loading

codecov bot commented Feb 26, 2026 •

edited

Loading

sureshanaparti Apr 2, 2026 •

edited

Loading

sureshanaparti Apr 2, 2026 •

edited

Loading

sureshanaparti Apr 2, 2026 •

edited

Loading