Improve skill eval harness and skill trigger accuracy (#394)

* fix(eval): serial execution by default and full process tree cleanup

- Default --num-workers to 1 (serial) to prevent resource exhaustion
  from concurrent claude -p processes
- Use start_new_session=True so each claude process gets its own
  process group
- Kill entire process group (SIGTERM → SIGKILL escalation) on
  timeout/early-exit to prevent orphaned node child processes
- Parallel execution still available via --num-workers N

* Improve skill descriptions for b2c-scapi-admin and b2c-am

- b2c-scapi-admin: rewrite description to emphasize backend integration
  use cases (ERP/OMS/WMS sync, data exports) and differentiate from
  b2c-config (CLI auth) and b2c-scapi-shopper (storefront APIs)
- b2c-am: merge b2c-users-roles into b2c-am, adding BM roles CRUD and
  permissions management. Updated description to cover both AM and BM
  user/role administration. Removed redundant b2c-users-roles skill.
- Updated b2c-am evals to reflect merged BM roles scope

* Add changeset for skill description improvements

* Improve b2c-site-import-export description and fix weak eval prompts

- b2c-site-import-export: tighten description to emphasize "site archive"
  (not generic "site data") to reduce false positives on CSV catalog
  imports. Rewrite weak custom-object-type prompt that overlapped with
  b2c-metadata to focus on the import workflow.
- b2c-job: rewrite two prompts that tested site archive content rather
  than job execution — now emphasize running/monitoring the job via CLI.

* Update changeset to cover all skill description improvements

Author: clavery

.changeset/improve-skill-descriptions.md

@@ -0,0 +1,5 @@
+---
+'@salesforce/b2c-agent-plugins': patch
+---
+
+Improve skill trigger accuracy: rewrite b2c-scapi-admin and b2c-site-import-export descriptions, merge b2c-users-roles into b2c-am, fix weak eval prompts for b2c-job

skills/b2c-cli/skills/b2c-am/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: b2c-am
-description: Manage Account Manager resources including API clients, users, roles, and organizations. Use this skill whenever the user needs to create or update API clients, onboard or offboard developers, assign Business Manager roles scoped to tenants, audit user permissions, look up organizations, or provision API clients for CI/CD pipelines. Also use when managing role assignments or querying Account Manager data -- even if they just say 'add a new developer' or 'set up an API client'.
+description: Manage users, roles, API clients, and organizations across Account Manager and Business Manager using the b2c CLI. Use this skill whenever the user needs to create or delete users, grant or revoke roles (AM or BM), onboard or offboard developers, audit permissions, look up organizations, create or update API clients for CI/CD pipelines, or manage BM role permissions on an instance. Also use when the user asks about user administration, role assignments, or permission audits — even if they just say "add a new developer", "set up an API client", or "who has admin access".
 ---
 
 # B2C Account Manager Skill
@@ -21,9 +21,10 @@ Account Manager commands work out of the box with no configuration. The CLI uses
 
 | Operations | Client Credentials (roles on API client) | User Auth (roles on user account) |
 |---|---|---|
-| Users & Roles | User Administrator | Account Administrator or User Administrator |
-| Organizations | Not supported -- use `--user-auth` | Account Administrator |
-| API Clients | Not supported -- use `--user-auth` | Account Administrator or API Administrator |
+| AM Users & Roles | User Administrator | Account Administrator or User Administrator |
+| AM Organizations | Not supported -- use `--user-auth` | Account Administrator |
+| AM API Clients | Not supported -- use `--user-auth` | Account Administrator or API Administrator |
+| BM Roles | OCAPI permissions for `/roles` resource | OCAPI permissions for `/roles` resource |
 
 Organization and API client management are only available with user authentication.
 
@@ -217,6 +218,53 @@ b2c am orgs get <org-id>
 b2c am orgs get "My Organization"
 ```
 
+## Business Manager Roles
+
+BM role commands operate on a specific Commerce Cloud instance (via `--server` or config).
+
+```bash
+# list BM roles on the configured instance
+b2c bm roles list
+
+# target a different instance
+b2c bm roles list --server my-sandbox.demandware.net
+
+# get role details (with user list)
+b2c bm roles get Administrator --expand users
+
+# create a custom role
+b2c bm roles create MyCustomRole --description "Custom role for content editors"
+
+# delete a custom role (system roles cannot be deleted)
+b2c bm roles delete MyCustomRole
+
+# grant a BM role to a user on the instance
+b2c bm roles grant user@example.com --role Administrator
+
+# revoke a BM role from a user
+b2c bm roles revoke user@example.com --role Administrator
+
+# all commands support --json for machine-readable output
+b2c bm roles list --json
+```
+
+### Business Manager Role Permissions
+
+Permissions use a file-based get/set workflow since the API replaces all permissions at once.
+
+```bash
+# view permission summary
+b2c bm roles permissions get Administrator
+
+# export permissions to a JSON file for editing
+b2c bm roles permissions get Administrator --output admin-perms.json
+
+# edit the file, then apply
+b2c bm roles permissions set Administrator --file admin-perms.json
+```
+
+The permissions JSON has four sections: `functional`, `module`, `locale`, and `webdav`. Each can be scoped to organization, site, or unscoped depending on type.
+
 ## Common Workflows
 
 ### User Onboarding

skills/b2c-cli/skills/b2c-am/evals/trigger-evals.json

@@ -5,8 +5,8 @@
   {"query": "A contractor left our team. I need to remove their Account Manager user and revoke all their role assignments across tenants.", "should_trigger": true},
   {"query": "I need to look up which organization ID our sandbox belongs to so I can assign the right roles to a new team member.", "should_trigger": true},
   {"query": "We're setting up a new project and need to create an API client with the SALESFORCE_COMMERCE_API role and scope it to two tenants.", "should_trigger": true},
+  {"query": "How do I grant a Business Manager user access to specific sites using site-level roles in BM itself?", "should_trigger": true},
   {"query": "How do I create a SLAS client for my headless storefront to handle shopper login and guest sessions?", "should_trigger": false},
   {"query": "I need to configure OAuth scopes for my SCAPI integration. Where do I set the allowed scopes on my client credentials?", "should_trigger": false},
-  {"query": "How do I grant a Business Manager user access to specific sites using site-level roles in BM itself?", "should_trigger": false},
   {"query": "I want to set up SSO with our corporate identity provider for Business Manager logins.", "should_trigger": false}
 ]

skills/b2c-cli/skills/b2c-job/evals/trigger-evals.json

@@ -1,9 +1,9 @@
 [
   {"query": "The product search on our sandbox isn't returning newly added products. I think the search index is stale and needs a rebuild. How do I trigger that?", "should_trigger": true},
-  {"query": "I have a local folder with metadata XML and site preferences that I need to push to our dev instance. How do I import it and wait for it to finish?", "should_trigger": true},
+  {"query": "I have a site archive zip ready and I need to run the import job on my sandbox and wait for it to complete. What's the CLI command and how do I check if it succeeded?", "should_trigger": true},
   {"query": "The nightly product export job keeps failing on our sandbox and I can't figure out why from Business Manager. Is there a way to run it and see the detailed logs?", "should_trigger": true},
   {"query": "I just updated a bunch of product descriptions via a data feed and now I need to make sure the search index picks them up before our QA team tests tomorrow morning. How do I kick off a reindex and confirm it completed?", "should_trigger": true},
-  {"query": "I need to export our entire site configuration — preferences, payment methods, shipping config — as a site archive that I can then import into a different sandbox. What's the process to trigger that export and download the result?", "should_trigger": true},
+  {"query": "I need to trigger a site export job on my instance and then download the resulting archive. How do I run the export and get the output files?", "should_trigger": true},
   {"query": "We have a scheduled import that runs at 2am but it seems to be stuck in a running state for over 3 hours. I need to check what's happening with it — is there a way to see the execution status and logs for currently running processes on the instance?", "should_trigger": true},
   {"query": "I need to write a batch processing script that runs nightly to clean up expired promotions. What's the job step pattern?", "should_trigger": false},
   {"query": "How do I set up a cron job on our CI server to automatically run our test suite every night?", "should_trigger": false},

skills/b2c-cli/skills/b2c-site-import-export/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: b2c-site-import-export
-description: Import and export site archives with metadata XML on B2C Commerce instances using the b2c CLI. Use this skill whenever the user needs to import site data, write metadata XML for custom attributes on products/orders/customers, define custom object types, configure site preferences via import, or work with site archive directory structure and IMPEX XML formats. Also use when the user asks about system-objecttype-extensions.xml, custom-objecttype-definitions.xml, preferences.xml, or any site archive import/export workflow — even if they just say they need to "push metadata" or "add a field to products".
+description: Import and export site archives containing metadata XML on B2C Commerce instances using the b2c CLI. Use this skill whenever the user needs to import a site archive directory or zip to an instance, export site configuration as XML, structure a site archive folder (sites/site_template/meta/), write or debug metadata XML files (system-objecttype-extensions.xml, custom-objecttype-definitions.xml, preferences.xml), or push custom attributes, custom object types, or site preferences to a sandbox via site import. Also use when an import job fails with schema validation errors — even if they just say "push metadata to the sandbox" or "import my XML files".
 ---
 
 # Site Import/Export Skill

skills/b2c-cli/skills/b2c-site-import-export/evals/trigger-evals.json

@@ -1,7 +1,7 @@
 [
   {"query": "I need to add a 'loyaltyTier' attribute to the Customer profile on our sandbox. I've got some XML written but I'm not sure about the right directory structure or how to push it up to the instance.", "should_trigger": true},
   {"query": "We have a folder with metadata XML and preferences files that define our feature flags. How do I get this imported onto our dev instance so the new attributes show up in Business Manager?", "should_trigger": true},
-  {"query": "I'm setting up a new custom object type for storing per-site API credentials. What does the XML format look like for the type definition and what does the import directory layout need to be?", "should_trigger": true},
+  {"query": "I wrote a custom-objecttype-definitions.xml for a new object type and I need to import it to the sandbox. How do I structure the site archive folder and run the import with the CLI?", "should_trigger": true},
   {"query": "I need to export the full site configuration from our staging instance so I can diff it against production before we go live. What's the best way to pull down all the metadata, preferences, and custom object definitions as XML files?", "should_trigger": true},
   {"query": "We're setting up a new sandbox and I need to replicate the entire configuration from another instance — site preferences, custom attributes, payment methods, shipping configs, the works. How do I package all that into a site archive and import it?", "should_trigger": true},
   {"query": "I have a sites/site_template/meta directory with a bunch of XML files but the import keeps failing with a schema validation error. The error message references system-objecttype-extensions.xml line 47. How do I debug what's wrong with my archive structure?", "should_trigger": true},

skills/b2c-cli/skills/b2c-users-roles/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: b2c-scapi-admin
-description: Integrate with SCAPI Admin APIs for server-to-server backend operations. Use this skill whenever the user needs to sync inventory via Admin APIs, manage orders or catalog data through SCAPI, build customer data integrations, configure Account Manager OAuth for admin scopes, or implement bulk data imports with NDJSON. Also use when querying admin endpoints for promotions, price books, or coupon management -- even if they just say 'pull orders from Commerce Cloud' or 'update product data via API'.
+description: Build backend integrations that sync data between B2C Commerce and external systems like ERPs, OMS, WMS, or CRMs using SCAPI Admin APIs. Use this skill whenever the user needs to pull or push orders, products, inventory, or customer data programmatically from a backend service, set up server-to-server authentication with Account Manager client credentials and admin OAuth scopes, implement bulk inventory imports with NDJSON, or call any Commerce API from a script or pipeline (not a storefront). Also use when building nightly data exports, warehouse sync jobs, or customer data integrations — even if they just say "pull orders into our ERP" or "sync inventory from the warehouse".
 ---
 
 # SCAPI Admin APIs