W-20917380 B2C MCP Telemetry

[allison-grady]

Summary

Sends telemetry data to verify that tool usage is correctly captured. Modifies the B2C MCP package to send telemetry data to Azure App Insights. Modifies the B2C Tooling package with a reusable telemetry tool.

SDK Operations (b2c-tooling-sdk)

Telemetry Operations:

• createTelemetry() - Creates a telemetry instance to allow sending telemetry data logs to Microsoft Azure

Key Features

• Collects anonymous usage data (server lifecycle, tool calls, command metrics, environment info) to improve the MCP server and CLI
• Sends events to Azure Application Insights: COMMAND_START, COMMAND_SUCCESS, TOOL_CALLED, SERVER_STOPPED, plus exception tracking
• Auto-disabled in development: bin/dev.js sets SFCC_DISABLE_TELEMETRY=true which skips telemetry initialization
• Production opt-out: Set SF_DISABLE_TELEMETRY=true or SFCC_DISABLE_TELEMETRY=true to disable
• Testing override: Set SFCC_APP_INSIGHTS_KEY to use a different App Insights instance (requires SFCC_DISABLE_TELEMETRY=false in dev mode)
• Implementation: Connection string in package.json (oclif.telemetry.connectionString), auto-initialized by BaseCommand from SDK, TOOL_CALLED events in src/server.ts, SERVER_STOPPED in src/commands/mcp.ts

Technical Implementation

• @salesforce/b2c-dx-mcp: Refactored telemetry to use BaseCommand auto-initialization from package.json, replaced NODE_ENV gating with SFCC_DISABLE_TELEMETRY=true in bin/dev.js, removed src/config.ts, updated README with new env vars (SF_DISABLE_TELEMETRY, SFCC_DISABLE_TELEMETRY, SFCC_APP_INSIGHTS_KEY), and updated tests
• @salesforce/b2c-tooling-sdk: Added Telemetry.isDisabled(), Telemetry.getConnectionString(), sendException(), and dataDir option; updated BaseCommand with auto-telemetry initialization from pjson.oclif.telemetry, exception tracking in catch(), and COMMAND_SUCCESS events in finally()

Documentation

• Added README documentation in b2c-dx-mcp

Testing

Unit Tests

• test/commands/mcp.test.ts: Verifies no --no-telemetry CLI flag exists (telemetry controlled via env var only)
• Verifies telemetry initialization passes telemetry instance to server
• Verifies server starts successfully without telemetry when disabled
• Verifies toolsets are added to telemetry attributes
• Verifies Telemetry.isDisabled() respects SF_DISABLE_TELEMETRY and SFCC_DISABLE_TELEMETRY
• Verifies Telemetry.getConnectionString() respects disable flags and SFCC_APP_INSIGHTS_KEY override
• Verifies telemetry lifecycle events (SERVER_STOPPED, sendException)
• test/server.test.ts: Verifies server accepts telemetry in options, works without telemetry, and sends TOOL_CALLED events correctly (success, error result, thrown exception, timing accuracy, disabled telemetry)

Manual Testing

• Verified telemetry events by running with explicit App Insights connection string:

SFCC_DISABLE_TELEMETRY=false SFCC_APP_INSIGHTS_KEY="{InstrumentationKey=...}" node ./packages/b2c-dx-mcp/bin/dev.js --toolsets SCAPI --allow-non-ga-tools

then starting and stopping b2c-dx via the Cursor Settings. confirmed events appeared in Azure Application Insights portal

---

• Tests pass (pnpm test)
• Code is formatted (pnpm run format)

[yhsieh1]

We should move telemetry to under b2c-tooling-sdk so that it can be shared in other packages. telemetry.ts will need to be refactored to move to SDK. Such as:

• Make it configurable - Remove hardcoded PROJECT, make paths/config injectable
• Split files - types.ts, telemetry.ts, index.ts
• Add factory function - createTelemetry(options: TelemetryOptions): Telemetry
• Add dependency - Add @salesforce/telemetry to SDK's package.json
• Add barrel exports - Export via SDK's package.json exports

We also need to think about how to provide the config.json that the caller b2c-dx-mcp need to provide. We probably can generated it during CI/CD under packages/b2c-dx-mcp/config.json

[yhsieh1]

@allison-grady Please update the PR description to have detailed information and test steps to give review more context and easy to understand the PR. You could reference other PRs such as #63

[clavery]

Hi @allison-grady I've got some ideas on how we should support this across all packages in this project. I worked this out as a claude plan and given the size I put it in this this gist: https://gist.github.com/clavery/5afe869ccfbfd9d8df34958f7ded07d1

can you @yhsieh1 review that? Happy to talk through it as well. I incorporated some of the smaller comments I made here into it.

Basically I want to pull this back as far as possible:

• Support initialize using our shared BaseCommand
• Leverage oclifs configuration in package.json for package specifics (connection string, etc). This also gives a convenient location if we do want to replace in CI/CD or other places
• Use oclif CLI dataDir function instead of a non-standard directory
• track command life cycle and exceptions in the spot where we are already doing that.

[yhsieh1]

Hi @allison-grady I've got some ideas on how we should support this across all packages in this project. I worked this out as a claude plan and given the size I put it in this this gist: https://gist.github.com/clavery/5afe869ccfbfd9d8df34958f7ded07d1

can you @yhsieh1 review that? Happy to talk through it as well. I incorporated some of the smaller comments I made here into it.

Basically I want to pull this back as far as possible:

• Support initialize using our shared BaseCommand
• Leverage oclifs configuration in package.json for package specifics (connection string, etc). This also gives a convenient location if we do want to replace in CI/CD or other places
• Use oclif CLI dataDir function instead of a non-standard directory
• track command life cycle and exceptions in the spot where we are already doing that.

@clavery @allison-grady The gist looks good but I wonder how to prevent local development pollute the production telemetry data? One approach is to disable it by default on local development. We can set SFCC_DISABLE_TELEMETRY=true either in package.json script or override it in dev.js.

[clavery]

We can set SFCC_DISABLE_TELEMETRY=true either in package.json script or override it in dev.js.

Just set it in dev.js like Allison had (but use our env vars not NODE_ENV; that was my only complaint about that)

You can set it in tests/e2e tests as well if it happens to trigger there

[allison-grady]

We can set SFCC_DISABLE_TELEMETRY=true either in package.json script or override it in dev.js.

To clarify, this will look like:
process.env.SFCC_DISABLE_TELEMETRY = process.env.SFCC_DISABLE_TELEMETRY || 'true';

[yhsieh1]

It looks good. Thank you!