Skip to content

Commit 1f24b81

Browse files
authored
perf(test): speed up CLI unit tests, silence stdout noise, fix flakey MCP tests (#260)
* perf(test): speed up CLI unit tests from ~8s to ~1.7s Two key changes: 1. Fix sandbox reset test poll-interval (5s → 0s): The test used poll-interval: 5 which caused a real 5-second sleep in waitForSandbox. Other sandbox tests already use poll-interval: 0 to avoid this. 2. Replace runCommand with shared Config in cip/report and scapi/schemas tests: runCommand from @oclif/test calls Config.load() on every invocation (~1s cold, ~40ms warm). Instead, load Config once in a before() hook and use config.runCommand() / Help.showHelp() directly. Added getSharedFullConfig() helper to test-setup.ts for reuse. * perf(test): silence stdout noise from AM command tests stubCommandConfigAndLogger now automatically stubs command.log, command.logToStderr, and ux.stdout so non-JSON command output (tables, formatted details) doesn't leak to the console during tests. * perf(test): speed up SDK unit tests from ~8.5s to ~2.7s Several changes to eliminate unnecessary real delays in tests: 1. Telemetry tests: Use fake timers to skip the 300ms real sleep in telemetry.stop(). Added stopTelemetryFast() helper that installs sinon fake timers, advances clock, then restores. (9 calls × 300ms) 2. Logs integration tests: Reduce pollInterval (1000→100ms) and delays (1500→200ms, 10000→5000ms timeout) since the test just verifies stop/discover behavior, not sustained polling. 3. MRT env tests: Use fake timers with shouldAdvanceTime for timeout and polling tests. Reduce pollInterval from 100→10ms. 4. Logs tail tests: Reduce pollInterval (100→10ms) and stop/wait delays (200→50ms) since these use MSW mocks, not real network. 5. Code watch tests: Reduce filesystem watcher ready delays from 200-300ms to 50ms. 6. Job command test: Stub mockInstance.webdav.get to reject immediately instead of timing out on a real network request. 7. CLI integration tests: Replace runCommand (which calls Config.load per invocation) with a shared Config instance loaded once in before() hook. * fix(test): fix type errors in base-command integration test Cast extraParams to Record<string, unknown> before accessing .query/.body to satisfy tsc -p test stricter type checking used in CI. * fix(test): revert mrt/env fake timers to avoid OOM in CI The sinon.useFakeTimers({shouldAdvanceTime: true}) with pollInterval: 10 caused rapid-fire MSW HTTP requests (50+ polls in a tickAsync burst), accumulating memory that led to OOM on CI runners with limited heap. Reverted to real timers with pollInterval: 100 and this.timeout(5000), which is still faster than the original tests and doesn't risk OOM. * fix(test): restore runCommand for cip/report and scapi/schemas tests Revert the shared Config + config.runCommand() optimization for these tests. The change to showHelp() and config.runCommand() skipped the oclif init hook and full run() pipeline, which reduced function coverage below the 70% threshold. The runCommand approach calls Config.load() per invocation but oclif caches internally, so subsequent calls are ~40ms. The sandbox reset poll-interval fix (5s→0s) remains the primary speed win. Retains: sandbox reset poll-interval fix, stdout silencing in stubCommandConfigAndLogger. * fix(test): fix flaky MCP signal handler tests that hang on CI The signal handler tests called command.run() which is async and sets up stdinClosePromise, then used setTimeout-based polling to wait for handlers to register. This was non-deterministic: - run() is synchronous after the stubbed connect() resolves, so handlers are registered immediately — no polling needed - stdinClosePromise was left dangling with only runPromise.catch(() => {}) as cleanup, which doesn't await the flush() promise chain - On CI (slower timing), the polling races and unresolved promises caused mocha to hang until the 60s timeout, then exit with code 1 or 2 Fixed by: - Awaiting command.run() directly (handlers register synchronously) - Triggering signal handlers immediately after run() - Awaiting stdinClosePromise to ensure flush() completes - Removing all setTimeout-based polling and manual cleanup * perf: cache ts-morph Project in page-designer-decorator analyzer Reuse a single ts-morph Project instance across analyzeComponent calls instead of creating a new one each time. The Project uses an in-memory file system and is stateless — source files are added with overwrite:true and removed in a finally block after analysis. This eliminates repeated TypeScript compiler initialization (~40ms local, ~700ms CI per call × 57 test invocations). MCP test suite drops from ~2s to ~440ms locally, expected ~52s to ~10-15s on CI. * ci: move docs build to separate workflow triggered only on docs/ changes The docs build (vitepress + typedoc) was running on every CI run regardless of what changed. Move it to a dedicated workflow with a paths filter on docs/ so it only runs when documentation files change. This removes a blocking step from the main CI pipeline for the vast majority of PRs that don't touch docs.
1 parent 1a15687 commit 1f24b81

17 files changed

Lines changed: 313 additions & 247 deletions

File tree

.claude/skills/testing/SKILL.md

Lines changed: 25 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -324,11 +324,29 @@ const customAuth = new MockAuthStrategy('custom-token');
324324

325325
## Silencing Test Output
326326

327-
Commands may produce console output (tables, formatted displays) even in tests. Use these helpers to keep test output clean.
327+
Commands produce console output (tables, formatted displays) during tests. **All test helpers are designed to keep test output clean by default** — no extra work needed in most cases.
328328

329-
### Using runSilent for Output Capture
329+
### stubCommandConfigAndLogger Silences Output Automatically
330330

331-
The `runSilent` helper uses oclif's `captureOutput` to suppress stdout/stderr:
331+
The `stubCommandConfigAndLogger` helper (used by AM and sandbox command tests) automatically stubs `command.log`, `command.logToStderr`, and `ux.stdout` so no output leaks to the console:
332+
333+
```typescript
334+
import { stubCommandConfigAndLogger } from '../../../helpers/test-setup.js';
335+
336+
it('displays client details in non-JSON mode', async () => {
337+
const command = new ClientGet([], {} as any);
338+
stubCommandConfigAndLogger(command); // stdout is silenced automatically
339+
stubJsonEnabled(command, false);
340+
// ... setup ...
341+
342+
const result = await command.run(); // No console noise
343+
expect(result.id).to.equal('client-123');
344+
});
345+
```
346+
347+
### Using runSilent for Other Cases
348+
349+
For commands that don't use `stubCommandConfigAndLogger`, use `runSilent` to capture stdout/stderr:
332350

333351
```typescript
334352
import { runSilent } from '../../helpers/test-setup.js';
@@ -344,24 +362,22 @@ it('returns data in non-JSON mode', async () => {
344362
});
345363
```
346364

347-
Use `runSilent` when:
348-
- Testing non-JSON output modes (tables, formatted displays)
349-
- The test doesn't need to verify console output content
350-
- You want clean test output with only pass/fail summary
351-
352365
### When Output Verification is Needed
353366

354-
If you need to verify console output, stub `ux.stdout` directly:
367+
If you need to verify console output content, stub `ux.stdout` directly **before** calling `stubCommandConfigAndLogger` (which checks if the stub already exists):
355368

356369
```typescript
357370
import { ux } from '@oclif/core';
358371

359372
it('prints table in non-JSON mode', async () => {
360373
const stdoutStub = sinon.stub(ux, 'stdout');
374+
stubCommandConfigAndLogger(command); // won't double-stub ux.stdout
361375

362376
await command.run();
363377

364378
expect(stdoutStub.called).to.be.true;
379+
const text = stdoutStub.firstCall.args[0];
380+
expect(text).to.include('expected content');
365381
});
366382
```
367383

.github/workflows/ci.yml

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -60,9 +60,6 @@ jobs:
6060
- name: Build packages
6161
run: pnpm -r run build
6262

63-
- name: Build documentation
64-
run: pnpm run docs:build
65-
6663
- name: Run SDK tests
6764
id: sdk-test
6865
working-directory: packages/b2c-tooling-sdk

.github/workflows/docs.yml

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
name: Docs Build
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
paths:
8+
- 'docs/**'
9+
pull_request:
10+
branches:
11+
- main
12+
paths:
13+
- 'docs/**'
14+
15+
permissions:
16+
contents: read
17+
18+
jobs:
19+
build-docs:
20+
runs-on: ubuntu-latest
21+
steps:
22+
- name: Checkout code
23+
uses: actions/checkout@v4
24+
25+
- name: Setup Node.js
26+
uses: actions/setup-node@v4
27+
with:
28+
node-version: 22.x
29+
30+
- name: Setup pnpm
31+
uses: pnpm/action-setup@v4
32+
with:
33+
version: 10.17.1
34+
35+
- name: Get pnpm store directory
36+
shell: bash
37+
run: echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_ENV
38+
39+
- name: Setup pnpm cache
40+
uses: actions/cache@v4
41+
with:
42+
path: ${{ env.STORE_PATH }}
43+
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
44+
restore-keys: |
45+
${{ runner.os }}-pnpm-store-
46+
47+
- name: Install dependencies
48+
run: pnpm install --frozen-lockfile
49+
50+
- name: Build packages
51+
run: pnpm -r run build
52+
53+
- name: Build documentation
54+
run: pnpm run docs:build

AGENTS.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -161,6 +161,8 @@ This catches prettier formatting, import ordering, class member ordering (`perfe
161161

162162
See [testing skill](./.claude/skills/testing/SKILL.md) for patterns on writing tests with Mocha, Chai, and MSW.
163163

164+
**Stdout in tests**: Command tests must not leak output to the console. `stubCommandConfigAndLogger()` silences `command.log`, `command.logToStderr`, and `ux.stdout` automatically. For other cases, use `runSilent()` from `test/helpers/test-setup.ts`.
165+
164166
## Changesets
165167

166168
This project uses [Changesets](https://github.com/changesets/changesets) for version management with **independent per-package versioning**. Each package versions independently based on its own changesets.

packages/b2c-cli/test/commands/sandbox/reset.test.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@ describe('sandbox reset', () => {
9292

9393
it('waits for sandbox to reach started state when --wait is set', async () => {
9494
const command = await setupCommand(
95-
{force: true, wait: true, 'poll-interval': 5, timeout: 60, json: true},
95+
{force: true, wait: true, 'poll-interval': 0, timeout: 60, json: true},
9696
{
9797
sandboxId: 'zzzz-001',
9898
},

packages/b2c-cli/test/helpers/test-setup.ts

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
*/
66

77
import type {Config} from '@oclif/core';
8+
import {ux} from '@oclif/core';
89
import {captureOutput} from '@oclif/test';
910
import sinon from 'sinon';
1011
import {isolateConfig, restoreConfig} from '@salesforce/b2c-tooling-sdk/test-utils';
@@ -116,6 +117,14 @@ export function stubCommandConfigAndLogger(command: any, accountManagerHost = 'a
116117
},
117118
configurable: true,
118119
});
120+
121+
// Silence stdout: stub command.log/logToStderr and ux.stdout to prevent
122+
// test noise from non-JSON command output (tables, formatted details, etc.)
123+
command.log = () => {};
124+
command.logToStderr = () => {};
125+
if (!Object.hasOwn(ux.stdout, 'isSinonProxy')) {
126+
sinon.stub(ux, 'stdout');
127+
}
119128
}
120129

121130
/**

packages/b2c-dx-mcp/src/tools/storefrontnext/page-designer-decorator/analyzer.ts

Lines changed: 51 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,23 @@ import path from 'node:path';
99
import {globSync} from 'glob';
1010
import {Project, InterfaceDeclaration, PropertySignature} from 'ts-morph';
1111

12+
/**
13+
* Lazily-initialized, reusable ts-morph Project for component analysis.
14+
* Creating a new Project (~40ms) on every analyzeComponent call is expensive;
15+
* reusing one with an in-memory file system avoids repeated TypeScript compiler init.
16+
*/
17+
let cachedProject: Project | undefined;
18+
19+
function getProject(): Project {
20+
if (!cachedProject) {
21+
cachedProject = new Project({
22+
useInMemoryFileSystem: true,
23+
skipAddingFilesFromTsConfig: true,
24+
});
25+
}
26+
return cachedProject;
27+
}
28+
1229
// ============================================================================
1330
// TYPE DEFINITIONS
1431
// ============================================================================
@@ -352,48 +369,49 @@ function parseComponentFile(filePath: string): ComponentInfo {
352369
};
353370
}
354371

355-
const project = new Project({
356-
useInMemoryFileSystem: true,
357-
skipAddingFilesFromTsConfig: true,
358-
});
372+
const project = getProject();
373+
const sourceFile = project.createSourceFile(filePath, content, {overwrite: true});
374+
375+
try {
376+
const interfaces = sourceFile.getInterfaces();
377+
const propsInterface = interfaces.find((i: InterfaceDeclaration) => i.getName().includes('Props'));
378+
379+
if (!propsInterface) {
380+
return {
381+
componentName: extractComponentName(content),
382+
interfaceName: null,
383+
hasDecorators: false,
384+
props: [],
385+
exportType: detectExportType(content),
386+
filePath,
387+
};
388+
}
359389

360-
const sourceFile = project.createSourceFile(filePath, content);
361-
const interfaces = sourceFile.getInterfaces();
362-
const propsInterface = interfaces.find((i: InterfaceDeclaration) => i.getName().includes('Props'));
390+
const props: PropInfo[] = propsInterface.getProperties().map((prop: PropertySignature) => {
391+
const name = prop.getName();
392+
const type = prop.getType().getText();
393+
const optional = prop.hasQuestionToken();
394+
395+
return {
396+
name,
397+
type,
398+
optional,
399+
isComplex: isComplexType(type),
400+
isUIOnly: isUIOnlyProp(name),
401+
};
402+
});
363403

364-
if (!propsInterface) {
365404
return {
366405
componentName: extractComponentName(content),
367-
interfaceName: null,
406+
interfaceName: propsInterface.getName(),
368407
hasDecorators: false,
369-
props: [],
408+
props,
370409
exportType: detectExportType(content),
371410
filePath,
372411
};
412+
} finally {
413+
project.removeSourceFile(sourceFile);
373414
}
374-
375-
const props: PropInfo[] = propsInterface.getProperties().map((prop: PropertySignature) => {
376-
const name = prop.getName();
377-
const type = prop.getType().getText();
378-
const optional = prop.hasQuestionToken();
379-
380-
return {
381-
name,
382-
type,
383-
optional,
384-
isComplex: isComplexType(type),
385-
isUIOnly: isUIOnlyProp(name),
386-
};
387-
});
388-
389-
return {
390-
componentName: extractComponentName(content),
391-
interfaceName: propsInterface.getName(),
392-
hasDecorators: false,
393-
props,
394-
exportType: detectExportType(content),
395-
filePath,
396-
};
397415
}
398416

399417
// ============================================================================

0 commit comments

Comments
 (0)