feat: add HTTP request/response tracing support#9125
Merged
knqyf263 merged 16 commits intoaquasecurity:mainfrom Jul 10, 2025
Merged
feat: add HTTP request/response tracing support#9125knqyf263 merged 16 commits intoaquasecurity:mainfrom
knqyf263 merged 16 commits intoaquasecurity:mainfrom
Conversation
- Add --trace-http flag for HTTP tracing with security warnings - Rename --trace to --trace-rego (keeping --trace as deprecated alias) - Implement comprehensive HTTP trace transport with sensitive data redaction - Add support for OCI and Docker manifest content types - Include CI environment protection for --trace-http flag - Add comprehensive test coverage for trace functionality
- Add WithWriter functional option to NewTraceTransport for configurable output - Implement comprehensive color support for HTTP methods, URLs, headers, and status codes - Add complete test coverage with exact output matching - Improve test infrastructure with RequestRecorder using functional options pattern - Normalize CRLF to LF in tests for cross-platform compatibility - Set consistent User-Agent in tests to avoid Go version dependency - Replace partial matching tests with complete output verification
- Add --trace-http flag documentation with security warning - Update --trace-rego flag documentation (renamed from --trace) - Regenerate CLI reference documentation across all commands
- Update debugging documentation to use --trace-rego instead of deprecated --trace - Reflect the flag rename in misconfiguration debugging examples
- Add HTTP request/response tracing section to troubleshooting guide - Include security warning about potential sensitive data exposure - Provide usage examples for debugging network issues - Clarify that Trivy attempts to redact sensitive information but exposure may still occur
The order of transport wrappers was incorrect, causing User-Agent headers to be set after the trace transport had already logged the request. Fixed by applying trace transport first, then user agent transport.
The --trace-http flag value can be included in telemetry data as it only indicates whether HTTP tracing is enabled or not.
# Conflicts: # docs/docs/advanced/telemetry-flags.md # pkg/flag/rego_flags.go
Update CLI documentation files to reflect recent flag changes.
DmitriyLewen
reviewed
Jul 3, 2025
Contributor
DmitriyLewen
left a comment
There was a problem hiding this comment.
LGTM
left 1 suggestion.
pkg/x/http/trace.go
Outdated
| }) | ||
|
|
||
| // Handle private keys | ||
| privateKeyPattern := regexp.MustCompile(`-----BEGIN (RSA |EC |DSA |OPENSSH )?PRIVATE KEY-----[\s\S]*?-----END (RSA |EC |DSA |OPENSSH )?PRIVATE KEY-----`) |
Contributor
There was a problem hiding this comment.
We can use secret scanner here (we can add this (if required) in another PR).
wdyt?
nikpivkin
reviewed
Jul 3, 2025
Replace example.com with example.invalid to use an invalid TLD as specified in RFC 6761. This makes it clearer that no external network communication should occur during tests.
Integrate Trivy's built-in secret scanner to improve secret detection and redaction capabilities in HTTP request/response tracing. Key changes: - Add secret scanner to traceTransport struct - Enhance redactBody function to use scanner findings - Replace asterisk patterns with colored redacted text - Add comprehensive test for private key detection - Optimize regex compilation for better performance This enhancement provides more accurate secret detection beyond basic pattern matching, leveraging Trivy's advanced secret scanning rules for better security in HTTP trace output.
Mark the --trace-http flag as internal to hide it from general users while keeping it available for maintainer debugging purposes. Changes: - Add Internal: true to TraceHTTPFlag to hide from help output - Remove CI environment check as flag is now hidden - Remove unused imports (strconv, xerrors) - Update CLI documentation to remove trace-http flag references This ensures the dangerous flag is only accessible to maintainers who explicitly know about it, reducing security risks from accidental usage by end users.
simar7
reviewed
Jul 10, 2025
simar7
reviewed
Jul 10, 2025
nikpivkin
approved these changes
Jul 10, 2025
Co-authored-by: DmitriyLewen <91113035+DmitriyLewen@users.noreply.github.com>
alexlebens
pushed a commit
to alexlebens/infrastructure
that referenced
this pull request
Jul 31, 2025
This PR contains the following updates: | Package | Update | Change | |---|---|---| | [mirror.gcr.io/aquasec/trivy](https://www.aquasec.com/products/trivy/) ([source](https://github.com/aquasecurity/trivy)) | minor | `0.64.1` -> `0.65.0` | --- ### Release Notes <details> <summary>aquasecurity/trivy (mirror.gcr.io/aquasec/trivy)</summary> ### [`v0.65.0`](https://github.com/aquasecurity/trivy/blob/HEAD/CHANGELOG.md#0650-2025-07-30) [Compare Source](aquasecurity/trivy@v0.64.1...v0.65.0) ##### Features - add graceful shutdown with signal handling ([#​9242](aquasecurity/trivy#9242)) ([2c05882](aquasecurity/trivy@2c05882)) - add HTTP request/response tracing support ([#​9125](aquasecurity/trivy#9125)) ([aa5b32a](aquasecurity/trivy@aa5b32a)) - **alma:** add AlmaLinux 10 support ([#​9207](aquasecurity/trivy#9207)) ([861d51e](aquasecurity/trivy@861d51e)) - **flag:** add schema validation for `--server` flag ([#​9270](aquasecurity/trivy#9270)) ([ed4640e](aquasecurity/trivy@ed4640e)) - **image:** add Docker context resolution ([#​9166](aquasecurity/trivy#9166)) ([99cd4e7](aquasecurity/trivy@99cd4e7)) - **license:** observe pkg types option in license scanner ([#​9091](aquasecurity/trivy#9091)) ([d44af8c](aquasecurity/trivy@d44af8c)) - **misconf:** add private ip google access attribute to subnetwork ([#​9199](aquasecurity/trivy#9199)) ([263845c](aquasecurity/trivy@263845c)) - **misconf:** added logging and versioning to the gcp storage bucket ([#​9226](aquasecurity/trivy#9226)) ([110f80e](aquasecurity/trivy@110f80e)) - **repo:** add git repository metadata to reports ([#​9252](aquasecurity/trivy#9252)) ([f4b2cf1](aquasecurity/trivy@f4b2cf1)) - **report:** add CVSS vectors in sarif report ([#​9157](aquasecurity/trivy#9157)) ([60723e6](aquasecurity/trivy@60723e6)) - **sbom:** add SHA-512 hash support for CycloneDX SBOM ([#​9126](aquasecurity/trivy#9126)) ([12d6706](aquasecurity/trivy@12d6706)) ##### Bug Fixes - **alma:** parse epochs from rpmqa file ([#​9101](aquasecurity/trivy#9101)) ([82db2fc](aquasecurity/trivy@82db2fc)) - also check `filepath` when removing duplicate packages ([#​9142](aquasecurity/trivy#9142)) ([4d10a81](aquasecurity/trivy@4d10a81)) - **aws:** update amazon linux 2 EOL date ([#​9176](aquasecurity/trivy#9176)) ([0ecfed6](aquasecurity/trivy@0ecfed6)) - **cli:** Add more non-sensitive flags to telemetry ([#​9110](aquasecurity/trivy#9110)) ([7041a39](aquasecurity/trivy@7041a39)) - **cli:** ensure correct command is picked by telemetry ([#​9260](aquasecurity/trivy#9260)) ([b4ad00f](aquasecurity/trivy@b4ad00f)) - **cli:** panic: attempt to get os.Args\[1] when len(os.Args) < 2 ([#​9206](aquasecurity/trivy#9206)) ([adfa879](aquasecurity/trivy@adfa879)) - **license:** add missed `GFDL-NIV-1.1` and `GFDL-NIV-1.2` into Trivy mapping ([#​9116](aquasecurity/trivy#9116)) ([a692f29](aquasecurity/trivy@a692f29)) - **license:** handle WITH operator for `LaxSplitLicenses` ([#​9232](aquasecurity/trivy#9232)) ([b4193d0](aquasecurity/trivy@b4193d0)) - migrate from `*.list` to `*.md5sums` files for `dpkg` ([#​9131](aquasecurity/trivy#9131)) ([f224de3](aquasecurity/trivy@f224de3)) - **misconf:** correctly adapt azure storage account ([#​9138](aquasecurity/trivy#9138)) ([51aa022](aquasecurity/trivy@51aa022)) - **misconf:** correctly parse empty port ranges in google\_compute\_firewall ([#​9237](aquasecurity/trivy#9237)) ([77bab7b](aquasecurity/trivy@77bab7b)) - **misconf:** fix log bucket in schema ([#​9235](aquasecurity/trivy#9235)) ([7ebc129](aquasecurity/trivy@7ebc129)) - **misconf:** skip rewriting expr if attr is nil ([#​9113](aquasecurity/trivy#9113)) ([42ccd3d](aquasecurity/trivy@42ccd3d)) - **nodejs:** don't use prerelease logic for compare npm constraints ([#​9208](aquasecurity/trivy#9208)) ([fe96436](aquasecurity/trivy@fe96436)) - prevent graceful shutdown message on normal exit ([#​9244](aquasecurity/trivy#9244)) ([6095984](aquasecurity/trivy@6095984)) - **rootio:** check full version to detect `root.io` packages ([#​9117](aquasecurity/trivy#9117)) ([c2ddd44](aquasecurity/trivy@c2ddd44)) - **rootio:** fix severity selection ([#​9181](aquasecurity/trivy#9181)) ([6fafbeb](aquasecurity/trivy@6fafbeb)) - **sbom:** merge in-graph and out-of-graph OS packages in scan results ([#​9194](aquasecurity/trivy#9194)) ([aa944cc](aquasecurity/trivy@aa944cc)) - **sbom:** use correct field for licenses in CycloneDX reports ([#​9057](aquasecurity/trivy#9057)) ([143da88](aquasecurity/trivy@143da88)) - **secret:** add UTF-8 validation in secret scanner to prevent protobuf marshalling errors ([#​9253](aquasecurity/trivy#9253)) ([54832a7](aquasecurity/trivy@54832a7)) - **secret:** fix line numbers for multiple-line secrets ([#​9104](aquasecurity/trivy#9104)) ([e579746](aquasecurity/trivy@e579746)) - **server:** add HTTP transport setup to server mode ([#​9217](aquasecurity/trivy#9217)) ([1163b04](aquasecurity/trivy@1163b04)) - supporting .egg-info/METADATA in python.Packaging analyzer ([#​9151](aquasecurity/trivy#9151)) ([e306e2d](aquasecurity/trivy@e306e2d)) - **terraform:** `for_each` on a map returns a resource for every key ([#​9156](aquasecurity/trivy#9156)) ([153318f](aquasecurity/trivy@153318f)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MS4xLjMiLCJ1cGRhdGVkSW5WZXIiOiI0MS4xLjMiLCJ0YXJnZXRCcmFuY2giOiJtYWluIiwibGFiZWxzIjpbImltYWdlIl19--> Reviewed-on: https://gitea.alexlebens.dev/alexlebens/infrastructure/pulls/1073 Co-authored-by: Renovate Bot <renovate-bot@alexlebens.net> Co-committed-by: Renovate Bot <renovate-bot@alexlebens.net>
yutatokoi
referenced
this pull request
in yutatokoi/trivy
Aug 12, 2025
Co-authored-by: DmitriyLewen <91113035+DmitriyLewen@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds a new
--trace-httpflag to enable HTTP request/response tracing for debugging purposes. It also renames the existing--traceflag to--trace-regofor clarity.Key features:
--traceto--trace-rego(with deprecated alias for backward compatibility)--trace-httpflag for HTTP request/response tracingSecurity Features
The
--trace-httpflag includes robust security measures:Usage information
Security Note: The
--trace-httpflag is intended for maintainer debugging only. It is hidden from normal help output (Internal: true) and should be used with caution as it may expose sensitive information in trace output despite redaction efforts.Before and After
Before
When debugging network issues or authentication problems, users had no visibility into HTTP requests/responses:
After
With
--trace-http, maintainers can see detailed HTTP traffic with sensitive data properly redacted:Secret Redaction Example
When processing requests with sensitive data, secrets are automatically detected and redacted:
Also, the existing
--traceflag has been renamed to--trace-regofor clarity:Implementation Details
Internal: truehides the flag from help outputRelated issues
Checklist