Supply chain security specialist #5
Description
Generate a new agent person with the following plan (you may have to adapt the plan to fit this project).
CLI Dependency Vulnerability Audit Agent Plan
Step 1: Environment Setup and Installation
• Ensure required tools and libraries are available: Use a scripting language like Python (3.8+ recommended) or Node.js for portability. For Python, install necessary packages (e.g. for HTTP requests and parsing manifests):
pip install requests toml PyYAML
(This installs an HTTP client and parsers for JSON/TOML/YAML as needed.)
• Set up API credentials via environment variables: Obtain API tokens/keys for services to handle rate limits. For example:
• GITHUB_TOKEN for GitHub API (a Personal Access Token with minimal scopes) – allows higher rate limits and access to the Advisory API .
• NVD_API_KEY for NVD (get a free key from NIST) – increases rate limit from 5 to 50 requests per 30 seconds .
• SNYK_TOKEN for Snyk (if you have access) – to query Snyk’s database; if unavailable, the tool will skip Snyk or use public data.
• Install CLI as a tool: Package the script as an executable (e.g. a Python package with a console entry point or a Node.js script). For example, if using Python, include a shebang (#!/usr/bin/env python3) and make the script executable. This ensures it can be invoked directly in shell or CI.
Step 2: Input Collection and Validation
• Accept manifest inputs: Support two modes of operation:
1. Single manifest scan – e.g. vuln-audit --file requirements.txt to scan one dependency file for known vulnerabilities.
2. Diff of two manifests – e.g. vuln-audit --old package-lock.json --new package-lock.json to compare before/after versions (such as before and after a commit or upgrade).
• Multiple language support: Recognize manifest types by filename or user flag:
• npm: package-lock.json or yarn.lock (JSON),
• Python: requirements.txt (text), Poetry’s poetry.lock (toml), etc.,
• Rust: Cargo.lock (toml),
• Go: go.mod/go.sum (module list),
• Java/Maven: pom.xml (XML) or Gradle build.gradle (groovy) – these may require parsing or use of dependency plugin output.
• Others: Recognize composer.lock (PHP Composer, JSON), Gemfile.lock (Ruby, plain text), etc.
• Validate input files: Check that files exist and are parseable. Provide user-friendly error messages if, for example, the file is not found or is of an unsupported format. If two manifests are provided, ensure they correspond to the same package ecosystem (to avoid comparing npm vs pip by mistake). If ecosystems differ, warn the user and handle separately (or abort).
• Usage consistency: The CLI interface should be the same across ecosystems. For instance:
vuln-audit --old path/to/old.lock --new path/to/new.lock --output report.md
should function similarly whether the files are npm, pip, Cargo, etc. This makes the tool easily usable in different projects.
Step 3: Determine Added, Removed, and Updated Dependencies
• Parse dependency manifests: For each input manifest (old and new), parse the dependency names and versions into an internal data structure (e.g. a dictionary mapping {package -> version}). Use format-specific logic:
• For JSON-based lockfiles (npm package-lock.json, Yarn, Composer), use a JSON parser to extract dependencies and versions. (E.g. for npm, parse the "dependencies" object recursively for name -> version mappings.)
• For plaintext lists (Python requirements.txt, Ruby Gemfile.lock), read line by line. Identify lines specifying a package and pinned version (for requirements.txt, lines like name==1.2.3).
• For Toml/Yaml (Cargo.lock, Poetry lock): utilize a TOML/YAML parser (toml or PyYAML in Python) to load and extract package name/version entries. For Cargo, parse sections under [[package]] to get name and version.
• For Go modules (go.mod), execute go list -m all or parse lines under require ( in the file to list modules and versions (since go.mod is not a strict key-value format).
• For Maven pom.xml, use an XML parser or XPath to find : and entries of direct dependencies. (Alternatively, instruct users to provide a dependency:list output or utilize tools to generate a lockfile or effective POM).
• Compute differences: If two manifests are provided, compare the old vs new dependency sets:
• Added – packages present in the new manifest but not in the old.
• Removed – packages in the old manifest but missing in the new.
• Updated – packages present in both, but with different version numbers. Determine if it’s an upgrade or downgrade by comparing versions (to highlight if a version was lowered, which might reintroduce old vulnerabilities).
Use set operations on package names and then compare versions for intersection. For example:
added_pkgs = new_deps.keys() - old_deps.keys()
removed_pkgs = old_deps.keys() - new_deps.keys()
common_pkgs = new_deps.keys() & old_deps.keys()
updated_pkgs = {pkg for pkg in common_pkgs if new_deps[pkg] != old_deps[pkg]}
• Handle transitive dependencies: By default, focus on direct dependencies listed in the manifest. (Lockfiles often include transitive dependencies as well; those changes will be picked up if listed. If not, an optional enhancement is to consider transitive updates by reading lockfile fully. For initial scope, scanning what’s explicitly in the lockfile is sufficient.)
• Output summary table (draft): Prepare a data structure for the summary of changes. Each entry: Package Name, Old Version (or “-” if added), New Version (or “-” if removed), and Change Type (“Added”, “Removed”, “Updated”). This will later be formatted into a Markdown table in the final report.
Step 4: Query Vulnerability Databases for Changes
For each dependency that was added or updated (since removed packages no longer affect the project), query public vulnerability databases to find any known security issues in the relevant versions. The agent will aggregate results from these sources: OSV, GitHub Advisory Database, NVD, and Snyk (if available). To avoid duplicate findings, use unique identifiers (like CVE or GHSA IDs) to merge results across sources. Ensure to respect API rate limits and use caching where possible (e.g., if multiple packages share a vulnerability or if multiple vulnerabilities share a CVE, avoid redundant calls).
• OSV.dev (Open Source Vulnerabilities): Use OSV’s public API as a primary source for open-source package vulnerabilities. OSV is an aggregator that includes data from many language ecosystems and sources (e.g. PyPI, npm, RustSec, GitHub Advisories) . Query OSV for each package version: the API allows querying by package name, ecosystem, and version to retrieve all matching vulnerabilities . For example:
curl -s -H "Content-Type: application/json" -d '{"package": {"name": "jinja2", "ecosystem": "PyPI"}, "version": "2.4.1"}'
https://api.osv.dev/v1/query
This returns any vulnerabilities affecting jinja2@2.4.1 (in JSON). The tool should parse the JSON response: for each vulnerability, note its ID (which may be a CVE, GHSA, or OSV ID), summary, and affected version range. OSV results often include a list of aliases (CVE IDs, etc.) and the versions in which the issue was introduced and fixed (if available). Use the affected data to confirm if the new version is vulnerable or if a fix is available in a later version  . To improve efficiency, use the batch query API (/v1/querybatch) to send multiple package queries in one request if a large number of dependencies changed (ensuring the JSON payload size remains reasonable).
• GitHub Advisory Database: Cross-reference OSV findings with GitHub’s Security Advisory Database for additional details like severity and CVSS. OSV entries that originate from GitHub will have a GHSA ID (e.g. “GHSA-xxxx”) as the vulnerability ID . If a GHSA ID is present (or a CVE with no GHSA), query GitHub’s advisory API:
• REST API: GitHub provides a REST endpoint to fetch a specific advisory by GHSA ID. For example:
curl -H "Accept: application/vnd.github+json" -H "Authorization: Bearer $GITHUB_TOKEN"
"https://api.github.com/advisories/GHSA-xxxx-xxxx-xxxx"
This returns a JSON with details of the advisory (summary, description, severity, published date, etc.) and a list of affected packages/versions  . Notably, it includes the advisory’s severity rating (GitHub’s classification) and CVSS score/vector if available  , as well as the vulnerabilities array which lists affected package name, ecosystem, vulnerable range, and first_patched_version (the version that contains a fix) . The agent should extract: severity level (e.g. low/medium/high/critical), CVSS score, and patched version from this data.
• GraphQL API (optional): For bulk operations, the GraphQL API can filter advisories by ecosystem or package. For example, a GraphQL query can list advisories for a given ecosystem and then filter client-side for the package name . However, since OSV already provides a direct package lookup, using the REST endpoint by specific GHSA or CVE is simpler for targeted queries.
• If no GHSA ID is known but a CVE is present, use the REST search: GET /advisories?cve_id=CVE-XXXX-YYYY&ecosystem=<ecosystem> which returns the advisory if it exists  . This can be done without auth (for public advisories) but using a token is recommended to avoid low rate limits.
• Rate limiting: The GitHub Advisory API allows a high number of requests when authenticated (GitHub’s standard rate limits for API apply). Use the provided token and consider bulk queries for efficiency. If many dependencies changed, avoid requesting the same advisory multiple times (cache GHSA IDs already fetched).
• NVD (National Vulnerability Database): For each CVE identifier obtained (from OSV or GH Advisory), query NVD’s CVE API for official data and CVSS scores. NVD’s API endpoint for a specific CVE ID is:
curl -H "Accept: application/json" "https://services.nvd.nist.gov/rest/json/cves/2.0?cveId=CVE-YYYY-NNNNN"
This returns the NVD record for that CVE in JSON , including detailed descriptions, references, and CVSS scores/metrics (v2 and v3, if available). The agent should extract the CVSS v3 base score and vector (NVD’s scoring) and any additional fields like Exploitability Subscore or Severity. NVD tags each CVE with a severity category (Low/Medium/High/Critical) based on CVSS v3 score. If the GitHub advisory lacked a CVSS score, use NVD’s. Also, NVD may have updated information on whether the vulnerability is being actively exploited or if it’s part of any known threat campaigns (sometimes mentioned in the description or references).
• Batching and caching: If multiple vulnerabilities share the same CVE (common in multiple packages if they depend on the same underlying library), query NVD once and reuse the data. NVD allows batching via filtering, but since we have specific IDs, it’s simplest to query each needed CVE.
• Rate limiting: NVD’s public API without a key permits only 5 requests per 30 seconds . Ensure to pause (sleep ~6 seconds between calls) or use the API key for up to 50 requests/30s . The agent should implement a short delay loop or queue if a lot of CVEs must be fetched sequentially, to stay within limits.
• Snyk Vulnerability Database (optional if available): Snyk’s database is a curated source that often includes vulnerabilities sooner or more comprehensively than others . While Snyk’s open API requires authentication and is typically scoped to projects, their advisory information can be accessed via their public website or API if credentials are provided. If a SNYK_TOKEN is configured (and possibly an Org ID), the agent can query Snyk’s API for issues by package using the Package URL (purl) format  . For example:
curl -H "Authorization: Bearer $SNYK_TOKEN" "https://api.snyk.io/rest/orgs/$ORG_ID/packages/pkg:npm/express@4.17.1/issues"
(This endpoint returns vulnerability issues for the specific package version in that org’s context  .)
If direct API use is not viable, the tool can fall back to scraping Snyk’s public vulnerability feed: Snyk’s website (https://security.snyk.io) allows searching by package name. The agent could fetch the HTML page for the package and parse known vulnerability entries. This is less structured, so prefer the API if possible.
• Data from Snyk: For each vulnerability, Snyk provides its own severity rating (which often aligns with CVSS categories), a description, and whether a fix is available (and in which version). Incorporate any Snyk-unique findings that weren’t already covered by OSV/GH/NVD. Snyk may identify vulnerabilities that have no CVE yet or are not in NVD , so this can catch newer issues.
• Rate limiting: The Snyk API has rate limits (varies by plan) – if using it, implement a delay or limit concurrent requests. If scraping the website, respect polite crawling (e.g. one request with a short pause, as the number of packages to check is small in a diff scenario).
• Consolidate results: Normalize all findings by linking them via common identifiers. For example, if OSV reports a vulnerability with alias CVE-2023-12345 and GHSA-abc, and NVD provides CVE-2023-12345 details, merge them so that the final report entry for CVE-2023-12345 includes data from all sources (OSV/GH description, NVD CVSS, Snyk additional info, etc.). Each changed dependency may have multiple vulnerabilities; preserve each as a separate entry. If a dependency is added new, consider that all its known vulnerabilities apply (since previously it wasn’t in the project) – highlight those as newly introduced risk. If a dependency is updated, focus on vulnerabilities in the new version (and possibly compare to old version’s vulnerabilities to see if the update fixed some issues but maybe introduced others – an advanced analysis could mention if the update fixed certain CVEs and introduced new ones).
Step 5: Prioritize Vulnerabilities (Scoring & Sorting)
Not all vulnerabilities are equal – the agent should prioritize them so the report and recommendations focus on the most critical issues first. Use multiple criteria for scoring: CVSS severity, exploit maturity, and availability of fixes.
• CVSS Severity: Use the CVSS v3 base score as a primary metric (0-10). If available, also note the severity category (e.g. Critical (9.0-10), High (7.0-8.9), Medium, Low). Rank vulnerabilities primarily by this score/category . For example, a Critical vulnerability (CVSS 9.8) should be ranked above a High (CVSS 7.5). If CVSS v3 is missing but v2 exists, map v2 to approximate severity or use the given severity label from GH/Snyk (which often have their own scoring).
• Exploit Maturity: Assess how likely or how far along exploitation is for each vulnerability. This can be estimated by:
• Checking if the vulnerability is on any known exploited list (for instance, CISA Known Exploited Vulnerabilities catalog) – if so, mark it as actively exploited (high priority).
• Using the presence of a public exploit or exploit code: if the CVE has references to exploit code (Metasploit modules, PoC links, etc. often found in NVD references), or if Snyk/GH advisory notes an exploit, increase priority.
• Optionally, incorporate EPSS (Exploit Prediction Scoring System) if accessible – EPSS provides a probability of exploitation. A high EPSS score would elevate priority. (EPSS data can be fetched by CVE from FIRST’s API or a cached dataset if the agent includes it.)
• If none of the above data is available, use vulnerability age and severity as a proxy: e.g. a critical bug from 5 years ago with many references might have exploit code in the wild, whereas a very recent issue might not yet.
• Patch Availability: Prefer to prioritize issues that have fixes available, because they can be resolved by upgrading, especially if severe. Paradoxically, one might also consider un-patchable issues (no fix yet) as high priority if severe, since they pose lingering risk – but the actionable item might be limited (e.g. find workarounds or remove the package). The agent should:
• Check if a fixed version is known for the vulnerability. OSV and GH advisories explicitly list the patched version or range . If first_patched_version or a similar field is present, that means upgrading to that version or later resolves the issue.
• If a fix is available, note the version. If no fix is available (no patched version and status is still open), flag that – the project maintainer might need to monitor or mitigate differently.
• Give slightly higher priority to vulnerabilities with a fix, because the team can act on them immediately (especially if severity is high). For vulnerabilities with no fix, still highlight them (they might consider replacing that dependency or applying a temporary patch).
• Combine into a priority score or list: The agent can assign a numeric score or ranking for each vulnerability change. For simplicity, one strategy is to sort by severity (Critical > High > Medium > Low), then within same severity, sort by whether an exploit is known (exploited in wild first), then by whether a fix is available (issues with fixes slightly higher as they are actionable). Alternatively, create a weighted score: e.g. Critical=5, High=4, Medium=3, Low=2, None=1; +1 if exploited, +1 if no fix (or minus 1 if no fix to denote more concern?), etc. The exact formula can be tuned, but the output should be an ordered list of vulnerabilities from most urgent to least.
• Prioritize at package level for actions: It may also help to aggregate by package: e.g. if one library has 3 high-severity issues and another has 1 medium, the library with multiple high issues might be a bigger risk. The action list (next step) will consider package-level fixes, so use the highest vulnerability in each package as representative for sorting the package in the action plan.
Step 6: Generate Markdown Report
Finally, compile all the collected information into a clear Markdown report. The report should be comprehensive yet easy to scan, using tables and lists as appropriate. The structure can be as follows:
1. Introduction (optional): A brief paragraph stating the scan context, e.g.: “The following report summarizes dependency changes between manifest A and manifest B, and known vulnerabilities associated with these changes.” Mention the date/time of scan and the tool name/version.
2. Summary Table of Dependency Changes:
Present a table of added, removed, and updated packages. For example:
Package Old Version New Version Change
express 4.16.1 4.17.1 Updated
lodash (none) 4.17.21 Added
left-pad 1.3.0 (none) Removed
Use bold or emphasis to highlight added packages (since new dependencies might introduce new risks) or critical changes. This table gives the reader a quick overview  . If there are many changes, consider grouping by change type or sorting alphabetically.
3. Vulnerability Details for Changed Packages:
For each changed dependency that has vulnerabilities, provide a detailed breakdown. This can be organized by package, each as a sub-section, e.g. “### Vulnerabilities in express (4.17.1):”. Under each:
• List each relevant vulnerability as a sub-bullet or table row, including:
• Identifier (CVE ID or GHSA ID) with a link to its database entry (NVD or OSV URL). For example: CVE-2022-12345 – Buffer Overflow in express-body-parser. If no CVE, use GHSA or Snyk ID.
• Severity & Score: e.g. “High (CVSS 7.6)” – include the severity label and score . If multiple sources differ, you might show NVD’s score vs GH’s (but usually they align; if not, note both).
• Description: A one-line summary of the vulnerability  – e.g. “Buffer overflow in X module allows remote code execution.” Keep it concise but clear. You can obtain this from NVD or GH advisory text.
• Affected versions: what versions of the package are vulnerable (e.g. “Affected: <=4.17.0”). And Fixed in: if known, specify the first patched version . For example, “Fixed in 4.17.1” (if the new version is 4.17.1 and it’s the fixed version, that implies the update actually resolved the issue rather than introduced it – highlight that as a positive if applicable).
• References: Provide links for further reading, such as the GitHub advisory page or NVD entry, or a Snyk page, or a blog post if relevant. For Markdown, you can include them as footnote-style links or inline links. For instance: “GH Advisory” or “NVD Entry”.
• If the package was added new, you might phrase it as “Newly introduced package X has the following known vulnerabilities:” to draw attention that these came with the new addition. If updated, “Package X was updated from v1 to v2; the new version has the following vulnerabilities (and mention if some from the old version were resolved, if that analysis was done).”
• Use bullet points for each vulnerability or a table if you prefer consistent columns (ID, severity, fix version, etc.). However, long text (descriptions) might be cleaner in bullets. Ensure each vulnerability entry is a few sentences at most, to maintain readability.
4. Prioritized Action List for the Developer:
This is the most actionable section – a list of recommended steps the team should take, ordered by priority (highest risk first). Use a numbered list for clear sequence:
1. Upgrade express to 4.17.2 or later. Reason: Version 4.17.1 is vulnerable to CVE-2022-12345 (High severity RCE) – a fix is available in 4.17.2 . Upgrading will resolve this issue.
2. Replace/Remove left-pad (if applicable). Reason: left-pad@1.3.0 was removed but if it were still present, for example, “No fix available for its vulnerability CVE-2021-0001 – consider removing this dependency or using an alternative.”
3. Monitor lodash (if newly added). Reason: “lodash@4.17.21 was added. It has known Moderate severity issues but no immediate fix needed; keep it updated.” (If there are high severity issues, it would be earlier in the list with an upgrade recommendation. If none, then no action needed other than awareness.)
• Each action item should reference the issue it addresses (CVE or package). Focus on remediation: e.g. “Update to version X”, “Apply patch Y (link)”, “Configure a workaround (if no fix, perhaps disabling a feature)”, or “Remove dependency if possible”. If an exploit is active, you might say “Update immediately – this vulnerability is being actively exploited.”
• Clearly state if the action will fully resolve the vulnerability or just mitigate. For example, “Upgrading will eliminate CVE-2022-12345. After upgrade, rerun this tool to ensure no new issues.”
• If multiple vulnerabilities can be fixed by a single action (often upgrading a package addresses all its known CVEs), one bullet suffices for that package grouping those CVEs by reference. Conversely, if a package has one fixable and one unfixable issue, note both: “Upgrade to X to fix A; note B has no fix – consider temporary mitigation.”
• The list should be in order of urgency: typically, critical vulns first. This ordering is informed by the scoring in Step 5 (e.g. all Critical issues sorted, then High, etc.). Make sure the numbering reflects priority (1 = highest priority).
5. Footer/Additional Notes:
Optionally, include any notes on methodology, such as: “Data sources: OSV, GitHub Advisory Database, NVD, Snyk. CVSS scores are from NVD or GitHub advisories. Exploit status based on public reports up to date of scan.” You can also mention the date of the vulnerability database data (if the tool has a cached database timestamp). Encourage the user to address issues promptly and consider running the tool regularly or integrating it into CI for continuous monitoring.
All sections should use proper Markdown formatting for readability (tables for structured data, bullet points for lists, bold to highlight package names or important points, backticks for file names or versions). The report is intended to be easy to read in a code review or CI context, so clarity is key.
Also, include the preserved citations (if this were an interactive report with sources) for traceability of information. For example, when referencing that “OSV is an aggregator of multiple databases including GitHub”, a citation was given , and similarly for other statements in this plan to show their source. These would not typically appear in the end-user report, but they ensure that every fact (e.g. rate limits, API usage) was backed by documentation during development of this plan.
Step 7: Extensibility and CI Integration
• Modular design for multiple ecosystems: Organize the code so that support for a new package manager can be added by writing a new parser module for its manifest and possibly adjusting how package naming is normalized for queries. For example, the agent could use a common Package URL (purl) format internally (which encodes ecosystem, name, version) to query APIs consistently. New ecosystems (Gradle, .NET, etc.) can be integrated by mapping their manifest data to this format. The vulnerability query logic mostly remains the same since OSV and others use ecosystem identifiers (like “Maven”, “NuGet”, etc.) . Ensure the ecosystem strings match what OSV/GH expect (e.g. use “PyPI” for Python packages, “npm” for Node, “Maven” for Java, “Go” for Golang, etc., as per OSV schema). This makes extending to new languages straightforward by adding the correct mapping.
• Extendable vulnerability sources: The agent is designed to pull from multiple sources. It can easily be extended to others if needed – e.g. adding Debian Security Tracker or OSS Index – by writing a new query function and merging results. The scoring and output steps are data-driven, so additional sources will automatically factor in (especially if they provide unique IDs like CVEs). The system already normalizes on CVE/GHSA, which most databases use, so combining is natural.
• Command-line interface consistency: Keep the CLI flags and output format consistent even as new features are added. Use subcommands or flags for new functionality (for example, a --format json flag could output machine-readable JSON instead of Markdown, if embedding in other tools, similar to OSV-Scanner’s format options  ). But by default, markdown is output to stdout for easy capture.
• Integration with Continuous Integration (CI): The tool should exit with a proper status code to facilitate CI outcomes. For instance, you might add an option like --fail-on-critical that exits with non-zero if any critical (or high) vulnerabilities are found, thus failing the build. Without such an option, it could always exit 0 to just produce a report and not break the build. This behavior should be configurable.
• In a CI pipeline (GitHub Actions, GitLab CI, Jenkins, etc.), you can use the CLI by installing it in the pipeline runner (e.g. pip install vuln-audit-tool) then running:
vuln-audit --old requirements.txt --new requirements.txt > vuln_report.md
(In this example, scanning a single file against itself would just list all current vulnerabilities in requirements.txt – useful for nightly scans).
• After generation, the Markdown report can be uploaded as a CI artifact or even used to comment on a pull request. For example, in GitHub Actions, one could run this tool on a Pull Request that updates dependencies, then use the resulting vuln_report.md to post a PR comment for developers to review. The report is already in Markdown, which renders nicely on these platforms.
• Rate limiting and performance in CI: If running in CI frequently, consider caching results for unchanged dependencies to speed up scans (e.g. maintain a local cache of “package@version -> vulnerabilities” for quick lookup, updating it when new versions are seen or a certain time has passed). Ensure to honor API usage policies – for instance, if hundreds of projects run this in parallel, you might hit OSV or NVD limits; using an API key and caching is important in that scenario.
• Self-update and data freshness: Make it easy to update the tool to get improvements or new data sources. Also, perhaps include a way to update local vulnerability databases if any offline mode is used (like allowing a user to supply an offline cache of NVD or OSV data, in case the tool is used in an environment without internet – though in CI it usually has internet). This makes it robust in various settings (corporate offline CI, etc.).
• Testing and validation: Include unit tests for parsing each supported manifest type and for the logic that computes diffs and risk rankings. Also test the CLI on a sample project in CI to ensure it integrates well (for example, a dry-run on a demo repository). This ensures future extensibility changes do not break existing functionality.
By following this plan, the resulting CLI agent will work across multiple package ecosystems, leverage the major vulnerability databases (OSV, GitHub, NVD, Snyk) for comprehensive coverage, and produce an actionable Markdown report. The consistent CLI interface and careful handling of environment setup, rate limits, and output formatting make it suitable to use both as a one-off audit tool and as an automated check within continuous integration pipelines. The developer using this tool will quickly understand what changed, what known security issues exist in those changes, and what to do about them – all in one consolidated report.