Skip to content

Latest commit

 

History

History
620 lines (443 loc) · 20.5 KB

File metadata and controls

620 lines (443 loc) · 20.5 KB

hq — HCL Query CLI

hq is a jq-like query tool for HCL2 files. It ships with python-hcl2 and supports structural queries, hybrid Python expressions, and full eval mode.

Mode preference (use the first one that works):

  1. Structural (default) — jq-like syntax with pipes, select, string functions, object construction; recommended.
  2. Hybrid (::) — structural path on the left, Python expression on the right. Only when structural can't express the transform.
  3. Eval (-e) — full Python expressions. Last resort — many operations are blocked for safety.

Structural Queries

hq queries HCL2 files using dot-separated paths. Segments match block types, then name labels, then body contents.

# Get all variable blocks
hq 'variable[*]' variables.tf

# Navigate into a specific resource
hq 'resource.aws_instance.main.ami' main.tf

# Output as JSON
hq 'variable[*]' variables.tf --json

# Output raw values only
hq 'resource.aws_instance.main.ami' main.tf --value

# Wildcard: all top-level blocks/attributes
hq '*' main.tf

# Index: first variable only
hq 'variable[0]' variables.tf

Path Grammar

path    := segment ("." segment)*
segment := (type_filter ":")? name "~"? ("[*]" | "[" INT "]" | "[select(PRED)]")?
name    := "*" | IDENTIFIER
type_filter := IDENTIFIER
  • name matches block types and attribute names
  • type:name matches only nodes of the given type (e.g. function_call:length)
  • name~ skips all block labels, going straight to the body (see below)
  • [*] selects all matches at that level
  • [N] selects the Nth match (zero-based)
  • [select(PRED)] filters matches using a predicate (see below)

Resolution Rules

  1. On a DocumentView/BodyView: segment matches block types and attribute names
  2. On a BlockView with unconsumed name labels: segment matches the next label
  3. On a BlockView with all labels consumed: delegates to body
  4. On an AttributeView: unwraps to the value expression
  5. On an ObjectView: segment matches keys
  6. On a TupleView: [N] or [*] selects elements

Skip Labels (~)

HCL blocks have labels (e.g. resource "aws_instance" "main"). Normally you consume them one segment at a time: resource.aws_instance.main. The ~ suffix explicitly skips all remaining labels and goes straight to the block body:

# Without ~: must name every label or use wildcard
hq 'resource.aws_instance.main.ami' main.tf

# With ~: skip all labels, access body directly
hq 'resource~.ami' main.tf

# All resource blocks regardless of labels
hq 'resource~[*]' main.tf

# Filter blocks by body content
hq 'resource~[select(.ami)]' main.tf

# Combine with wildcards
hq '*~[*] | .block_type' main.tf --value

Pipes

Chain stages with |. Each stage feeds its results into the next. Between stages, attributes unwrap to their values and blocks unwrap to their bodies (for non-path stages like builtins and select).

# Navigate through block body, then extract an attribute
hq 'resource.aws_instance.main | .ami' main.tf --value

# Select with pipe
hq 'variable[*] | select(.default)' variables.tf --json

# Builtins
hq 'x | keys' file.tf --json
hq 'x | length' file.tf --value

Property accessors — when a pipe stage like .name or .block_type doesn't match a structural path, it falls back to Python properties on the view:

View Type Available Properties
BlockView .block_type (e.g. "resource"), .labels (all labels including type), .name_labels (labels after the block type, e.g. ["aws_instance", "main"])
AttributeView .name (attribute name), .value (serialized value)
FunctionCallView .name (function name), .args (argument list), .has_ellipsis
ForTupleView .iterator_name, .second_iterator_name, .iterable, .value_expr, .has_condition, .condition
ForObjectView same as ForTupleView plus .key_expr, .has_ellipsis
ConditionalView .condition, .true_val, .false_val
TupleView .elements
ObjectView .keys, .entries
All views .type (short string like "block", "attribute", etc.)
# Get block types
hq 'resource[*] | .block_type' main.tf --value

# Get name labels (labels after the block type)
hq 'resource[*] | .name_labels' main.tf --json

# Get all labels including block type
hq 'resource[*] | .labels' main.tf --json

# Get attribute names
hq '*[*] | .name' main.tf --value

# Get function call names
hq '*..function_call:*[*] | .name' main.tf --value

# Chain with builtins
hq 'resource[*] | .labels | length' main.tf --value

Pipeline Semantics

Between pipe stages:

  • AttributeView → unwraps to value node (ObjectView, TupleView, etc.)
  • BlockView → unwraps to body (BodyView)
  • ExprTermRule → unwraps to inner expression

This means label traversal should be done within a single stage:

# Good: label traversal in one stage, pipe at body boundary
hq 'resource.aws_instance.main | .ami' main.tf

# Won't work: labels split across pipe stages
# hq 'resource | .aws_instance | .main | .ami' main.tf

Select Predicates

Filter results without eval. Two syntactic positions:

Bracket syntax — inline in a path segment (works with type qualifiers too):

hq '*[select(.name == "x")]' file.tf --value
hq 'variable[select(.default)]' variables.tf
hq '*..function_call:*[select(.args[2])]' file.tf  # functions with >2 args

Pipe stage — as a pipeline stage:

hq 'variable[*] | select(.default)' variables.tf --json
hq 'resource[*] | select(.block_type == "resource")' main.tf

Predicate grammar:

predicate  := or_expr
or_expr    := and_expr ("or" and_expr)*
and_expr   := not_expr ("and" not_expr)*
not_expr   := "not" not_expr | comparison
comparison := accessor (comp_op literal)? | any_all | has_expr
any_all    := ("any" | "all") "(" accessor ";" predicate ")"
has_expr   := "has" "(" STRING ")"
accessor   := "." IDENT ("." IDENT)* ("[" INT "]")? ("|" BUILTIN_OR_FUNC)?
BUILTIN    := "keys" | "values" | "length" | "not"
FUNC       := ("contains" | "test" | "startswith" | "endswith") "(" STRING ")"
literal    := STRING | NUMBER | "true" | "false" | "null"
comp_op    := "==" | "!=" | "<" | ">" | "<=" | ">="

Without a comparison operator, the accessor is an existence/truthy check.

Builtin transforms in accessors — append | builtin to apply a transform before comparing:

# Functions with more than 2 arguments
hq '*..function_call:*[select(.args | length > 2)] | .name' file.tf --value

# Blocks with more than 2 labels
hq '*[select(.labels | length > 2)]' file.tf

String functions (jq-compatible) — filter by substring, regex, or prefix/suffix:

# contains — substring match
hq 'module~[select(.source | contains("docker"))]' dir/ --value

# test — regex match
hq 'resource.aws_instance~[select(.ami | test("^ami-[0-9]+"))]' dir/ --value

# startswith / endswith
hq '*[select(.name | startswith("prod-"))]' file.tf
hq '*[select(.path | endswith("/api"))]' file.tf

has("key") — jq-compatible key existence check:

# Blocks with a "tags" attribute
hq 'resource~[select(has("tags"))]' main.tf

# Equivalent to: select(.tags)

Postfix not — jq-style postfix negation (equivalent to prefix not):

# Blocks without tags
hq 'resource~[select(.tags | not)]' main.tf

# Equivalent to: select(not .tags)

any / all — iterate over a list-valued accessor and test a predicate on each element (jq-style any(generator; condition)):

# Tuples that contain function calls
hq '*..tuple:*[*] | select(any(.elements; .type == "function_call"))' file.tf

# Tuples where ALL elements are plain nodes
hq '*..tuple:*[*] | select(all(.elements; .type == "node"))' file.tf

# Combine with boolean operators
hq '*..tuple:*[*] | select(any(.elements; .type == "function_call" or .type == "tuple"))' file.tf

Virtual accessor .type — returns the view type as a short string:

View Class .type value
DocumentView "document"
BodyView "body"
BlockView "block"
AttributeView "attribute"
ObjectView "object"
TupleView "tuple"
ForTupleView "for_tuple"
ForObjectView "for_object"
FunctionCallView "function_call"
NodeView "node"
# Filter to only object-valued attributes, then get keys
hq '*[select(.type == "attribute")] | select(.type == "object") | keys' file.tf --json

# Or use the type qualifier syntax
hq 'attribute:*[*] | select(.type == "object") | keys' file.tf --json

Type Qualifiers

Prefix a segment with type: to match only nodes of that type. Most useful with recursive descent:

# Find all function calls named "length" anywhere in the document
hq '*..function_call:length' file.tf

# Get the arguments of a specific function call
hq '*..function_call:length | .args' file.tf --json

# All function calls (wildcard name)
hq '*..function_call:*[*]' file.tf

# Filter top-level to only blocks
hq 'block:*[*]' file.tf

After resolving to a FunctionCallView, you can navigate into it:

Segment Behavior
args All arguments
args[*] All arguments (explicit select-all)
args[N] Nth argument (zero-based)

Builtins

Terminal transforms available as pipe stages:

Builtin Description
keys Object → key list; Body/Document → block type + attribute names; Block → labels
values Object → values; Tuple → elements; Body → blocks + attributes
length Tuple/Object/Body → count; others → 1

Append [*] to unpack list results into individual pipeline items:

hq 'tags | keys' file.tf --json       # one JSON array
hq 'tags | keys[*]' file.tf --value   # one key per line
hq 'items | length' file.tf --value

Object Construction {...} (jq-style)

Extract multiple fields into a JSON object per result. Matches jq syntax:

# Shorthand — field name = key name
hq 'module~[*] | {source, cpu, memory}' dir/ --json
# Output: {"source": "...", "cpu": 2, "memory": 512}

# Renamed keys
hq 'resource[*] | {type: .block_type, name: .name_labels}' main.tf --json
# Output: {"type": "resource", "name": ["aws_instance", "main"]}

# Combine with select
hq 'resource~[select(has("tags"))] | {name: .name_labels, tags}' main.tf --json

Optional Operator (?)

Append ? to a query to exit 0 even when no results are found:

hq 'nonexistent?' file.tf --value; echo "exit: $?"  # exit: 0
hq 'x?' file.tf --value                              # prints value, exit: 0

The ? is a CLI-level concern only — it is not stripped in eval mode. It works with all query forms including [select()].

Input: Multiple Files and Globs

hq accepts multiple FILE arguments — files, directories, and glob patterns:

# Multiple files
hq 'resource[*]' file1.tf file2.tf --json

# Directory (walks for .tf, .hcl, .tfvars)
hq 'variable[*]' modules/

# Mix files and directories
hq 'resource[*]' main.tf modules/ --json

# Glob patterns (expanded by hq, not the shell)
hq 'resource[*]' 'modules/**/*.tf' --json

# Stdin (default when no FILE given)
echo 'x = 1' | hq 'x' --value

Glob patterns containing *, ?, or [ are expanded by hq itself (using Python's glob.glob with recursive=True), so they work even when the shell doesn't expand them (e.g. when quoted or used by agents).

When multiple files are queried, output lines are prefixed with filename: (like grep). Use --no-filename to suppress.

Output Modes

Flag Behavior
(default) HCL reconstruction via to_hcl(), str() for primitives
--json to_dict() then json.dumps(). Array for multiple results
--value to_dict() for views, str() for primitives. Auto-unwraps single-key dicts (e.g. attribute → inner value). One per line
--raw Like --value but strips surrounding "quotes" from strings — ideal for shell piping
--ndjson One JSON object per line (newline-delimited JSON). Ideal for streaming/piping large result sets

Compact JSON for Non-TTY

When stdout is not a TTY (e.g. piped to another command), JSON output is compact (no indentation) by default. When stdout is a TTY, JSON is pretty-printed with 2-space indent. Override with --json-indent N.

NDJSON

--ndjson emits one JSON object per line, flushed immediately. Useful for streaming large monorepos:

# One JSON per result, compact
hq 'resource[*]' dir/ --ndjson

# With source location
hq 'resource[*]' dir/ --ndjson --with-location

# Pipe to jq for further processing
hq 'resource[*]' dir/ --ndjson | jq '.tags'

Cannot be combined with --value or --raw.

JSON Provenance (__file__)

When querying multiple files with --json or --ndjson, dict results automatically include a "__file__" key indicating the source file. Use --no-filename to suppress.

Source Location (--with-location)

Add --with-location to --json or --ndjson output to include source file and line numbers:

hq 'resource[*]' main.tf --json --with-location
{
  "__file__": "main.tf",
  "__line__": 3,
  "__end_line__": 7,
  "__column__": 1,
  "__end_column__": 2,
  "ami": "\"ami-12345\""
}

Comments (--with-comments)

Add --with-comments to --json or --ndjson output to include comments in the serialized output (uses SerializationOptions(with_comments=True)):

hq 'resource[*]' main.tf --json --with-comments

Exit Codes

Code Meaning
0 Query matched at least one result (or ? suffix used)
1 No results found
2 HCL parse error
3 Query syntax error or unsafe expression
4 I/O error (file not found, permission denied)

When querying multiple files, "worst error wins" — but only when no results. If any file produces results, exit 0 (grep-like semantics: a single unparseable file in a 100-file directory shouldn't mask success).

Parallel Processing

When querying 20+ files with --json or --ndjson, hq automatically uses multiprocessing for faster results. Each file is parsed and queried in a separate worker process, with results merged at the end.

# Auto-parallel (default for 20+ files with --json/--ndjson)
hq 'resource[*]' large-monorepo/ --ndjson

# Force serial processing
hq 'resource[*]' large-monorepo/ --ndjson --jobs 0

# Explicit worker count
hq 'resource[*]' large-monorepo/ --ndjson --jobs 8

Parallel mode is used when all of these are true:

  • 20+ files to process
  • --json or --ndjson output mode
  • Not reading from stdin
  • Not using --eval or --describe
  • --jobs is not 0 or 1

Text output modes (--value, --raw, default HCL) always run serially to preserve file ordering.

Agent Tips

  • Use distinct exit codes to distinguish "no results" (1) from "bad query" (3) from "file not found" (4)
  • Pipe output through --ndjson for streaming; compact JSON is default for non-TTY
  • Use --with-location for IDE/editor integration (file + line numbers)
  • Quote glob patterns to prevent shell expansion: hq 'query' 'modules/**/*.tf'
  • Use ? suffix for optional queries that shouldn't fail on empty results
  • Large repos: --ndjson with auto-parallel gives the best throughput

Diff

Compare two HCL files structurally:

hq file1.tf --diff file2.tf
hq file1.tf --diff file2.tf --json

Hybrid Queries

Note: Most queries should use structural mode (pipes, select, object construction). Only reach for hybrid mode when you need a Python transform that structural mode can't express.

Use :: to split a structural path (left) from a Python eval expression (right). The expression runs once per result from the structural path, with _ bound to each result.

# Get name_labels for all variables
hq 'variable[*]::name_labels' variables.tf

# Get block_type
hq 'variable[*]::block_type' variables.tf --value

# Call methods
hq 'resource.aws_instance[*].tags::entries()' main.tf

# Use builtins
hq 'variable[*]::len(_.name_labels)' variables.tf --value

Expression normalization (right of ::):

Input Normalized
name_labels _.name_labels
.foo _.foo
_.foo _.foo (unchanged)
len(_.x) len(_.x) (unchanged)
doc.blocks() doc.blocks() (unchanged)

Eval Mode

Note: Eval mode is a last resort. Many operations (comprehensions, imports, f-strings) are blocked for safety. Prefer structural queries with pipes, select, and object construction.

Use -e to treat the entire query as a Python expression. doc is bound to the DocumentView.

# Access specific block attributes
hq -e 'doc.blocks("variable")[0].attribute("default").value' variables.tf --json

# Sort blocks
hq -e 'sorted(doc.blocks("variable"), key=lambda b: b.name_labels[0])' variables.tf

# Filter blocks
hq -e 'list(filter(lambda b: b.attribute("default"), doc.blocks("variable")))' variables.tf

# Find by predicate
hq -e 'doc.find_by_predicate(lambda n: n.type == "attribute" and n.name == "ami")' main.tf

Safe Eval Namespace

  • Variables: doc (DocumentView), _ (per-result in hybrid mode)
  • Builtins: len, str, int, float, bool, list, tuple, type, isinstance, sorted, reversed, enumerate, zip, range, min, max, print, any, all, filter, map
  • Allowed: attribute access (except dunder attributes), method calls, subscripts, lambdas, comparisons, boolean/arithmetic ops, keyword arguments
  • Blocked: imports, comprehensions, assignments, f-strings, walrus operator, exec/eval/__import__, dunder attribute access (__class__, __subclasses__, etc.)

Introspection

--describe — Show type info and available API for query results:

hq --describe 'variable[*]' variables.tf
{
  "results": [
    {
      "type": "BlockView",
      "properties": ["block_type", "labels", "name_labels", "body", "raw", "parent_view"],
      "methods": ["blocks(...)", "attributes(...)", "attribute(...)"],
      "summary": "block_type='variable', labels=['variable', 'name']"
    }
  ]
}

--schema — Dump the full view API hierarchy as JSON (no QUERY or FILE needed):

hq --schema

All Flags

Flag Description
-e, --eval Treat QUERY as a Python expression
--json Output as JSON
--value Output raw values only (auto-unwraps attributes)
--raw Output raw strings (strip surrounding quotes)
--ndjson One JSON object per line (newline-delimited)
--json-indent N JSON indentation width (default: 2 for TTY, compact otherwise)
--with-location Include __file__, __line__, __end_line__ in JSON output
--with-comments Include comments in JSON output
--describe Show type and available properties/methods
--schema Dump full view API schema as JSON
--diff FILE2 Structural diff against FILE2
--no-filename Suppress filename prefix when querying directories
-j N, --jobs N Parallel workers (default: auto for 20+ files, 0 or 1 = serial)
--version Show version and exit

Error Output

Errors are printed to stderr. When --json, --describe, or --schema is active, errors are JSON:

{"error": "query_syntax", "message": "Invalid path segment: '123' in '123invalid'", "query": "123invalid"}
{"error": "unsafe_expression", "message": "comprehensions are not allowed", "expression": "[x for x in _]"}
{"error": "parse_error", "message": "Unexpected token ..."}

Real-World Examples

For a comprehensive collection of validated, task-oriented examples (discovery, compliance, cost analysis, deployment, etc.), see hq Examples.

A few syntax-focused examples showing feature combinations:

# Skip labels, unpack keys — tag keys across all resources
hq 'resource~[*] | .tags | keys[*]' main.tf --value

# Select + string functions — modules sourcing "docker"
hq 'module~[select(.source | contains("docker"))]' dir/ --value

# Recursive descent + type qualifier — all function calls named "length"
hq '*..function_call:length | .args' file.tf --json

# Object construction — multiple fields per result
hq 'resource[*] | {type: .block_type, name: .name_labels}' main.tf --json

# Hybrid mode — Python expression on structural results
hq 'variable[select(.default)]::name_labels[0] + " = " + str(_.attribute("default").value)' variables.tf --value

See Also