1. Mental model
What jq does
jq reads JSON values, runs a filter program against each input value, and writes zero or more output values.
Filters compose
Every expression is a filter. Pipe filters with |, build values with literals, and use iterators such as .[] to produce multiple results.
When jq is a good fit
Use jq for JSON APIs, log records, config files, object reshaping, quick joins, validation, shell-safe extraction, and compact reporting.
Key distinction: jq's | passes JSON values between filters, not text streams. If the left side emits multiple values, the right side runs once for each value.
2. Quick start
| Task | Command | Notes |
| Pretty-print JSON | jq '.' file.json | Also validates JSON. |
| Compact JSON | jq -c '.' file.json | One output value per line. |
| Extract a field | jq '.name' file.json | Missing field returns null. |
| Raw string output | jq -r '.name' file.json | Good for shell variables and plain text. |
| Array items | jq '.items[]' file.json | Emits each item separately. |
| Filter array items | jq '.items[] | select(.active)' file.json | select keeps truthy values. |
| Project fields | jq '.items[] | {id, name}' file.json | Shorthand for {id: .id, name: .name}. |
| Map an array | jq '.items | map(.name)' file.json | Returns one array. |
| Count items | jq '.items | length' file.json | Works on arrays, objects, and strings. |
| Default missing values | jq '.name // "unknown"' file.json | // uses the right side for null or false. |
Common shape
jq 'filter' file.json
jq -r '.items[] | "\(.id)\t\(.name)"' file.json
Pipeline shape
curl -s https://api.example.com/users | jq '.data[] | {id, email}'
kubectl get pods -o json | jq -r '.items[].metadata.name'
3. Invocation and options
| Option | Use | Example |
-r, --raw-output | Print strings without JSON quotes. | jq -r '.url' response.json |
-c, --compact-output | Print each output on one line. | jq -c '.items[]' data.json |
-s, --slurp | Read all input values into one array. | jq -s 'add' shards/*.json |
-n, --null-input | Run once with null input. | jq -n '{ok: true}' |
-R, --raw-input | Read lines as strings instead of JSON. | printf '%s\n' a b | jq -R '.' |
-f, --from-file | Read jq program from a file. | jq -f report.jq data.json |
-e, --exit-status | Set exit code from final output truthiness. | jq -e '.ok' response.json |
-S, --sort-keys | Sort object keys in output. | jq -S '.' config.json |
-M, --monochrome-output | Disable color. | jq -M '.' file.json > out.json |
-C, --color-output | Force color output. | jq -C '.' file.json | less -R |
--arg name value | Pass a shell string as $name. | jq --arg env prod '.env == $env' file.json |
--argjson name json | Pass parsed JSON as $name. | jq --argjson limit 10 '.n > $limit' file.json |
--slurpfile name file | Bind parsed file values as an array. | jq --slurpfile cfg cfg.json '. + $cfg[0]' data.json |
--rawfile name file | Bind file contents as a string. | jq --rawfile body msg.txt '.body = $body' payload.json |
Quoting: In POSIX shells, put jq programs in single quotes. Use --arg and --argjson for shell values instead of interpolating into the filter text.
Input modes
jq '.items[]' file.json
jq -s 'map(.items[])' *.json
jq -Rn '[inputs | select(length > 0)]' lines.txt
jq -n --arg name "$USER" '{user: $name, generated: now}'
4. Paths and traversal
| Filter | Meaning | Example |
. | Identity: current input value. | jq '.' file.json |
.key | Object field access. | jq '.user.email' file.json |
."key-name" | Field with punctuation or spaces. | jq '."content-type"' headers.json |
.["key-name"] | Bracket field access. | jq '.["content-type"]' headers.json |
.[0] | Array index. | jq '.[0].name' users.json |
.[-1] | Last array item. | jq '.[-1]' events.json |
.[1:4] | Slice from index 1 up to 4. | jq '.[0:10]' results.json |
.[] | Iterate array values or object values. | jq '.items[]' file.json |
.[]? | Optional iteration without errors. | jq '.items[]?' file.json |
.. | Recursive descent. | jq '.. | objects | .id? // empty' file.json |
paths | Emit paths to all values. | jq 'paths | join(".")' file.json |
getpath($p) | Read value at path array. | jq 'getpath(["user","email"])' file.json |
setpath($p; v) | Set value at path array. | jq 'setpath(["meta","ok"]; true)' file.json |
delpaths([$p]) | Delete path arrays. | jq 'delpaths([["debug"],["trace"]])' file.json |
Safe traversal
jq '.user?.email // "none"' file.json
jq '.items[]? | .metadata?.name // empty' file.json
jq '.. | objects | select(has("id")) | .id' file.json
Optional access: ? suppresses errors from missing or wrong-shaped values. Pair it with // empty when you want no output instead of null.
5. Arrays
| Filter | Use | Example |
length | Array length. | jq '.items | length' file.json |
map(f) | Apply filter to each item and keep an array. | jq '.items | map(.name)' file.json |
select(f) | Keep values where filter is truthy. | jq '.items[] | select(.active)' file.json |
first, last | First or last item. | jq '.items | first' file.json |
.[0:5] | Take first five. | jq '.items[0:5]' file.json |
reverse | Reverse order. | jq '.items | reverse' file.json |
sort | Sort scalar values. | jq '.tags | sort' file.json |
sort_by(f) | Sort objects by computed key. | jq '.items | sort_by(.created_at)' file.json |
unique | Deduplicate sorted-compatible values. | jq '.tags | unique' file.json |
unique_by(f) | Deduplicate by computed key. | jq '.items | unique_by(.id)' file.json |
add | Add numbers, concatenate arrays, or merge objects. | jq '.amounts | add' file.json |
flatten | Flatten nested arrays. | jq '.groups | flatten' file.json |
any, all | Boolean checks. | jq '.items | any(.active)' file.json |
contains([x]) | Array contains all requested values. | jq '.tags | contains(["prod"])' file.json |
Array transformations
jq '.items | map(select(.enabled) | {id, name})' file.json
jq '.items | sort_by(.score) | reverse | .[0:10]' file.json
jq '[.items[] | select(.status == "failed") | .id]' file.json
jq '.items | group_by(.team) | map({team: .[0].team, count: length})' file.json
6. Objects
| Filter | Use | Example |
keys | Sorted object keys. | jq 'keys' object.json |
keys_unsorted | Keys in original order where possible. | jq 'keys_unsorted' object.json |
has("key") | Check field existence. | jq 'has("email")' user.json |
in(obj) | Check whether input key exists in object. | jq '.field | in({"id": true})' file.json |
to_entries | Object to {key,value} array. | jq 'to_entries' object.json |
from_entries | {key,value} array to object. | jq 'from_entries' entries.json |
with_entries(f) | Transform object entries. | jq 'with_entries(.key |= ascii_downcase)' object.json |
map_values(f) | Transform each object value. | jq 'map_values(tostring)' object.json |
pick(.a, .b) | Project selected paths in newer jq versions. | jq 'pick(.id, .name)' user.json |
del(.key) | Remove a field. | jq 'del(.password)' user.json |
+ {...} | Merge object fields. | jq '. + {active: true}' user.json |
* {...} | Recursive object merge. | jq '.defaults * .override' config.json |
Object reshaping
jq '{id, name, email: .profile.email}' user.json
jq 'del(.debug, .internal, .password)' user.json
jq 'with_entries(select(.value != null))' object.json
jq '.items | map({key: .id, value: .}) | from_entries' file.json
Projection shorthand: {id, name} means {id: .id, name: .name}. Use explicit keys when renaming or computing values.
7. Types and operators
| Filter | Meaning | Example |
type | Return type name. | jq 'type' file.json |
numbers | Pass only numbers. | jq '.. | numbers' file.json |
strings | Pass only strings. | jq '.. | strings' file.json |
arrays | Pass only arrays. | jq '.. | arrays' file.json |
objects | Pass only objects. | jq '.. | objects' file.json |
nulls | Pass only null values. | jq '.. | nulls' file.json |
booleans | Pass only booleans. | jq '.. | booleans' file.json |
tostring | Convert value to string. | jq '.id | tostring' file.json |
tonumber | Parse string as number. | jq '.count | tonumber' file.json |
+ - * / % | Arithmetic and overloaded JSON operations. | jq '.price * .quantity' file.json |
==, != | Equality checks. | jq '.status == "ok"' file.json |
<, <=, >, >= | Ordering comparisons. | jq '.score >= 90' file.json |
and, or, not | Boolean logic. | jq '.active and (.score > 0)' file.json |
// | Alternative/default operator. | jq '.name // .id // "unknown"' file.json |
Truthiness: Only false and null are false. Empty strings, empty arrays, empty objects, and zero are truthy.
8. Conditionals and errors
| Pattern | Use | Example |
if ... then ... else ... end | Branch on a condition. | jq 'if .ok then .data else empty end' |
select(expr) | Keep inputs where expression is truthy. | jq '.items[] | select(.kind == "pod")' |
empty | Produce no output. | jq '.value // empty' |
error("message") | Fail deliberately. | jq 'if .id then . else error("missing id") end' |
try expr catch fallback | Recover from errors. | jq 'try .count / .total catch null' |
expr? | Shortcut for optional error suppression. | jq '.items[]? | .name?' |
halt_error(code) | Exit with explicit error status. | jq 'if .ok then . else halt_error(1) end' |
Useful branches
jq 'if type == "array" then length else 0 end' file.json
jq '.items[] | select((.status // "") | test("^5"))' file.json
jq 'try (.payload | fromjson) catch {error: "bad payload"}' file.json
9. Variables and bindings
| Syntax | Use | Example |
expr as $x | ... | Bind a computed value. | jq '.user.id as $id | .events[] | select(.user_id == $id)' |
$name | Use a variable. | jq --arg name alex '.name == $name' |
$ARGS.named | Access named arguments. | jq --arg env prod '$ARGS.named.env' |
$ARGS.positional | Access args after --args or --jsonargs. | jq -n --args '$ARGS.positional' a b |
$__loc__ | Location object for debugging. | jq 'error($__loc__)' |
label $out | break $out | Break out of a control context. | jq 'label $done | .[] | if .stop then break $done else . end' |
Binding examples
jq 'INDEX(.users[]; .id) as $users | .orders[] | .user = $users[.user_id]' data.json
jq --arg id "$id" '.items[] | select(.id == $id)' file.json
jq --argjson patch "$patch" '. + $patch' file.json
Scope: as $x does not change the current input. It stores the left-side value and continues with the original input on the right side.
10. Strings and regex
| Filter | Use | Example |
length | String length. | jq '.name | length' file.json |
ascii_downcase | Lowercase ASCII letters. | jq '.email | ascii_downcase' file.json |
ascii_upcase | Uppercase ASCII letters. | jq '.code | ascii_upcase' file.json |
split(str) | Split string into array. | jq '.path | split("/")' file.json |
join(str) | Join array into string. | jq '.tags | join(",")' file.json |
startswith(str) | Prefix check. | jq '.url | startswith("https://")' file.json |
endswith(str) | Suffix check. | jq '.file | endswith(".json")' file.json |
contains(str) | Substring check. | jq '.message | contains("timeout")' file.json |
test(regex) | Regex boolean. | jq '.email | test("@example\\.com$")' file.json |
match(regex) | Regex match object. | jq '.text | match("id=(?<id>[0-9]+)")' file.json |
capture(regex) | Named captures as object. | jq '.text | capture("id=(?<id>[0-9]+)")' file.json |
sub(regex; repl) | Replace first match. | jq '.name | sub(" "; "_")' file.json |
gsub(regex; repl) | Replace all matches. | jq '.name | gsub("[^A-Za-z0-9]+"; "-")' file.json |
interpolation | Build strings with \(...). | jq -r '"\(.id): \(.name)"' file.json |
String recipes
jq -r '.items[] | "\(.id)\t\(.name)\t\(.status)"' file.json
jq '.slug = (.title | ascii_downcase | gsub("[^a-z0-9]+"; "-") | gsub("(^-|-$)"; ""))' file.json
jq '.logs[] | capture("^(?<level>[A-Z]+) (?<msg>.*)$")' file.json
11. Dates and numbers
| Filter | Use | Example |
now | Current Unix timestamp. | jq -n 'now' |
fromdateiso8601 | ISO 8601 string to timestamp. | jq '.created_at | fromdateiso8601' file.json |
todateiso8601 | Timestamp to ISO 8601 string. | jq '.ts | todateiso8601' file.json |
floor, ceil, round | Round numbers. | jq '.ratio | round' file.json |
min, max | Minimum or maximum array item. | jq '.scores | max' file.json |
min_by(f), max_by(f) | Minimum or maximum object by key. | jq '.items | max_by(.score)' file.json |
range(n) | Generate numbers. | jq -n '[range(5)]' |
range(a; b; step) | Generate stepped numbers. | jq -n '[range(0; 10; 2)]' |
Calculations
jq '.items | map(.price * .quantity) | add' order.json
jq '.events[] | select((now - (.created_at | fromdateiso8601)) < 86400)' events.json
jq '.scores as $s | {min: ($s | min), max: ($s | max), avg: (($s | add) / ($s | length))}' file.json
Numeric precision: jq is excellent for JSON-sized numbers, but avoid using it as the source of truth for high-precision decimal finance calculations unless you have verified your jq build and workflow.
12. Updates and assignment
| Syntax | Meaning | Example |
.a = v | Set path to value computed from whole input. | jq '.active = true' user.json |
.a |= f | Update path using old path value as input. | jq '.count |= . + 1' file.json |
+=, -=, *=, /= | Arithmetic or overloaded updates. | jq '.tags += ["new"]' file.json |
//= | Set a default only when current value is null or false. | jq '.name //= "unknown"' file.json |
del(path) | Delete paths. | jq 'del(.debug, .items[].temp)' file.json |
|= empty | Delete selected array items or object fields. | jq '(.items[] | select(.bad)) |= empty' file.json |
walk(f) | Recursively transform values. | jq 'walk(if type == "object" then del(.debug) else . end)' file.json |
Update examples
jq '.items[] |= (.status //= "pending")' file.json
jq '(.items[] | select(.id == "a1") | .enabled) = true' file.json
jq '.metadata.updated_at = (now | todateiso8601)' file.json
jq 'walk(if type == "object" then with_entries(select(.value != null)) else . end)' file.json
= versus |=: With =, the right side sees the original whole input. With |=, the right side sees the value at the path being updated.
13. Aggregation, grouping, and joins
| Filter | Use | Example |
group_by(f) | Group array values by key. | jq '.items | group_by(.team)' file.json |
reduce stream as $x (init; update) | Fold many values into one. | jq 'reduce .items[] as $i (0; . + $i.count)' file.json |
foreach stream as $x (init; update; extract) | Fold while emitting intermediate values. | jq 'foreach .[] as $x (0; . + $x; .)' nums.json |
INDEX(stream; expr) | Build lookup object by key. | jq 'INDEX(.users[]; .id)' data.json |
JOIN(index; stream; expr) | Join stream values against an index. | jq 'JOIN(INDEX(.users[]; .id); .orders[]; .user_id)' data.json |
transpose | Rows to columns. | jq 'transpose' matrix.json |
combinations | Cartesian combinations. | jq 'combinations' arrays.json |
Grouping and counting
jq '.items
| group_by(.status)
| map({status: .[0].status, count: length})' file.json
jq 'reduce .items[] as $item
({}; .[$item.status] += 1)' file.json
jq '.items
| sort_by(.team)
| group_by(.team)
| map({team: .[0].team, total: (map(.amount) | add)})' file.json
Grouping note: group_by(f) returns groups sorted by key. Add a separate sort_by step when you want the ordering to be explicit in a longer pipeline.
15. Slurp, streams, and large files
| Mode | Use | Example |
default | Run filter once per JSON input value. | jq '.id' many-values.json |
-s | Slurp all input values into one array. | jq -s 'length' *.json |
-c | Write newline-delimited JSON values. | jq -c '.items[]' big.json |
inputs | Read remaining input values from inside a -n program. | jq -n '[inputs.id]' |
--stream | Parse large JSON as path/value events. | jq --stream 'select(length == 2)' huge.json |
fromstream | Rebuild values from stream events. | jq --stream 'fromstream(1 | truncate_stream(inputs))' huge.json |
truncate_stream(n) | Drop leading path elements from stream events. | jq --stream 'truncate_stream(1)' huge.json |
--unbuffered | Flush output after each JSON value. | tail -f log.jsonl | jq --unbuffered -c 'select(.level == "error")' |
JSON Lines
jq -c '.items[]' response.json > items.jsonl
jq -s 'map(select(.ok))' items.jsonl
jq -n '[inputs | select(.status == "failed")]' items.jsonl
Large-file rule: If the input is a huge top-level array, .items[] still requires parsing the JSON document. Prefer JSON Lines when you control the format, or use --stream for truly large single documents.
16. Functions, scripts, and modules
| Syntax | Use | Example |
def name: expr; | Define a no-argument function. | def clean: with_entries(select(.value != null)); clean |
def name($x): expr; | Define a function with a value argument. | def older_than($n): select(.age > $n); |
def name(f): expr; | Define a function taking a filter argument. | def keep(f): map(select(f)); |
include "mod"; | Load module definitions. | include "dates"; |
import "mod" as m; | Load module under namespace. | import "util" as u; u::clean |
-L dir | Add module search directory. | jq -L jq-lib -f report.jq data.json |
-f file.jq | Keep longer filters in files. | jq -f transform.jq input.json |
Script file example
def compact_user:
{
id,
name,
email: (.email // .profile.email // null)
};
.users | map(compact_user)
Reusable filters
def non_null_object:
with_entries(select(.value != null));
def count_by(f):
reduce .[] as $x ({}; .[($x | f | tostring)] += 1);
.items | count_by(.status)
17. Shell integration
| Task | Command | Notes |
| Store extracted string | name=$(jq -r '.name' file.json) | Use -r for shell strings. |
| Test JSON condition | jq -e '.ok' response.json > /dev/null | Good for scripts and CI. |
| Pass string safely | jq --arg name "$name" '.name = $name' file.json | Never concatenate untrusted shell text into jq code. |
| Pass JSON safely | jq --argjson patch "$patch" '. + $patch' file.json | $patch must be valid JSON. |
| Update a file atomically | tmp=$(mktemp) && jq '.x=1' file.json > "$tmp" && mv "$tmp" file.json | jq does not edit files in place. |
| Read from curl | curl -fsS "$url" | jq -r '.data[].id' | Use curl failure flags in scripts. |
| Use with find | find . -name '*.json' -print0 | xargs -0 jq -c '.id' | Handles spaces in paths. |
| Generate JSON payload | jq -n --arg msg "$msg" '{message: $msg}' | Avoid manual JSON string escaping. |
Robust script patterns
if jq -e '.status == "ready"' response.json > /dev/null; then
echo ready
fi
payload=$(jq -n \
--arg name "$name" \
--argjson count "$count" \
'{name: $name, count: $count}')
curl -fsS -H 'Content-Type: application/json' -d "$payload" "$url"
18. Recipes and gotchas
| Goal | Filter | Notes |
| List every scalar path | paths(scalars) as $p | [$p | join("."), getpath($p)] | Pair with @tsv for reports. |
| Remove null fields | with_entries(select(.value != null)) | Top-level object only. |
| Remove nulls recursively | walk(if type == "object" then with_entries(select(.value != null)) else . end) | Requires walk. |
| Flatten selected fields | .items[] | [.id, .user.email, .status] | @tsv | Use -r. |
| Make object from array | map({key: .id, value: .}) | from_entries | Later duplicate keys win. |
| Join two arrays by id | INDEX(.users[]; .id) as $u | .orders[] | .user = $u[.user_id] | Indexes users for fast lookup. |
| Count by key | reduce .items[] as $i ({}; .[$i.status] += 1) | Missing count starts as null; null + 1 becomes 1. |
| Keep only keys | {id, name, status} | Projection shorthand. |
| Rename keys | {user_id: .id, display_name: .name} | Explicit object construction. |
| Find duplicate ids | .items | group_by(.id) | map(select(length > 1) | .[0].id) | Returns ids that appear more than once. |
Common one-liners
jq -r '.items[] | select(.status == "failed") | .name' file.json
jq '.items |= map(select(.enabled))' file.json
jq '.items | map(.amount) | add // 0' file.json
jq -r 'paths(scalars) as $p | [($p | join(".")), (getpath($p) | tostring)] | @tsv' file.json
jq -s '.[0] * .[1]' defaults.json override.json
jq -n --slurpfile users users.json --slurpfile orders orders.json '{users: $users[0], orders: $orders[0]}'
Gotcha: generators
.items[] emits multiple values. Wrap with brackets, as in [.items[] | .id], when you need one array result.
Gotcha: raw output
Use -r only when the result is meant to be text. Without it, jq preserves valid JSON output.
Gotcha: shell quoting
Use single quotes around filters in Unix shells and pass shell data with --arg or --argjson.
Gotcha: in-place edits
jq writes to stdout. For file updates, write to a temporary file and move it into place only after jq succeeds.