Text Processing and File OperationsLesson 4.3

Parsing CSV and JSON in Bash scripts

IFS-based CSV parsing, limitations of Bash CSV parsing, jq for JSON, jq filters and select, extracting values from API responses, iterating JSON arrays with jq, jq as a dependency

CSV Parsing with IFS

Bash can parse simple CSV using IFS, but it breaks on quoted fields containing commas. For production CSV, use a real parser like Python or csvkit.

# Simple CSV — no quoted fields
while IFS=',' read -r name email role; do
  echo "User: $name ($email) — $role"
done < users.csv

# Skip header line
tail -n +2 users.csv | while IFS=',' read -r name email role; do
  echo "$name,$email"
done

JSON with jq

jq is the standard tool for JSON in Bash. Install it: apt install jq or brew install jq.

# Extract a field
echo '{"name":"Alice","age":30}' | jq '.name'
# Output: "Alice"

# Raw string output (no quotes)
echo '{"name":"Alice"}' | jq -r '.name'
# Output: Alice

# Filter array elements
echo '[{"id":1,"active":true},{"id":2,"active":false}]' \
  | jq '.[] | select(.active == true) | .id'
# Output: 1

# From an API response
curl -s https://api.github.com/repos/bash/bash \
  | jq -r '.description'

# Iterate JSON array in a loop
jq -r '.users[] | "\(.name) \(.email)"' data.json \
  | while IFS=' ' read -r name email; do
    echo "Sending welcome email to $email"
  done

For null safety: use // "default" in jq — .field // "n/a" returns n/a if .field is null or missing.