How to pretty-print json in a shell script

Pretty-printing JSON in a shell script involves using tools like jq or python to format the JSON data in a human-readable way. jq is a lightweight and flexible command-line JSON processor that is widely used for this purpose, while Python’s json.tool module can also be utilized for pretty-printing JSON. Both methods provide a straightforward approach to make JSON output more readable directly from the shell.

Using jq for Pretty-Printing

Installation: First, ensure that jq is installed on your system. You can install it using package managers like apt, brew, or yum.

# For Debian/Ubuntu
sudo apt-get install jq

# For macOS
brew install jq

# For Red Hat/CentOS
sudo yum install jq

Basic Usage: To pretty-print a JSON file or JSON string using jq, use the following command. This command reads the JSON data and formats it with proper indentation and spacing.

# Pretty-print a JSON file
jq . input.json

# Pretty-print a JSON string
echo '{"name":"John","age":30,"city":"New York"}' | jq .

Examples: Here’s an example of how you can pretty-print JSON data from a file or string using jq.

# From a JSON file
jq . mydata.json

# From a JSON string
echo '{"name":"Alice","age":25,"city":"Wonderland"}' | jq .

Using Python for Pretty-Printing

Using Python’s json.tool: If jq is not available, you can use Python’s built-in json.tool module to pretty-print JSON. Python is commonly available on most systems, making this a convenient option.

# Pretty-print a JSON file
python -m json.tool input.json

# Pretty-print a JSON string
echo '{"name":"John","age":30,"city":"New York"}' | python -m json.tool

Example: Here’s how to use Python to pretty-print JSON data.

# From a JSON file
python -m json.tool mydata.json

# From a JSON string
echo '{"name":"Alice","age":25,"city":"Wonderland"}' | python -m json.tool

Combining Commands

Shell Scripting: You can combine these commands in a shell script to automate the process of pretty-printing JSON data. This can be useful for processing multiple JSON files or for integrating into larger automation scripts.

#!/bin/bash

# Function to pretty-print JSON using jq
pretty_print_json() {
    local json_data=$1
    echo "$json_data" | jq .
}

# Read JSON data from a file or input
json_data=$(cat input.json)
pretty_print_json "$json_data"

Using Python in a Script: Similarly, you can use Python within a shell script to pretty-print JSON data.

#!/bin/bash

# Function to pretty-print JSON using Python
pretty_print_json() {
    local json_data=$1
    echo "$json_data" | python -m json.tool
}

# Read JSON data from a file or input
json_data=$(cat input.json)
pretty_print_json "$json_data"

Handling Large JSON Files

Efficient Processing: For large JSON files, both jq and Python can handle the data efficiently, but jq is generally faster and more memory-efficient for processing large amounts of JSON data.

# Using jq for large JSON files
jq . largefile.json

# Using Python for large JSON files
python -m json.tool largefile.json

Streaming with jq: jq also supports streaming processing, which is useful for very large JSON files that cannot be loaded into memory all at once.

jq --stream . largefile.json

Error Handling

Validating JSON: Both jq and Python will throw errors if the JSON data is not valid. This can be used to validate JSON files before further processing.

# Using jq to validate JSON
jq . input.json > /dev/null

# Using Python to validate JSON
python -m json.tool input.json > /dev/null

Handling Errors in Scripts: In a shell script, you can handle errors gracefully by checking the exit status of the commands.

#!/bin/bash

# Function to pretty-print JSON using jq with error handling
pretty_print_json() {
    local json_data=$1
    echo "$json_data" | jq . || { echo "Invalid JSON data"; exit 1; }
}

# Read JSON data from a file or input
json_data=$(cat input.json)
pretty_print_json "$json_data"

Summary

Pretty-Printing Tools: jq and Python’s json.tool module are powerful tools for pretty-printing JSON in a shell script. Both methods ensure that the JSON data is formatted in a readable way.

Installation and Usage: jq requires installation via package managers, while Python’s json.tool is readily available with Python installations. Basic usage involves piping JSON data to these tools for pretty-printing.

Scripting and Automation: These commands can be easily integrated into shell scripts to automate JSON processing tasks, handle large files, and manage errors effectively.

Best Practices: Use jq for efficient and fast processing, especially with large JSON files. Use Python for its convenience and availability on most systems. Handle errors to ensure the JSON data is valid before processing.