You can use jq in-built function unique or unique_by() to remove duplicate entries from your JSON data. Below is an example for your reference –
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
cat << EOF > data.json [ { "id": 300, "name": "debjeet", "did": [2,1,2,5,4,2,5] }, { "name": "debjeet", "id": 100 }, { "name": "debjeet", "id": 100 } ] EOF cat data.json | jq '.[0].did | unique' ## returns [1,2,4,5] cat data.json | jq 'unique_by(.id)' ## returns output with unique id value |