This is the raw data I pull from my api:
{
"observations": [
{
"winddir": 292,
"humidity": 92,
"qcStatus": 1,
"imperial": {
"temp": 20,
"heatIndex": 20,
"dewpt": 40,
"windChill": 20,
"windSpeed": 3,
"windGust": 3,
"pressure": 29.71,
"precipRate": 0,
"precipTotal": 0.01,
"elev": 1905
}
}
]
}
This is the command that I run:
curl -s 'https://api.myawesomeapidata.com' | jq -r '.observations[].winddir, .observations[].humidity, .observations[].imperial.temp'
This is the output:
292
92
20
This is the output I would like:
Wind Direction: 292
Humidity: 92
Temperature: 20
But I would be fine if this were the output:
winddir: 292
humidity: 92
temp: 20
As you can see, I would like the key to appear in front of the value. Preferably allowing me to change the name of the key (Wind Direction) before printing, but I would also be fine with the original key name (winddir).
Try this using String interpolation
… | jq -r '.observations[]
| "Wind Direction: \(.winddir)"
, "Humidity: \(.humidity)"
, "Temperature: \(.imperial.temp)"
'
Wind Direction: 292
Humidity: 92
Temperature: 20
Demo
Yet you can keep the original key names through use of
jq -r '.observations[] | {winddir},{humidity},(.imperial| {temp})| "\(keys[]) : \(.[])"'
which results
winddir : 292
humidity : 92
temp : 20
Demo
Related
I am basically wanting to update multiple scholars for an NFT game (axie infinity). It requires a JSON file that looks like this:
{
"name": "Scholar 1",
"ronin": "ronin:<account_s1_address>",
"splits": [
{
"persona": "Manager",
"percentage": 44,
"ronin": "ronin:<manager_address>"
},
{
"persona": "Scholar",
"percentage": 40,
"ronin": "ronin:<scholar_1_address>"
},
{
"persona": "Other Person",
"percentage": 6,
"ronin": "ronin:<other_person_address>"
},
{
"persona": "Trainer",
"percentage": 10,
"ronin": "ronin:<trainer_address>"
}
]
}
But since there are multiple scholars/players, I wanted to know if there was anyway to format something on a CSV file that if I convert or import it using a JSON tool it will look like like the JSON above?
Your help is much appreciated.. Thank you!
PS:
The first lines:
"name": "Scholar 1",
"ronin": "ronin:<account_s1_address>",
"splits":
Would need to be repeated since again there are multiple scholars, i.e. Scholar 1, Scholar 2, Scholar 3...
CSV file structure is column-base, if Axie infinity require JSON file, you can create a CSV file by Excel or Google sheet and convert to JSON.
there is a similar answer to convert CSV to JSON
starting from this CSV that has this structure
name
ronin
id_persona
persona
percentage
split_ronin
Scholar 1
ronin:<account_s1_address>
1
Manager
44
ronin:<manager_address>
Scholar 1
ronin:<account_s1_address>
2
Scholar
40
ronin:<scholar_1_address>
Scholar 1
ronin:<account_s1_address>
3
Other Person
6
ronin:<other_person_address>
Scholar 1
ronin:<account_s1_address>
4
Trainer
10
ronin:<trainer_address>
you can run this Miller command
mlr --c2j reshape -r "^(p|s)" -o k,v then \
put '$k="splits".".".${id_persona}.".".$k' then \
cut -x -f id_persona then \
reshape -s k,v out.csv
to have
[
{
"name": "Scholar 1",
"ronin": "ronin:<account_s1_address>",
"splits": [
{
"persona": "Manager",
"percentage": 44,
"split_ronin": "ronin:<manager_address>"
},
{
"persona": "Scholar",
"percentage": 40,
"split_ronin": "ronin:<scholar_1_address>"
},
{
"persona": "Other Person",
"percentage": 6,
"split_ronin": "ronin:<other_person_address>"
},
{
"persona": "Trainer",
"percentage": 10,
"split_ronin": "ronin:<trainer_address>"
}
]
}
]
Some notes:
reshape -r "^(p|s)" -o k,v, to transform the input from wide to long;
put '$k="splits".".".${id_persona}.".".$k', to create values that I will use as field names (splits.1.persona,splits.1.percentage,splits.1.split_ronin,splits.2.persona,splits.2.percentage, ....
cut -x -f id_persona, to remove the field id_persona;
reshape -s k,v, to transform all from long to wide.
The real goal is to build, starting from that input, this kind of CSV
+-----------+----------------------------+------------------+---------------------+-------------------------+------------------+---------------------+---------------------------+------------------+---------------------+------------------------------+------------------+---------------------+-------------------------+
| name | ronin | splits.1.persona | splits.1.percentage | splits.1.split_ronin | splits.2.persona | splits.2.percentage | splits.2.split_ronin | splits.3.persona | splits.3.percentage | splits.3.split_ronin | splits.4.persona | splits.4.percentage | splits.4.split_ronin |
+-----------+----------------------------+------------------+---------------------+-------------------------+------------------+---------------------+---------------------------+------------------+---------------------+------------------------------+------------------+---------------------+-------------------------+
| Scholar 1 | ronin:<account_s1_address> | Manager | 44 | ronin:<manager_address> | Scholar | 40 | ronin:<scholar_1_address> | Other Person | 6 | ronin:<other_person_address> | Trainer | 10 | ronin:<trainer_address> |
+-----------+----------------------------+------------------+---------------------+-------------------------+------------------+---------------------+---------------------------+------------------+---------------------+------------------------------+------------------+---------------------+-------------------------+
and than use it to create the final JSON output
I would like to add some values from json file separated by pipe. It's working well so far until a value is a number and not a string.
Here what I've done so far: jq -r '.content[] | {seasonTitle, number, name} | join("|")' file.json
I've tried to convert number to string without any success jq -r '.content[] | {seasonTitle, "episodeNumber|tostring", name} | join("|")' file.json
Actual Result:
Top Master||Last Chance / Season 12
Top Master||Épisode 8 / Season 12
Top Master||Épisode 7 / Season 12
Expected Result:
Top Master|236|Last Chance / Season 12
Top Master|235|Épisode 8 / Season 12
Top Master|234|Épisode 7 / Season 12
Here the file.json
{
"page": 0,
"size": 3,
"count": 3,
"content": [
{
"name": "Last Chance / Season 12",
"releaseDate": "2008",
"duration": 2100,
"episodeNumber": 236,
"title": "Last Chance / Season 12",
"seasonTitle": "Top Master"
},
{
"name": "Épisode 8 / Season 12",
"releaseDate": "2008",
"duration": 7320,
"episodeNumber": 235,
"title": "Épisode 8 / Season 12",
"seasonTitle": "Top Master"
},
{
"name": "Épisode 7 / Season 12",
"releaseDate": "2008",
"duration": 7200,
"episodeNumber": 234,
"title": "Épisode 7 / Season 12",
"seasonTitle": "Top Master"
}
]
}
You are using join to concatenate values of different types, which works fine under jq v1.6:
.content[] | {seasonTitle, episodeNumber, name} | join("|")
Top Master|236|Last Chance / Season 12
Top Master|235|Épisode 8 / Season 12
Top Master|234|Épisode 7 / Season 12
Demo
However, with jq v1.5 it doesn't, and you need to convert non-strings to strings using tostring. As you are using a shortcut to create an object for join, introducing this conversion sacrifices the conciseness of your solution. So either stick with it:
.content[] | {seasonTitle, episodeNumber: (.episodeNumber | tostring), name} | join("|")
Or use an array instead, as you are going for the values only anyway:
.content[] | [.seasonTitle, (.episodeNumber | tostring), .name] | join("|")
I have a two-fold issue and looking for clues as to how to approach it.
I have a json file that is formatted as such:
{
"code": 2000,
"data": {
"1": {
"attribute1": 40,
"attribute2": 1.4,
"attribute3": 5.2,
"attribute4": 124
"attribute5": "65.53%"
},
"94": {
"attribute1": 10,
"attribute2": 4.4,
"attribute3": 2.2,
"attribute4": 12
"attribute5": "45.53%"
},
"96": {
"attribute1": 17,
"attribute2": 9.64,
"attribute3": 5.2,
"attribute4": 62
"attribute5": "51.53%"
}
},
"message": "SUCCESS"
}
My goals are to:
I would first like to sort the data by any of the attributes.
There are around 100 of these, I would like to grab the top 5 (depending on how they are sorted), then...
Output the data in a table e.g.:
These are sorted by: attribute5
---
attribute1 | attribute2 | attribute3 | attribute4 | attribute5
40 |1.4 |5.2|124|65.53%
17 |9.64|5.2|62 |51.53%
10 |4.4 |2.2|12 |45.53%
*also, attribute5 above is a string value
Admittedly, my knowledge here is very limited.
I attempted to mimick the method used here:
python sort list of json by value
I managed to open the file and I can extract the key values from a sample row:
import json
jsonfile = path-to-my-file.json
with open(jsonfile) as j:
data=json.load(j)
k = data["data"]["1"].keys()
print(k)
total=data["data"]
for row in total:
v = data["data"][str(row)].values()
print(v)
this outputs:
dict_keys(['attribute1', 'attribute2', 'attribute3', 'attribute4', 'attribute5'])
dict_values([1, 40, 1.4, 5.2, 124, '65.53%'])
dict_values([94, 10, 4.4, 2.2, 12, '45.53%'])
dict_values([96, 17, 9.64, 5.2, 62, '51.53%'])
Any point in the right direction would be GREATLY appreciated.
Thanks!
If you don't mind using pandas you could do it like this
import pandas as pd
rows = [v for k,v in data["data"].items()]
df = pd.DataFrame(rows)
# then to get the top 5 values by attribute can choose either ascending
# or descending with the ascending keyword and head prints the top 5 rows
df.sort_values('attribute1', ascending=True).head()
This will allow you to sort by any attribute you need at any time and print out a table.
Which will produce output like this depending on what you sort by
attribute1 attribute2 attribute3 attribute4 attribute5
0 40 1.40 5.2 124 65.53%
1 10 4.40 2.2 12 45.53%
2 17 9.64 5.2 62 51.53%
I'll leave this answer here in case you don't want to use pandas but the answer from #MatthewBarlowe is way less complicated and I recommend that.
For sorting by a specific attribute, this should work:
import json
SORT_BY = "attribute4"
with open("test.json") as j:
data = json.load(j)
items = data["data"]
sorted_keys = list(sorted(items, key=lambda key: items[key][SORT_BY], reverse=True))
Now, sorted_keys is a list of the keys in order of the attribute they were sorted by.
Then, to print this as a table, I used the tabulate library. The final code for me looked like this:
from tabulate import tabulate
import json
SORT_BY = "attribute4"
with open("test.json") as j:
data = json.load(j)
items = data["data"]
sorted_keys = list(sorted(items, key=lambda key: items[key][SORT_BY], reverse=True))
print(f"\nSorted by: {SORT_BY}")
print(
tabulate(
[
[sorted_keys[i], *items[sorted_keys[i]].values()]
for i, _ in enumerate(items)
],
headers=["Column", *items["1"].keys()],
)
)
When sorting by 'attribute5', this outputs:
Sorted by: attribute5
Column attribute1 attribute2 attribute3 attribute4 attribute5
-------- ------------ ------------ ------------ ------------ ------------
1 40 1.4 5.2 124 65.53%
96 17 9.64 5.2 62 51.53%
94 10 4.4 2.2 12 45.53%
I'm trying to parse JSON and store certain values as metrics in Graphite.
In order to make my Graphite more user-friendly I have to form a metric name, that contains some values from its object.
I got working solution on bash loops + jq, but it's really slow. So I'm asking for help :)
Here is my input:
{
...
},
"Johnny Cage": {
"firstname": "Johnny",
"lastname": "Cage",
"height": 183,
"weight": 82,
"hands": 2,
"legs": 2,
...
},
...
}
Desired output:
mk.fighter.Johnny.Cage.firstname Johnny
mk.fighter.Johnny.Cage.lastname Cage
mk.fighter.Johnny.Cage.height 183
mk.fighter.Johnny.Cage.weight 82
mk.fighter.Johnny.Cage.hands 2
mk.fighter.Johnny.Cage.legs 2
...
With single jq command:
Sample input.json:
{
"Johnny Cage": {
"firstname": "Johnny",
"lastname": "Cage",
"height": 183,
"weight": 82,
"hands": 2,
"legs": 2
}
}
jq -r 'to_entries[] | (.key | sub(" "; ".")) as $name
| .value | to_entries[]
| "mk.fighter.\($name).\(.key) \(.value)"' input.json
To get $name as a combination of inner firstname and lastname keys replace (.key | sub(" "; ".")) as $name with "\(.value.firstname).\(.value.lastname)" as $name
The output:
mk.fighter.Johnny.Cage.firstname Johnny
mk.fighter.Johnny.Cage.lastname Cage
mk.fighter.Johnny.Cage.height 183
mk.fighter.Johnny.Cage.weight 82
mk.fighter.Johnny.Cage.hands 2
mk.fighter.Johnny.Cage.legs 2
My current code as seen below attempts to construct a request payload (body), but isn't giving me the desired result.
library(df2json)
library(rjson)
y = rjson::fromJSON((df2json::df2json(dataframe)))
globalparam = ""
req = list(
Inputs = list(
input1 = y
)
,GlobalParameters = paste("{",globalparam,"}",sep="")#globalparam
)
body = enc2utf8((rjson::toJSON(req)))
body currently turns out to be
{
"Inputs": {
"input1": [
{
"X": 7,
"Y": 5,
"month": "mar",
"day": "fri",
"FFMC": 86.2,
"DMC": 26.2,
"DC": 94.3,
"ISI": 5.1,
"temp": 8.2,
"RH": 51,
"wind": 6.7,
"rain": 0,
"area": 0
}
]
},
"GlobalParameters": "{}"
}
However, I need it to look like this:
{
"Inputs": {
"input1": [
{
"X": 7,
"Y": 5,
"month": "mar",
"day": "fri",
"FFMC": 86.2,
"DMC": 26.2,
"DC": 94.3,
"ISI": 5.1,
"temp": 8.2,
"RH": 51,
"wind": 6.7,
"rain": 0,
"area": 0
}
]
},
"GlobalParameters": {}
}
So basically global parameters have to be {}, but not hardcoded. It seemed like a fairly simple problem, but I couldn't fix it. Please help!
EDIT:
This is the dataframe
X Y month day FFMC DMC DC ISI temp RH wind rain area
1 7 5 mar fri 86.2 26.2 94.3 5.1 8.2 51 6.7 0.0 0
2 7 4 oct tue 90.6 35.4 669.1 6.7 18.0 33 0.9 0.0 0
3 7 4 oct sat 90.6 43.7 686.9 6.7 14.6 33 1.3 0.0 0
4 8 6 mar fri 91.7 33.3 77.5 9.0 8.3 97 4.0 0.2 0
This is an example of another data frame
> a = data.frame("col1" = c(81, 81, 81, 81), "col2" = c(72, 69, 79, 84))
Using this sample data
dd<-read.table(text=" X Y month day FFMC DMC DC ISI temp RH wind rain area
1 7 5 mar fri 86.2 26.2 94.3 5.1 8.2 51 6.7 0.0 0", header=T)
You can do
globalparam = setNames(list(), character(0))
req = list(
Inputs = list(
input1 = dd
)
,GlobalParameters = globalparam
)
body = enc2utf8((rjson::toJSON(req)))
Note that globalparam looks a bit funny because we need to force it to a named list for rjson to treat it properly. We only have to do this when it's empty.