I have json data:
[{
"dataType": "detox",
"hLogging": 0.5,
"reading": 63.9,
"minValue": 25,
"dataValue": [{
"time": 143221019,
"value": 44
}, {
"time": 1433521119,
"value": 66
}, {
"time": 1433521319,
"value": 22
}]
}, {
"dataType": "epox",
"hLogging": 3,
"reading": 61.0,
"min"Value: 0,
"dataValue": [{
"time": 1433521019,
"value": 55
}, {
"time": 1433521119,
"value": 66
}, {
"time": 1433521219,
"value": 77
}, {
"time": 1433521319,
"value": 88
}]
}]
There are two data types which have their own data value. This data value contains time which is in unix epoch time. I need to convert it into normal date time value. For this I started by parsing the data using for:
for item in range(len(json_data['dataValue'])):
print(json_data['dataValue'][item]['time'])
But this throws error:
string indices must be integers
Probably because json data has string values but then how can I approach to get the values of time and convert it into normal date time format. Also the time values are not fix in datavalue, it can be 3,4,5...nth items, so need to include range. Please suggest a good way.
The json data which you have is not actually json data but the list having element as json data. So you have to call the index of the list before parsing the json data.
for item in range(len(json_data[0]['dataValue'])):
print(json_data[0]['dataValue'][item]['time'])
Hope this helps! Cheers!
Related
The JSON output I am trying to parse:
{
"success": true,
"data": {
"aa": [
{
"timestamp": 123456,
"price": 1
},
{
"timestamp": 123457,
"price": 2
],
"bb": [
{
"timestamp": 123456,
"price": 3
},
{
"timestamp": 123457,
"price": 4
}
]
}
}
So after banging my head against the wall a million times, I just removed the "success": true", line from the output and I could easily do jq stuff with it. Otherwise if I ran for example:
cat jsonfile.json | jq -c .[].aa
I would get:
Cannot index boolean with string "aa"
Which makes sense, since the first key is boolean. But I have no clue how to skip it while processing with jq.
Goal is to filter only timestamp and price of "aa", without giving any care about the "success": true key/value pair.
You need to select the data field first: jq .data.aa[]
I have a json object served from an api as follows:
{
"workouts": [
{
"id": 92527291,
"starts": "2021-06-28T15:42:44.000Z",
"minutes": 30,
"name": "Indoor Cycling",
"created_at": "2021-06-28T16:12:57.000Z",
"updated_at": "2021-06-28T16:12:57.000Z",
"plan_id": null,
"workout_token": "ELEMNT BOLT A1B3:59",
"workout_type_id": 12,
"workout_summary": {
"id": 87540207,
"heart_rate_avg": "152.0",
"calories_accum": "332.0",
"created_at": "2021-06-28T16:12:58.000Z",
"updated_at": "2021-06-28T16:12:58.000Z",
"power_avg": "185.0",
"distance_accum": "17520.21",
"cadence_avg": "87.0",
"ascent_accum": "0.0",
"duration_active_accum": "1801.0",
"duration_paused_accum": "0.0",
"duration_total_accum": "1801.0",
"power_bike_np_last": "186.0",
"power_bike_tss_last": "27.6",
"speed_avg": "9.73",
"work_accum": "332109.0",
"file": {
"url": "https://cdn.wahooligan.com/wahoo-cloud/production/uploads/workout_file/file/FPoJBPZo17BvTmSomq5Y_Q/2021-06-28-154244-ELEMNT_BOLT_A1B3-59-0.fit"
}
}
}
],
"total": 55,
"page": 1,
"per_page": 1,
"order": "descending",
"sort": "starts"
}
I want to get the data into a dataframe. However, lots of the columns seem to have a dtype of object. I assume that this is because some of the numeric values in the json are double quoted. What is the best and most efficient way to avoid this (the json potentially has many workouts elements)?
Is it to fix the returned json? Or to iterate through the dataframe columns and convert the objects to floats?
Thank you
Martyn
IIUC, you can try:
df = pd.json_normalize(json_data, meta=[
'total', 'page', 'per_page', 'order', 'sort'], record_path='workouts').convert_dtypes()
Try using pandas.to_numeric. Here are the docs.
https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.to_numeric.html
I have a table with a JSON column. in this column I have a blockchain hash data.
for example this JSON:
{ "017989w06d3f902f1f362dfg48f862dba6a605229e99859a91d854f93ac13894": {
"transaction": {
"block_id": 648895,
"id": 568135560,
"hash": "017989w06d3f902f1f362dfg48f862dba6a605229e99859a91d854f93ac13894",
"date": "2020-01-14",
"time": "2020-01-14 11:37:37",
"size": 198,
},
"inputs": [
{
"block_id": 648859,
"transaction_id": 567456558,
"index": 4,
"transaction_hash": "8aa2c6c9a804mate29790e03fac462782d99f16614732f82a5214786926e1397",
"date": "2020-01-13",
"time": "2020-01-13 23:15:37",
"value": 300830,
"value_usd": 33.2264,
"recipient": "1LcrmomE74BPzBTdduE8WHU2ox4QAFEpQi",
}
],
"outputs": [
{
"block_id": 648445,
"transaction_id": 568146680,
"index": 0,
"transaction_hash": "017989w06d3f902f1f362dfg48f862dba6a605229e99859a91d854f93ac13894",
"date": "2020-01-14",
"time": "2020-01-14 11:37:37",
"value": 300048,
"value_usd": 31.9397,
"recipient": "12UJZqf4sDGRNb9uYBABJkMyX91iLjDViT",
}
]}}
I used below query:
SELECT *, JSON_VALUE(d.json_data,'$.017989w06d3f902f1f362dfg48f862dba6a605229e99859a91d854f93ac13894.transaction.size') as jj
FROM BlockChain as d
but I have an error
Msg 13607, Level 16, State 4, Line 39
JSON path is not properly formatted. Unexpected character '0' is found at position 2.
Does anyone have any idea?
The path
'$.017989w06d3f902f1f362dfg48f862dba6a605229e99859a91d854f93ac13894.transaction.size'
cannot have a node starting with a 0. So enclose it in quotes:
'$."017989w06d3f902f1f362dfg48f862dba6a605229e99859a91d854f93ac13894".transaction.size'.
You also have a problem with your actual JSON, in that it has trailing commas, which is not supported in SQL Server, nor in the vast majority of parsers and browsers, as it is against the spec.
If you have different key names for each value, then you need to break out the JSON with OPENJSON:
SELECT b.*, j.[key] AS hash, JSON_VALUE(j.value,'$.transaction.size') as jj
FROM BlockChain as d
CROSS APPLY OPENJSON(d.json_data) AS j
(Cannot summarize the problem in a single statement, hence the ambiguous title)
I create a JSON structure via Angular Typescript, wherein when a user interacts with certains parts of the component the JSON Structure gets updated.
Steps
Initially, the JSON under consideration is by default set to the following:
{
"keyword": {
"value": "product",
"type": "main"
}
}
For example, a user chooses some parameter Name. Once the user complies to certain steps in the UI, the JSON structure gets updated to the following:
{
"keyword": {
"value": "product",
"type": "main"
},
"Name": {
"value": " <hasProperty> Name",
"type": "dataprop"
}
}
Once the user selects a numeric value for a parameter like dryTime, the JSON gets updated to the following:
{
"20": { // WHY WOULD 20 be here?
"value": "<hasValue> 20",
"type": "fValue"
},
"keyword": {
"value": "Varnish",
"type": "main"
},
"Name": {
"value": " <hasProperty> Name",
"type": "dataprop"
},
"dryingTime": {
"value": " <hasProperty> dryingTime",
"type": "dataprop"
}
}
I understand that a JSON is an unordered data structure. But a previous implementation of something similar actually worked well, i.e., the value 20 here was 20.0 before and it was displayed after dryingTime in my JSON.
The order is critical for me as I parse all the Keys in the above mentioned JSON using a for loop and store it in an array. This array needs to show all the keys in the order of the User Interaction.
Where am I going wrong here if I decide to stay with JSON and not with an array to store such interactions?
Yes, JSON fields are unordered. JSON array is ordered.
If you want to keep the order of elements insterted, you could build your JSON like so:
{
"keyword": {
"value": "Varnish",
"type": "main"
},
"props": [
{
"name": "dryingTime",
"value": 20
},
{
"name": "anotherOrderedField",
"value": "fieldValue"
}
]
}
{
"gameId": 32,
"participantIdentities": [
{
"player": {
"id": "123",
"name": "xxx",
},
"participantId": 1
},
{
"player": {
"id": "123",
"name": "yyyy",
},
"participantId": 2
}
]
"gameDuration": 143,
}
I am trying to print names in this json file in python 3
list_id = []
for info in matchinfo['participantIdentities']['player']['name']:
list_id.append(info)
But I get the following error below
TypeError: list indices must be integers or slices, not str
How do I get the content of 'name'?
There are several issues:
You provided an invalid JSON. matchinfo['participantIdentities'] should be a list but the JSON you provided is missing a closing ]
matchinfo['participantIdentities'] is a list, so you should either provide an index (matchinfo['participantIdentities'][0]['player']['summonerId'] for example) or iterate over all matchinfo['participantIdentities'] entries.
You are trying to access a key that doesn't even exist (at least in the JSON you provided). There is no 'summonerId' key anywhere.