Related
I want to convert json to csv.
I use tool "jq" for this
jq
Here json
{
"total": 2040,
"created_at": "2022-01-27T09:50:59+0200",
"project": "my project",
"issues": [
{
"key": "key_1",
"component": "my_component",
"textRange": {
"startLine": 35,
"endLine": 35,
"startOffset": 46,
"endOffset": 84
},
"flows": [],
"status": "OPEN",
"creationDate": "2022-01-24T06:42:58+0200",
"updateDate": "2022-01-24T06:42:58+0200",
"type": "BUG",
"scope": "MAIN"
},
{
"key": "key2",
"component": "my component 2",
"textRange": {
"startLine": 34,
"endLine": 34,
"startOffset": 3,
"endOffset": 52
},
"flows": [
{
"locations": [
{
"component": "some component",
"textRange": {
"startLine": 35,
"endLine": 35,
"startOffset": 3,
"endOffset": 50
},
"msg": "any message"
}
]
},
{
"locations": [
{
"component": "another component",
"textRange": {
"startLine": 36,
"endLine": 36,
"startOffset": 3,
"endOffset": 71
},
"msg": "message custom"
}
]
},
{
"locations": [
{
"component": "Alarm.java",
"textRange": {
"startLine": 37,
"endLine": 37,
"startOffset": 3,
"endOffset": 76
},
"msg": "message number 2"
}
]
},
{
"locations": [
{
"component": "Alarm.java",
"textRange": {
"startLine": 38,
"endLine": 38,
"startOffset": 3,
"endOffset": 50
},
"msg": "message number 3"
}
]
}
]
},
{
"key": "my_key3",
"component": "my component 3",
"textRange": {
"startLine": 548,
"endLine": 548,
"startOffset": 14,
"endOffset": 15
},
"flows": [],
"status": "OPEN",
"creationDate": "2022-01-21T17:16:06+0200",
"updateDate": "2022-01-21T17:16:06+0200",
"type": "CODE_SMELL",
"scope": "LOCAL"
}
]
}
And here convert to csv only some flelds:
jq -r '.issues[] | [.key ,.component, .textRange[], .status] | #csv' test.json
And here result:
"1","my_component",35,35,46,84,"OPEN"
"2","my component 2",34,34,3,52,
Nice.
Now I want to convert field (array) flows. I try this:
jq -r '.issues[] | [.key ,.component, .textRange[], .flows[], .status] | #csv' test.json
And here result:
"1","my_component",35,35,46,84,"OPEN"
jq: error (at test.json:91): object ({"locations...) is not valid in a csv row
How fix convert? Array (locations) in array (flows)
The result must be like this (first line are fields's names):
total,created_at,project,issues,key,component,textRange,startLine,endLine,startOffset,endOffset,status,creationDate,updateDate,type,scope,flows,locations,component,textRange,startLine,endLine,startOffset,endOffset,msg
2040,2022-01-27T09:50:59+0200,my project,,key_1,my_component,,35,35,46,84,OPEN,2022-01-24T06:42:58+0200,2022-01-24T06:42:58+0200,BUG,MAIN,,,,,,,,,
,,,,key2,my component 2,,34,34,3,52,,,,,,,,some component,,35,35,3,50,any message
,,,,,,,,,,,,,,,,,,another component,,36,36,3,71,message custom
,,,,,,,,,,,,,,,,,,Alarm.java,,37,37,3,76,message number 2
,,,,,,,,,,,,,,,,,,Alarm.java,,38,38,3,50,message number 3
,,,,my_key3,my component 3,,548,548,14,15,OPEN,2022-01-21T17:16:06+0200,2022-01-21T17:16:06+0200,CODE_SMELL,LOCAL,,,,,,,,,
It's important to show values not only issues but also fields's values: total,created_at,project
Do you want the objects in .flows[].locations[] to be appended as separate rows?
jq -r '.issues[]
| .key as $key | ., .flows[].locations[]
| [$key ,.component, .textRange[], .status]
| #csv
'
"1","my_component",35,35,46,84,"OPEN"
"2","my component 2",34,34,3,52,
"2","some component",35,35,3,50,
"2","another component",36,36,3,71,
"2","Alarm.java",37,37,3,76,
"2","Alarm.java",38,38,3,50,
Demo
Seems you need stringify, then you can use tostring such as
jq -r '.issues[]
| [.key ,.component, .textRange[], (.flows[].locations[]|tostring), .status]
| #csv'
Demo
I request data with axios.get(...), successful get the return as nested objects. I can access the data.message, data.cnt, data.cod but I can't access the properties of the "city". kept getting undefine or typeerror.
I want to access the properties of the city object nested within the response data. like this "data.city.name" but errors.
{
"cod": "200",
"message": 0.0051,
"cnt": 40,
"list": [
{
"dt": 1545318000,
"main": {
"temp": 282.74,
"temp_min": 282.167,
"temp_max": 282.74,
"pressure": 1012.86,
"sea_level": 1020.54,
"grnd_level": 1012.86,
"humidity": 84,
"temp_kf": 0.57
},
"weather": [
{
"id": 500,
"main": "Rain",
"description": "light rain",
"icon": "10d"
}
],
"clouds": { "all": 44 },
"wind": { "speed": 5.67, "deg": 243.503 },
"rain": { "3h": 0.25 },
"sys": { "pod": "d" },
"dt_txt": "2018-12-20 15:00:00"
},
{
"dt": 1545739200,
"main": {
"temp": 282.628,
"temp_min": 282.628,
"temp_max": 282.628,
"pressure": 1037.58,
"sea_level": 1045.29,
"grnd_level": 1037.58,
"humidity": 100,
"temp_kf": 0
},
"weather": [
{
"id": 500,
"main": "Rain",
"description": "light rain",
"icon": "10d"
}
],
"clouds": { "all": 76 },
"wind": { "speed": 2.02, "deg": 212.503 },
"rain": { "3h": 0.13 },
"sys": { "pod": "d" },
"dt_txt": "2018-12-25 12:00:00"
}
],
"city": {
"id": 2643743,
"name": "London",
"coord": { "lat": 51.5073, "lon": -0.1277 },
"country": "GB",
"population": 1000000
}
}
Use JSON.parse();
Example:
var data = JSON.parse(yourJSONObject)
console.log(data.city.name)
May not full represent what you need, but this should show how to set a default state, fetch your results, and then pass them into components. Your object is fine, I feel like you might be trying to access it before Axios completes.
export class DataView extends Component {
constructor(props) {
super(props);
this.state = {
isLoading: true,
data: {}
};
this.defaultState = {
// default properties of state
};
}
componentWillMount() {
this.fetchData();
}
fetchData = () => {
await axios.get( ${apiEndpoint}${inputCity}&appid=${apiKey} )
.then( data => {
this.setState({
data,
...this.defaultState,
isLoading: false
});
});
};
render() {
return (
<Fragment>
{this.state.isLoading ? ( // Check if it's still loading and load nothing if true
<Fragment />
) : (
<Component withProps={this.state.data}></Component> // Data can be accessed if isLoading: false
)}
</Fragment>
);
}
}
if you are getting data.message, then you must get the city name with data.city.name
I think you have some errors in your code because i can access the data and the json is also valid too.
try to console.log() the response and see what's going on.
Seems to work fine with data.city.name
const data = {
"cod": "200",
"message": 0.0051,
"cnt": 40,
"list": [
{
"dt": 1545318000,
"main": {
"temp": 282.74,
"temp_min": 282.167,
"temp_max": 282.74,
"pressure": 1012.86,
"sea_level": 1020.54,
"grnd_level": 1012.86,
"humidity": 84,
"temp_kf": 0.57
},
"weather": [
{
"id": 500,
"main": "Rain",
"description": "light rain",
"icon": "10d"
}
],
"clouds": { "all": 44 },
"wind": { "speed": 5.67, "deg": 243.503 },
"rain": { "3h": 0.25 },
"sys": { "pod": "d" },
"dt_txt": "2018-12-20 15:00:00"
},
{
"dt": 1545739200,
"main": {
"temp": 282.628,
"temp_min": 282.628,
"temp_max": 282.628,
"pressure": 1037.58,
"sea_level": 1045.29,
"grnd_level": 1037.58,
"humidity": 100,
"temp_kf": 0
},
"weather": [
{
"id": 500,
"main": "Rain",
"description": "light rain",
"icon": "10d"
}
],
"clouds": { "all": 76 },
"wind": { "speed": 2.02, "deg": 212.503 },
"rain": { "3h": 0.13 },
"sys": { "pod": "d" },
"dt_txt": "2018-12-25 12:00:00"
}
],
"city": {
"id": 2643743,
"name": "London",
"coord": { "lat": 51.5073, "lon": -0.1277 },
"country": "GB",
"population": 1000000
}
}
document.write(data.city.name);
getData = async () => {
const { inputCity } = this.state;
try {
const data = await axios.get(${apiEndpoint}${inputCity}&appid=${apiKey});
this.setState({ data });
console.log(this.state.data);
return data;
} catch (err) {
console.error(err);
}
}
I have some json from eurostat, which looks like this:
{
"version": "2.0",
"label": "Principaux agrégats des administrations publiques, y compris recettes et dépenses",
"href": "http://ec.europa.eu/eurostat/wdds/rest/data/v2.1/json/fr/gov_10a_main?unit=PC_GDP&na_item=TE§or=S13&time=2008&time=2009&time=2010&time=2011&time=2012&time=2013&time=2014&time=2015&time=2016&time=2017&geo=DE&geo=AT&geo=BE&geo=BG&geo=CY&geo=HR&geo=FI",
"source": "Eurostat",
"updated": "2018-10-26",
"status": {
"57": "b"
},
"extension": {
"datasetId": "gov_10a_main",
"lang": "FR",
"description": null,
"subTitle": null,
"status": {
"label": {
"b": "rupture de série"
}
}
},
"class": "dataset",
"value": {
"0": 49.9,
"1": 54.1,
"2": 52.8,
"3": 50.9,
"4": 51.2,
"5": 51.6,
"6": 52.4,
"7": 51.1,
"8": 50.3,
"9": 49.2,
"10": 50.3,
"11": 54.2,
"12": 53.3,
"13": 54.5,
"14": 55.9,
"15": 55.8,
"16": 55.3,
"17": 53.7,
"18": 53,
"19": 52.2,
"20": 37.1,
"21": 39.4,
"22": 36.2,
"23": 33.8,
"24": 34.5,
"25": 37.7,
"26": 43.1,
"27": 40.5,
"28": 35.1,
"29": 35.1,
"30": 38.4,
"31": 42.1,
"32": 42,
"33": 42.3,
"34": 41.9,
"35": 41.9,
"36": 48.8,
"37": 40.6,
"38": 38,
"39": 37.5,
"40": 43.6,
"41": 47.6,
"42": 47.3,
"43": 44.7,
"44": 44.3,
"45": 44.7,
"46": 44,
"47": 43.7,
"48": 43.9,
"49": 43.9,
"50": 48.3,
"51": 54.8,
"52": 54.8,
"53": 54.4,
"54": 56.2,
"55": 57.5,
"56": 58.1,
"57": 57.1,
"58": 55.9,
"59": 54,
"60": 45.3,
"61": 48.3,
"62": 48,
"63": 48.5,
"64": 47.8,
"65": 47.6,
"66": 48.1,
"67": 48.3,
"68": 46.9,
"69": 45
},
"dimension": {
"unit": {
"label": "unit",
"category": {
"index": {
"PC_GDP": 0
},
"label": {
"PC_GDP": "Pourcentage du produit intérieur brut (PIB)"
}
}
},
"sector": {
"label": "sector",
"category": {
"index": {
"S13": 0
},
"label": {
"S13": "Administrations publiques"
}
}
},
"na_item": {
"label": "na_item",
"category": {
"index": {
"TE": 0
},
"label": {
"TE": "Total des dépenses des administrations publiques"
}
}
},
"geo": {
"label": "geo",
"category": {
"index": {
"AT": 0,
"BE": 1,
"BG": 2,
"CY": 3,
"DE": 4,
"FI": 5,
"HR": 6
},
"label": {
"AT": "Autriche",
"BE": "Belgique",
"BG": "Bulgarie",
"CY": "Chypre",
"DE": "Allemagne (jusqu'en 1990, ancien territoire de la RFA)",
"FI": "Finlande",
"HR": "Croatie"
}
}
},
"time": {
"label": "time",
"category": {
"index": {
"2008": 0,
"2009": 1,
"2010": 2,
"2011": 3,
"2012": 4,
"2013": 5,
"2014": 6,
"2015": 7,
"2016": 8,
"2017": 9
},
"label": {
"2008": "2008",
"2009": "2009",
"2010": "2010",
"2011": "2011",
"2012": "2012",
"2013": "2013",
"2014": "2014",
"2015": "2015",
"2016": "2016",
"2017": "2017"
}
}
}
},
"id": [
"unit",
"sector",
"na_item",
"geo",
"time"
],
"size": [
1,
1,
1,
7,
10
]
}
I would like to produce a csv file.
First, I need to concatenate .status with .value by string (sorry for my poor json knowledge) --> "status":{"57": "b"} with "value":{"57": 57.1}.
Second, I need to produce the same table as the original one (downloaded from eurostat).
I try many jq commands, like:
.status,.value | to_entries
I'm far from finding a solution.
Any help? I think map or map_values/group_by command are needed, but I don't really understand these functions.
EDIT :
I download data from eurostat.
I use their web service here, where I can download data in json format.
I would like to reproduce in shell same table as original, with jq. In my exemple, it should look like :
GEO/TIME,2010,2011,2012,2013,2014,2015,2016,2017
Belgique,"53,3","54,5","55,9","55,8","55,3","53,7","53,0","52,2"
Bulgarie,"36,2","33,8","34,5","37,7","43,1","40,5","35,1","35,1"
"Allemagne (jusqu'en 1990, ancien territoire de la RFA)","47,3","44,7","44,3","44,7","44,0","43,7","43,9","43,9"
Croatie,"48,0","48,5","47,8","47,6","48,1","48,3","46,9","45,0"
Chypre,"42,0","42,3","41,9","41,9","48,8","40,6","38,0","37,5"
Finlande,"54,8","54,4","56,2","57,5","58,1","57,1","55,9","54,0"
But json contain metadata, and Finlande must have 57,1b value.
I hope it's more clear with this edit.
And many thanks for your help.
Your question doesn't indicate very precisely what output you want, but hopefully you'll be able to adapt the following:
.value as $dict
| .status
| to_entries
| map( [.key, .value, $dict[.key]] )
| .[]
| #csv
With your input, and invoking jq with the -r option, this produces:
"57","b",57.1
I'm trying to access to a JSON array returned by a OpenWeatherMap request.
The JSON data is:
{ "city":
{ "id": 3171457, "name": "New York", "coord": { "lon": 0.32898, "lat": 4.802662 }, "country": "US", "population": 0, "sys": { "population": 0 } },
"cod": "200", "message": 0.0317, "cnt": 40,
"list": [ { "dt": 1483552800, "main": { "temp": 277.28, "temp_min": 275.705, "temp_max": 277.28, "pressure": 1013.85, "sea_level": 1021.42, "grnd_level": 1013.85, "humidity": 93, "temp_kf": 1.57 }, "weather": [ { "id": 800, "main": "Clear", "description": "clear sky", "icon": "01n" } ], "clouds": { "all": 0 }, "wind": { "speed": 4.74, "deg": 269.002 }, "sys": { "pod": "n" }, "dt_txt": "2017-01-04 18:00:00" },
...
If i use in a template {{payload.list}} I see a list of [object Object]s.
But if I use {{payload.list[0]}}, or {{payload.list[0].main}}, I see nothing - my guess was a single [object Object].
How can i access to the first member and inner members?
The template node uses the mustache format, documented here: https://mustache.github.io/mustache.5.html
To access array values, you should use the following syntax:
{{ payload.list.0 }}
and
{{ payload.list.0.main }}
I've written the following groovy script in SOAPUI to execute an assertion on a JSON response.
I'm having difficulty writing the assertion to extract and assert on Weather > main > Clouds property and value of the JSON response.
Can someone please assist with correcting my code to extract the value I want?
Thanks!
import groovy.json.JsonSlurper
def json = '''{
"coord": {
"lon": -0.13,
"lat": 51.51
},
"weather": [
{
"id": 801,
"main": "Clouds",
"description": "few clouds",
"icon": "02n"
}
],
"base": "stations",
"main": {
"temp": 281.644,
"pressure": 1027.43,
"humidity": 100,
"temp_min": 281.644,
"temp_max": 281.644,
"sea_level": 1035.14,
"grnd_level": 1027.43
},
"wind": {
"speed": 3.33,
"deg": 43.5005
},
"clouds": {
"all": 12
},
"dt": 1476231232,
"sys": {
"message": 0.0084,
"country": "GB",
"sunrise": 1476253200,
"sunset": 1476292372
},
"id": 2643743,
"name": "London",
"cod": 200
}'''
def result = new JsonSlurper().parseText(json)
log.info(result)
assert result.weather.main == "Clouds"
Weather is an array of maps, as I see. So, you need to select an item or groovy will return to you an array of main.
assert result.weather.first().main == "Clouds"
assert result.weather.main == ["Clouds"]
weather in your json is array. You can access it elements as a regular array
assert result.weather.main[0] == "Clouds"
assert result?.weather?.main?.getAt(0) == "Clouds"
second prefered because it null safe
Looks it's a trivial issue.
weather is an array(in []) and that is why the assertion fails.
"weather": [
{
"id": 801,
"main": "Clouds",
"description": "few clouds",
"icon": "02n"
}
],
If you do result.weather.main, then it returns a list which contains an element Clouds. But,not a single value as you expected.
So, you can do:
assert result.weather[0].main == 'Clouds', 'Not matching the expected result' or
assert result.weather.main == ['Clouds'] or
assert result.weather.main.contains('Clouds')
For suppose if weather is below (example with more elements in json array):
"weather": [
{
"id": 801,
"main": "Clouds",
"description": "few clouds",
"icon": "02n"
},
{
"id": 802,
"main": "CloudApps",
"description": "few clouds",
"icon": "03n"
}
],
Then assertion can be done
assert result.weather.main == ['Clouds', 'CloudApps']