I'm fairly new to PowerShell and I am trying to convert this JSON file to CSV.
Using some simple code like this:
$sourceFilePath = "Records.json"
$destinationFilePath = "Records.csv"
((Get-Content -Path $sourceFilePath -Raw) | ConvertFrom-Json) | Export-CSV $destinationFilePath -NoTypeInformation
Results in:
Because I am getting System.Object[] I think I need to do some kind of expansion or "ForEach" of the contact_ids, site_ids, first name aliases, and aliases. I've done some searching and I can't seem to wrap my head around the examples I've been given. Also, "first name aliases" has spaces in it that add to the complexity.
I don't have any control over the source JSON. Here it is as an example:
[
{
"mdm id" : "947b2a12-3aac-480a-a0bf-626aff106f48",
"first name" : "Claude",
"last name" : "Bones",
"contact_ids" : [
"CTP70499"
],
"site_ids" : [
"5015"
],
"first name aliases" : [
"Claude"
],
"aliases" : [
"Claude Bones"
],
"createdDate" : "2020-06-03T19:59:08Z",
"updatedDate" : "2020-06-03T20:48:27Z",
"veevaID" : "V0B000000000569"
},
{
"mdm id" : "bce21b05-0b28-4ebb-a34d-761c1a397821",
"first name" : "Daniel",
"last name" : "Smith",
"contact_ids" : [
"CTP699"
],
"site_ids" : [
"5015"
],
"first name aliases" : [
"Dan",
"Danial",
"Danne",
"Danny",
"Daniel"
],
"aliases" : [
"Daniel Smith"
],
"createdDate" : "2020-06-03T19:59:08Z",
"updatedDate" : "2020-06-03T20:48:27Z",
"veevaID" : "V0B000000000566"
}
]
The values that are arrays will need to be combined into a single value. Regardless of how many values it may have, if it's designated as an array [] in json, it will need to be manipulated. There are several articles and custom functions written on the internet. Your example could be handled with this bit of code.
$JSONdata = #'
[
{
"mdm id" : "947b2a12-3aac-480a-a0bf-626aff106f48",
"first name" : "Claude",
"last name" : "Bones",
"contact_ids" : [
"CTP70499"
],
"site_ids" : [
"5015"
],
"first name aliases" : [
"Claude"
],
"aliases" : [
"Claude Bones"
],
"createdDate" : "2020-06-03T19:59:08Z",
"updatedDate" : "2020-06-03T20:48:27Z",
"veevaID" : "V0B000000000569"
},
{
"mdm id" : "bce21b05-0b28-4ebb-a34d-761c1a397821",
"first name" : "Daniel",
"last name" : "Smith",
"contact_ids" : [
"CTP699"
],
"site_ids" : [
"5015"
],
"first name aliases" : [
"Dan",
"Danial",
"Danne",
"Danny",
"Daniel"
],
"aliases" : [
"Daniel Smith"
],
"createdDate" : "2020-06-03T19:59:08Z",
"updatedDate" : "2020-06-03T20:48:27Z",
"veevaID" : "V0B000000000566"
}
]
'# | ConvertFrom-Json
$JSONdata | foreach {
$record = [ordered]#{}
foreach($property in $_.psobject.Properties)
{
if($property.value -is [string])
{
$record.Add($property.name,$property.value)
}
else
{
$record.Add($property.name,($property.value -join ', '))
}
}
[PSCustomObject]$record
} | ConvertTo-Csv -NoTypeInformation
Output
"mdm id","first name","last name","contact_ids","site_ids","first name aliases","aliases","createdDate","updatedDate","veevaID"
"947b2a12-3aac-480a-a0bf-626aff106f48","Claude","Bones","CTP70499","5015","Claude","Claude Bones","2020-06-03T19:59:08Z","2020-06-03T20:48:27Z","V0B000000000569"
"bce21b05-0b28-4ebb-a34d-761c1a397821","Daniel","Smith","CTP699","5015","Dan, Danial, Danne, Danny, Daniel","Daniel Smith","2020-06-03T19:59:08Z","2020-06-03T20:48:27Z","V0B000000000566"
Simply change it to Export-Csv
Related
I have a JSON file:
{
"JSONS" : [
{
"id" : "ToRemove",
"First" : [
{
"id" : "geo",
"Name" : "Person1",
"model" : [
],
"adjustments" : [
{
"uid" : "3",
"name" : "4s",
"value" : "1"
},
{
"uid" : "5",
"name" : "3s",
"value" : "6"
}
]
},
{
"id" : "Meters",
"Dictionary" : "4.2"
},
{
"id" : "Moon",
"Filter" : "0.5",
"Saturn" : {
"s" : "0",
"v" : "1"
}
}
]
}
]
}
I would like to delete entire node, if the "id", in this example, contains "ToRemove" string. Everyting between { and }, including those lines also, to make the final JSON consistent.
This is a screenshot what I want to get rid of.
I only found how to delete properties, but not entire nodes. I've tried to appli something like this:
$ToRemove = Get-Content $SourceFile | ConvertFrom-Json
$ToRemove.PSObject.Object.Remove('id:','ToRemove')
$ToRemove | ConvertTo-Json -Depth 100 | Out-File $DestFile
but of course it didn't work.
How to delete the entire node? I would love to use an array to put all strings I would like to delete.
Based on your comment, you can remove that object having the property id = ToRemove by filtering where id is not equal to ToRemove and assigning that result to the .JSONS property:
$json = Get-Content path\to\json.json -Raw | ConvertFrom-Json
$json.JSONS = #($json.JSONS.Where{ $_.id -ne 'ToRemove' })
$json | ConvertTo-Json
The end result in this case would be an empty array for the .JSONS property:
{
"JSONS": []
}
.PSObject.Properties.Remove(...) wouldn't be useful in this case because what it does is remove properties from one object but what you want to do is filter out an entire object based on a condition.
You should be able to use just plain PowerShell, like this:
{
"JSONS" : [
{
"id" : "ToRemove",
"First" : [
{
"id" : "geo",
"Name" : "Person1",
"model" : [
]
},
{
"id" : "Meters",
"Dictionary" : "4.2"
}
]
},
{
"id" : "DontRemove",
"First" : []
}
]
}
$json = Get-Content -Path $SourceFile | ConvertFrom-Json
$json.JSONS = $json.JSONS | Where-Object { $_.Id -ne "ToRemove" }
$json | ConvertTo-Json -Depth 100 | Out-File -Path $DestFile
I wish to reduce with jq
[ {
"context" : "app:swagger,dev:8080",
"parent" : null,
"beans" : [ {
"bean" : "app",
"aliases" : [ ],
"scope" : "singleton",
"type" : "com.example.App",
"resource" : "null",
"dependencies" : [ "environment" ]
}, {
"bean" : "environment",
"aliases" : [ ],
"scope" : "singleton",
"type" : "com.example.Environment",
"resource" : "null",
"dependencies" : [ ]
}
},
...
}]
to
app --> environment
...
The problem statement appears to be under-specified, but the following seems to be either a solution or very close to one:
jq -r '.[] | .beans[] | "\(.bean) --> \(.dependencies[])"' input.json
I am working with a JSON file similar to the one below:
{ "Response" : {
"TimeUnit" : [ 1576126800000 ],
"metaData" : {
"errors" : [ ],
"notices" : [ "query served by:1"]
},
"stats" : {
"data" : [ {
"identifier" : {
"names" : [ "apiproxy", "response_status_code", "target_response_code", "target_ip" ],
"values" : [ "IO", "502", "502", "7.1.143.6" ]
},
"metric" : [ {
"env" : "dev",
"name" : "sum(message_count)",
"values" : [ 0.0]
} ]
} ]
} } }
My object is to display a mapping of the identifier and values like :
apiproxy=IO
response_status_code=502
target_response_code=502
target_ip=7.1.143.6
I have been able to parse both names and values with
.[].stats.data[] | (.identifier.names[]) and .[].stats.data[] | (.identifier.values[])
but I need help with the jq way to map the values.
The whole thing can be done in jq using the -r command-line option:
.[].stats.data[]
| [.identifier.names, .identifier.values]
| transpose[]
| "\(.[0])=\(.[1])"
I have the following sample data in MongoDB:
{
"_id" : ObjectId("54833e93ade1a1521a2a2fe8"),
"fname" : "yumi",
"mname" : "sakura",
"lname" : "kirisaki",
"consultations" : [
{
"medications" : [
"paracetamol",
"ibuprofen",
"carbocisteine"
],
"diagnosis" : [
"sore throat",
"fever",
"cough"
],
"date" : ISODate("2014-12-01T16:00:00Z")
},
{
"medications" : [
"paracetamol",
"carbocisteine",
"afrin"
],
"diagnosis" : [
"cough",
"colds",
"fever"
],
"date" : ISODate("2014-12-11T16:00:00Z")
}
]
}
{
"_id" : ObjectId("54833e93ade1a1521a2a2fe9"),
"fname" : "james",
"mname" : "legaspi",
"lname" : "reyes",
"consultations" : [
{
"medications" : [
"zanamivir",
"ibuprofen",
"paracetamol"
],
"diagnosis" : [
"influenza",
"body aches",
"headache"
],
"date" : ISODate("2014-10-22T16:00:00Z")
},
{
"medications" : [
"carbocisteine",
"albuterol",
"ibuprofen"
],
"diagnosis" : [
"asthma",
"cough",
"headache"
],
"date" : ISODate("2014-11-13T16:00:00Z")
}
]
}
I am trying to query patients with zanamivir AND ibuprofen AND cough:
db.patient.find({
$and:
[
{"consultations.medications":["zanamivir", "ibuprofen"]},
{"consultations.diagnosis":"cough"}
]
}).pretty()
So, in the short sample data, I was hoping james would be returned since he is the only one with zanamivir medication.
Nothing is happening when I enter the above query in cmd. It just goes to the next line (no syntax errors, etc.)
How must I go about the query?
You need the use the $all operator.
db.patient.find({
"consultations.medications": { "$all" : [ "zanamivir", "ibuprofen" ]},
"consultations.diagnosis": "cough"
})
Pretty simple, it's just your first part of the query.
db.patient.find({
$and:[
{"consultations.medications":["zanamivir", "ibuprofen"]},
{"consultations.diagnosis":"cough"}]})
Asking Mongodb to find consultations.medications against ["zanamivir", "ibuprofen"] is asking it to find someone whose medications are equal to ['zanamivir', 'ibuprofen'].
If you want to find people who have had zanamivir and ibuprofen medicated you need to tweak the query to this:
db.patient.find({
$and:[
{"consultations.medications":"zanamivir"},
{"consultations.medications":"ibuprofen"},
{"consultations.diagnosis":"cough"}]})
Enjoy!
On the Actor/Movie demo graph, cypher returns column names in a separate array.
MATCH (n:Person) RETURN n.name as Name, n.born as Born ORDER BY n.born LIMIT 5
results:
{ "columns" : [ "Name", "Born" ], "data" : [ [ "Max von Sydow", 1929 ], [ "Gene Hackman", 1930 ], [ "Richard Harris", 1930 ], [ "Clint Eastwood", 1930 ], [ "Mike Nichols", 1931 ] ]}
Is it possible to get each node properties tagged instead?
{ "nodes" : [ ["Name": "Max von Sydow", "Born": 1929 ], ...] }
If I return the node instead of selected properties, I get way too many properties.
MATCH (n:Person) RETURN n LIMIT 5
results:
{ "columns" : [ "n" ], "data" : [ [ { "outgoing_relationships" : "http://localhost:7474/db/data/node/58/relationships/out", "labels" : "http://localhost:7474/db/data/node/58/labels", "data" : { "born" : 1929, "name" : "Max von Sydow" }, "all_typed_relationships" : "http://localhost:7474/db/data/node/58/relationships/all/{-list|&|types}", "traverse" : "http://localhost:7474/db/data/node/58/traverse/{returnType}", "self" : "http://localhost:7474/db/data/node/58", "property" : "http://localhost:7474/db/data/node/58/properties/{key}", "outgoing_typed_relationships" : "http://localhost:7474/db/data/node/58/relationships/out/{-list|&|types}", "properties" : "http://localhost:7474/db/data/node/58/properties", "incoming_relationships" : "http://localhost:7474/db/data/node/58/relationships/in", "extensions" : { }, "create_relationship" : "http://localhost:7474/db/data/node/58/relationships", "paged_traverse" : "http://localhost:7474/db/data/node/58/paged/traverse/{returnType}{?pageSize,leaseTime}", "all_relationships" : "http://localhost:7474/db/data/node/58/relationships/all", "incoming_typed_relationships" : "http://localhost:7474/db/data/node/58/relationships/in/{-list|&|types}" } ], ... ]}
You can use the new literal map syntax in Neo4j 2.0 and do something like:
MATCH (n:Person)
RETURN { Name: n.name , Born: n.born } as Person
ORDER BY n.born
LIMIT 5