Powershell - Create array of arrays for Google sheets append api - json

I am tryign to use this google sheets append api to update values in bulk. I've used it the past for single rows updates and thats been fine , but i have a large data set I want to append in chunks. Using the
https://sheets.googleapis.com/v4/spreadsheets/$MYSHEETID/values/ExternallyExposed!A1%3AB10:append?insertDataOption=INSERT_ROWS&responseValueRenderOption=UNFORMATTED_VALUE&valueInputOption=RAW
Using the API explorer this is the expected JSON that google expects, note that Values is an array of arrays. Im having difficulty with the values array of arrays section. I cant seem to get it to just be
values : [ [a,b],[c,d],[e,f] ]
i just keep getting something like values : [a,b], values: [c,d]
{
"majorDimension": "ROWS",
"range": "ExternallyExposed!A1:B10",
"values": [
[
"a",
"b"
],
[
"e",
"f"
],
[
"g",
"h"
]
]
}
I've tried various things like
$x = #(
,#("a","b"),
,#("c","D"),
,#("e","F")
) | convertto-json does not produce the expected results.
or
$values= #()
$values+= [pscustomobject]#{
values = #(
"data"
"data"
)
}
gives me
{
"values": [
"data",
"data",
"data"
]
},
{
"values": [
"data",
"data",
"data"
]
}
]

I got help elsewhere, just posting the answer incase others have the same issue.
$values = ("a","b"),("c","d"),("e","f")
$build = [PSCustomObject]#{
'majorDimension' = 'ROWS'
'range' = 'ExternallyExposed!A1:B10'
'values' = $values
}
$json = $build | ConvertTo-Json
The values can also be built this way iterating though a loop.
foreach ($item in $items)
{
$values += ,("a","b")
}

Related

Updating Nested JSON Array with new key and value from another key

I have have a JSON file where I have IDs with tasks. Some tasks can be empty. I want to put the ID into the tasks where tasks are not empty.
[
{
"id": 1961126,
"tasks": [
{
"id": 70340700,
"title": "Test1",
},
{
"id": 69801130,
"title": "Test15A",
}
]
},
{
"id": 1961126,
"tasks": []
}
]
I would like to get the tasks array updated to look like
[
{
"id": 1961126,
"tasks": [
{
**"sId":1961126,**
"id": 70340700,
"title": "Test1",
},
{
**"sId":1961126,**
"id": 69801130,
"title": "Test15A",
}
]
},
{
"id": 1961126,
"tasks": []
}
]
I can't figure out how to get the id from the object into the nested array. Here is what I have come up with
jq 'map(.tasks[0]|select( . != null )|.sId = .id)' file.json
This is only pulling in the same id. I have tired to put in [].id but I get a error Cannot index number with string "id". I am still learning how to deal with nested arrays and objects.
Save the ID in a variable and add it as a new field to each array member.
jq 'map(.id as $sId | .tasks[] += {$sId})' file.json
Demo
Note #1: Get rid of the final , within each object (see the Demo), as it's not proper JSON.
Note #2: Object fields generally have no order, but if you want to have the propagated ID shown first, as seen in your expected output, you could try to replace += {$sId} (which by itself is shorthand for |= . + {$sId}) with |= {$sId} + . to flip the order of generation (Demo). Although there is no guarantee that it stays that way with further processing.

Replace column value with null in json file using Powershell

Here is how my json file looks like :
{
"count": 12,
"name": "Daily Ticket",
"columnNames": [
"User",
"Channel",
"Date",
"# of Closed Incidents",
"Open",
"Response",
"Remark",
"Closed"
],
"rows": [
[
"abc",
"Service Web",
"\u00272020-06-13 00:00:00\u0027",
"1",
"0",
"0",
"this is a text,please replace with null",
"1"
],
[
"xyz",
"Email",
"\u00272020-06-13 00:00:00\u0027",
"21",
"1",
"0",
"this is a text,please replace with null",
"7"
]
]
}
I want to replace all the values in columns of Remark with null and convert into a csv file using powershell. Please help to achieve this.
I want column names as header and rows as rows separated with comma in csv.
My output csv file should look like below one:
User,Channel,Date,# of Closed Incidents,Open,Response,Remark,Closed
abc,Service Web,\u00272020-06-13 00:00:00\u0027,1,0,0,,1
xyz,Email,\u00272020-06-13 00:00:00\u0027,1,0,0,,1
To convert this json into a CSV file is not that difficult.
Just load the JSON, convert it into an object and loop through the properties building an array of new objects you can save as CSV:
$json = Get-Content -Path 'D:\Test\DailyTicket.json' -Raw | ConvertFrom-Json
$headers = $json.columnNames
$result = foreach ($row in $json.rows) {
# just a precaution to not run into index errors when there are
# more items in the array than there are headers or vice-versa
$items = [math]::Min($row.Count, $headers.Count)
# create a new empty (ordered) hashtable
$hash = [ordered]#{}
for ($i = 0; $i -lt $items; $i++) {
# fill the hashtable, except for iten 'Remark'
$hash[$headers[$i]] = if ($headers[$i] -ne 'Remark') { $row[$i] } else { $null }
}
# If you insist on keeping the apostrophe characters in the date field in unicode format `\u0027`
# $hash['Date'] = $hash['Date'] -replace "'", '\u0027'
# output a PSObject to be collected in array $result
[PsCustomObject]$hash
}
# output on screen
$result | Format-Table -AutoSize
# output to CSV file
$result | Export-Csv -Path 'D:\Test\DailyTicket.csv' -NoTypeInformation
Resulting CSV file:
"User","Channel","Date","# of Closed Incidents","Open","Response","Remark","Closed"
"abc","Service Web","'2020-06-13 00:00:00'","1","0","0",,"1"
"xyz","Email","'2020-06-13 00:00:00'","21","1","0",,"7"

Adding array objects to a JSON with a property name in powershell

I've to retrieve products and it's details and save it together in a json file. Unfortunately the name and details cannot be retrieved from the same API, I gotta use 2 different API. So I run a code inside a for loop to retrieve the list of product names as well as its details as follows:
For($i=0; $i -lt $cart.total; $i++)
{
$query = 'dummy value with the cart number'
$productName = Invoke-RestMethod -Uri 'https://abc' -Body $query -Method Post
$details = Invoke-RestMethod -Uri 'https://xyz' -Body $query -Method Post
}
All this works, and below is sample data in $productname and $details.
$productName
"iPhone"
$details
"data":[
{"Moduleid": "1", "propertyName": "camera", "value": "12MP"},
{"Moduleid" : "43", "propertyName":"battery", "value": "4000MAj"}
]
However what I am trying to do is to save the $productName and $details into a json as follows:
{
"iPhone 7":[
{"Moduleid": "1", "propertyName": "camera", "value": "12MP"},
{"Moduleid" : "423", "propertyName":"battery", "value": "3000MAh"}
],
"S10+": [
{"Moduleid": "21", "propertyName": "camera", "value": "12MP"},
{"Moduleid" : "43", "propertyName":"battery", "value": "4000MAh"}
]
}
I've tried several ways but couldn't reach any conclusion. Please let me know how can these to variable datas be put into a json format as shown above.
Thank you
The question doesn't make clear what each query is returning. Generally speaking, you need to convert the return JSON from both queries into objects then merge the objects into the objects you really want. The run those objects through ConvertTo-Json which should do the trick. The source object structure is what results in the JSON text etc...
It would look something like below, but again, I have to guess at the starting point, for example I'm sure that $productName isn't a flat array, but is itself JSON.
# For test
$productName = "iPhone","S10+"
# For test
$details =
#"
{
"Data": [{
"Moduleid": "1",
"propertyName": "camera",
"value": "12MP"
},
{
"Moduleid" : "43",
"propertyName": "battery",
"value": "4000MAj"
}]
}
"# |
ConvertFrom-Json
$Results =
For( $i = 0; $i -lt $productName.count; ++$i )
{
[PSCustomObject]#{
$productName[$i] = $details.data[$i]
}
}
$Results | ConvertTo-Json
This would give back:
[
{
"iPhone": {
"Moduleid": "1",
"propertyName": "camera",
"value": "12MP"
}
},
{
"S10+": {
"Moduleid": "43",
"propertyName": "battery",
"value": "4000MAj"
}
}
]
Not quite what you're looking for, but should at least demonstrate the point.
Note: Normally I would look for a common property between the 2
returns to relate the data. Absent that I used the array index. Again
part of the reason this doesn't look right.
If you can please edit the question to include the return JSON from both queries, then we can refine the above.
I created a hashtable like:
$hashTable = [HashTable]::New(0, [StringComparer]::Ordinal)
This is so that the keys are case sensitive. Normally if you create it as #{}, it won't be case sensitive.
Then i mapped the key and value to the hash table as follows,
$hashTable.Add($productName.data.name,$details.data)
Now, I convert it to Json as I wanted it using ConvertTo-Json
$hashTable| ConvertTo-Json | Out-File $outputPath

Create a new json string from jq output elements

My jq command returns objects in brackets but without comma separators. But I would like to create a new json string from it.
This call finds all elements of arr that have a FooItem in them and then returns texts from the nested array at index 3:
jq '.arr[] | select(index("FooItem")) | .[3].texts'
on this json (The original has more elements ):
{
"arr": [
[
"create",
"w199",
"FooItem",
{
"index": 0,
"texts": [
"aBarfoo",
"avalue"
]
}
],
[
"create",
"w200",
"NoItem",
{
"index": 1,
"val": 5,
"hearts": 5
}
],
[
"create",
"w200",
"FooItem",
{
"index": 1,
"texts": [
"mybarfoo",
"bValue"
]
}
]
]
}
returns this output:
[
"aBarfoo",
"avalue"
]
[
"mybarfoo",
"bValue"
]
But I'd like to create a new json from these objects that looks like this:
{
"arr": [
[
"aBarfoo",
"avalue"
],
[
"mybarfoo",
"bValue"
]
]
}
Can jq do this?
EDIT
One more addition: Considering that texts also has strings of zero length, how would you delete those/not have them in the result?
"texts": ["",
"mybarfoo",
"bValue",
""
]
You can always embed a stream of (zero or more) JSON entities within some other JSON structure by decorating the stream, that is, in the present case, by wrapping the STREAM as follows:
{ arr: [ STREAM ] }
In the present case, however, we can also take the view that we are simply editing the original document, and accordingly use a variation of the map(select(...)) idiom:
.arr |= map( select(index("FooItem")) | .[3].texts)
This latter approach ensures that the context of the "arr" key is preserved.
Addendum
To filter out the empty strings, simply add another map(select(...)):
.arr |= map( select(index("FooItem"))
| .[3].texts | map(select(length>0)))

How to use jq to reconstruct complete contents of json file, operating only on part of interest?

All the examples I've seen so far "reduce" the output (filter out) some part. I understand how to operate on the part of the input I want to, but I haven't figured out how to output the rest of the content "untouched".
The particular example would be an input file with several high level entries "array1", "field1", "array2", "array3" say. Each array contents is different. The specific processing I want to do is to sort "array1" entries by a "name" field which is doable by:
jq '.array1 | sort_by(.name)' test.json
but I also want this output as "array1" as well as all the other data to be preserved.
Example input:
{
"field1": "value1",
"array1":
[
{ "name": "B", "otherdata": "Bstuff" },
{ "name": "A", "otherdata": "Astuff" }
],
"array2" :
[
array2 stuff
],
"array3" :
[
array3 stuff
]
}
Expected output:
{
"field1": "value1",
"array1":
[
{ "name": "A", "otherdata": "Astuff" },
{ "name": "B", "otherdata": "Bstuff" }
],
"array2" :
[
array2 stuff
],
"array3" :
[
array3 stuff
]
}
I've tried using map but I can't seem to get the syntax correct to be able to handle any type of input other than the array I want to be sorted by name.
Whenever you use the assignment operators (=, |=, +=, etc.), the context of the expression is kept unchanged. So as long as your top-level filter(s) are assignments, in the end, you'll get the rest of the data (with your changes applied).
In this case, you're just sorting the array1 array so you could just update the array.
.array1 |= sort_by(.name)