Powershell Nested Json doesn't look like a Json - json

I am currently trying to prepare a JSON body for an API call, which should look something like this
curl -XPOST -H 'Authorization: Bearer ***API*KEY***' -H 'Content-Type: application/json' http://127.0.0.1:9000/api/alert -d '{
"title": "Other alert",
"description": "alert description",
"type": "external",
"source": "instance1",
"sourceRef": "alert-ref",
"severity": 3,
"tlp": 3,
"artifacts": [
{ "dataType": "ip", "data": "127.0.0.1", "message": "localhost" },
{ "dataType": "domain", "data": "thehive-project.org", "tags": ["home", "TheHive"] },
],
"caseTemplate": "external-alert"
}'
The problem however is that my json body which I create with powershell has weird characters and don't see where the problem is. This is my JSON Body
{
"tlp": 1,
"source": "Test",
"title": "Test Alert1",
"artifacts": "{\r\n \"dataType\": \"ip\",\r\n \"data\": \"127.0.0.1\"\r\n}",
"type": "external",
"sourceRef": "1",
"description": "Test",
"severity": 1
}
I create this JSON body as follows. I have already tried CustomObjects and Hashtables, but always get the same output
$body = #{
title = "$title"
description = "$Alert_Description"
type ="external"
source ="$Source"
sourceRef ="$SourceRef"
severity = $Severity
tlp = $tlp
$artifacts = [PSCustomObject]#{
dataType=ip
data=127.0.0.1}
}| ConvertTo-Json
$JsonBody = $body | ConvertTo-Json
Can somebody give me a hint? I have no clue anymore

First, fix your hashtable so that it creates the intended JSON representation, and only apply ConvertTo-Json once:
$body = [ordered] #{
title = $title
description = $Alert_Description
type = 'external'
source = $Source
sourceRef = $SourceRef
severity = $Severity
tlp = $tlp
artifacts = , [pscustomobject] #{
dataType = 'ip'
data = '127.0.0.1'
}
} | ConvertTo-Json
Variables don't need enclosing in "..." unless you explicitly want to convert their values to strings.
Literal textual information such as ip does require quoting ('...', i.e. a verbatim string literal, is best).
The artifacts property is an array in your sample JSON, so the unary form of ,, the array constructor operator is used to wrap the [pscustomobject] instance in a single-element array.
If you have multiple objects, place , between them; if the array was constructed previously and stored in a variable named, say, $arr, use artifacts = $arr.
Use of [ordered], i.e. the creation of an ordered hashtable isn't strictly necessary, but makes it easier to compare the resulting JSON representation to the input hashtable.
Generally keep in mind that explicit use of a -Depth argument with ConvertTo-Json is situationally required in order to ensure that properties aren't truncated - see this post for more information. However, with the hashtable in your question this happens not to be a problem.
Second, since you're sending the json to an external program, curl (note that in Windows PowerShell you'd have to use curl.exe in order to bypass the built-in curl alias for Invoke-WebRequest, an additional escaping step is - unfortunately - required up to at least v7.1 - though that may change in v7.2: The " instances embedded in the value of $body must manually be escaped as \", due to a long-standing bug in PowerShell:
curl -XPOST -H 'Authorization: Bearer ***API*KEY***' -H 'Content-Type: application/json' `
http://127.0.0.1:9000/api/alert -d ($body -replace '([\\]*)"', '$1$1\"')
Note: With your particular sample JSON, -replace '"', '\"' would do.
See this answer for more information.

Slap a "-Depth 50" on your ConvertTo-Json, that should solve your issue.
Like here: ConvertTo-Json flattens arrays over 3 levels deep

Related

Non variable recognition on Syspass API Bash Script

When i input a variable on the curl with the json indexed, it takes the string value of the variable, if i scaped the double-quotes, it returns me a sintax error because of in a json request u have to input the data with double-quotes.
Example:
#!/bin/bash
token="x"
tokenPass="x"
name="prueba"
url="https://prueba.prueba.prueba"
user="prueba"
pass="prueba"
notes="prueba"
curl -k -H "Content-Type:Application/json" -d '{"jsonrpc": "2.0", "method": "account/create", "params": { "authToken": "$token", "tokenPass": "$tokenPass", "name": "$name" , "categoryId": "7", "clientId": "9","login":"$user","url": "$url" ,"pass": "$pass","notes": "$notes"}, "id": 1}' -o- https://syspass.prueba.es/api.php
The curl json request works if i input the data manually, but with the variables, when i create the account, the account is named as the string value of the variable, i mean, if the variable is named $name , the account created is name $name. Any help please?
Also i tried to input the variable: "${variable}" and either works
Try this:
curl -k -H "Content-Type:Application/json" -d '{"jsonrpc": "2.0", "method": "account/create", "params": { "authToken": "'${token}'", "tokenPass": "'${tokenPass}'", "name": "'${name}'" , "categoryId": "7", "clientId": "9","login":"'${user}'","url": "'${url}'" ,"pass": "'${pass}'","notes": "'${notes}'"}, "id": 1}' -o- https://syspass.prueba.es/api.php
A bit of explanation:
Text that is between single quotes ' will not be interpreted by bash and that is why the variables will not be inserted. You have to break the string up...
An other suggestion is that you should use curly braces around variables: When do we need curly braces around shell variables?
I explain it here for the variable token. The other variables can be treated in the same way.
Assuming that the variable token contains the string foo bar baz, then you want curl to receive as its 5th argument the following string:
{"jsonrpc": "2.0", "method": "account/create", "params": { "authToken": "foo bar baz", "tokenPass":... rest omitted for brevity}
Since this has to become a single argument, you have to quote the hole string somehow, and because this string contains many double-quotes, it is easier to use single quotes for wrapping. However, this would prevent parameter expansion. To make this happen, the variables must be outside the single quotes. However, they must also be inside double quotes, lest you become a victim of word splitting, if the variables contain spaces. This yields the following solution:
curl .... '{"jsonrpc": "2.0", "method": "account/create", "params": { "authToken": "'"$token"'", "tokenPass": ... rest omitted for brevity}'
Note the "'" in front of $token: The first double quote is the one which will be part of the expanded "foo bar baz". It must be literal, because curl wants to see it. The next single quote terminates the initial single quote. The next double quote tells the shell to expand $token, but without attempting word splitting.
If there are many parameters to expand, the resulting expression is difficult to debug and maintain. An alternative is to set up a HERE-document:
# Set up Parameter 5 for curl
read p5 <<END
{"jsonrpc": "2.0", "method": "account/create", "params": { "authToken": "$token", "tokenPass":... rest omitted for brevity}
END
and then use it as
curl .... -d "$p5"
This requires the separated step of setting up the argument, but in the end is much more readable, and you don't have to worry about word splitting either.
I'd use jq to generate the JSON string (mostly, to handle the quoting for you, JSON strings are not trivial), and for readability (and for the sanity of the next maintainer) add some newlines and indentation:
data=$(
jq -n \
--arg token "$token" \
--arg tokenPass "$tokenPass" \
--arg name "$name" \
--arg url "$url" \
--arg user "$user" \
--arg pass "$pass" \
--arg notes "$notes" \
'{
jsonrpc: "2.0",
method: "account/create",
params: {
authToken: $token,
tokenPass: $tokenPass,
name: $name,
categoryId: "7",
clientId: "9",
login: $user,
url: $url ,
pass: $pass,
notes: $notes
},
id: 1
}'
)
curl -k \
-H "Content-Type:Application/json" \
-d "$data" \
-o- \
https://syspass.prueba.es/api.php

Inputting JSON data in Powershell

Currently, I'm attempting to call upon an API to run a POST, with JSON data as the body. So I was wondering if anyone would be able to tell me how I need to format the text below inside the variable $postParams. I'm pretty new at working with JSON so I'm having so trouble with this.
Currently, I only have the following and don't know what to do about the second line on.
$postParams = #{name='Example'}
Here's is the entire data I was hoping to add to $postParams. So if you could help me with the 2nd, 4th, and 8th that'd be awesome. Thanks!
{
"name":"Example",
"template":{"name":"Template"},
"url":"http://localhost",
"page":{"name":"Landing Page"},
"smtp":{"name":"Sending Profile"},
"launch_date":"2019-10-08T17:20:00+00:00",
"send_by_date":null,
"groups":[{"name":"test group"}]
}
You'll need a here-string and ConvertFrom-Json.
here-string:
Quotation marks are also used to create a here-string. A here-string is a single-quoted or double-quoted string in which quotation marks are interpreted literally. A here-string can span multiple lines. All the lines in a here-string are interpreted as strings, even though they are not enclosed in quotation marks.
The resulting code:
# Use a PowerShell here string to take JSON as it is
$jsonString = #"
{
"name":"Example",
"template":{"name":"Template"},
"url":"http://localhost",
"page":{"name":"Landing Page"},
"smtp":{"name":"Sending Profile"},
"launch_date":"2019-10-08T17:20:00+00:00",
"send_by_date":null,
"groups":[{"name":"test group"}]
}
"#
# Pipe the string to create a new JSON object
$jsonObject = $jsonString | ConvertFrom-Json
# The resulting JSON object has properties matching the properties in the orig. JSON
$jsonObject.name
$jsonObject.url
# Nested property
$jsonObject.template.name
# Nested property in array
$jsonObject.groups[0].name
I've posted an online version of the above code at tio.run, so you can play around with it.
If you want to update several properties of the $jsonObject you can do the following:
$jsonObject.name = "NEW NAME"
$jsonObject.url = "NEW URL"
$jsonObject | ConvertTo-Json
ConvertTo-Json will take your object and create an appropriate JSON string:
{
"name": "NEW NAME",
"template": {
"name": "Template"
},
"url": "NEW URL",
"page": {
"name": "Landing Page"
},
"smtp": {
"name": "Sending Profile"
},
"launch_date": "2019-10-08T17:20:00+00:00",
"send_by_date": null,
"groups": [
{
"name": "test group"
}
]
}
If you $jsonObject has more than two levels of depth, use the -Depth parameter, otherwise not all object information will be included in the JSON string.
ConvertTo-Json:
-Depth
Specifies how many levels of contained objects are included in the JSON representation. The default value is 2.
Here is a tio.run link to a ConvertTo-Json example.
Hope that helps.
I can't test it currently, but try this.
$postParams = #'
{
"name":"Example",
"template":{"name":"Template"},
"url":"http://localhost",
"page":{"name":"Landing Page"},
"smtp":{"name":"Sending Profile"},
"launch_date":"2019-10-08T17:20:00+00:00",
"send_by_date":null,
"groups":[{"name":"test group"}]
}
'#
Make a hashtable, then convert to JSON:
$Hashtable = #{
Key1 = "Value1"
Key2 = "Value2"
}
$Json = $Hashtable | ConvertTo-Json

Changing Slack CURL call into PowerShell

I've in the middle of writing a translation of a working Slack script from Bash to Powershell, and I'm stumbled on what is literally the meat of the problem; how to use Invoke-WebRequest to replace curl.
This is the curl command that works successfully in Bash on *nix systems:
curl \
-X POST \
-H "Content-type: application/json" \
--data "{
\"attachments\":
[
{
\"mrkdwn_in\": [\"text\"],
\"color\": \"$COLOUR\",
\"text\": \"$MESSAGE\",
}
]
}" \
https://hooks.slack.com/services/tokenpart1/tokenpart2
Note the $COLOUR and $MESSAGE variables are derived from elsewhere in the script (not the section I'm having trouble with).
I can't get it to work in PowerShell. My translation so far is:
$Body = #{
"attachments" = "[
{
"mrkdwn_in": ["text"],
"color": "$Colour",
"text": "$Message",
]"
}
$BodyJSON = $Body |convertto-JSON
Invoke-WebRequest -Headers -Method Post -Body "$BodyJSON" -Uri "https://hooks.slack.com/services/tokenpart1/tokenpart2" -ContentType application/json
This results in the following error:
At C:\path-to-file-etc.ps1:53 char:11
+ "mrkdwn_in": ["text"],
+ ~~~~~~~~~~~~~~~~~~~~~
Unexpected token 'mrkdwn_in": ["text"],
"color": "$COLOUR",
"text": "$MESSAGE",
}
]"' in expression or statement.
At C:\path-to-file-etc.ps1:53 char:11
+ "mrkdwn_in": ["text"],
+ ~
The hash literal was incomplete.At
C:\path-to-file-etc.ps1:53 char:11
+ "mrkdwn_in": ["text"],
+ ~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordEx
ception
+ FullyQualifiedErrorId : UnexpectedToken
Process exited with code 1
Process exited with code 1
Step Notify (PowerShell) failed
I have pretty much zero experience in Powershell. Also because this script has to be able to dropped onto a whole variety of boxes,I won't use any libraries or custom cmdlets or anything like that. Out of box methods or die.
Since you're using ConvertTo-Json anyway, it's simpler to construct the entire input as a (nested) custom object / hashtable, and let ConvertTo-Json handle all JSON formatting:
$Body = [pscustomobject] #{
attachments = , [pscustomobject] #{
mrkdwn_in = , 'text'
color = $Colour
text = $Message
}
}
$BodyJson = $Body | ConvertTo-Json -Depth 3
Note: You could substitute [ordered] for [pscustomobject] to use a hashtable with ordered keys instead; even omitting either would work, though seeing the entries in the resulting JSON in different order may be confusing.
Note the use of ,, the array-construction operator, to ensure that the attachments and mrkdwn_in entries are treated as arrays.
Additionally, since ConvertTo-Json only fully serializes to a default depth of 2 levels, -Depth 3 must be used to ensure that the value of entry mrkdwn_in is serialized as an array.
As for what you tried:
Your immediate problem (in addition to the missing } that Jeff Zeitlin points out in a comment on the question): You've neglected to escape the embedded " chars. in your multi-line string.
Therefore, as documentation topic Get-Help about_Quoting_Rules discusses, you can either use `" to embed double quotes inside "..." or use a here-string (#"<nl>....<n>"#).
Even with the syntax problems fixed, your code wouldn't work as intended, however, because ConvertTo-Json wouldn't preserve the pre-formatted JSON in the attachments entry and instead treat the string value as a string literal that needs escaping; here's a simple example:
#{
foo = "[
1,
2
]"
} | ConvertTo-Json
The above yields:
{
"foo": "[\n 1, \n 2 \n ]"
}

Powershell & Curl - Using variables inside single-quoted JSON body

I am currently trying to automate new user creation in our Zendesk ticketing system using Powershell and Curl. The problem I am running into is that the curl json body is enclosed by single quotes and I need to reference a variable inside that body. Here is what I have:
$Firstname = "Test"
$Lastname = "User"
$email= 'user#test.org'
curl.exe https://mydomain.zendesk.com/api/v2/users.json -H "Content-Type: application/json" -X POST -d '{\"user\": {\"name\": \"$Firstname $Lastname\", \"email\": \"$email\"}}' -v -u myuser:mypass
This works fine if I type in regular text values inside the json, but how can I get it to recognize the variables $Firstname, $Lastname and $email?
Try the following:
$Firstname = "Test"
$Lastname = "User"
$email= 'user#test.org'
$json=#"
{\"user\": {\"name\": \"$Firstname $Lastname\", \"email\": \"$email\"}}
"#
curl.exe https://mydomain.zendesk.com/api/v2/users.json -d $json -H 'Content-Type: application/json' -X POST -v -u myuser:mypass
Using a double-quoted here-string #"<newline>...<newline>"# makes specifying embedded " instances easy (no escaping for the sake of PowerShell's own syntax required), while still expanding variable references - see the docs online or Get-Help about_Quoting_Rules.
You're clearly aware of the - unfortunate - additional need to \-escape the " instances, but just to explain why that is needed:
When passing arguments to an external program, PowerShell, after its own parsing and interpolation, conditionally wraps the resulting arguments in double quotes when concatenating them to form the argument list (a single string) to pass to the external utility. Unfortunately, this can result in embedded " instances getting discarded, and the only way to preserve them reliably is to \-escape them - see this answer for more information.
If you wanted to do it inline with a regular double-quoted string, you'd have to escape the " instances for PowerShell too (as `"), resulting in the awkward combination \`":
"{\`"user\`": {\`"name\`": \`"$Firstname $Lastname\`", \`"email\`": \`"$email\`"}}"
Afterthought:
Ryan himself points out in a comment that using a hashtable to construct the data and then converting it to JSON with ConvertTo-Json and feeding it to curl via stdin is an alternative that avoids the quoting headaches:
# Create data as PS hashtable literal.
$data = #{ user = #{ name = "$Firstname $Lastname"; email = "$adUsername#mydomain.org" } }
# Convert to JSON with ConvertTo-Json and pipe to `curl` via *stdin* (-d '#-')
$data | ConvertTo-Json -Compress | curl.exe mydomain.zendesk.com/api/v2/users.json -d '#-' -H "Content-Type: application/json" -X POST -v -u myuser:mypass
I think we can use here-string as json body for Invoke-RestMethod as below
$bufferTime = 5
$requestBody = #"
{
"size": 0,
"aggs": {
"last_x_min": {
"filter": {
"range": {
"#timestamp": {
"gte": "now-$($bufferTime)m",
"lte": "now"
}
}
},
"aggs": {
"value_agg": {
"avg": {
"field": "time-taken"
}
}
}
}
}
}
"#
$esResponse = Invoke-RestMethod -URI http://locahost:9200 -TimeoutSec 15 -Method Post -ContentType 'application/json' -Body $requestBody
This is the script I use to query Elasticsearch. No need to escape double quotes.

Why won't jsontool parse my array?

I'm having trouble parsing this json, which contains an array as its outermost element:
response=[ { "__type": "File", "name": "...tfss-ea51ec70-d3a8-45e5-abbf-294f2c2c81f0-myPicture.jpg", "url": "http://files.parse.com/ac3f079b-cacb-49e9-bd74-8325f993f308/...tfss-ea51ec70-d3a8-45e5-abbf-294f2c2c81f0-myPicture.jpg" } ]
for blob in $response
do
url=$(echo $blob | json url)
done
But this last json parsing gives a bunch of errors:
json: error: input is not JSON: Bad object at line 2, column 1:
^
Do I need to do something special to turn a JSON array into a bash array, or vice versa?
You should quote the value of reponse to protect it from the shell trying to interpret it:
response='[ { "__type": "File", "name": "...tfss-ea51ec70-d3a8-45e5-abbf-294f2c2c81f0-myPicture.jpg", "url": "http://files.parse.com/ac3f079b-cacb-49e9-bd74-8325f993f308/...tfss-ea51ec70-d3a8-45e5-abbf-294f2c2c81f0-myPicture.jpg" } ]'
And you can't expect the shell to be able to parse JSON, so for blob in $response isn't going to work out. According to http://trentm.com/json/, using -a should handle the array:
while read url ; do
# use $url here...
done < <(echo "$response" | json -a url)
After putting your json in a file and piping it through jsontool it works just fine, so I'm guessing your file has some weird whitespace which is breaking in jsontool.
Try this instead:
cat your_file.json | sed 's/[[:space:]]\+$//' | json