I am using PowerShell to run a php script on a daily basis. I can't seem to get the json return back from Invoke-RestMethod. How can that be accomplished?
Here is the code I am using:
$limit = 200;
for ($i=1; $i -le $limit; $i++)
{
Write-Host $i started of $limit
$Site = "http://example.com/updates"
$Info = Measure-Command{
$data = #{call_type = 'powershell'}
$resp = (Invoke-RestMethod -Method Post -URI $Site -Body $data -TimeoutSec 500).results
}
Write-Host resp - On $resp.date, $resp.num were downloaded
Write-Host $i took $Info.TotalSeconds s
Write-Host ---------------------------------------
}
The PHP script is passing back a json string like:
{date: '2014-09-30', num: 127569}
You don't need .results. Invoke-RestMethod converts JSON into a [PSObject] automatically.
If you want to get the raw JSON then use Invoke-WebRequest, but since you reference $resp.date and $resp.num it seems like you wanted it to do the conversion for you after all.
Related
I am badly struck by this problem. I request you to answer or give a hint. I am running out of options.
I am calling an azure runbook upon high CPU utilization via a WebHook. My problem is inside runbook data is not getting decoded properly. For example, the below line is not printing anything.
Write-Output $WebHookData.RequestHeader
Wheras IF i try to explictly convert the data to JSON, like this
*$WebhookData = ConvertFrom-Json $WebhookData*
then it is a throwing error.
ConvertFrom-Json : Invalid JSON primitive: . At line:6 char:31 +
$WebhookData = $WebhookData | ConvertFrom-Json
By the way, I am trying to use the runbook available on Azure gallery {Vertically scale up an Azure Resource Manager VM with Azure Automation}
My Webhook is called from alert created on VM.
A very strange observation:
Working WebHood Example (found in an example) {"WebhookName":"test1","RequestBody":" [\r\n {\r\n \"Message\": \"Test Message\"\r\n }\r\n****]****"
Not Working(the data sent upon calling runbook from VM):
{"WebhookName":"test2","RequestBody":" {\"schemaId\":\"AzureMonitorMetricAlert\"}}
Thanks
I was getting the same error. From my testing, it appears that when performing a "Test" of the runbook, the Webhook data is received as plain text, but when invoked remotely it comes through already formatted as JSON. Here was my solution to cover both scenarios and so far has been working well...
Param (
[object] $WebhookData
)
# Structure Webhook Input Data
If ($WebhookData.WebhookName) {
$WebhookName = $WebhookData.WebhookName
$WebhookHeaders = $WebhookData.RequestHeader
$WebhookBody = $WebhookData.RequestBody
} ElseIf ($WebhookData) {
$WebhookJSON = ConvertFrom-Json -InputObject $WebhookData
$WebhookName = $WebhookJSON.WebhookName
$WebhookHeaders = $WebhookJSON.RequestHeader
$WebhookBody = $WebhookJSON.RequestBody
} Else {
Write-Error -Message 'Runbook was not started from Webhook' -ErrorAction stop
}
I tried with a webhook, the script Write-Output $WebHookData.RequestHeader should work fine.
And if I use ConvertFrom-Json $WebhookData, I can reproduce your issue, not sure why it occurred, according to the doc, the $WebhookData is also in a JSON format, if it is accepted, you could use ConvertFrom-Json -InputObject $WebhookData.RequestBody, it will work fine.
My runbook:
param
(
[Parameter (Mandatory = $false)]
[object] $WebhookData
)
if ($WebhookData) {
Write-Output $WebhookData.RequestHeader
$Body = ConvertFrom-Json -InputObject $WebhookData.RequestBody
Write-Output $Body
} else
{
Write-Output "Missing information";
exit;
}
The powershell script I used to send a webhook:
$uri = "https://s5events.azure-automation.net/webhooks?token=xxxxxxxxxxxx"
$vms = #(
#{ Name="vm01";ResourceGroup="vm01"},
#{ Name="vm02";ResourceGroup="vm02"}
)
$body = ConvertTo-Json -InputObject $vms
$header = #{ message="StartedbyContoso"}
$response = Invoke-WebRequest -Method Post -Uri $uri -Body $body -Headers $header
$jobid = (ConvertFrom-Json ($response.Content)).jobids[0]
Output:
I had the same problem use following to get webhookdata if using test pane with Alert json as input
if(-Not $WebhookData.RequestBody){
$WebhookData = (ConvertFrom-Json -InputObject $WebhookData)
}
$RequestBody = ConvertFrom-JSON -InputObject $WebhookData.RequestBody
So I'm trying to generate a streaming dataset in Power BI, so that I can have a tab in teams that constantly updates with data from my companies database. The way I pull the data is through our platform API that generates a JSON and a URL based on the query. As far as I can find, the only way to really do this is through Powershell. I want to pull out the hours and numbers fields from the JSON and then push that data to the dashboard in Power Bi, but I have absolutely no experience with any of this, so I'm completely baffled as to where to start.
Here's the powershell code I have so far, but it throws an error in regard to pulling the URL info.
$request = 'http://wya.works/rta_develop/xmlServlet?&command=retrieve&sql=select%20%5B%24Hours%5D%2C%20%5B%24Date%20Worked%5D%20from%20%5B%21HOURS%5D%20&attributesOnly=Date%20Worked%2C%20Hours&contentType=JSON&referer=&time=1595271424377&key=696a666d'
Invoke-WebRequest $request
$json.RECORD | % {
$hours = 0
$date = ""
$_.Field | % {
if ($_ -match '\d{1,2}/\d{1,2}\/d{4}'){
$date = "$_"
}
else {
$hours = $_
}
}
New-Object -TypeName psobject -Property (#{
Date = $date
Hours = $hours
})
}
$endpoint = "https://api.powerbi.com/beta/d6cdaa23-930e-49c1-9d2a-0fbe648551b2/datasets/34eaea1e-73b6-4759-ac8b-aaae51708654/rows?noSignUpCheck=1&key=Ur9E0GDrhkp4EwJOF4bCbg7EO7aIve54urjB8M%2BHevG1%2F6pDgRJ47Fvkmx4b%2FcMowlhV18ZYyVtF9pfG%2BM1EQA%3D%3D"
$payload = #{
"Hours" =$hours
"Date Worked" =$date
}
Invoke-RestMethod -Method Post -Uri "$endpoint" -Body (ConvertTo-Json #($paylojnjlkhad))
This is a snippet from the JSON file and all of the fields I need (numeric: Hours) (date: Date Worked) are labeled the same, which makes this a lot more difficult.
{"COUNT":"332","DISPLAY_LIST_START":"1","DISPLAY_LIST_STOP":"332","STOP":"332","RECORD":[{"SESSION_ID":"HxNI-Zuc1B2EFAzTS8hx6w7Ek_dbrNCMhYVNI3Ta","FIELD":["6",["04/23/2018"]]},{"SESSION_ID":"HxNI-Zuc1B2EFAzTS8hx6w7Ek_dbrNCMhYVNI3Ta","FIELD":["6",["04/24/2018"]]},{"SESSION_ID":"HxNI-Zuc1B2EFAzTS8hx6w7Ek_dbrNCMhYVNI3Ta","FIELD":["6",["04/26/2018"]]},{"SESSION_ID":"HxNI-Zuc1B2EFAzTS8hx6w7Ek_dbrNCMhYVNI3Ta","FIELD":["6",["04/30/2018"]]},{"SESSION_ID":"HxNI-Zuc1B2EFAzTS8hx6w7Ek_dbrNCMhYVNI3Ta","FIELD":["6",["05/01/2018"]]},{"SESSION_ID":"HxNI-Zuc1B2EFAzTS8hx6w7Ek_dbrNCMhYVNI3Ta","FIELD":["4",["05/02/2018"]]}
I need to use the URL instead of the actual file, because my company's platform runs on blockchain and is constantly updated.
If you want to make an API call per pair of $date and $hours, you can do the following. This will make one call per RECORD value.
$endpoint = "https://api.powerbi.com/beta/d6cdaa23-930e-49c1-9d2a-0fbe648551b2/datasets/34eaea1e-73b6-4759-ac8b-aaae51708654/rows?noSignUpCheck=1&key=Ur9E0GDrhkp4EwJOF4bCbg7EO7aIve54urjB8M%2BHevG1%2F6pDgRJ47Fvkmx4b%2FcMowlhV18ZYyVtF9pfG%2BM1EQA%3D%3D"
$json.RECORD | Foreach-Object {
$hours = 0
$date = ""
$_.Field | Foreach-Object {
if ($_ -match '\d{1,2}/\d{1,2}/\d{4}'){
$date = $_
}
else {
$hours = $_
}
}
$payload = #{
"Hours" = $hours
"Date Worked" = $date
}
Invoke-RestMethod -Method Post -Uri $endpoint -Body (ConvertTo-Json $payload)
}
If you want to make one API call after creating an array of $date and $hours pairs, you can do the following. This will only make one API call.
$endpoint = "https://api.powerbi.com/beta/d6cdaa23-930e-49c1-9d2a-0fbe648551b2/datasets/34eaea1e-73b6-4759-ac8b-aaae51708654/rows?noSignUpCheck=1&key=Ur9E0GDrhkp4EwJOF4bCbg7EO7aIve54urjB8M%2BHevG1%2F6pDgRJ47Fvkmx4b%2FcMowlhV18ZYyVtF9pfG%2BM1EQA%3D%3D"
$payloadArray = $json.RECORD | Foreach-Object {
$hours = 0
$date = ""
$_.Field | Foreach-Object {
if ($_ -match '\d{1,2}/\d{1,2}/\d{4}'){
$date = $_
}
else {
$hours = $_
}
}
$payload = #{
"Hours" = $hours
"Date Worked" = $date
}
$payload
}
Invoke-RestMethod -Method Post -Uri $endpoint -Body (ConvertTo-Json $payloadArray)
Exlanation:
If you want to have a payload per field pair, you will need to make an API call after $date and $hours are both set. Then repeat that process for each pair. If your payload supports an array structure, then you can create a collection of those pairs.
You are only doing the API call after all fields have been processed without creating a collection, which means you only get the last pair of $date and $hours values. As you iterate through your RECORD items, you are overwriting $date and $hours before the API call is ever made.
So I'm trying to pull the number of hours worked and the date worked from a table in my companies database to make a chart in Power BI through a streaming data set. I'm using powershell to parse a JSON file
Here's a JSON sample:
{"COUNT":"334","DISPLAY_LIST_START":"1","DISPLAY_LIST_STOP":"334","STOP":"334","RECORD":[{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["04/23/2018"]]},{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["04/24/2018"]]},{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["04/26/2018"]]},{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["04/30/2018"]]},{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["05/01/2018"]]},{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["4",["05/02/2018"]]},{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["05/03/2018"]]},{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["05/07/2018"]]},
I know it's not the best in terms of organization, but it's all I have to work with.
Here's the powershell code I have so far:
Invoke-WebRequest -Uri "http://wya.works/rta_develop/xmlServlet?&command=retrieve&sql=select%20%5B%24Hours%5D%2C%20%5B%24Date%20Worked%5D%20from%20%5B%21HOURS%5D%20&attributesOnly=Date%20Worked%2C%20Hours&contentType=JSON&referer=&time=1595443368507&key=696a6768"
$endpoint = "https://api.powerbi.com/beta/d6cdaa23-930e-49c1-9d2a-0fbe648551b2/datasets/91466553-d719-420c-9e3e-73e748379263/rows?noSignUpCheck=1&key=SU5GRBBWuuEIDSjqHW5hdgJzSMCQ3qUQ9mGrBDanjgpExv6woY1Sa1c3PC1Wk3WHHn1N%2FEpIuVgzHHcw0JXwYw%3D%3D"
$json.RECORD | Foreach-Object {
Write-Output "Checking Records"
$hours = 0
$date = ""
$json.FIELD | Foreach-Object{
Write-Output "Checking Field"
if ($_ -match '\d{1,2}/\d{1,2}\/d{4}'){
$date = $_
}
else {
$hours = $_
}
}
$payload = #{
"Hours" = $hours
"Date Worked" =$date
}
}
Invoke-RestMethod -Method Post -Uri "$endpoint" jlk-Body (ConvertTo-Json #($payload))
I need to parse through each record and pull the values of the hours (the numeric value in the JSON) and the Date (the date value).
When I run the code I don't get any errors, but it doesn't seem to be reaching the -match or the else statements. I tried logging the output on both and it returns nothing.
Is there something wrong with my loops?
I'm brand new to powershell and most of this code I got from help from other people, but I understand what its doing for the most part.
Also, anyone who knows about streaming datasets, will pulling this this way even give me what I want?
Store the Invoke-Webrequest values in your $json first. You missed that point; that's why you are getting Null.I don't know it is a typo or a miss.
Secondly, you $json.RECORD is wrong because it doesnt have any Record in the response. What you are looking for is basically the content. $json.content is going to give you the content of numbers.
$json=Invoke-WebRequest -Uri "http://wya.works/rta_develop/xmlServlet?&command=retrieve&sql=select%20%5B%24Hours%5D%2C%20%5B%24Date%20Worked%5D%20from%20%5B%21HOURS%5D%20&attributesOnly=Date%20Worked%2C%20Hours&contentType=JSON&referer=&time=1595443368507&key=696a6768"
Your endpoint and invoke-restmethod has nothing to do with your json parsing. First handle the response in the loop and see what is the outcome you are getting. I have structured it but I have not worked on the JSON sample data as if now:
$json=Invoke-WebRequest -Uri "http://wya.works/rta_develop/xmlServlet?&command=retrieve&sql=select%20%5B%24Hours%5D%2C%20%5B%24Date%20Worked%5D%20from%20%5B%21HOURS%5D%20&attributesOnly=Date%20Worked%2C%20Hours&contentType=JSON&referer=&time=1595443368507&key=696a6768"
$json.content | Foreach-Object {
Write-Output "Checking Records"
$hours = 0
$date = ""
$json.FIELD | Foreach-Object{
Write-Output "Checking Field"
if ($_ -match '\d{1,2}/\d{1,2}\/d{4}'){
$date = $_
}
else {
$hours = $_
}
}
$payload = #{
"Hours" = $hours
"Date Worked" =$date
}
}
$endpoint = "https://api.powerbi.com/beta/d6cdaa23-930e-49c1-9d2a-0fbe648551b2/datasets/91466553-d719-420c-9e3e-73e748379263/rows?noSignUpCheck=1&key=SU5GRBBWuuEIDSjqHW5hdgJzSMCQ3qUQ9mGrBDanjgpExv6woY1Sa1c3PC1Wk3WHHn1N%2FEpIuVgzHHcw0JXwYw%3D%3D"
Invoke-RestMethod -Method Post -Uri "$endpoint" jlk-Body (ConvertTo-Json #($payload))
The root of your problem is this:
$json.RECORD | Foreach-Object {
$json.FIELD | Foreach-Object{
...
}
}
Your outer foreach-object is looping over each item in the RECORD array, but the inner foreach-object is trying to loop over top-level FIELD properties that don't actually exist!
However, I think you'll hit more problems further along your code if you try to troubleshoot what you've already got, and there's an easier way to do what you're trying to do...
First, your sample json isn't quite valid - there's a trailing comma and some unclosed brackets so I'm going to reformat it with some line breaks so it's easier to read, and then fix it.
Note - you'll get the text from your call to Invoke-WebRequest, or as #Lee_Dailey suggested, Invoke-RestMethod.
# reformat the json, fix it, and assign it to a variable using a "Here-String"
# (see https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_quoting_rules?view=powershell-7#here-strings)
$text = #"
{
"COUNT":"334",
"DISPLAY_LIST_START":"1",
"DISPLAY_LIST_STOP":"334",
"STOP":"334",
"RECORD":[
{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["04/23/2018"]]},
{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["04/24/2018"]]},
{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["04/26/2018"]]},
{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["04/30/2018"]]},
{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["05/01/2018"]]},
{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["4",["05/02/2018"]]},
{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["05/03/2018"]]},
{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["05/07/2018"]]}
]
}
"#
Then, we'll parse it - i.e. convert it from a string into an object model with well-defined properties:
# parse the json text
$json = $text | ConvertFrom-Json
And then process each record in turn:
$json.RECORD | ForEach-Object {
# get the number of hours worked. this is the first value in the FIELD array
# (note - the array starts at index 0 because it's zero-indexed)
$hours = $_.FIELD[0] # e.g. "6"
# get the "date worked". we need to get the second value (index 1) in
# the FIELD array, but this is a nested array, so once we've got the
# inner array we need to get the first value (index 0 again) from that
$date = $_.FIELD[1][0] # e.g. "04/23/2018"
# now we can build the payload...
$payload = #{
"Hours" = $hours
"Date Worked" = $date
}
# ...and invoke the api for this record
$endpoint = "https://api.powerbi.com/beta/d6cdaa23-930e-49c1-9d2a-0fbe648551b2/datasets/91466553-d719-420c-9e3e-73e748379263/rows?noSignUpCheck=1&key=SU5GRBBWuuEIDSjqHW5hdgJzSMCQ3qUQ9mGrBDanjgpExv6woY1Sa1c3PC1Wk3WHHn1N%2FEpIuVgzHHcw0JXwYw%3D%3D"
Invoke-RestMethod -Method Post -Uri "$endpoint" -Body (ConvertTo-Json #($payload))
}
Note the api call is inside the foreach loop, otherwise you end up calculating the $payload for each RECORD but only ever actually calling the api for the last one.
(I've also removed a spurious "jlk" from your final line, which is probably a typo).
I'm having an issue with getting a script to work the way I intend but admittedly I'm not very good at for loops in general. I'm trying to loop through my hashtable and convert and perform an invoke-restmethod for every 500 items in the hastable
for ($every500 in $params) {
$params1 = $params | ConvertTo-Json
$params2 = '{"update"' + ":" + $($params1) + "}"
Invoke-RestMethod -Uri $website -Headers $header -ContentType "application/json" -Method POST -Body $params2
}
I'm not very good with for loops and the last one I made I had a lot of help with. As you can see from my example below I'm lost in all this.
So far I've been searching forums and looking for something that would point in the right direction but I'm coming up empty.
Can someone please show me how to loop through a hashtable and perform a call after every X amount items?
$params = Import-Csv C:\tesrfolder\test.csv |
ForEach-Object {
#{ # new hashtable here
stock_quantity = $_.Stock
id = $_.ID
stock_status = $_.'In stock?'
}# output is implicit
}
$params1 = $params | ConvertTo-Json
$params2 = '{"update"' + ":" + $($params1) + "}"
# $count = $params1.count
# For ($i = $count; $i -lt 500; ) {
# }
# foreach ($keys in $params) { $params -lt 10 }
As you can see above, not a clue on what I'm doing lol
I normally don't have to deal with large amounts of data so I never really practiced my for loops and now it's come to bite me in the butt.
This is what I have before trying to loop for every 500
$website = "https://www.mywebsite.com/"
$params = Import-Csv C:\testfolder\test.csv |
ForEach-Object {
#{ # new hashtable here
stock_quantity = $_.Stock
id = $_.ID
stock_status = $_.'In stock?'
}# output is implicit
}
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("user:pass")))
$header = #{
Authorization = ("Basic {0}" -f $base64AuthInfo);
}
$params1 = $params | ConvertTo-Json
$params2 = '{"update"' + ":" + $($params1) + "}"
Invoke-RestMethod -Uri $website -Headers $header -ContentType "application/json" -Method POST -Body $params2
I'm creating my hashtable from a CSV, and what I'd like to achieve is at every 500 items in my $params hastable is to convert those keys to json, then add {"update": at the top and } at the end of each converted table followed by an invoke-restmethod for each conversion.
In my head I keep picturing something like this
for ($every500 in $params) {
$params1 = $params | ConvertTo-Json
$params2 = '{"update"' + ":" + $($params1) + "}"
Invoke-RestMethod -Uri $website -Headers $header -ContentType "application/json" -Method POST -Body $params2
}
I just can't seem to wrap my head around getting this to work, I know I'm missing parts to this puzzle I just don't know what it is.
Any help would be appreciated!
Here is a way to create a hash from a sample of CSV records. This does not make a REST call.
$samples = 500
$data = Import-Csv -Path '.\stock.csv'
$n = $data.count
for ($i = 0; $i -lt [Math]::Ceiling($n/$samples); $i++) {
$ht = #{ # new hashtable here
stock_quantity = $data[$i * $samples].Stock
id = $data[$i * $samples].ID
stock_status = $data[$i * $samples].'In stock?'
}
}
I need to run 1000 API calls for 1000 different URI's. I tried to run 1000 different URI in one go as shown below but the API has a limit of 1000 lines.
Is there a way to run this code 1000 times producing 1000 separate JSON files or in one JSON file without me having to run the URI with the powershell script indivdiually.
$access_token ="dfdfgdfgf45h5676rdffghfghfbazbfgjvdrtwsffrgr"
$LogPath ='C:\StackOVerflow'
$URI = 'https://api.web.ser/audits/a_80weqwepeopeoA'
'https://api.web.ser/audits/a_B0980kdfl;skfd'
'https://api.web.ser/audits/a_Csdfsdfsjlksdc'
'https://api.web.ser/audits/a_Tlksdjfalkjdff'
'https://api.web.ser/audits/a_Fldkfjlsfjdfdl'
'https://api.web.ser/audits/a_Fsdfsdfsdfsd34'
'https://api.web.ser/audits/a_G34gfgf4ffsdrf'
'https://api.web.ser/audits/a_Haere34534fdgf'
'https://api.web.ser/audits/a_Ifdl023lererew'
'https://api.web.ser/audits/a_Afrererl45645w'
$headers = #{“authorization” = “Bearer $access_token”}
$result = Invoke-RestMethod -Uri $URI -Headers $headers |ConvertTo-Json| out-file "$LogPath\id_ExtractLogs$(((get-date).ToString("yyyyMMdd"))).json"
$result
You cannot pass an array of uri to the Invoke-RestMethod cmdlet; they must be iterated.
Here's a working solution:
$uri = 'https://api.web.ser/audits'
$reportList = #(
'a_80weqwepeopeoA'
'a_B0980kdfl;skfd'
# .. etc.
)
$token = 'secret'
$headers = #{ Headers = #{ Authorization = "Bearer $token" }}
$logPath = 'C:\StackOverflow'
foreach ($report in $reportList) {
$stamp = Get-Date -Format FileDateTime
Invoke-RestMethod -Uri "$uri/$report" -OutFile "$logPath\id_$stamp.json" #headers
}