Im getting a stream from shopify with powershell and converting it json readable text.
It was working fine for a couple of weeks but now is not returning the same data format.
$uri = "https://yourshop.myshopify.com/admin/api/2021-10/orders.json?limit=250"
$apikey = ""
$password = ""
$headers = #{"Authorization" = "Basic "+[System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($apikey+":"+$password))}
$products = Invoke-RestMethod -Uri $uri -FollowRelLink -contentType "application/json" -Method Get -Header $headers -Body $Body -MaximumFollowRelLink 5
$products | ConvertTo-Json -Depth 5 | Out-File C:\json\ordersdepth5.json -encoding utf8
it used to return neat json
{
"orders": [
{
"id": 4059713634402,
"admin_graphql_api_id": "gid://shopify/Order/4059713634402",
"app_id": 580111,
"browser_ip": "120.154.83.138",
"buyer_accepts_marketing": true,
"cancel_reason": null,
"cancelled_at": null,
but now it returns
["{\"orders\":[{\"id\":4122462257250,\"admin_graphql_api_id\":\"gid:\\/\\/shopify\\/Order\\/4122462257250\",\"
A bunch of slashes and rubbish
Is there an issue with the data stream? what should I look at.
Shopify wouldn't change formatting unless there is an api change?
I fixed the issue by moving the data into powershell to convert from and then convert to json.
$products | ConvertFrom-Json -AsHashtable | ConvertTo-Json -Depth 5 -Compress |Out-File C:\json\ordersdepth2.json -encoding utf8
Related
I have a script that gets data from a public API. I try to parse a value from the Json response into a variable. However it seems that when I Write-Host the variable it has replaced — with â.
Code:
$SetData = Invoke-RestMethod -Uri "https://mtgjson.com/api/v5/2XM.json" -ContentType "application/json" -Method GET
$Card = $SetData.data.cards | Where-Object { $_.name -eq "Adaptive Automaton" -and $_.isPromo -ne "true"}
Write-Host $Card.type -ForegroundColor Cyan
Output:
Artifact Creature — Construct
It looks like the strings returned by the Invoke-RestMethod here are encoded in 'ISO-8859-1' and not as you would expect in UTF-8.
This means you need to convert to UTF-8 where needed, something like this:
$encoding = [System.Text.Encoding]::GetEncoding('ISO-8859-1')
$SetData = Invoke-RestMethod -Uri "https://mtgjson.com/api/v5/2XM.json" -ContentType "application/json" -Method GET
$Card = $SetData.data.cards | Where-Object { $_.name -eq "Adaptive Automaton" -and !$_.isPromo}
# convert the string in '$Card.type' from encoding 'ISO-8859-1' into 'UTF-8'
$cardType = ([System.Text.Encoding]::UTF8).GetString($encoding.GetBytes($Card.type))
Write-Host $cardType -ForegroundColor Cyan
Output
Artifact Creature — Construct
To convert the whole json to UTF-8, You could use Invoke-WebRequest rather than Invoke-RestMethod:
$encoding = [System.Text.Encoding]::GetEncoding('ISO-8859-1')
$SetData = Invoke-WebRequest -Uri "https://mtgjson.com/api/v5/2XM.json" -Method Get
# convert $SetData.Content to UTF-8 and convert that from JSON
$content = ([System.Text.Encoding]::UTF8).GetString($encoding.GetBytes($SetData.Content)) | ConvertFrom-Json
$Card = $content.data.cards | Where-Object { $_.name -eq "Adaptive Automaton" -and !$_.isPromo}
Write-Host $Card.type -ForegroundColor Cyan
I have the following call to an API in a powershell script (Powershell 4.0):
$Json = Invoke-WebRequest -Uri $RequestURL -UseBasicParsing -Headers $headers -ContentType 'application/json; charset=utf-8' -Method POST -Body $postParams -TimeoutSec 40
...and the content of the response (which is a string in JSON format) is written to a file:
Set-Content $path -Value $Json.Content
An example of a typical response...
{
"MyArray": [{
"MyField": "A1",
"MyField2": "A2"
}, {
"MyField": "B1",
"MyField2": "B2"
}]
}
All well and good, but now I have a requirement to parse the returned content as JSON and query some properties from within this Powershell script.
I presume I need to convert my string to 'proper' JSON and then to a powershell object in order to access the properties...so I have tried combinations of ConvertTo-Json and ConvertFrom-Json but can't ever seem to access it in anything other than a string. For example...
$x = $Json.Content | ConvertTo-Json
Write-Host $x.MyArray[0].MyField
$y = $x | ConvertFrom-Json
Write-Host $y[0].MyArray[0].MyField
In both cases above I get an error "Cannot index into a null array" suggesting that MyArray is null.
How do I convert my $Json response object into an object I can drill down into?
See ConvertFrom-Json
Converts a JSON-formatted string to a custom object or a hash table.
The ConvertFrom-Json cmdlet converts a JavaScript Object Notation
(JSON) formatted string to a custom PSCustomObject object that has a
property for each field in the JSON string.
Once you get the response converted to custom object or a hash table, you can access the individual properties
The link includes coding examples
This seems to work...
[void][System.Reflection.Assembly]::LoadWithPartialName("System.Web.Extensions")
$x = (New-Object -TypeName System.Web.Script.Serialization.JavaScriptSerializer -Property #{MaxJsonLength=67108864}).DeserializeObject($Json.Content)
Write-Host $x.MyArray[0].MyField
...although not sure why yet.
I have the following PowerShell script that uses cURL in Windows 10 and works perfectly:
$Body = #{
'data' = #{
'CID' = 15;
'HID' = 37;
'Type' = "TYPE1";
'downloadOn' = "NEXT_CONTACT";
'AutomationEnabled' = "True";
}
}
$CurlArgument = '-s', '-X', 'PATCH',
'-H', 'Content-Type: application/json',
$URL,
'-H',
$AuthBearer,
'-d',
(($Body | ConvertTo-Json) -replace '"', '\"')
Write-Host "$Section cURL command took" ("{0:n1}" -f (Measure-Command {$UpdateResponse = & $CURLEXE #CurlArgument}).TotalSeconds) "Seconds" -ErrorAction SilentlyContinue
I can't use cURL, I need to use native Invoke-WebRequest on my production servers. I need to convert the above into a Invoke-WebRequest command, which I have done, as follows:
$Body = #{
'data' = #{
'CID' = 15;
'HID' = 37;
'Type' = "TYPE1";
'downloadOn' = "NEXT_CONTACT";
'AutomationEnabled' = "True";
}
}
(($Body | ConvertTo-Json) -replace '"', '\"')
$Method = "PATCH"
$Header = #{"Accept" = "*/*" ; "Cache-Control" = "no-cache" ; "Host" = "myURL"; "accept-encoding" = "gzip,deflate"; "Authorization" = "Bearer $SessionToken" }
$ContentType = "application/json"
Write-Host "$Section Invoke-WebRequest command took" ("{0:n1}" -f (Measure-Command { $UpdateResponse = Invoke-WebRequest -Method $Method -Uri $URL -Header $Header -ContentType $ContentType -Body $Body }).TotalSeconds) "Seconds"
When I run the Invoke-WebRequest, I get the following error, i.e. A JSONObject text must begin with '{':
Invoke-WebRequest : {"status":"FAILURE","errors":...."message":{"5011":"A JSONObject text must begin with '{' at 1 [character 2 line 1]"}}]}
My $Body looks like this i.e. it begins with '{' :
{
\"data\": {
\"downloadOn\": \"NEXT_CONTACT\",
\"HID\": 37,
\"AutomationEnabled\": \"True\",
\"CID\": 15,
\"Type\": \"TYPE1\"
}
}
I tried with and without "-replace '"', '\"'", from this post"
cURL to PowerShell - Double hash table in --data?
Looking at my $Body JSON "object"(?), I can see this:
Name Value
---- -----
data {downloadOn, HID, AutomationEnabled, CID...}
Looking at my Value, I can see it is listed as follows:
Name Value
---- -----
downloadOn NEXT_CONTACT
HID 37
AutomationEnabled True
CID 15
Type TYPE1
Instead of sending -Body $Body,I thought maybe I should just sent the values, as follows (which also failed) with the same message.
-Body $Body.Values
I did a heap of searching last night, but I am at a loss on how to convert that into a successful Invoke-WebRequest, and any help would be appreciated.
You are sending $Body as a Hashtable, try converting it to JSON
$Body = $Body | ConvertTo-Json
If you send $Body before the above line and again after to an echo service you'll see the difference
Invoke-RestMethod -Method 'POST' -Uri 'https://postman-echo.com/post' -ContentType 'application/json' -Body $Body
I'm trying to manipulate a json object and send it as content into the body of a put / post web request. The source of my json is a file on my disk.
This is my Powershell script:
$urlBase = 'https://mysite.myapp.com/service/api/Item/'
$myJson = (Get-Content 'file.json' | ConvertFrom-JSON)
# Then I manipulate my object
$id = $myJson.id
$myJson.version = '1.2.3.4'
# Request
$response = Invoke-RestMethod -Uri ($urlBase + $id) -Method Put -Body $myJson -ContentType 'application/json' -Headers $hdrs
When I execute my script y get this error message:
Invoke-RestMethod : The remote server returned an error: (400) Bad Request.
At line:18 char:17
+ ... $response = Invoke-RestMethod -Uri ($urlBase + $id) -Method Put -Body ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
If I change my $myJson asignment for this the request works fine...
$myJson = Get-Content 'file.json'
... , but then I can't manipulate my json before send it.
Edited:
If I try to convert back using ConvertTo-Json I get the same error:
$convertedBack = $myJson | ConvertTo-Json
# Request
$response = Invoke-RestMethod -Uri ($urlBase + $id) -Method Put -Body $convertedBack -ContentType 'application/json' -Headers $hdrs
As pointed out in the comments: you need to convert your object back to JSON using the ConvertTo-Json cmdlet.
I see that you've tried that now and had the same problem. So I ask you this: is the value of $convertedBack exactly what you expected? Dump it to file and check!
The reason I am suspicious of this detail is that ConvertTo-Json has a little gotcha in it. Specifically the -Depth parameter which can cause some data loss.
-Depth
Specifies how many levels of contained objects are included in the JSON representation. The default value is 2.
Example Without -Depth
$basicJsonObject = #"
{
"name": "George",
"properties": {
"mood": "jovial",
"coffee": {
"hasCoffee": true,
"mugContents": {
"milk": false,
"doubleShot": true
}
}
}
}
"#
$psObject = ConvertFrom-Json -InputObject $basicJsonObject
Write-Host "Freshly Imported"
Write-Host "DoubleShot = $($psObject.properties.coffee.mugContents.doubleShot)"
$convertedBack = ConvertTo-Json -InputObject $psObject
$reConverted = ConvertFrom-Json -InputObject $convertedBack
Write-Host "Re-Converted"
Write-Host "DoubleShot = $($reConverted.properties.coffee.mugContents.doubleShot)"
Results
Freshly Imported
DoubleShot = True
Re-Converted
DoubleShot =
Example With -Depth
Change one line of code:
$convertedBack = ConvertTo-Json -InputObject $psObject -Depth 5
Results
Freshly Imported
DoubleShot = True
Re-Converted
DoubleShot = True
Note how the new results include the value from the $reConverted variable. This is because the data is not lost further upstream!
The following is a snippet from my PowerShell script where the values for the parameters $book and $author are not getting substituted. Please suggest some techniques that I can apply to fix it or share some code that can help me out.
$body = #{
version = '1.0'
inactive = 'false'
yml = { "Service1:\n book: $book\n author: $author\n "} | ConvertFrom-Json
} | ConvertTo-Json
$request = Invoke-WebRequest -UseBasicParsing -Method Post -Uri $uri -Body
$body -Headers $headers -ContentType $contentType
$response = ConvertFrom-Json -InputObject $request.Content
You have some weird stuff going on in this line
... yml = { "Service1:\n book: $book\n author: $author\n "} | ConvertFrom-Json } | ConvertTo-Json
Because it says "do a script block with this body, and try to convert the script block to JSON".
So, if you want to have a JSON string in yml field, you have two options.
Write the proper JSON string yourself:
#{...put the rest of your elements here...; yml = "{Service1:'', book:'$book', author: '$author'}"
Populate a hashtable first and then convert it to JSON string:
#{...put the rest of your elements here...; yml = #{Service1=''; book='$book'; author='$author'} } | ConvertTo-Json