I found a nice little JSON API from Kayaposoft which will give a true/false value if the given day is a work day or not (e.g. Sunday the 14th of April; isWorkDay: false). This API is also able to honor our local holidays, like Vappu in Finland, etc.
So, as I try to script this in PowerShell (being the beginner that I am), I quickly realized a problem with the code below. While the code works, it isn't very practical and is dependent on each variable twice (once to set it up and once in the loop).
Is there any way to beautify this code? Can it be made to be more practical and/or to not use each variable twice?
$date = (Get-Date).AddDays(11)
$jsonDate = $date.ToString('dd-MM-yyyy')
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$jsonRequest = Invoke-WebRequest "https://kayaposoft.com/enrico/json/v2.0/?action=isWorkDay&date=$jsonDate&country=fin" | ConvertFrom-Json
while ($jsonRequest.isWorkDay -ne $true) {
$date = $date.AddDays(1)
$jsonDate = $date.ToString('dd-MM-yyyy')
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$jsonRequest = Invoke-WebRequest "https://kayaposoft.com/enrico/json/v2.0/?action=isWorkDay&date=$jsonDate&country=fin" | ConvertFrom-Json
Write-Host $jsonDate
Write-Host $jsonRequest
}
Results:
22-04-2019
#{isWorkDay=False}
23-04-2019
#{isWorkDay=True}
Something like this:
$date = (Get-Date)
# not sure if you really need this ?!?
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
# do { ... } while (...) - execute at least once
do {
$jsonDate = $date.ToString('dd-MM-yyyy')
$jsonRequest = Invoke-WebRequest "https://kayaposoft.com/enrico/json/v2.0/?action=isWorkDay&date=$jsonDate&country=fin" | ConvertFrom-Json
$date = $date.AddDays(1)
Write-Host $jsonRequest
Write-Host $jsonDate
}
while ($jsonRequest.isWorkDay -ne $true)
Related
So I'm trying to generate a streaming dataset in Power BI, so that I can have a tab in teams that constantly updates with data from my companies database. The way I pull the data is through our platform API that generates a JSON and a URL based on the query. As far as I can find, the only way to really do this is through Powershell. I want to pull out the hours and numbers fields from the JSON and then push that data to the dashboard in Power Bi, but I have absolutely no experience with any of this, so I'm completely baffled as to where to start.
Here's the powershell code I have so far, but it throws an error in regard to pulling the URL info.
$request = 'http://wya.works/rta_develop/xmlServlet?&command=retrieve&sql=select%20%5B%24Hours%5D%2C%20%5B%24Date%20Worked%5D%20from%20%5B%21HOURS%5D%20&attributesOnly=Date%20Worked%2C%20Hours&contentType=JSON&referer=&time=1595271424377&key=696a666d'
Invoke-WebRequest $request
$json.RECORD | % {
$hours = 0
$date = ""
$_.Field | % {
if ($_ -match '\d{1,2}/\d{1,2}\/d{4}'){
$date = "$_"
}
else {
$hours = $_
}
}
New-Object -TypeName psobject -Property (#{
Date = $date
Hours = $hours
})
}
$endpoint = "https://api.powerbi.com/beta/d6cdaa23-930e-49c1-9d2a-0fbe648551b2/datasets/34eaea1e-73b6-4759-ac8b-aaae51708654/rows?noSignUpCheck=1&key=Ur9E0GDrhkp4EwJOF4bCbg7EO7aIve54urjB8M%2BHevG1%2F6pDgRJ47Fvkmx4b%2FcMowlhV18ZYyVtF9pfG%2BM1EQA%3D%3D"
$payload = #{
"Hours" =$hours
"Date Worked" =$date
}
Invoke-RestMethod -Method Post -Uri "$endpoint" -Body (ConvertTo-Json #($paylojnjlkhad))
This is a snippet from the JSON file and all of the fields I need (numeric: Hours) (date: Date Worked) are labeled the same, which makes this a lot more difficult.
{"COUNT":"332","DISPLAY_LIST_START":"1","DISPLAY_LIST_STOP":"332","STOP":"332","RECORD":[{"SESSION_ID":"HxNI-Zuc1B2EFAzTS8hx6w7Ek_dbrNCMhYVNI3Ta","FIELD":["6",["04/23/2018"]]},{"SESSION_ID":"HxNI-Zuc1B2EFAzTS8hx6w7Ek_dbrNCMhYVNI3Ta","FIELD":["6",["04/24/2018"]]},{"SESSION_ID":"HxNI-Zuc1B2EFAzTS8hx6w7Ek_dbrNCMhYVNI3Ta","FIELD":["6",["04/26/2018"]]},{"SESSION_ID":"HxNI-Zuc1B2EFAzTS8hx6w7Ek_dbrNCMhYVNI3Ta","FIELD":["6",["04/30/2018"]]},{"SESSION_ID":"HxNI-Zuc1B2EFAzTS8hx6w7Ek_dbrNCMhYVNI3Ta","FIELD":["6",["05/01/2018"]]},{"SESSION_ID":"HxNI-Zuc1B2EFAzTS8hx6w7Ek_dbrNCMhYVNI3Ta","FIELD":["4",["05/02/2018"]]}
I need to use the URL instead of the actual file, because my company's platform runs on blockchain and is constantly updated.
If you want to make an API call per pair of $date and $hours, you can do the following. This will make one call per RECORD value.
$endpoint = "https://api.powerbi.com/beta/d6cdaa23-930e-49c1-9d2a-0fbe648551b2/datasets/34eaea1e-73b6-4759-ac8b-aaae51708654/rows?noSignUpCheck=1&key=Ur9E0GDrhkp4EwJOF4bCbg7EO7aIve54urjB8M%2BHevG1%2F6pDgRJ47Fvkmx4b%2FcMowlhV18ZYyVtF9pfG%2BM1EQA%3D%3D"
$json.RECORD | Foreach-Object {
$hours = 0
$date = ""
$_.Field | Foreach-Object {
if ($_ -match '\d{1,2}/\d{1,2}/\d{4}'){
$date = $_
}
else {
$hours = $_
}
}
$payload = #{
"Hours" = $hours
"Date Worked" = $date
}
Invoke-RestMethod -Method Post -Uri $endpoint -Body (ConvertTo-Json $payload)
}
If you want to make one API call after creating an array of $date and $hours pairs, you can do the following. This will only make one API call.
$endpoint = "https://api.powerbi.com/beta/d6cdaa23-930e-49c1-9d2a-0fbe648551b2/datasets/34eaea1e-73b6-4759-ac8b-aaae51708654/rows?noSignUpCheck=1&key=Ur9E0GDrhkp4EwJOF4bCbg7EO7aIve54urjB8M%2BHevG1%2F6pDgRJ47Fvkmx4b%2FcMowlhV18ZYyVtF9pfG%2BM1EQA%3D%3D"
$payloadArray = $json.RECORD | Foreach-Object {
$hours = 0
$date = ""
$_.Field | Foreach-Object {
if ($_ -match '\d{1,2}/\d{1,2}/\d{4}'){
$date = $_
}
else {
$hours = $_
}
}
$payload = #{
"Hours" = $hours
"Date Worked" = $date
}
$payload
}
Invoke-RestMethod -Method Post -Uri $endpoint -Body (ConvertTo-Json $payloadArray)
Exlanation:
If you want to have a payload per field pair, you will need to make an API call after $date and $hours are both set. Then repeat that process for each pair. If your payload supports an array structure, then you can create a collection of those pairs.
You are only doing the API call after all fields have been processed without creating a collection, which means you only get the last pair of $date and $hours values. As you iterate through your RECORD items, you are overwriting $date and $hours before the API call is ever made.
So I'm trying to pull the number of hours worked and the date worked from a table in my companies database to make a chart in Power BI through a streaming data set. I'm using powershell to parse a JSON file
Here's a JSON sample:
{"COUNT":"334","DISPLAY_LIST_START":"1","DISPLAY_LIST_STOP":"334","STOP":"334","RECORD":[{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["04/23/2018"]]},{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["04/24/2018"]]},{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["04/26/2018"]]},{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["04/30/2018"]]},{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["05/01/2018"]]},{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["4",["05/02/2018"]]},{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["05/03/2018"]]},{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["05/07/2018"]]},
I know it's not the best in terms of organization, but it's all I have to work with.
Here's the powershell code I have so far:
Invoke-WebRequest -Uri "http://wya.works/rta_develop/xmlServlet?&command=retrieve&sql=select%20%5B%24Hours%5D%2C%20%5B%24Date%20Worked%5D%20from%20%5B%21HOURS%5D%20&attributesOnly=Date%20Worked%2C%20Hours&contentType=JSON&referer=&time=1595443368507&key=696a6768"
$endpoint = "https://api.powerbi.com/beta/d6cdaa23-930e-49c1-9d2a-0fbe648551b2/datasets/91466553-d719-420c-9e3e-73e748379263/rows?noSignUpCheck=1&key=SU5GRBBWuuEIDSjqHW5hdgJzSMCQ3qUQ9mGrBDanjgpExv6woY1Sa1c3PC1Wk3WHHn1N%2FEpIuVgzHHcw0JXwYw%3D%3D"
$json.RECORD | Foreach-Object {
Write-Output "Checking Records"
$hours = 0
$date = ""
$json.FIELD | Foreach-Object{
Write-Output "Checking Field"
if ($_ -match '\d{1,2}/\d{1,2}\/d{4}'){
$date = $_
}
else {
$hours = $_
}
}
$payload = #{
"Hours" = $hours
"Date Worked" =$date
}
}
Invoke-RestMethod -Method Post -Uri "$endpoint" jlk-Body (ConvertTo-Json #($payload))
I need to parse through each record and pull the values of the hours (the numeric value in the JSON) and the Date (the date value).
When I run the code I don't get any errors, but it doesn't seem to be reaching the -match or the else statements. I tried logging the output on both and it returns nothing.
Is there something wrong with my loops?
I'm brand new to powershell and most of this code I got from help from other people, but I understand what its doing for the most part.
Also, anyone who knows about streaming datasets, will pulling this this way even give me what I want?
Store the Invoke-Webrequest values in your $json first. You missed that point; that's why you are getting Null.I don't know it is a typo or a miss.
Secondly, you $json.RECORD is wrong because it doesnt have any Record in the response. What you are looking for is basically the content. $json.content is going to give you the content of numbers.
$json=Invoke-WebRequest -Uri "http://wya.works/rta_develop/xmlServlet?&command=retrieve&sql=select%20%5B%24Hours%5D%2C%20%5B%24Date%20Worked%5D%20from%20%5B%21HOURS%5D%20&attributesOnly=Date%20Worked%2C%20Hours&contentType=JSON&referer=&time=1595443368507&key=696a6768"
Your endpoint and invoke-restmethod has nothing to do with your json parsing. First handle the response in the loop and see what is the outcome you are getting. I have structured it but I have not worked on the JSON sample data as if now:
$json=Invoke-WebRequest -Uri "http://wya.works/rta_develop/xmlServlet?&command=retrieve&sql=select%20%5B%24Hours%5D%2C%20%5B%24Date%20Worked%5D%20from%20%5B%21HOURS%5D%20&attributesOnly=Date%20Worked%2C%20Hours&contentType=JSON&referer=&time=1595443368507&key=696a6768"
$json.content | Foreach-Object {
Write-Output "Checking Records"
$hours = 0
$date = ""
$json.FIELD | Foreach-Object{
Write-Output "Checking Field"
if ($_ -match '\d{1,2}/\d{1,2}\/d{4}'){
$date = $_
}
else {
$hours = $_
}
}
$payload = #{
"Hours" = $hours
"Date Worked" =$date
}
}
$endpoint = "https://api.powerbi.com/beta/d6cdaa23-930e-49c1-9d2a-0fbe648551b2/datasets/91466553-d719-420c-9e3e-73e748379263/rows?noSignUpCheck=1&key=SU5GRBBWuuEIDSjqHW5hdgJzSMCQ3qUQ9mGrBDanjgpExv6woY1Sa1c3PC1Wk3WHHn1N%2FEpIuVgzHHcw0JXwYw%3D%3D"
Invoke-RestMethod -Method Post -Uri "$endpoint" jlk-Body (ConvertTo-Json #($payload))
The root of your problem is this:
$json.RECORD | Foreach-Object {
$json.FIELD | Foreach-Object{
...
}
}
Your outer foreach-object is looping over each item in the RECORD array, but the inner foreach-object is trying to loop over top-level FIELD properties that don't actually exist!
However, I think you'll hit more problems further along your code if you try to troubleshoot what you've already got, and there's an easier way to do what you're trying to do...
First, your sample json isn't quite valid - there's a trailing comma and some unclosed brackets so I'm going to reformat it with some line breaks so it's easier to read, and then fix it.
Note - you'll get the text from your call to Invoke-WebRequest, or as #Lee_Dailey suggested, Invoke-RestMethod.
# reformat the json, fix it, and assign it to a variable using a "Here-String"
# (see https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_quoting_rules?view=powershell-7#here-strings)
$text = #"
{
"COUNT":"334",
"DISPLAY_LIST_START":"1",
"DISPLAY_LIST_STOP":"334",
"STOP":"334",
"RECORD":[
{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["04/23/2018"]]},
{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["04/24/2018"]]},
{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["04/26/2018"]]},
{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["04/30/2018"]]},
{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["05/01/2018"]]},
{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["4",["05/02/2018"]]},
{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["05/03/2018"]]},
{"SESSION_ID":"c_a7FdTFicmxBJh9kln4V6gKxz_QErcufE7URF9m","FIELD":["6",["05/07/2018"]]}
]
}
"#
Then, we'll parse it - i.e. convert it from a string into an object model with well-defined properties:
# parse the json text
$json = $text | ConvertFrom-Json
And then process each record in turn:
$json.RECORD | ForEach-Object {
# get the number of hours worked. this is the first value in the FIELD array
# (note - the array starts at index 0 because it's zero-indexed)
$hours = $_.FIELD[0] # e.g. "6"
# get the "date worked". we need to get the second value (index 1) in
# the FIELD array, but this is a nested array, so once we've got the
# inner array we need to get the first value (index 0 again) from that
$date = $_.FIELD[1][0] # e.g. "04/23/2018"
# now we can build the payload...
$payload = #{
"Hours" = $hours
"Date Worked" = $date
}
# ...and invoke the api for this record
$endpoint = "https://api.powerbi.com/beta/d6cdaa23-930e-49c1-9d2a-0fbe648551b2/datasets/91466553-d719-420c-9e3e-73e748379263/rows?noSignUpCheck=1&key=SU5GRBBWuuEIDSjqHW5hdgJzSMCQ3qUQ9mGrBDanjgpExv6woY1Sa1c3PC1Wk3WHHn1N%2FEpIuVgzHHcw0JXwYw%3D%3D"
Invoke-RestMethod -Method Post -Uri "$endpoint" -Body (ConvertTo-Json #($payload))
}
Note the api call is inside the foreach loop, otherwise you end up calculating the $payload for each RECORD but only ever actually calling the api for the last one.
(I've also removed a spurious "jlk" from your final line, which is probably a typo).
So I have this script that goes out and finds all the software versions installed on machines and lets people know what software and when it was installed across several VMs.
I want to put this on a Dashboard provider we use but they have a specific format in which to use it.
it does produce a valid JSON however I just found out it's not in the format the company wishes.
which would be:
{"table": [["header1", "header2"], ["row1column1", "row1column2"], ["row2column1", "row2column2"]]}
My first thought would be to produce a header row as a beginning variable and then individual variables for each component but that feels very tedious and laborious to create variables for each individual row of data (Date, Name of Software, etc). then at the end combine them into 1 and convert to json
My script is this:
[CmdletBinding()]
Param (
[Parameter(ValueFromPipeline = $true,
ValueFromPipelinebyPropertyName = $true)]
[Alias("Servers")]
[string[]]$Name = (Get-Content "c:\utils\servers.txt")
)
Begin {
}
Process {
$AllComputers = #()
#Gather all computer names before processing
ForEach ($Computer in $Name) {
$AllComputers += $Computer
}
}
End {
ForEach ($Computer in $AllComputers) {
write-output "Checking $computer"
if ($computer -like "*x86*") {
$data = Invoke-Command -cn $computer -ScriptBlock {Get-ItemProperty HKLM:\Software\Microsoft\Windows\CurrentVersion\Uninstall\* | Select-Object #{Label = "ServerName"; Expression = {$env:computername}}, DisplayName, Publisher, DisplayVersion, InstallDate | Where-object { $_.Publisher -match "Foobar" }}
$jsondata += $data
}
else {
$data = Invoke-Command -cn $computer -ScriptBlock { Get-ItemProperty HKLM:\Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\* | Select-Object #{Label = "ServerName"; Expression = {$env:computername}}, DisplayName, Publisher, DisplayVersion, InstallDate | Where-object { $_.Publisher -match "foobar" } }
$jsondata += $data
}
}
$jsondata | ConvertTo-Json -depth 100 | Out-File "\\servername\C$\Utils\InstalledApps.json"
}
From the sample output format provided I would conclude that you are looking for an array of array. There is a "bug" using ConvertTo-Json when trying to do this but since we need it inside a table object anyway. I will show an example using your code but just on my local computer. Integrating this into your code should not be an issue.
# gather the results
$results = Get-ItemProperty HKLM:\Software\Microsoft\Windows\CurrentVersion\Uninstall\* | Where-object { $_.Publisher -match "The" } | Select-Object #{Label = "ServerName"; Expression = {$env:computername}}, DisplayName, Publisher, DisplayVersion, InstallDate
# Prepare an array of arrays for the output.
$outputToBeConverted = #()
# build the header
$header = ($results | Get-Member -MemberType NoteProperty).Name
$outputToBeConverted += ,$header
# Add the rows
Foreach($item in $results){
# Create a string array by calling each property individually
$outputToBeConverted += ,[string[]]($header | ForEach-Object{$item."$_"})
}
[pscustomobject]#{table=$outputToBeConverted} | ConvertTo-Json -Depth 5
Basically it is making a jagged array of arrays where the first member is your "header" and each row is built manually from the items in the $results collection.
You will see the unary operator , used above. That is done to prevent PowerShell from unrolling the array. Without that you could end up with one long array in the output.
I have a powershell script in which I do the following
$somePSObjectHashtables = New-Object Hashtable[] $somePSObject.Length;
$somePSObjects = Import-CSV $csvPath
0..($somePSObject.Length - 1) | ForEach-Object {
$i = $_;
$somePSObjectHashtables[$i] = #{};
$somePSObject[$_].PSObject.Properties | ForEach-Object {
$somePSObjectHashtables[$i][$_.Name] = $_.Value;
}
}
I need to do this because I want to make several distinct copies of the data in the CSV to perform several distinct manipulations. In a sense I'm performing an "INNER JOIN" on the resulting array of PSObject. I can easily iterate through $somePSObjectHashtables with a ForEach-Object and call Hashtable.Clone() on each member of the array. I can then use New-Object PSObject -Property $someHashTable[$i] to get a deep copy of the PSObject.
My question is, is there some easier way of making the deep copy, without an intermediary Hashtable?
Note that here is a shorter, maybe a bit cleaner version of this (that I quite enjoy):
$data = Import-Csv .\test.csv
$serialData = [System.Management.Automation.PSSerializer]::Serialize($data)
$data2 = [System.Management.Automation.PSSerializer]::Deserialize($serialData)
Note:
However, weirdly, it does not keep the ordering of ordered hashtables.
$data = [ordered] #{
1 = 1
2 = 2
}
$serialData = [System.Management.Automation.PSSerializer]::Serialize($data)
$data2 = [System.Management.Automation.PSSerializer]::Deserialize($serialData)
$data2
Will output:
Name Value
---- -----
2 2
1 1
While with other types it works just fine:
$data = [PsCustomObject] #{
1 = 1
2 = 2
}
$data = #(1, 2, 3)
For getting really deep copies we can use binary serialization (assuming that all data are serializable; this is definitely the case for data that come from CSV):
# Get original data
$data = Import-Csv ...
# Serialize and Deserialize data using BinaryFormatter
$ms = New-Object System.IO.MemoryStream
$bf = New-Object System.Runtime.Serialization.Formatters.Binary.BinaryFormatter
$bf.Serialize($ms, $data)
$ms.Position = 0
$data2 = $bf.Deserialize($ms)
$ms.Close()
# Use deep copied data
$data2
Here's an even shorter one that I use as a function:
using namespace System.Management.Automation
function Clone-Object ($InputObject) {
<#
.SYNOPSIS
Use the serializer to create an independent copy of an object, useful when using an object as a template
#>
[psserializer]::Deserialize(
[psserializer]::Serialize(
$InputObject
)
)
}
I am trying to retrieve more than 250 records from ServiceNow using Powershell cmdlet Invoke-RestMethod.
Is there a powershell script that I can use ?
Old topic but perhaps this answer can still be helpful.
I found that if I requested more than a certain number of results, seemingly nothing would be returned. Here's what works for me (change 300 to whatever number you want and remove any of the conditions after the ampersand):
$restapiuri = "https://yourserver.service-now.com/api/now/table/incident?sysparm_limit=300&sysparm_query=active=true^ORDERBYDESCnumber"
Use whatever method you prefer for credentials:
$SNowUser = “account username”
$SNowPass = ConvertTo-SecureString –String “Password” –AsPlainText -Force
$SNowCreds = New-Object –TypeName System.Management.Automation.PSCredential –ArgumentList $SNowUser, $SNowPass
Next should be fairly familiar. We're building the request, invoking it and assigning the results to a variable ($completeticket). Without adding " | out-string" you may see no results.
I'm also splitting the results into individual incidents by finding unique text in the first line of the results and assigning that to a variable ($separatetickets) and then iterating through each of them ($separateticket).
$i = 0
$headers = Get-HttpBasicHeader $Credentials
Invoke-RestMethod -uri $restapiuri -Headers $headers -Method GET -ContentType "application/json" |
% {
$completeticket = $_.result | Out-String
$separatetickets = $completeticket -split "whatever the first line of your record is"
foreach ($separateticket in $separatetickets) {
$i++
Write-Host
Write-Host "$i" -ForegroundColor White
Write-Host "$separateticket" -ForegroundColor Magenta
}
}