returned data from rest-API as JSON where the data/time is formated as
{
"userKey": ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤,
"userId": "¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤",
"userEmail": "¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤",
"userPrincipalName": "¤¤¤¤¤¤¤¤¤¤¤¤¤¤",
"displayName": "¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤",
"intuneLicensed": true,
"isDeleted": true,
"startDateInclusiveUTC": "2017-10-16T00:00:00Z",
"endDateExclusiveUTC": "9999-12-31T00:00:00Z",
"isCurrent": true,
"rowLastModifiedDateTimeUTC": "2017-10-17T00:24:11.8233333Z",
"PictureUrl": "¤¤¤¤¤¤¤¤¤¤¤¤.dk"
}
date/time is not formated properly, the data is from Intune DataWareHouse API
the data is requested via PowerShell my Question is it a PowerShell Shell Bug or this how JSON handel Data Formats
The Date Shold Appear as
MM/DD/YY HH:MM:SS
see the picture
The date is simply a string containing the date as ISO-8601 (because JSON doesn't have a DateTime type). Parse the date, and then format it as a different string:
PS> [datetime]::parse("9999-12-31T00:00:00Z").Tostring('MM/dd/yy HH:mm:ss')
12-31-99 01:00:00
i did a Convertfrom-json and then looping through every Note Property, if there is a better approach feel free to comment :)
$IntuneCollectionData | get-member -type NoteProperty | foreach-object {
if ($_.value -match "(\d{4}-[01]\d-[0-3]\dT[0-2]\d:[0-5]\d:[0-5]\d\.\d+)|(\d{4}-[01]\d-[0-3]\dT[0-2]\d:[0-5]\d:[0-5]\d)|(\d{4}-[01]\d-[0-3]\dT[0-2]\d:[0-5]\d)")
{
$NewDateTime = [datetime]::parse("$($_.value)").Tostring('MM/dd/yy HH:mm:ss')
$_.value = $NewDateTime
}
}
Related
I have retrieved json log data from a rest API as follows
[
{
"id": "6523276",
"type": "logs",
"attributes": {
"created-at": "2022-02-22T10:50:26Z",
"action": "delete",
"resource-name": "DocumentABC.docx",
"user-name": "Joe Smith"
}
},
{
"id": "6523275",
"type": "logs",
"attributes": {
"created-at": "2022-02-22T10:03:22Z",
"action": "create",
"resource-name": "Document123.docx",
"user-name": "Joe Smith"
}
},
{
"id": "6523274",
"type": "logs",
"attributes": {
"created-at": "2022-02-22T06:42:21Z",
"action": "open",
"resource-name": "123Document.docx",
"user-name": "Joe Smith"
}
}
]
I need to Post the json to another web app but I only want the last hour of logs.
In the json example above, the current time was 2022-02-22T10:55:22Z, therefore I'm only interested in the first two log entries.
For example
[
{
"id": "6523276",
"type": "logs",
"attributes": {
"created-at": "2022-02-22T10:50:26Z",
"action": "delete",
"resource-name": "DocumentABC.docx",
"user-name": "Joe Smith"
}
},
{
"id": "6523275",
"type": "logs",
"attributes": {
"created-at": "2022-02-22T10:03:22Z",
"action": "create",
"resource-name": "Document123.docx",
"user-name": "Joe Smith"
}
}
]
Here is my powershell v7 script
$json = $json | ConvertFrom-Json
$filterTime = (Get-date).AddHours(-1)
$RFCfilterTime = [Xml.XmlConvert]::ToString($filterTime,[Xml.XmlDateTimeSerializationMode]::Utc)
$Filteredjson = $json | Where-Object $json.attributes[0] -ge $RFCfilterTimefilterDate
$jsonToPost = ConvertTo-Json -InputObject #($Filteredjson) -Depth 5
The problem is ConvertFrom-Json changes the 'created-at' from RFC3339 format to 'datetime' format.
Therefore the Where-Object filter doesn't work...
id type attributes
-- ---- ----------
6523276 logs #{created-at=22/02/2022 10:50:26 AM; action…
6523275 logs #{created-at=22/02/2022 10:03:22 AM; action…
6523274 logs #{created-at=22/02/2022 6:42:21 AM; action=…
How do I change all of the 'created-at' objects back to RCF3339 format?
Is the
$json | Where-Object $json.attributes[0] -ge $RFCfilterTimefilterDate
statement being used correctly?
Is there any easier way altogether?
Your approach should work in principle, but there was a problem with your Where-Object statement - see the bottom section.
Mathias' answer shows how to work with the [datetime] instances that result from ConvertTo-Json's parsing directly, but a bit more work is required:
Indeed, in PowerShell (Core) v6+ ConvertFrom-Json (which with JSON web services is used implicitly by Invoke-RestMethod) automatically deserializes ISO 8601-format date-time strings such as "2022-02-22T10:03:22Z" into [datetime] System.DateTime instances, and, conversely, on (re)serialization with ConvertTo-Json, [datetime] instances are (re)converted to ISO 8601 strings.
While this enables convenient chronological comparisons with other [datetime] instances, such as returned by Get-Date, there is a major pitfall: Only [datetime] instances that have the same .Kind property value compare meaningfully (possible values are Local, Utc, and Unspecified, the latter being treated like Local in comparisons).
Unfortunately, as of PowerShell 7.2.1, you don't get to control what .Kind of [datetime] instances Convert-FromJson constructs - it is implied by the specific date-time string format of each string recognized as an ISO 8601 date.
Similarly, on (re)serialization with ConvertTo-Json, the .Kind value determines the string format.
See this answer for details.
In your case, because your date-time strings have the Z suffix denoting UTC, [datetime] instances with .Kind Utc are constructed.
Therefore, you need to ensure that your comparison timestamp is a Utc [datetime] too, which calling .ToUniversalTime() on the Local instance that Get-Date outputs ensures:
# Note the need for .ToUniversalTime()
$filterTime = (Get-Date).ToUniversalTime().AddHours(-1)
# Note: Only works as intended if all date-time strings are "Z"-suffixed
$filteredData = $data | Where-Object { $_.attributes.'created-at' -ge $filterTime }
However, at least hypothetically a given JSON document may contain differing date-time string formats that result in different .Kind values.
The way to handle this case - as well as the case where the string format is consistent, but not necessarily known ahead of time - you can use the generally preferable [datetimeoffset] (System.DateTimeOffset) type, which automatically recognizes timestamps as equivalent even if their expression (local vs. UTC) is different:
# Note the use of [datetimeoffset]
[datetimeoffset] $filterTime = (Get-Date).AddHours(-1)
# With this approach the specific format of the date-time strings is irrelevant,
# as long as they're recognized as ISO 8601 strings.
$filteredData = $data |
Where-Object { [datetimeoffset] $_.attributes.'created-at' -ge $filterTime }
Note: Strictly speaking, it is sufficient for the LHS of the comparison to be of type [datetimeoffset] - a [datetime] RHS is then also handled correctly.
Potential future improvements:
GitHub issue #13598 proposes adding a -DateTimeKind parameter to ConvertFrom-Json, so as to allow explicitly requesting the kind of interest, and to alternatively construct [datetimeoffset] instances.
As for what you tried:
Is the $json | Where-Object $json.attributes[0] -ge $RFCfilterTimefilterDate
statement being used correctly?
No:
You're using simplified syntax in which the LHS of the comparison (the -Property parameter) must be the name of a single (non-nested) property directly available on each input object.
Because nested property access is required in your case, the regular script-block-based syntax ({ ... }) must be used, in which case the input object at hand must be referenced explicitly via the automatic $_ variable.
.attributes[0] suggests you were trying to access the created-at property by index, which, however, isn't supported in PowerShell; you need to:
either: spell out the property's name, if known: $_.attributes.'created-at' - note the need to quote in this case, due to use of the nonstandard - char. in the name.
or: use the intrinsic .psobject member that provides reflection information about any given object: $_.attributes.psobject.Properties.Value[0]
Thus, with spelling out the property name - and with making sure that the LHS [datetime] value is represented as an ISO 8601-formatted string too, via .ToString('o') - your statement should have been:
$json | Where-Object {
$_.attributes.'created-at'.ToString('o') -ge $RFCfilterTimefilterDate
}
The fact that newer version of ConvertFrom-Json implicitly parses timestamps as [datetime] is actually to your advantage - [datetime] values are comparable, so this simply means you can skip the step where you convert the threshold value to a string:
$data = $json | ConvertFrom-Json
$filterTime = (Get-Date).AddHours(-1)
$filteredData = $data | Where-Object {$_.attributes.'created-at' -ge $filterTime}
$jsonToPost = ConvertTo-Json -InputObject #($filteredData) -Depth 5
I'll say first that I'm not very experienced with PowerShell, so many things are not obvious to me.
I've got the following command:
$JSON = Get-ADUser -Filter "SAMAccountName -eq 'jdray'" -Properties SAMAccountName,
Name,
DisplayName,
Manager,
SAMAccountType,
pwdLastSet,
Created,
AccountExpirationDate,
lastLogonTimestamp,
DistinguishedName|Select-object -Property SAMAccountName,
Name,
#{name='Manager';expression={ ([ADSI]"LDAP://$($_.manager)").displayname }},
Enabled,
DistinguishedName,
Created,
AccountExpirationDate,
LastLogonTimestamp,
PasswordLastSet|ConvertTo-Json
The output of this is:
{
"SAMAccountName": "jdray",
"Name": "Ray, James",
"Manager": "Lady, Boss",
"Enabled": true,
"DistinguishedName": "CN=Ray\\, James,OU=...[yadda yadda yadda]...DC=com",
"Created": "\/Date(1554387070000)\/",
"AccountExpirationDate": "\/Date(1643702400000)\/",
"LastLogonTimestamp": 132732539928955494,
"pwdLastSet": 132729945927805005
}
First of all, I'm confused why AccountExpirationDate and Created have one format, but LastLogonTimestamp and pwdLastSet are formatted differently.
What I really want is to have all the DateTime objects output in a human-readable DateTime format. I tried -ExpandProperty, but couldn't figure out how to make it work. It appears to add a property named DateTime to the end of the output, formatted reasonably: "DateTime": "Thursday, April 4, 2019 7:11:10 AM", but it doesn't name it. The Microsoft documentation on the subject is very light on useful examples, as is the rest of the web.
I tried following the #{} format for the Manager expression, which I copied from someone else's script. Unfortunately my PowerShell isn't strong enough to figure out a reasonable expression. In C#, something like Created.ToShortDateString() would suffice (presuming Created is a DateTime).
So, how do I consistently output named, human-readable DateTime property values in a JSON array?
EDIT
Using a proposed answer below, I modified the four timestamp fields using the same pattern:
#{n='Created';e={[DateTime]::FromFileTime($_.Created)}},
#{n='AccountExpirationDate';e={[DateTime]::FromFileTime($_.AccountExpirationDate)}},
#{n='LastLogonTimestamp';e={[DateTime]::FromFileTime($_.lastLogonTimestamp)}},
#{n='pwdLastSet';e={[DateTime]::FromFileTime($_.pwdLastSet)}}
Here's the (frustrating) result I got:
"Created": "\/Date(1554387070000)\/",
"AccountExpirationDate": "\/Date(1643702400000)\/",
"LastLogon": {
"value": "\/Date(1628780392895)\/",
"DateTime": "Thursday, August 12, 2021 7:59:52 AM"
},
"pwdLastSet": 132729945927805005
Also of note, here's the output for the four DateTime fields before it gets translated to JSON:
Created : 4/4/2019 7:11:10 AM
AccountExpirationDate : 2/1/2022 12:00:00 AM
LastLogonTimestamp : 8/12/2021 7:59:52 AM
PasswordLastSet : 8/9/2021 7:56:32 AM
Why does that formatting work for one DateTime instance but not the rest?
How do I extract DateTime from the format of the pwdLastSet field, which is clearly a different animal?
How do I get a simple property:value pair, such as "LastLogonTimestamp": "Thursday, August 12, 2021 7:59:52 AM"?
Use DateTime.FromFileTime(LastLogonTimestamp)
$JSON=Get-ADUser -Filter {Enabled -eq $true} -Properties Name,Manager,LastLogon |
Select-Object Name,Manager,#{n='LastLogon';e={[DateTime]::FromFileTime($_.LastLogon)}}
I am trying to read JSON data which is stored in VM notes. Below is the command i execute to get the VM notes
Get-VM testbox |format-list Notes
The output is
Notes : {
"Program": "AAA",
"Project": "BBBB"
}
I want to read the value of Program into a variable. How can i do it ?
Use ConvertFrom-JSON to parse the JSON-value in your notes-property. I'd store the converted notes in a variable just in case you need to access Project or another part of the json later. Try:
$vm = Get-VM testbox
$notes = $vm.Notes | ConvertFrom-JSON
$mynewvar = $notes.program
Your json is not valid, if it was this would work:
(Get-VM -VMName TestBox).notes | ConvertFrom-Json
Valid Json:
{
"Notes":
{
"Program": "AAA",
"Project": "BBBB"
}
}
Totally untested (and can't test right now) but something like:
Get-VM testbox | Select-Object -ExpandProperty Notes | ConvertFrom-Json
Documentation for ConvertFrom-Json here
Powershell command-let ConvertTo-Json has the following limitations
1) It returns Enum values as Integers instead of their text
2) It doesn't return the date in a readable format
For point #1 see below, Status and VerificationMethod Properties
PS C:\Windows\system32> Get-Msoldomain | ConvertTo-Json
{
"ExtensionData": {
},
"Authentication": 0,
"Capabilities": 5,
"IsDefault": true,
"IsInitial": true,
"Name": "myemail.onmicrosoft.com",
"RootDomain": null,
"Status": 1,
"VerificationMethod": 1
}
To handle this, I changed my command as below
PS C:\Windows\system32> Get-Msoldomain | ConvertTo-Csv | ConvertFrom-Csv | ConvertTo-Json
{
"ExtensionData": "System.Runtime.Serialization.ExtensionDataObject",
"Authentication": "Managed",
"Capabilities": "Email, OfficeCommunicationsOnline",
"IsDefault": "True",
"IsInitial": "True",
"Name": "curtisjmspartnerhotmail.onmicrosoft.com",
"RootDomain": "",
"Status": "Verified",
"VerificationMethod": "DnsRecord"
}
Now you see, that the enums are being returned with their text values above (Status and VerificationMethod) instead of their integer values.
However, There are a few limitations with this approach:
1) ConvertTo-Csv doesn't retain the Arrays or Complex Objects, and
outputs them as their Class Names (Watch the ExtensionData Properties
in both the outputs). In the second output, we tend to lose the data,
and just get the className
System.Runtime.Serialization.ExtensionDataObject as a string
2) Both ConvertTo-Csv and ConvertFrom-Csv are not the script-level
commandlets, but they are command-level commandlets, which means that
we can't use them at the end of the script , but they will have to be
used with the individual commands like I am doing above. WHEREAS,
ConvertTo-Json need not be applied at the commmandLevel, but just
applied once for the script output.
My question is:
1) How do I still use the convertTo-Json, so that all my enum properties are returned with their texts and not integers, and ALSO the Complex Objects or Arrays are not lost? In the approach I have used, the complex objects are getting lost
2) Also, it should be generic enough so that It can be applied at the end of the script, and not at the command level
ConvertTo-Json and ConvertTo-Csv are both forms of serializing objects in some sort of text representation and both are useful in different use cases.
ConvertTo-Csv is perhaps best used for 2-dimensional data that can be expressed in a table such as a spreadsheet. Hence it is awkward to try to convert "complex" objects, i.e. those with properties that contain other structured data, into a simple table. In such cases PowerShell represents such data as the full name of the data type.
ConvertTo-Json is capable of serializing more complicated objects, since the format allows for nested arrays/data structures, e.g. the ExtensionData property in your example. Note that you may need to use the -Depth parameter to ensure that deeply nested data is serialized correctly.
So the problem really comes down to how the ConvertTo-Json cmdlet serializes enums, which can be demonstrated with:
[PS]> (Get-Date).DayOfWeek
Tuesday
[PS]> (Get-Date).DayOfWeek.GetType()
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True DayOfWeek System.Enum
[PS]> Get-Date | Select DayOfWeek | ConvertTo-Json
{
"DayOfWeek": 2
}
So before you convert to JSON you need to ensure that the DayOfWeek property (in this example) or Status and VerificationMethod properties (from your example) are converted to their string equivalents first.
You can do this using an expression with Select-Object to convert the data as it passes down the pipe. Note that you do need to include all the properties that you want included in the final JSON:
[PS]> Get-Date |
Select DateTime,#{Label="DayOfWeek";Expression={$_.DayOfWeek.ToString()}} |
ConvertTo-Json
{
"DateTime": "13 June 2017 10:33:51",
"DayOfWeek": "Tuesday"
}
So in your case you'd need something like this:
[PS]> Get-Msoldomain |
Select-Object ExtensionData,IsDefault,IsInitial,Name,RootDomain `
,{Label="Authentication";Expression={$_.Authentication.ToString()}} `
,{Label="Capabilities";Expression={$_.Capabilities.ToString()}} `
,{Label="Status";Expression={$_.Status.ToString()}} `
,{Label="VerificationMethod";Expression={$_.VerificationMethod.ToString()}} |
ConvertTo-Json
#puneet, following your comment on my other answer, here is an example of how you might build up a new object, based on an existing one, with the Enum types converted to strings.
The idea is to create a new "empty" object, then loop through all the properties of the original object and add them to the new one, but if any of the original properties are Enums, then those are converted to strings.
$data = [PSCustomObject]#{}
(Get-Date).PSObject.Properties | Select Name,Value | Foreach-Object {
if($_.Value.GetType().BaseType.FullName -eq "System.Enum"){
$data | Add-Member -MemberType NoteProperty -Name $_.Name -Value $_.Value.ToString()
}
else {
$data | Add-Member -MemberType NoteProperty -Name $_.Name -Value $_.Value
}
}
$data | ConvertTo-Json
You may want to finesse this a little for your own application, but hopefully the idea behind it is clear. Definitely check to see that all the properties are being treated correctly in the JSON output.
to keep enum,array and date when converting psObject to json, you can use newtonsoft. a sample here https://github.com/chavers/powershell-newtonsoft using Nerdy Mishka powershell module.
$obj = New-Object pscustomobject -Property #{Enum = (Get-DAte).DayOfWeek; int = 2; string = "du text"; array = #("un", "deux", "trois"); obj= #{enum = (Get-DAte).DayOfWeek; int = 2; string = "du text"; array = #("un", "deux", "trois")}}
Import-Module Fmg-PrettyJson
$settings = Get-NewtonsoftJsonSettings
$enumconv = "Newtonsoft.Json.Converters.StringEnumConverter"
$e = New-Object $enumconv
$settings.Converters.Add($e)
Set-NewtonsoftJsonSettings $settings
$obj | ConvertTo-NewtonsoftJson
return:
{
"array": [
"un",
"deux",
"trois"
],
"enum": "Thursday",
"int": 2,
"obj": {
"enum": "Thursday",
"array": [
"un",
"deux",
"trois"
],
"int": 2,
"string": "du text"
},
"string": "du text"
}
In Powershell, when I use the invoke-webrequest cmdlet in my script, I get data returned in a JSON with a variety of data types. However, integers and dates do not seem to work correctly with the convertfrom-json cmdlet.
Here is an example of the code:
$json1 = Invoke-webrequest -URI $URI -Certificate $cert -Headers $header | convertfrom-json
A field located here should have numbers in it, and they do in the response before the pipeline to convertfrom-json:
Number = $json1.workers.workAssignments.Number
So, how can I extract the number and date fields? Preferably I would like to loop through and add every employee and every attribute to a datatable, disconnected recordset, pscustomobject, or psobject & then export to CSV.
I have coded each of the above objects several different times, and none of them have I gotten the integers or dates to export correctly. So, in lieu of posting a bunch of my solutions I figured I would keep it basic with the above and ask how to get the variable out of $json1 that contains integers and into some object to export to CSV.
I am not married to convertfrom-json whatsoever.
Example JSON:
{
"workers": [ {
"associateOID": "xxxxxxxx",
"workerID": {
"idValue": "xxxxxxxx"
},
"legalName": {
"givenName": "Lebron",
"middleName": "King",
"familyName1": "James",
"formattedName": "James, Lebron King"
},
"birthDate": "1985-01-01",
"baseRemuneration": {
"payPeriodRateAmount": {
"amountValue": 25,000,000,
"currencyCode": "USD"
},
"effectiveDate": "2016-01-01"}
When using convertfrom-json, that amount in "Amount Value" results in Zero no matter what method I use.