I'm beginner in power shell ( HTML ) and I need some help. I want to get a variable from the McAfee website. In this page there is a table where you can download the latest .dat files.
I only need the version number - now it's 8963 - from the first table ( Download V2 Virus Definition Updates (DATs) ) and this result needs to be saved in a variable because I want to compare it with other variable from another script.
Now I'm able to query all of the tables with all of the data:
$r = Invoke-WebRequest -Uri https://www.mcafee.com/enterprise/en-us/downloads/security-updates.html
$r.parsedhtml.getelementsbytagname("TR") |
% { ( $_.children | ?{ $_.tagName -eq "td"} | % innerText ) }
Write-Host
Unfortunately it's too much information for me, because list all of the data which is in a table.
The retrived data in pic.: data
Thanks for the help.
Well this isn't the cleanest method, but if you expect the first entry to allwayse be the one supplying the Filename of the needed .exe you could use this:
($r.parsedhtml.getelementsbytagname("TR") |% { ( $_.children | ?{ $_.tagName -eq "td"} | % innerText ) } | Select-Object -First 1).Split('xdat')[0]
This basically takes the first String of your query, splits the string on the position of xdat and than takes everything before xdat.
The Output then is 8963.
If you want to do it the more correct way, you should look out for the correct html-attribute to filter for.
Related
Hi I have a script that reads a csv file, creates a json file, checks the users in the file against a service, then i get the result as a json file.
I take that result and finds the users i csv file and creates a new file.
I do that with a where-object
But i need to add some extra values on every user before i export it to csv
This is my 2 lines for finding users and then export
$matches = $users | where-object { $_.number -in $response.allowedItemIds } | Select-Object -Property Number,Surname,Forename,Emailaddress
$matches | Export-Csv -path $Saved$savefile -NoTypeInformation -Append
Is that possible or do i need to do a for each?
Cheers
Assuming I've interpretted your question correctly, you should be able to use PowerShell's Calculated Properties for this purpose.
For example, if you wanted to add a field called "Date" and set the current Date/Time to each user row, you could do the following:
$matches = $users | where-object { $_.number -in $response.allowedItemIds } | Select-Object -Property Number,Surname,Forename,Emailaddress, #{Name="Date";Expression={Get-Date}}
The Expression value can either be a static value such as "StaticValue", a variable such as $i (useful if used as part of a loop, for example) or more complex value that is returned from other cmdlets (as in my example above)
I am currently writing a script for the following problem:
Initial Problem
Data was exported from an Audit System into a CSV. The CSV itself consists of several columns of which one column has JSON data inside. Sadly there arent many options to influence the export / structure of the export. Since the amount of data included there is tough to filter and to analyse, the exported CSVs (when needed) need to be transformed so that only relevant columns and JSON keys are remaining within the new to-be-exported CSV. It has to be a new CSV as the file needs to potentially be shared. A textfile to-be-imported contains the relevant JSON keys that should remain in the to-be-exported CSV.
About the JSON: The keys can vary based on the events that are exported. Lets say there are 3-4 different variants but the textfile to be imported contains for all 3-4 keys the relevant subkeys that need to be included as new column in the export. If the subkey does not exist its okay if that particular column is empty in the export.
Initial Thoughts
a) Import the CSV and the file that is listing the relevant JSON keys that should be kept
b) Expand the JSON
c) Select the JSON entries that are relevant
d) Merge everything into new columns
e) Export again into a new file
What Questions are open / Where are the Problems?
I was writing some piece of code (I started my PS experience just 2 days ago) and encountered/wondering the following:
Are there any recommendations to improve the code since I am sure to do my very recent PS adventure there are probably many obvious things that have to be improved
Is there a way to make the export straight into a CSV format without the manual -join and then using Out-File?. I noticed that for my final test cases (I cannot share those because the data is extremely hard to anonymize) I didnt manage to come up with a delimiter (tried ",", ";" and "´t") that doesnt seem to be included in parts of the imported cells. Excel (when importing from text) doesnt seem to have an issue tho loading and parsing the data as CSV and to recognize the columns and boundaries correctly.
Happy to hear any tips!
Current Code
### Configure variables
$inputPath = "C:\Users\test\Downloads\inputTest.csv"
$formatTemplate = "C:\Users\test\Downloads\templateTest.txt"
$outputPath = "C:\Users\test\Downloads\outputTest.csv"
### Load the columns from template file to perform transformation depending on the required AuditData Fields. The file contains a list of relevant JSON keys from the Audit Data columns
$selectedAuditDataFields = Get-Content $formatTemplate
### Load CSV, select needed columns and expand the JSON entries within the AuditData column
$importCsvCompact = Import-Csv -Path $inputPath -Delimiter "," | Select-Object -Property CreationDate, UserIds, Operations, #{name = "AuditData"; Expression = {$_.AuditData | ConvertFrom-Json }}
### Calculate the number of Rows (import and export CSV have same number of rows) and Columns (3 standard columns plus template columns) for the CSV to be exported
$exportCsvNumberOfRows = $importCsvCompact.Count
$exportCsvNumberOfColumns = $selectedAuditDataFields.Length + 3
### Add header to to-be-exported-CSV
$header = [object[]]::new($exportCsvNumberOfColumns);
$header[0] = "CreationDate"
$header[1] = "UserIds"
$header[2] = "Operations"
for($columnIncrement = 3; $columnIncrement -ne $exportCsvNumberOfColumns; $columnIncrement++) {
$header[$columnIncrement] = ($selectedAuditDataFields[$columnIncrement-3])
}
$toAppend = $header -join ","
$toAppend | Out-File -FilePath $outputPath -Append
### initiate array for each transformed row and initiate counter of current row
$processingRowCounter = 0
### traverse each row of the CSV import and setup the new column structure
### connect the 3 standard columns with a subset of the expanded JSON entries (based on the imported template)
$importCsvCompact | ForEach-Object {
$csvArrayColumn = [object[]]::new($exportCsvNumberOfColumns);
$csvArrayColumn[0] = $importCsvCompact.CreationDate[$processingRowCounter]
$csvArrayColumn[1] = $importCsvCompact.UserIds[$processingRowCounter]
$csvArrayColumn[2] = $importCsvCompact.Operations[$processingRowCounter]
for($columnIncrement = 3; $columnIncrement -ne $exportCsvNumberOfColumns; $columnIncrement++) {
$csvArrayColumn[$columnIncrement] = $importCsvCompact.AuditData.($selectedAuditDataFields[$columnIncrement-3])[$processingRowCounter]
}
$processingRowCounter++
$directExport = $csvArrayColumn -join ","
$directExport | Out-File -FilePath $outputPath -Append
Write-Host "Processed $processingRowCounter Rows..."
}
Testfiles
templateTest.txt
https://easyupload.io/vx7k75
inputTest.csv
https://easyupload.io/ab77q9
Current Version based on Feedback
### Configure variables
$inputPath = "C:\Users\forstchr\Downloads\inputTest.csv"
$formatTemplate = "C:\Users\forstchr\Downloads\templateTest.txt"
$outputPath = "C:\Users\forstchr\Downloads\outputTest.csv"
### Load the columns from template file to perform transformation depending on the required AuditData Fields. The file contains a list of relevant JSON keys from the Audit Data columns
$selectedAuditDataFields = Get-Content $formatTemplate
### Calculate the number of Rows (import and export CSV have same number of rows) and Columns (3 standard columns plus template columns) for the CSV to be exported
$exportCsvNumberOfRows = $importCsvCompact.Count
$exportCsvNumberOfAuditColumns = $selectedAuditDataFields.Length
###Load CSV, select needed columns and expand the JSON entries within the AuditData column
Import-csv -Path $inputPath -Delimiter "," | Select-Object -Property CreationDate, UserIds, Operations, #{name = "AuditData"; Expression = {$_.AuditData | ConvertFrom-Json }} | % {
[pscustomobject]#{
'CreationDate' = $_.CreationDate
'UserIds' = $_.UserIds
'Operations' = $_.Operations
# the next part is not correct but hopefully displays what I am trying to achieve with the JSON in the AuditData column
for($auditFieldIncrement = 0; $auditFieldIncrement -ne $exportCsvNumberOfAuditColumn; $auditFieldIncrement++) {
'$selectedAuditDataFields[$auditFieldIncrement]' = $_.AuditData.($selectedAuditDataFields[$auditFieldIncrement])
}
}
} | Export-csv $outputPath
I have had to produce a "cleansed" csv file in one project. My general approach was as follows: import the existing csv data, and send it through the pipeline.
Foreach-object, do some processing, storing the results in variables. The last step in processing creates a hashtable typecast as a pscustomobject, and this result in passed through the pipeline. The output of the second pipeline is fed to Export-csv. Export-csv does all the joining and the commas for me, and also encloses the output fields in quotes, making them strings.
Here is a code snippet that illustrates the approach. The cleansing consists of reformatting dates so that they use a standard 14 digit format, and reformatting currency amounts so that they don't contain dollar signs. But that is not relevant to you.
Import-csv checking.csv | % {
$balance += [decimal]$(Get-Amount $_.AMOUNT)
[pscustomobject]#{
'TRNTYPE' = Get-Type $_.AMOUNT
'DTPOSTED' = (Get-Date $_.DATE).Tostring('yyyyMMddHHmmss')
'TRNAMT' = Get-Amount $_.AMOUNT
'FITID' = $fitid++ #this is a stopgap
'NAME' = $_.DESCRIPTION
'MEMO' = $memo
}
} |
Export-csv transactions.csv
Get-Type is a function that yields 'CREDIT' or 'DEBIT' depending on the sign of the amount. Get-Amount is a function that gives a numeric amount without commas and dollar signs. Those functions are defined at the beginning of the script. Note that, when you call a powershell function, there are no parentheses involved. That was a big jolt to me, but it's actually a feature of powershell.
hoping someone can help out as im banging my head against the wall a little here ...
I have a simple CSV file, 2 columns, 1 for HostName and 1 for Password.
For example;
HostName,Password
Computer1,Password1
Computer2,Password2
etc
What I need to do is read the password from the CSV, based on the HostName of the device the script is running on.
This is for BitLocker PIN encryption and so instead of me specifying the PIN in my current PowerShell script, it would read the value from the CSV and encrypt using that.
Cheers guys,
In a very general way, you can grab your object (CSV) file and filter with where-object. Once that is stuffed into a variable, you can just call the property with a "."
$Comp1 = Get-MyCSV | Where {$_.HostName -eq Computer1}
$Comp1.Password
Resolved now thanks for the input chaps.
$InSecureString = Import-Csv "C:\EncryptionPasswords.csv"
$BitLockerPassword = $InSecureString.Where({$PSItem.ComputerName -eq $env:COMPUTERNAME}).Password
$BitLockerPassword
Then called the $BitLockerPassword variable and converted to SecureString.
An alternative to using Where-Object every time you need to retrieve a password based on hostname, would be to create a hashtable from the values:
$LookupTable = #{}
Import-Csv .\passwords.csv |ForEach-Object {
$LookupTable[$_.HostName] = ConvertTo-SecureString $_.Password -AsPlainText -Force
}
Now you can retrieve the corresponding password as a secure string directly from the hashtable:
$MachineName = $env:ComputerName
Enable-BitLocker -MountPoint "C:" -Pin $LookupTable[$MachineName] -TPMandPinProtector
I'm writing a script which displays the last 7 backups of my server. The scripts outputs the following: TimeCreated, Level, ID, Message,Size in a html table.
my script:
$lastsevendays = (get-date) - (new-timespan -day 7)
$MyObject = Get-WinEvent -FilterHashTable #{LogName='Microsoft-Windows-Backup'; StartTime=$lastsevendays}
$BackupSize = #{ Name = "BackupSize"; Expression = { (($objFSO.GetFolder("C:\Users\LaBackup").Size) / 1MB) } }
$MyObject |
Select 'TimeCreated', 'LevelDisplayName','ID', 'Message', $BackupSize |
ConvertTo-HTML -Head $Header |
Out-file C:\Users\script.htm
I want the script to run everyday and automatically remove backups row which date back from more than 7 seven.
I think i'm supposed to select row which have "Date" less or equal than seven days ago,
and delete them to add more recent row.
I have no idea how i could do this as, ConvertTo-HTML seems to regenerate a whole table each time it's called.
Some ideas??
thanks in advance
Give this a try:
Get-WinEvent -LogName Microsoft-Windows-Backup | Where-Object -FilterScript { $_.TimeCreated -gt [DateTime]::Now.AddDays(-7); }
This uses the Where-Object cmdlet to filter events that are older than 7 days in the Microsoft-Windows-Backup log.
EDIT
I guess I misunderstood what you were looking for. All I did was change the -lt operator to -gt.
I am new to powershell. Currently we are in need of a poweshell script to compare two large (100000 rows and n columns (n > 300, also column headers are Dates corresponding to each wednesday). The value of n keeps on incrementing each week in the file. We need to compare the files (current week and last week), and need to make sure that the only difference between the two files is the last column.
I have gone through some Forums and Blogs and I could do only Little due to my ignorance.
If there is a way to drop the last column from a csv file in powershell, we may be able to make use of the below script below to compare the previous week's file and the current week's file after droping the last column from current week's file.
It would be really helpful if someone can help me here with your hard earned knowledge
[System.Collections.ArrayList]$file1Array = Get-Content "C:\Risk Management\ref_previous.csv"|Sort-Object
[System.Collections.ArrayList]$file2Array = Get-Content "C:\Risk Management\ref_current.csv"|Sort-Object
$matchingEntries = #()
foreach ($entry in $file1Array) {
if ($file2Array.Contains($entry)) {
$matchingEntries += $entry
}
}
foreach ($entry in $matchingEntries){
$file1Array.Remove($entry)
$file2Array.Remove($entry)
}
Cheers,
Anil
Assuming that the column name you want to exclude is LastCol (adjust to your actual column name):
$previous = Import-csv "C:\Risk Management\ref_previous.csv" | Select-Object -Property * -ExcludeProperty LastCol | Sort-Object;
$current = Import-csv "C:\Risk Management\ref_current.csv" | Sort-Object;
Compare-Object $previous $current;
This will drop the last column from each of the input files and indicate whether the remaining content differs.
Based on the answer that alroc gave, you should be able to get the last column name using a split operation on the first line of the CSV file, and then using that on the -ExcludeProperty parameter.
However, the Compare-Object command on this doesn't work for me, but it does pull back the right data into each variable.
$CurrentFile = "C:\Temp\Current.csv"
$PreviousFile = "C:\Temp\Previous.csv"
$CurrentHeaders = gc $CurrentFile | Select -First 1
$CurrentHeadersSplit = $CurrentHeaders.Split(",")
$LastColumn = $CurrentHeadersSplit[-1] -Replace '"'
$Current = Import-Csv $CurrentFile | Select -Property * -ExcludeProperty $LastColumn | Sort-Object
$Previous = Import-Csv $PreviousFile | Sort-Object
Compare-Object $Current $Previous
The import-csv and export-csv both give the opportunity to exclude columns.
The import-csv has the -header option and you simply name the incoming headers and exclude the last columns header. If there are 10 columns, only name 9. The last column will be excluded.
For export-csv, select the columns you'd like to write out ( |select col1,col2,col3|export-csv... ) and don't select the column you're trying to exclude.