Select (or split) column from SQLCMD output - json

How do I select the columns from SQLCMD output in PowerShell v1? I'm trying to make JSON output using Write-Output in PowerShell v1.
Query output at the end.
$_ returns both columns. If only we could use $_.name and $_.jobid, but they both returns empty lines. Fixing this would be the preferred solution.
Here is the PowerShell command:
Write-Output '{"data":[';
(SQLCMD -S 'x.x.x.x' -U 'user' -P 'passwors' -i "C:\query.sql" -W) | %{
try {
($coma + '{"{#JOBID}":"' + $_ + '",' + '"{#JOBNAME}":"' + $_ + '"}');
$coma=',';
} catch {}
};
Write-Output "]}"
What it returns:
{"data":[
,{"{#JOBID}":"12345-aaaa-1234-5678-000000000000000 Clear DB entries","{#JOBNAME}":"12345-aaaa-1234-5678-000000000000000 Clear DB entries"}
,{"{#JOBID}":"12345-bbbb-1234-5678-000000000000000 TempLog DB","{#JOBNAME}":"12345-bbbb-1234-5678-000000000000000 TempLog DB"}
]}
What I expect:
{"data":[
,{"{#JOBID}":"12345-aaaa-1234-5678-000000000000000","{#JOBNAME}":"Clear DB entries"}
,{"{#JOBID}":"12345-bbbb-1234-5678-000000000000000","{#JOBNAME}":"TempLog DB"}
]}
I'm not sure how to use split with tab delimiter ($_ -split "t") for both job_id and name. My attempts either returned both column names as one and in some cases it returned empty.
Here is the query and its output on a command line:
PS C:\> SQLCMD -S 'x.x.x.x' -U 'user' -P 'password' -i "C:\query.sql" -W
job_id name
12345-aaaa-1234-5678-000000000000000 Clear DB entries
12345-bbbb-1234-5678-000000000000000 TempLog DB
(2 rows affected)
I know about ConvertTo-Json on version 3, but I want to get it working on PowerShell v1 so it'd be helpful to those who can't upgrade for whatever reason.

The workaround I have come up with is to use ($_ -split ' ')[1..100] for second column #JOBNAME and ($_ -split ' ')[0] for first column #JOBID.
Note: This only works because the JOBID column values is phrased as one single word. It doesn't work for other queries if the first column has random number of words.
Here is the final command and output :
$coma=''; Write-Output '{"data":[';
(SQLCMD -S 'x.x.x.x' -U 'user' -P 'passwors' -i "C:\query.sql" -W) | %{
try {
($coma + '{"{#JOBID}":"' + ($_ -split ' ')[0] + '",' + '"{#JOBNAME}":"' + ($_ -split ' ')[1..100] + '"}');
$coma=',';
} catch {}
};
Write-Output "]}"
Output:
{"data":[
,{"{#JOBID}":"12345-aaaa-1234-5678-000000000000000","{#JOBNAME}":"Clear DB entries"}
,{"{#JOBID}":"12345-bbbb-1234-5678-000000000000000","{#JOBNAME}":"TempLog DB"}
]}
There is also Invoke-SQLcmd method but it takes 30 seconds to add snap-in and 2 seconds to execute query.
Add-PSSnapin SqlServerCmdletSnapin100; $coma=''; Write-Output '{"data":[';
(Invoke-Sqlcmd -ServerInstance 'x.x.x.x' -username 'user' -password 'password' -inputfile "C:\query.sql") | %{
try {
($coma + '{"{#JOBID}":"' + $_.job_id + '",' + '"{#JOBNAME}":"' + $_.name + '"}');
$coma=',';
} catch {}
};
Write-Output "]}"
If you have Powershell v3 then you could just use Sqlcmd... | ConvertTO-Json

Another method, seems reliable. Thanks to Bacon Bits answer.
$coma=''; Write-Output '{"data":[';
(SQLCMD -S 'x.x.x.x' -U 'user' -P 'password' -i "C:\query.sql" -W -m 1 -s `"`t`") | ConvertFrom-Csv -Delimiter "`t" | Select-Object -Skip 1 | %{
try {
($coma + '{"{#JOBID}":"' + $_.job_id + '",' + '"{#JOBNAME}":"' + $_.name + '"}');
$coma=',';
} catch {}
};
Write-Output "]}"
If your data contains tabs, you'll need a different separator. The Select-Object -Skip 1 is to skip the underline row that sqlcmd always creates below the header.
Also be aware that you should use the -w parameter on sqlcmd to prevent any incorrect wrapping. Also beware that null values are always output as a literal string NULL.
Again Powershell v3 or Invoke-SQLcmd is recommended over this method.

Related

find specific field and insert new value with jq

I have this command in a bash script
kubectl get svc --selector='app.kubernetes.io/component=sentinel' --all-namespaces -o json |
jq -r '
.items
| map(
{label:.metadata.name,
sentinels: [{host:(.metadata.name + "." + .metadata.namespace + "." + "svc" + "." + "cluster" + "." + "local"),port: .spec.ports[0].port}],
sentinelName:"mymaster",
sentinelPassword: ""
dbIndex: 0
})
| {connections: . }' /
>local.json
which produces something like this output
{
"connections": [
{
"label": "Redis1",
"sentinels": [
{
"host": "Redis1.default.svc.cluster.local",
"port": 26379
}
],
"sentinelName": "mymaster",
"sentinelPassword": "",
"dbIndex": 0
},
{
"label": "Redis2",
"sentinels": [
{
"host": "Redis2.development.svc.cluster.local",
"port": 26379
}
],
"sentinelName": "mymaster",
"sentinelPassword": "",
"dbIndex": 0
}
]
}
This config file is injected into container via an init-container so Redis-Commander fetches the redis instances without the user having to manually input any connection config data. This works fine however but one of the instances requires a sentinelPassword value.
I can fetch the password using kubectl get secret but I'm trying to figure out how to insert that password into the config file for the particular instance that requires it.
I've been trying something along the lines of this but getting my jq syntax wrong. Any help or alternative ways of going around would be appreciated.
#store output in var
JSON=$(kubectl get svc --selector='app.kubernetes.io/component=sentinel' --all-namespaces -o json |
jq -r '
.items
| map(
{label:.metadata.name,
sentinels: [{host:(.metadata.name + "." + .metadata.namespace + "." + "svc" + "." + "cluster" + "." + "local"),port: .spec.ports[0].port}],
sentinelName:"mymaster",
sentinelPassword: "",
dbIndex: 0
})
| {connections: . }')
# Find instance by their host value which is unique. (Can't figure out how to do this bit)
if $JSON host name contains "Redis2.development.svc.cluster.local"
#then do something like this
"$JSON" | jq '.[]'| .sentinelPassword = "$password" #var stored from kubectl get secret cmd
#save output to file
"$JSON">/local.json
Assuming for the moment that you want to invoke jq on the sample JSON (local.json) as shown, you could run:
password=mypassword
< local.json jq --arg password "$password" '
.connections[] |= if any(.sentinels[].host; index("Redis2.development.svc.cluster.local"))
then .sentinelPassword = $password else . end'
However, if possible, it would probably be better to invoke jq just once.

Import CSV and updating specific lines

So I have a script that runs at logon to search for PST's on a users machine, then copies them to a holding area waiting for migration.
When the search/copy is complete it outputs to a CSV that looks something like this:
Hostname,User,Path,Size_in_MB,Creation,Last_Access,Copied
COMP1,user1,\\comp1\c$\Test PST.pst,20.58752,08/12/2015,08/12/2015,Client copied
COMP1,user1,\\comp1\c$\outlook\outlook.pst,100,08/12/2015,15,12,2015,In Use
The same logon script has an IF to import the CSV if the copied status is in use and makes further attempts at copying the PST into the holding area. If it's successful it exports the results to the CSV file.
My question is, is there anyway of getting it to either amend the existing CSV changing the copy status? I can get it to add the new line to the end, but not update.
This is my 'try again' script:
# imports line of csv where PST file is found to be in use
$PST_IN_USE = Import-CSV "\\comp4\TEMPPST\PST\$HOSTNAME - $USER.csv" | where { $_.copied -eq "In Use" }
ForEach ( $PST_USE in $PST_IN_USE )
{ $NAME = Get-ItemProperty $PST_IN_USE.Path | select -ExpandProperty Name
$NEW_NAME = $USER + "_" + $PST_IN_USE.Size_in_MB + "_" + $NAME
# attempts to copy the file to the pst staging area then rename it.
TRY { Copy-Item $PST_IN_USE.Path "\\comp4\TEMPPST\PST\$USER" -ErrorAction SilentlyContinue
Rename-Item "\\comp4\TEMPPST\PST\$USER\$NAME" -NewName $NEW_NAME
# edits the existing csv file replacing "In Use" with "Client Copied"
$PST_IN_USE.Copied -replace "In Use","Client Copied"
} # CLOSES TRY
# silences any errors.
CATCH { }
$PST_IN_USE | Export-Csv "\\comp4\TEMPPST\PST\$HOSTNAME - $USER.csv" -NoClobber -NoTypeInformation -Append
} # CLOSES ForEach ( $PST_USE in $PST_IN_USE )
This is the resulting CSV
Hostname,User,Path,Size_in_MB,Creation,Last_Access,Copied
COMP1,user1,\\comp1\c$\Test PST.pst,20.58752,08/12/2015,08/12/2015,Client copied
COMP1,user1,\\comp1\c$\outlook\outlook.pst,100,08/12/2015,15,12,2015,In Use
COMP1,user1,\\comp1\c$\outlook\outlook.pst,100,08/12/2015,15,12,2015,Client copied
It's almost certainly something really simple, but if it is, it's something I've yet to come across in my scripting. I'm mostly working in IF / ELSE land at the moment!
If you want to change the CSV file, you have to write it completely again, not just appending new lines. In your case this means:
# Get the data
$data = Import-Csv ...
# Get the 'In Use' entries
$inUse = $data | where Copied -eq 'In Use'
foreach ($x in $inUse) {
...
$x.Copied = 'Client Copied'
}
# Write the file again
$data | Export-Csv ...
The point here is, you grab all the lines from the CSV, modify those that you process and then write the complete collection back to the file again.
I've cracked it. It's almost certainly a long winded way of doing it, but it works and is relatively clean too.
#imports line of csv where PST file is found to be in use
$PST_IN_USE = Import-CSV "\\comp4\TEMPPST\PST\$HOSTNAME - $USER.csv" | where { $_.copied -eq "In Use" }
$PST_IN_USE | select -ExpandProperty path | foreach {
# name of pst
$NAME = Get-ItemProperty $_ | select -ExpandProperty Name
# size of pst in MB without decimals
$SIZE = Get-ItemProperty $_ | select -ExpandProperty length | foreach { $_ / 1000000 }
# path of pst
$PATH = $_
# new name of pst when copied to the destination
$NEW_NAME = $USER + "_" + $SIZE + "_" + $NAME
TRY { Copy-Item $_ "\\comp4\TEMPPST\PST\$USER" -ErrorAction SilentlyContinue
TRY { Rename-Item "\\comp4\TEMPPST\PST\$USER\$NAME" -NewName $NEW_NAME -ErrorAction SilentlyContinue | Out-Null }
CATCH { $NEW_NAME = "Duplicate exists" }
$COPIED = "Client copied" }
CATCH { $COPIED = "In use" ; $NEW_NAME = " " }
$NEW_FILE = Test-Path "\\comp4\TEMPPST\PST\$HOSTNAME - $USER 4.csv"
IF ( $NEW_FILE -eq $FALSE )
{ "Hostname,User,Path,Size_in_MB,Creation,Last_Access,Copied,New_Name" |
Set-Content "\\lccfp1\TEMPPST\PST\$HOSTNAME - $USER 4.csv" }
"$HOSTNAME,$USER,$PATH,$SIZE,$CREATION,$LASTACCESS,$COPIED,$NEW_NAME" |
Add-Content "\\comp4\TEMPPST\PST\$HOSTNAME - $USER 4.csv"
} # CLOSES FOREACH #
$a = Import-CSV "\\comp4\TEMPPST\PST\$HOSTNAME - $USER.csv" | where { $_.copied -ne "in use" }
$b = Import-Csv "\\comp4\TEMPPST\PST\$HOSTNAME - $USER 4.csv"
$a + $b | export-csv "\\comp4\TEMPPST\PST\$HOSTNAME - $USER 8.csv" -NoClobber -NoTypeInformation
Thanks for the help. Sometimes it takes a moments break and a large cup of coffee to see things a different way.

How to deal with automated duplicate user removal

I have the following:
#(Import-Csv C:\Users\Administrator\Desktop\dbs\Monday.csv) +
#(Import-Csv C:\Users\Administrator\Desktop\dbs\Tuesday.csv) +
#(Import-Csv C:\Users\Administrator\Desktop\dbs\Wednesday.csv) +
#(Import-Csv C:\Users\Administrator\Desktop\dbs\Thursday.csv) +
#(Import-Csv C:\Users\Administrator\Desktop\dbs\Friday.csv) |
sort first_name,last_name,phone1 -Unique |
Export-Csv C:\Users\Administrator\Desktop\dbs\joined.csv
Import-Module ActiveDirectory
#EDIT PATH SO IT POINTS TO DB FILE \/
$newUserList = Import-Csv C:\Users\Administrator\Desktop\dbs\joined.csv
ForEach ($item in $newUserList){
$fname = $($item.first_name)
$lname = $($item.last_name)
$phone = $($item.phone1)
$username=$fname+$lname.substring(0,1)
# Puts Domain name into a Placeholder.
$domain='#csilab.local'
# Build the User Principal Name Username with Domain added to it
$UPN=$username+$domain
# Create the Displayname
$Name=$fname+" "+$lname
$newusers1 = (New-ADUser -GivenName $fname -Surname $lname -HomePhone $phone -Name $Name -DisplayName $Name -SamAccountName $username -AccountPassword (ConvertTo-SecureString "1NewPassword" -asplaintext -force) -ChangePasswordAtLogon $true -UserPrincipalName $UPN -Path "ou=test,dc=csi,dc=lab" -Enabled $true -PassThru) |
# I need this block to check for duplicates missed by the csv sort & merge
# as well as any in the destination OU itself as the script will run daily
# with inevitable possibility that user is unique to the csv but not the AD.
$newusers1 | Get-ADUser -Filter * -SearchBase "OU=Active Users,DC=csilab,DC=local" |
Sort-Object -Unique |
Remove-ADUser -confirm:$false
However I run it and get:
Get-ADUser : The input object cannot be bound to any parameters for the command
either because the command does not take pipeline input or the input and its
properties do not match any of the parameters that take pipeline input.
At C:\Users\Administrator\Desktop\Team2.ps1:40 char:14
+ $newusers1 | Get-ADUser -Filter * -SearchBase "OU=Active Users,DC=csilab,DC=loca ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (CN=Bethanie Cut...csilab,dc=local:PSObject) [Get-ADUser], ParameterBindingException
+ FullyQualifiedErrorId : InputObjectNotBound,Microsoft.ActiveDirectory.Management.Commands.GetADUser
I also worry even if it did work that it'd delete the unique users instead of duplicates.
get-AdUser $username | Move-ADObject -TargetPath 'OU=Active Users,dc=csilab,dc=local'
}
What can I do to ensure all users are there without any originals getting deleted, just the duplicates?
You still have an empty pipe at the end of your New-ADUser statement, which would cause your script to fail with an "empty pipe element is not allowed" error, but oh well ...
To avoid collisions just check if an account already exists before you try to create it, and create it only if it doesn't:
$username = $fname + $lname.substring(0,1)
...
if (Get-ADUser -Filter "SamAccountName -eq '$username'") {
Write-Host "Account $username already exists."
} else {
New-ADUser -SamAccountName $username -Name $Name ... -PassThru
}
Also, you're overcomplicating the CSV handling. Simply process a list of the files via ForEach-Object:
$domain = '#csilab.local'
Set-Location 'C:\Users\Administrator\Desktop\dbs'
'Monday.csv', 'Tuesday.csv', 'Wednesday.csv', 'Thursday.csv', 'Friday.csv' |
ForEach-Object { Import-Csv $_ } |
Sort-Object first_name, last_name, phone1 -Unique |
ForEach-Object {
$fname = $_.first_name
$lname = $_.last_name
$phone = $_.phone1
...
}
You might want to use Try/Catch:
Try {$newusers1=(New-ADUser–GivenName$fname–Surname$lname-HomePhone$phone–Name$Name-DisplayName $Name –SamAccountName$username–AccountPassword(ConvertTo-SecureString"1NewPassword"-asplaintext-force) -ChangePasswordAtLogon$true–UserPrincipalName$UPN–Path"dc=csi,dc=lab"-Enabled$true)}
Catch {
If ($_.Exception.Message -match "account already exists")
{
#do whatever here, eg $NewUsers1 = Get-ADUser $Name
}
}
Also, if you can't see the user when browsing via ADUC it could be that you are connected to a different DC.
As mentioned above, the newuser1 variable will be null if the command failed. It will not load with the other user automatically, and it would be scary bad if it did. You need to decide what to do if the account already exist, that may simply be loading the variable with the other account, or doing something like appending a "1" to $name and retrying the command.

PowerShell Import CSV to add Active Directory Security Groups

I have a CSV from a client with a few columns in it. A column for Email, SamAccountName, SecurityGroups.
In the Security Group column, each Security Group is separated by a ','
Each row would look like below
User1 User1#email.com SecurityGroup1, SecurityGroup2, SecurityGroup3
User2 User2#email.com SecurityGroup1, SecurityGroup2, SecurityGroup3
User3 User3#email.com SecurityGroup13, SecurityGroup5, SecurityGroup6
A list of users in different Security groups, how do I go about reading the CSV and only adding the Security Groups each user needs.
Below is what I have so far, when I run it in WhatIf mode it adds all the security groups to one user.
Clear
$ADGroup = $null
$i = 0
$ADGroup = import-csv 'File.csv'
$sADGroup = $ADGroup.'Groups required'
Import-CSV "TestUsers.csv" | % {
$SAM = $_.SamAccountName
$ADUser = Get-Aduser -filter {SamAccountName -eq $SAM} -Properties * | Select SamAccountName, MemberOf
Write-Host 'SamAccountName :' $_.SamAccountName ' - ADUser - ' $SAM
$sADGroup | ForEach-Object{
Write-HOst $ADGroup.ADgroup[$i]
Add-ADGroupMember $sADGroup.split(",")[$i] -Members $ADUser.SamAccountName -WhatIf
$i += 1
}
}
Here is the final script, using AD-ADPrincipalGroupMembership was the find, instead of using Add-ADGroupMember
Import-Csv 'testusers.csv' |
ForEach-Object {
add-content $LogFile "Adding User to Group - $($_.Name)"
Try{
Add-ADPrincipalGroupMembership -identity $_.'SamAccountName' -memberof ($_.'Groups' -split', ')
}
Catch {
Add-content $LogFile "Error adding user to group - $($_.Name)"
}
}

How to update AD Info attribute to list groups user was deleted from?

I have this script that will only list a user's groups on the ISE screen where the data can be copied and pasted elsewhere, but I'm trying to get the group membership names written into the Telephone Notes tab (or Info field). I'm thinking next that these probably need to be turned into string values since I'm getting errors about multi properties not allowed. Here is what I've been trying, but I keep getting errors. Thanks
Import-Module ActiveDirectory
$Users= Import-csv "C:\Scripts\UsersSAM-DisplayName.csv"
ForEach ($User in $Users) {
$SamAccountName=$User.SamAccountName
$DisplayName=$User.DisplayName
$TableFormat= #{E={$_.Name};L="$($DisplayName) - $($SamAccountName)"}
Get-ADUser -Identity $SamAccountName -Properties MemberOf | % {$_.MemberOf } | % {Get-ADGroup -Identity $_ } | % { Set-ADUser -Identity $SamAccountName -add #{info="$_.name"}} | Select Name |
Format-Table $TableFormat }
I figured this out. What they wanted was to first write out a terminated user's groups, then remove those. I did it like this and this code includes the semi-colon so if the user comes back, all you need to do to add them back to all the groups is copy and paste those from the output stored in the Telephones Tab, Notes field. I've also used a trimmed down version of this to export a user's groups to speed up duplicating a user's groups so they match with others on the same team. Hope this helps someone.
Import-csv "$Terms" | % {
$user = Get-ADUser -LDAPFilter ("(sAMAccountName=" + $_.samaccountname + ")") -Properties samaccountname,enabled,name,memberof,distinguishedname,info
#Grab all user group names
$user | ForEach-Object {
$grps = $_.MemberOf | Get-ADGroup | ForEach-Object {$_.Name} | Sort-Object
$arec = $_.Name,$_.SamAccountName
$aline = ($grps -join ";")
#Add info to Notes field Telephone Tab
Get-ADPrincipalGroupMembership -Identity $user | %{
If ($_.SamAccountName -ne "Domain Users") {
$Userinfo=$user.info
Set-ADUser $User -replace #{info= "$Userinfo | $a | Terminated via automated process | $aline"}
#Remove User Groups Process in Telephones Tab Notes Field.
Remove-ADPrincipalGroupMembership -Identity $user -MemberOf $_.SamAccountName -Confirm:$false
(" "+ $a +" [" + $User.samaccountname + "], Removed from group [" + $_.samaccountname + "]. ") | Out-File -FilePath $ErrorLog -Append
}
}}}