I have this table in normal string format ,
I want to convert this string to json object in PowerShell. ConvertTo-Json is not giving in correct format.
The answer depends somewhat on the true format of the table. If I assume this is tab delimited and that each column name doesn't have spaces I could pull it out something like:
$String =
#"
test test2 first others versions
------------------------------------------
Decimal 1 2 5 p
Decimal 1 3 8 p
Decimal 1 2 4 i
Decimal 2 2 6 p
Decimal 5 4 6 k
Decimal 2 5 2 p
"#
$String = $String -split "\r?\n"
$Headers = $String[0] -split "\s"
$Objects =
$String[2..($String.Count -1)] |
ConvertFrom-Csv -Header $Headers -Delimiter "`t" |
ConvertTo-Json
Above, -split the big string into lines, then look at the header line and -split it to get an array of column headers. Now skipping the first 2 elements in the $String array convert the remaining lines to objects using ConvertFrom-Csv and using the previously extracted $Headers array.
Note: This segment may also work and may be preferred for readability:
$Objects =
$String |
Select-Object -Skip 2 |
ConvertFrom-Csv -Header $Headers -Delimiter "`t" |
ConvertTo-Json
Note: Splitting on white space ( "\s" ) may cause issues if the field data may have whitespace itself.
However, given the ambiguity, a more certain approach might be more reliable, I would use the known start and end positions of the table's fields to do this.
Continuing with the above example string:
$String =
#"
test test2 first others versions
------------------------------------------
Decimal 1 2 5 p
Decimal 1 3 8 p
Decimal 1 2 4 i
Decimal 2 2 6 p
Decimal 5 4 6 k
Decimal 2 5 2 p
"#
$String = $String -Split "\r?\n"
$String |
Select-Object -Skip 2 |
ForEach-Object{
[PSCustomObject]#{
test = $_.SubString(0,7)
test2 = $_.SubString(8,1)
first = $_.SubString(14,1)
others = $_.SubString(20,1)
versions = $_.SubString(26,1)
}
}
Again, these positions may change depending if the columns are separated by spaces or tabs. My sample data may not be the same as yours and you may need to play with those positions. That said this is a very useful technique for deal with output from traditional console applications, very much worth knowing...
Note: Thanks Neko Nekoru; I added '?' to the RegEx to accommodate both Unix & Windows line ending styles.
Related
Given a TSV file with col2 that contains either a field or record separator (FS/RS) being respectively a tab or a carriage return which are escaped/surrounded by quotes.
$ printf '%b\n' 'col1\tcol2\tcol3' '1\t"A\tB"\t1234' '2\t"CD\nEF"\t567' | \cat -vet
col1^Icol2^Icol3$
1^I"A^IB"^I1234$
2^I"CD$
EF"^I567$
+------+---------+------+
| col1 | col2 | col3 |
+------+---------+------+
| 1 | "A B" | 1234 |
| 2 | "CD | 567 |
| | EF" | |
+------+---------+------+
Is there a way in sed/awk/perl or even (preferably) miller/mlr to transform those pesky characters into spaces in order to generate the following result:
+------+---------+------+
| col1 | col2 | col3 |
+------+---------+------+
| 1 | "A B" | 1234 |
| 2 | "CD EF" | 567 |
+------+---------+------+
I cannot get miller 6.2 to make the proper transformation (tried with DSL put/gsub) because it doesn't recognize the tab or CR/LF being part of the columns which breaks the field number:
$ printf '%b\n' 'col1\tcol2\tcol3' '1\t"A\tB"\t1234' '2\t"CD\nEF"\t567' | mlr --opprint --barred --itsv cat
mlr : mlr: CSV header/data length mismatch 3 != 4 at filename (stdin) line 2.
A good library cleanly handles things like embedded newlines and quoted separators (in fields)
In a Perl script with Text::CSV
use warnings;
use strict;
use Text::CSV;
my $file = shift // die "Usage: $0 filename\n";
my $csv = Text::CSV->new( { binary => 1, sep_char => "\t", auto_diag => 1 } );
open my $fh, '<', $file or die "Can't open $file: $!";
while (my $row = $csv->getline($fh)) {
s/\s+/ /g for #$row; # collapse multiple spaces, tabs, newlines
$csv->say(*STDOUT, $row);
}
Note the many other options for the constructor that can help handle various irregularities.
This can fit in a one-liner; its functional interface (with csv) is particularly well suited for that.
if you run
printf '%b\n' 'col1\tcol2\tcol3' '1\t"A\tB"\t1234' '2\t"CD\nEF"\t567' | \
mlr --c2t --fs "\t" clean-whitespace
col1 col2 col3
1 A B 1234
2 CD EF 567
I'm using mlr 6.2.
A way to do it in miller 5 is to use simply the put verb:
printf '%b\n' 'col1\tcol2\tcol3' '1\t"A\tB"\t1234' '2\t"CD\nEF"\t567' | \
mlr --tsv put -S 'for (k in $*) {$[k] = gsub($[k], "\n", " ")}' then clean-whitespace
perl -MText::CSV_XS=csv -e'
csv
in => *ARGV,
on_in => sub { s/\s+/ /g for #{$_[1]} },
sep_char => "\t";
'
Or s/[\t\n]/ /g if you prefer.
Can be placed all on one line.
Input is accepted from file named by argument or STDIN.
With GNU awk for multi-char RS, RT, and gensub():
$ awk -v RS='"([^"]|"")*"' '{ORS=gensub(/[\n\t]/," ","g",RT)} 1' file
col1 col2 col3
1 "A B" 1234
2 "CD EF" 567
The above just uses RS to isolate each "..." string and saves it in RT, then replaces every \n or \t in that string with a blank and saves the result in ORS, then prints the record.
you absolutely don't need gawk to get this done - here's one that works for mawk, gawk, or macos nawk :
INPUT
--before--
col1 col2 col3
1 "A B" 1234
2 "CD
EF" 567
CODE
{m,n,g}awk '
BEGIN {
1 __=substr((OFS=FS="\t\"")(FS)(ORS=_)\
(RS = "^$"),_+=_^=_<_,_)
}
END {
1 printbefore()
3 for (_^=_<_; _<=NF; _++) {
3 sub(/[\t-\r]+/, ($_~__)?" ":"&", $_)
}
1 print
}
1 function printbefore(_)
{
1 printf("\n\n--before--\n%s\n------"\
"------AFTER------\n\n", $+_)>("/dev/stderr")
}
OUTPUT
———AFTER (using mawk)------
col1 col2 col3
1 "A B" 1234
2 "CD EF" 567
strip out the part about printbefore() that's more for debugging purposes, then it's just
{m,n,g}awk '
BEGIN { __=substr((OFS=FS="\t\"") FS \
(ORS=_) (RS="^$"),_+=_^=_<_,_)
} END {
for(--_;_<=NF;_++) {
sub(/[\t-\r]+/, $_~__?" ":"&",$_) } print }'
I'm trying to use PowerShell to convert a json with the following format:
{"Value1":[89,91,88,71,59],"Value2":[53,58,54,53,58]}
To a CSV with the following format:
Value1,Value2
89,53
91,58
88,54
71,53
59,58
The issue I'm running into is that this code that I've cobbled together
Get-Content 'C:\test\test.json' | ConvertFrom-Json |
select #{n='value1';e={-split $_.value1}},
#{n='value2';e={-split $_.value2}} |
Export-Csv -NoTypeInformation 'C:\test\test.csv'
is making a CSV with the following format:
Value1,Value2
89 91 88 71 59,53 58 54 53 58
Does anybody have any ideas on what I could do here? I'm really new to PowerShell and I'm getting pretty lost. I've read a ton of similar items on here and other sites, but none that exactly fit the issue I'm running into.
Thanks
There might be more sophisticated solutions but at least for this simple example the following snippet works I think:
$Var = '{"Value1":[89,91,88,71,59],"Value2":[53,58,54,53,58]}' | ConvertFrom-Json
$Var.Value1.Count
for ($i = 0; $i -lt $Var.Value1.Count; $i++) {
[PSCustomObject]#{
Value1 = $Var.Value1[$i]
Value2 = $Var.Value2[$i]
}
}
The output looks like this:
Value1 Value2
------ ------
89 53
91 58
88 54
71 53
59 58
There is a JSON URL which produces dynamic content, and there is one particular part of the JSON URL which I am trying to separate two values which have no title or name associated to them other than the parent title (accountUsage) and give them each a unique title, which I can then call upon in PowerShell.
Any ideas how to achieve this?
I need to convert this
accountUsage : #{10.10.2018=5; 09.10.2018=0; 08.10.2018=0; 07.10.2018=0; 06.10.2018=0; 05.10.2018=8; 04.10.2018=1; 03.10.2018=0;
02.10.2018=0; 01.10.2018=0}
Into this:
date
----
10.10.2018
value
----
5
date
----
09.10.2018
value
----
0
$json = '{"accountUsage":{"06.10.2018":0,"09.10.2018":0,"04.10.2018":1,"08.10.2018":0,"02.10.2018":0,"07.10.2018":0,"03.10.2018":0,"05.10.2018":8,"10.10.2018":5,"01.10.2018":0}}'
$data = $json | ConvertFrom-Json
$data.accountUsage | Get-Member -MemberType NoteProperty | ForEach-Object {
$key = $_.Name
[PSCustomObject]#{
date = $key
value = $data.accountUsage.$key
}
}
gives me a list of date/value pairs:
date value
---- -----
06.10.2018 0
09.10.2018 0
04.10.2018 1
08.10.2018 0
02.10.2018 0
07.10.2018 0
03.10.2018 0
05.10.2018 8
10.10.2018 5
01.10.2018 0
See this earlier answer of mine for some more insight into this.
I'm reading a fixed-width file with 4000 rows though substrings, and assigning each substring to a header in a csv. But I'm not sure how to save the csv.
An example row am reading:
$line = ABC 7112123207/24/16Smith Timpson Head Coach 412-222-0000 00011848660 ELl CAAN HIGH SCHOOL 325 N Peal AVE. Smith Timpson Head Coach COLORADO CITY AZ 86021 01 FALL MALE 07/29/16EQ15031 1977904 BUDDY'S ALL STARS INC. BUDDY ALL STARS N V12V70R16 1.00V12V70R16
I've the csv with the headers.
$csvheaders = import-csv temp.csv
foreach ($Line in (Get-Content $FILE.FullName))
{
foreach($csh in $csvheaders)
{
$csh.GROUP = $line.Substring(0,10).Trim()
$csh.NUMBER = $line.Substring(10,8).Trim()
$csh.DATE=$line.Substring(18,8).Trim()
$csh.CONTACT_FIRST=$line.Substring(26,35).Trim()
$csh.CONTACT_LAST=$line.Substring(61,35).Trim()
}
}
I would need the csv output as:
Group Number Date Contact_First Contact_Last
ABC 71121232 07/24/16 Smith Timpson
There is a Export-Csv cmdlet:
Get-Content $FILE.FullName | ForEach-Object {
[PSCustomObject]#{
Group = $_.Substring(0,10).Trim()
Number = $_.Substring(10,8).Trim()
Date = $_.Substring(18,8).Trim()
Contact_First = $_.Substring(26,35).Trim()
Contact_Last = $_.Substring(61,35).Trim()
}
} | Export-Csv -Path 'Your_Output_Path.csv' -NoTypeInformation
Note: You probably need to specify a tab delimiter for the Export-Csv cmdlet.
I am trying to write simple PowerShell code to read two columns values from CSV and print in manner like
1
201
2
202
.
.
CSV File:
PRODID DEVID Name
1 201 Application
2 202 Product
3 203 Component
4 204 Author
5 205 Version
Powershell Code:
$DEVID = Import-Csv C:\test\install.csv | % {$_.DEVID}
$PRODID = Import-Csv C:\test\install.csv | % {$_.PRODID}
ForEach ($DEVI in $DEVID)
{
Write-Host $DEVI
ForEach ($PRODI in $PRODID)
{[enter image description here][1]
Write-Host $PRODI
}
}
But I am not getting expected output, though I have tried break, continue syntax.
Can anyone help me in this case please?
You only need to import your csv once. Then just iterate over it and output your desired records:
Import-Csv 'C:\test\install.csv' | Foreach {
$_.PRODID; $_.DEVID;
}
Output:
1
201
2
202
3
203
4
204
5
205
If this doesn't work, you have to show us your csv file.