parse csv to make more readable - csv

I am using PsInfo from sysinternals to return drive information in the form of a csv.
When I run this command.
psinfo -c -d volume
I get the following output.
PCName,,C:,Fixed,NTFS,,930.97 GB,705.81 GB,75.8%,D:,CD-ROM,,,,,0.0%,G:,Fixed,NTFS,My Book,1862.98 GB,889.71 GB,47.8%
My goal here is to parse that output to achieve this format:
PCNAME,,
,,C:,Fixed,NTFS,,930.97 GB,705.81 GB,75.8%
,,G:,Fixed,NTFS,My Book,1862.98 GB,889.71 GB,47.8%
So when the output is read by a spreadsheet viewer it appears readable and organized.
I've tried using regex to match the drive letter bit I don't know how to correctly capture the output.
Edit: (forgot to actually post my code...)
This is what I have so far.
ECHO OFF
For /f "tokens=1*" %%x in ('net view ^| find "\\"') do (psinfo %%x filter -d -c >> out.csv & echo. >> out.csv)
::used to remove blanmk lines from output
#jrepl "^[ \t]*(.*?)[ \t]*$" "$1?$1:false" /jmatch /f out.csv /o -
pause

#ECHO OFF
SETLOCAL enabledelayedexpansion
FOR /f "delims=" %%a IN ('psinfo -c -d volume') DO SET "var=%%a"
SET "var2=!var:%%=\!"
SET "retained="
:again
REM FOR /f "tokens=delims==:" %%a IN () DO
FOR /f "tokens=1*delims=:" %%a IN ("%var2%") DO (
CALL :FORMAT "%%a"
SET "var2=%%b"
IF NOT "%%b"=="" GOTO again
)
GOTO :EOF
:FORMAT
SET "data=%~1"
IF "%data:~-1%"=="\" (SET "data=%retained%:%data%") ELSE (SET "data=%retained%:%data:~,-2%")
IF DEFINED retained (
ECHO ,,!data:\=%%!
) ELSE (
ECHO %data:~1%,
)
SET "retained=%~1"
SET retained=%retained:~-1%
GOTO :eof
substitutes \ for % for ease of processing.

This worked for me. Given this example you should be able to play around with the format of the output you desire.
#echo off
FOR /F "tokens=1* delims=," %%G IN ('psinfo -c -d volume') DO (
SET "pcname=%%G"
SET "driveinfo=%%H"
)
set driveinfo="%driveinfo:,=","%"
:LOOP
FOR /F "tokens=1-7* delims=," %%G IN ("%driveinfo%") DO (
echo "%pcname%",%%G,%%H,%%I,%%J,%%K,%%L,%%M
SET "driveinfo=%%N"
)
IF DEFINED driveinfo GOTO LOOP
Output
"COCO","C:","Fixed","NTFS","","930.97 GB","705.81 GB","75.8%"
"COCO","G:","Fixed","NTFS","My Book","1862.98 GB","889.71 GB","47.8%"
And here is the code changed to replicate your exact needed Output.
#echo off
FOR /F "tokens=1* delims=," %%G IN ('psinfo -c -d volume') DO (
SET "pcname=%%G"
SET "driveinfo=%%H"
)
set driveinfo="%driveinfo:,=","%"
echo %pcname%,,
:LOOP
FOR /F "tokens=1-7* delims=," %%G IN ("%driveinfo%") DO (
echo ,,%%~G,%%~H,%%~I,%%~J,%%~K,%%~L,%%~M
SET "driveinfo=%%N"
)
IF DEFINED driveinfo GOTO LOOP
And here is doing it in a couple of less lines.
#echo off
setlocal enabledelayedexpansion
FOR /F "tokens=1* delims=," %%G IN ('psinfo -c -d volume') DO (
echo %%G,,
SET "driveinfo=%%H"
)
set "drive2=echo ,,!driveinfo:%%=%%&echo,,!"
%drive2:~0,-1%

You can get the same information with Powershell, eg with the Get-Volume command. This returns, everything except the ration eg:
DriveLetter FileSystemLabel FileSystem DriveType HealthStatus OperationalStatus SizeRemaining Size
----------- --------------- ---------- --------- ------------ ----------------- ------------- ----
D CD-ROM Healthy Unknown 0 B 0 B
C Win7 NTFS Fixed Healthy OK 43.59 GB 238.03 GB
H Projects NTFS Fixed Healthy OK 96.73 GB 375.24 GB
Get-Volume returns volume information objects. The shell just displays the most common ones.
You can select only the properties you want, and add more columns with the select command:
Get-Volume | where {$_.DriveType -ne 'CD-ROM'}
| select DriveLetter, FileSystemLabel,FileSystem,SizeRemaining, Size,
#{Name="Ratio";Expression={"{0:P}" -f ($_.SizeRemaining/$_.Size)}}
|Format-Table
DriveLetter FileSystemLabel FileSystem SizeRemaining Size Ratio
----------- --------------- ---------- ------------- ---- -----
C Win7 NTFS 46793633792 255584587776 18,31%
H Projects NTFS 103853056000 402911129600 25,78%
I'm cheating a bit here, by filtering out the CD ROM with where
If the results are too long to fit in the window, Powershell displays them as a list of name/values. The final Format-Table is used to force Powershell to format the results as a table
I can store the table to a file with a normal redirection :
Get-Volume | where {$_.DriveType -ne 'CD-ROM'}
| select DriveLetter, FileSystemLabel,FileSystem,SizeRemaining, Size,
#{Name="Rate";Expression={"{0:P}" -f ($_.SizeRemaining/$_.Size)}}
|Format-Table > somefile.txt
or I can export the results to a real CSV with Export-CSV
Get-Volume | where {$_.DriveType -ne 'CD-ROM'}
| select DriveLetter, FileSystemLabel,FileSystem,SizeRemaining, Size,
#{Name="Rate";Expression={"{0:P}" -f ($_.SizeRemaining/$_.Size)}}
| Export-CSV somefile.csv

The following code snippet reformats the output of psinfo -c -d volume as you wish. Since it seems that you want it filtered by drive type (Fixed) according to your sample data, I implemented such a feature; to not filter anything, simply change line set "FILTER=Fixed" to set "FILTER=":
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem // Define constants here:
set "FILTER=Fixed"
(set LF=^
%= empty line =%
)
for /F "tokens=1,* delims=, eol=," %%K in ('psinfo -c -d volume') do (
echo(%%K,,
set "LINE=%%L"
setlocal EnableDelayedExpansion
set ^"LINE=!LINE:%%,=%%^%LF%%LF%!^"
for /F "delims= eol=," %%E in ("!LINE!") do (
endlocal
if defined FILTER (
for /F "tokens=2 delims=, eol=," %%F in ("%%E") do (
if "%%F"=="%FILTER%" (
echo(,,%%E
)
)
) else (
echo(,,%%E
)
setlocal EnableDelayedExpansion
)
endlocal
)
endlocal
exit /B

Related

BATCH - Remove some characters from filename

I've files with this name:
FirstPart_SecondPart_ThirdPart.zip_FourthPart_FifthPartX.csv
where X is a one-digit number.
It should be renamed via batch scripting as:
FirstPart_SecondPart_ThirdPart.zip_FifthPartX.csv
So I'd like to remove the FourthPart. Please note that all the parts ALWAYS HAVE the same lenght. FirstPart is always 7 digits, SecondPart is always 9 digits...etc...
Here is what I've tried:
ren \*.zip_*FifthPart?.csv *.zip_FifthPart?.csv
But it does not work.
Please any help?
Within the appropriate directory
…at the Command prompt:
For /F "EOL=_Tokens=1-4*Delims=_" %A In ('Dir/B/A-D "*_*_*_*_*.csv"') Do #Ren "%A_%B_%C_%D_%E" "%A_%B_%C_%E"
…in a batch file:
#For /F "EOL=_Tokens=1-4*Delims=_" %%A In ('Dir/B/A-D "*_*_*_*_*.csv"'
) Do #Ren "%%A_%%B_%%C_%%D_%%E" "%%A_%%B_%%C_%%E"
Because you are already certain of the file name format and character numbers I feel that the best approach would be to utilise Where instead of Dir.
For example, in a batch file:
FirstPart is always 7 digits, SecondPart is always 9 digits, ThirdPart is always 6 digits, FourthPart is always 5 digits and FifthPart is always 8 digits.
#For /F "EOL=_Tokens=1-4*Delims=_" %%A In (
'Where .:"???????_?????????_??????_?????_????????.csv"'
) Do #Ren "%%A_%%B_%%C_%%D_%%E" "%%A_%%B_%%C_%%E"
You can use a for /F loop to split and rebuild the file names:
for /F "delims= eol=|" %%F in ('dir /B /A:-D "*_*_*_*_*.csv"') do (
for /F "tokens=1-4* delims=_ eol=_" %%A in ("%%F") do (
ren "%%F" "%%A_%%B_%%C_%%E"
)
)
Given that none of the parts contain underscores (_) on their own and none of them are empty, a single loop is sufficient:
for /F "tokens=1-4* delims=_ eol=_" %%A in ('dir /B /A:-D "*_*_*_*_*.csv"') do (
ren "%%A_%%B_%%C_%%D_%%E" "%%A_%%B_%%C_%%E"
)
Here is an approach with an additional filter for file names using findstr in order to exclude files that do not match the name specifications:
for /F "tokens=1-4* delims=_ eol=_" %%A in ('
dir /B /A:-D "*_*_*_*_*.csv" ^| findstr /I "^[^_][^_]*_[^_][^_]*_[^_][^_]*_[^_][^_]*_[^_].*\.csv$"
') do (
ren "%%A_%%B_%%C_%%D_%%E" "%%A_%%B_%%C_%%E"
)
Two possibilities to solve this:
(1) As all parts always have the same length, you can just use substrings to cut the FourthPart_ part:
#ECHO OFF
SETLOCAL EnableDelayedExpansion
FOR /F "tokens=*" %%G IN ('DIR /B *.csv') DO (
SET new_filename=%%G
SET new_filename=!new_filename:~0,35!!new_filename:~46!
ECHO REN "%%G" "!new_filename!"
)
(2) Alternatively, if FourthPart_ is a static string, you might also get away with removing the FourthPart_ using search & replace:
#ECHO OFF
SETLOCAL EnableDelayedExpansion
FOR /F "tokens=*" %%G IN ('DIR /B *.csv') DO (
SET new_filename=%%G
SET new_filename=!new_filename:FourthPart_=!
ECHO REN "%%G" "!new_filename!"
)
These batch files will only output the commands to be issued. Remove the ECHO once you've inspected the output and you're confident it does what you want.

How to split CSV column values into other columns using batch script

I'm new to batch files and this is a tricky question. In stores.csv there is a column called 'Image' which stores vertical-line-delimited image URLs as values. There are also additional columns called 'AltImage2', 'AltImage3', etc. How can I split the vertical-line-delimited string into columns that start with 'AltImage' for each row in the CSV? 'AltImage' columns only go to AltImage5, and there may not be five image URLs in a given row. I would also like to keep the first image URL in the 'Image' column if possible.
Example of headers and single row of data:
Company,Title,Image,AltImage2,AltImage3,AltImage4,AltImage5
Testco,U2X40,image1.png|image2.png|image3.png
Desired result after running batch:
Company,Title,Image,AltImage2,AltImage3,AltImage4,AltImage5
Testco,U2X40,image1.png,image2.png,image3.png
So far I've tried this:
for /f "tokens=3 delims=, " %%a in ("stores.csv") do (
echo run command here "%%a"
)
But cannot even echo the values in the Image column.
Here is a solution using Bash script (unfortunately I need batch): How do I split a string on a delimiter in Bash?
#echo off
setlocal
< stores.csv (
rem Read and write the header
set /P "header="
call echo %%header%%
rem Process the rest of lines
for /F "tokens=1-3 delims=|" %%a in ('findstr "^"') do echo %%a,%%b,%%c
)
I think this handles your parsing problem. Pay attention to quotes and the usebackq option.
for /f "skip=1 tokens=3 delims=," %%a in (stores.csv) do for /f "tokens=1-5 delims=|" %%b in ("%%a") do echo %%b %%c %%d %%e %%f
Here's a fuller solution to play with. There may be a more elegant way to handle optional commas. And you'll have to handle directing the output to whichever place is appropriate.
#echo off
setlocal enabledelayedexpansion
echo Company,Title,Image,AltImage2,AltImage3,AltImage4,AltImage5
for /f "skip=1 tokens=3 delims=," %%a in (stores.csv) do (
for /f "tokens=1-5 delims=|" %%b in ("%%a") do (
set line=%%b
if not "%%c"=="" set line=!line!,
set line=!line!%%c
if not "%%d"=="" set line=!line!,
set line=!line!%%d
if not "%%e"=="" set line=!line!,
set line=!line!%%e
if not "%%f"=="" set line=!line!,
set line=!line!%%f
echo !line!
)
)
read the file line by line and replace | with , (you have to escape the | and use delayed expansion:
#echo off
setlocal enabledelayedexpansion
(
for /f "delims=" %%a in (old.csv) do (
set line=%%a
echo !line:^|=,!
)
)>new.csv

Combined csv via cmd

I'm currently working on a way to combine 3 csv files and have the following script to do so:
Script
#echo off
ECHO Set working directory
pushd %~dp0
setlocal ENABLEDELAYEDEXPANSION
set cnt=1
for %%i in (*.csv) do (
if !cnt!==1 (
for /f "delims=" %%j in ('type "%%i"') do echo %%j >> combined.csv
) else if %%i NEQ combined.csv (
for /f "skip=1 delims=" %%j in ('type "%%i"') do echo %%j >> combined.csv
)
REM increment count by 1
set /a cnt+=1
)'
This works like a charm and also strips the header of the other 2 csv files in the working dir.
The script now outputs combined.csv, which is nice but I would like the script to output NL2 ddmmyyyy.csv.
The issue I have and can't seem to figure out is how to make the name of the output file incremental and date-based.
Code can parse the date elements from the DATE variable. The format of the date will vary based on locale. Mine is set to YYYY-MM-DD, but yours may be different.
M:>echo %DATE%
2016-01-27
8:23:58.28 \\SWPDCENDWXTK01\D$ M:\DW_Devl\Paul\phs_dmx_config\bin
M:>SET FNDATE=%DATE:~8,2%%DATE:~5,2%%DATE:~0,4%
8:24:08.54 \\SWPDCENDWXTK01\D$ M:\DW_Devl\Paul\phs_dmx_config\bin
M:>ECHO %FNDATE%
27012016
Inside the loop, set a variable and use it for the output.
SET COMBO_FILENAME=NL!cnt!%FNDATE%.csv
ECHO %%j >>!COMBO_FILENAME!

Merge CSV Files (Without Duplicates) in Batch

I want to merge two similar CSV files using batch. I have found a file that was working perfectly and now is not. This may have been due to renaming the CSV files that were used.
I want what is demonstrated below:
File 1:
name1,group1,data1
name2,group2,data2
name3,group3,data3
File 2:
name1,group1,data1,time1
name2,group2,data2,time2
Merged file:
name1,group1,data1,time1
name2,group2,data2,time2
name3,group3,data3
(Note that the fourth column was not filled in by name3 and was subsequently not on file 2.)
The following code was modified from: http://www.experts-exchange.com/OS/Microsoft_Operating_Systems/MS_DOS/Q_27694997.html.
#echo off
set "tmpfile=%temp%\importlist.tmp"
set "csvfile=importlist.csv"
copy nul "%tmpfile%" >nul
echo.
echo Processing all CSV files...
set "header="
for %%a in (%1) do (
if not "%%a"=="%csvfile%" (
set /p =Processing %%a...<nul
for /f "tokens=1* usebackq delims=," %%b in ("%%a") do (
if /i "%%b"=="Keyword" (
if not defined header (
set /p =Found header...<nul
set "header=%%b,%%c"
)
) else (
title [%%a] - %%b,%%c
findstr /b /c:"%%b" /i "%tmpfile%">nul || echo %%b,%%c>>"%tmpfile%"
)
)
echo OK
)
)
echo Finished processing all CSV files
echo.
echo Creating %csvfile%
echo %header%>"%csvfile%"
set /p =Sorting data...<nul
sort "%tmpfile%">>"%csvfile%"
echo OK
del "%tmpfile%"
echo Finished!
title Command Prompt
exit /b
The problem is that when executed it just creates a sorted CSV with all the data from the first file and not the second.
I have attempted to get it working by putting quotation marks around the parameter (%1 - "directory*.csv") to no avail.
you might try this
#echo off &setlocal disabledelayedexpansion
for /f "delims=" %%a in (file1.csv) do set "#%%~a=7"
for /f "tokens=1-4delims=," %%a in (file2.csv) do (
set "lx1=#%%~a,%%~b,%%~c"
setlocal enabledelayedexpansion
if defined !lx1! (
endlocal
set "#%%~a,%%~b,%%~c="
) else (
endlocal
)
set "#%%~a,%%~b,%%~c,%%~d=4"
)
(for /f "delims==#" %%a in ('set #') do echo %%~a)>merge.csv
type merge.csv
It doesn't work, if you have = or # in your data.

Batch script: update database with new sql files

So I am trying to create a batch file which I can execute in order to update multiple databases with sql files. I already created a first part which is executing the sql files in all the databases. It is not very elaborate but it does the job. My problem is that all the files are always executed.
What I would like to have is a check between the execution date/time of the batch script and the last modified date/time of every sql scripts. Thanks to this, I can only execute the sql scripts which were modified/created after the execution of the batch script. I tried quite hard but didn't manage to script this functionality. I'm not really good with batch scripts ^^
Thanks !
The Script
#echo off
set "mySQLexe=C:\wamp\bin\mysql\mysql5.5.24\bin\mysql.exe"
for /f "tokens=*" %%A in (databases.txt) do (
for %%G in (*.sql) do (
%mySQLexe% -hlocalhost -uroot %%A < %%G
)
)
REM Save execution of batch file
#echo %date% > executiontime.txt
#echo %time% >> executiontime.txt
Try this:
#echo off
setlocal enabledelayedexpansion
set "mySQLexe=C:\wamp\bin\mysql\mysql5.5.24\bin\mysql.exe"
for /f "tokens=*" %%A in (databases.txt) do (
for /f "tokens=*" %%G in ('dir /b *.sql') do (
Call :FileModTime "%%G" sTime
Call :FileModTime "%%~dpf0" bTime
if !sTime! GTR !bTime! (
echo %mySQLexe% -hlocalhost -uroot %%A < %%G
)
)
)
exit /b
:FileModTime File [TimeVar]
:: Created by dbenham from dostips ::
:: Converts current time to epox time ::
setlocal disableDelayedExpansion
for %%F in ("%~1") do set "file=%%~fF"
set "time="
for /f "skip=1 delims=,. tokens=2" %%A in (
'wmic datafile where "name='%file:\=\\%'" get lastModified /format:csv 2^>nul'
) do set "ts=%%A"
set "ts=%ts%"
if defined ts (
set /a "yy=10000%ts:~0,4% %% 10000, mm=100%ts:~4,2% %% 100, dd=100%ts:~6,2% %% 100"
set /a "dd=dd-2472663+1461*(yy+4800+(mm-14)/12)/4+367*(mm-2-(mm-14)/12*12)/12-3*((yy+4900+(mm-14)/12)/100)/4"
set /a "ss=(((1%ts:~8,2%*60)+1%ts:~10,2%)*60)+1%ts:~12,2%-366100-%ts:~21,1%((1%ts:~22,3%*60)-60000)"
set /a "ts=ss+dd*86400"
)
endlocal & if "%~2" neq "" (set "%~2=%ts%") else echo(%ts%
Remove the echo once you have tested it and made sure it is working.