How to drop all but last cell in CSV using CMD - csv

my goal is to write a script that will monitor process memory usage and run % based comparison on it to determine if there is a memory leak in the said process.
I am using the following command to get the momory usage of the process:
tasklist /fi "imagename eq %PROCESS%" /FO csv | findstr K
SAMPLE:
"cmd.exe","11640","Console","1","3,160 K"
This gives me a CSV file with last cell being the memory usage. I have two problems that I need help with.
Problem 1) How do I drop all but the last cell so that I can then assign the Kb used to a variable for comparison.
Problem 2) How do I get rid of the comma in the number? That kind of makes using comma as delim hard :/
Is there a better command than tasklist for this? I just need the raw number that the program is using, it can be in KB or MB.
Id love to be able to not have dependencies, but if I have to have dependencies I can include them with the batch.
Also is there any way for findstr to not return the entire line?
Thanks for any help! Ive been trying to get this solved for two days now with not much luck.

#ECHO OFF
SETLOCAL
FOR /f "delims=" %%i IN (memcsv.csv) DO CALL :process %%i
GOTO :EOF
:process
SET memsize=%~5
SET memsize=%memsize:,=%
ECHO memsize found = %memsize%
GOTO :eof
This should get your output into a variable called memsize.
It uses a file memcsv.csv as input, but you could replace mmcsv.csv with
'tasklist /fi "imagename eq %PROCESS%" /FO csv ^| findstr Mem'
to operate directly on the output of FINDSTR. Your resultant line would thus be
FOR /f "delims=" %%i IN ('tasklist /fi "imagename eq %PROCESS%" /FO csv ^| findstr Mem') DO CALL :process %%i
which, for ease of legibility could be entered as
FOR /f "delims=" %%i IN (
'tasklist /fi "imagename eq %PROCESS%" /FO csv ^| findstr Mem'
) DO CALL :process %%i
Note that the line-breaks are specific - before and after the single-quote.
Also that the single-quotes are REQUIRED and that there is a caret (^) before the pipe (|) which tells cmd that the pipe is part of the command to be executed, not part of the FOR command
Edit to add explanation of HOW.
The ouput of the tasklist...|findstr... can be used as input to a for/f as if it was a file. All you need do is to surround the command with SINGLE-QUOTES and ensure that redirectors like | < > are "escaped" by a caret.
FOR /F "reads" the "file" line-by-line, assigning (by default) the first "token" in the line to the "metavariable" (the loop-control variable, %%i in the above case). This behaviour canbe modified by the addition of control-clauses to the FOR/F. You may use `tokens=x,y,z" for instance to assign token number x, number y and number z to %%i, %%j, %%k respectively.
TOKENS are counted from 1 and have a value of the line contents up to a (series of) delimiter(s). By default, delimiters are spaces, commas, semicolons and TABs, so a line
TOKEN_ONE TOKEN_2,TOKEN_THREE;Token_FOUR
when seen by
for /f "tokens=1,3,4" %%i in (filecontainingaboveline) do
would set %%i=TOKEN_ONE %%j=TOKEN_THREE %%k=Token_FOUR
Using "delims=" turns OFF the delimiters and hence the ENTIRE line is assigned to the metavariable.
HENCE, in the above code, the entire line is assigned to %%i and delivered to the subroutine :process.
From :process's point-of-view, it has been given the argument ** "cmd.exe","11640","Console","1","3,160 K"** which it interprets as a sequence of 5 parameters separated by commas - and a comma (or any other separator) WITHIN "quotes" is data, not a separator.
Parameter number 5 is accessed by %5 - and that is "3,160 K" - including the quotes and comma.
The variable is set to the value of the fifth parameter - the tilde (~) means "remove enclosing quotes." Hence memsize acquires a value of 3,160 K
The next SET replaces the string after the colon in the nominated variable with the string after the = - replace commas with nothing, and assign the result to the memsize variable.
The goto :eof means 'go to the physical end-of-file.` It is very specific - the colon MUST be present. Reaching end-of-file terminates a subroutine or batch-process.
To remove the last 2 characters of the variable, you could use
SET var=%var:0,~-2%
where var is the variable-name.
SEE
SET /?
from the prompt for documentation.
Also GOTO /? and FOR/? for more details on these commands...

Related

Batch if else not executing consistently

I am trying to set a variable using if else statements. The problem is that the else if "%%D"GEQ "%%G" (set var=%%G) is not executing consistently. I am not sure what is wrong.
Here is my code:
#echo on
SETLOCAL EnableDelayedExpansion
FOR /F "tokens=1-8* delims=," %%A IN (results.csv) DO (
if "%%D" equ 0 (
set var=0
) else if "%%D" GEQ "%%G" (
set var=%%G
) else (set var=%%D)
set "y=%%A,%%B,%%C,%%D,%%E,%%F,%%G,%%H,!var!"
echo !y!>>final.csv
)
Here is a sample of my input file.
"01185901095","11","0379-0005","50","001","0","3","3"
"01185901215","11","0379-0013","138","001","0","4","2"
Here is the output I get in final.csv
"01185901095","11","0379-0005","50","001","0","3","3","3"
"01185901215","11","0379-0013","138","001","0","4","2","138"
My expected output is:
"01185901095","11","0379-0005","50","001","0","3","3","3"
"01185901215","11","0379-0013","138","001","0","4","2","4"
Line 2 in the output is the problem . %%D is greater than %%G so I expect the value of %%G (or 138 is greater than 4 so I expect 4)
There should be read first my answer on Symbol equivalent to NEQ, LSS, GTR, etc. in Windows batch files to get full knowledge on how the internal command IF of cmd.exe makes a string comparison which is done here and not an integer comparison as expected.
The condition if "%%D" equ 0 becomes if ""50"" EQU 0 on processing the first line of the input CSV file results.csv which results in the comparison of the string ""50"" with the string 0 because of the four double quotes around the value 50. The first character " of first string ""50"" has the decimal byte value 34 while the first character 0 of the second string 0 as the decimal byte value 48. The used function lstrcmpW exits for that reason already on comparing the first character of the two strings with −1. This integer value is compared next on being equal with the integer value 0 independent the fact that there is specified right to the comparison operator EQU by chance here also 0. The result of this condition is always false independent on which string is assigned to the loop variable D read from the file results.csv because of the double quotes around %%D resulting always in comparing the character " with the character 0.
There is next executed always if "%%D" GEQ "%%G" which on processing the first line of results.csv results in the comparison of the string ""50"" with the string ""3"". The first two characters of both compared strings are equal. The third character 5 of the first string has with decimal byte value 53 a greater value than third character 3 of the second string with decimal byte value 51 and for that reason the result of the string comparison is the integer value 1 which is greater the integer value 0 and for that reason the second condition is by chance correct true for the first line of results.csv.
But on processing the second line of results.csv is compared the string ""138"" with the string ""4"" on which the third character of first string has a lower byte value than the third character of the second string. The result of the string comparison is in this case −1 which is less than the integer value 0. The comparison result is false for the second condition on processing the values in second line of the CSV file although the integer value 138 would be greater than the integer value 4.
The solution is not using " around the loop variable references at all and additionally remove the double quotes around the values read from the CSV file to really run integer value comparisons and not string comparisons which means using %%~D and %%~G.
#echo off
setlocal EnableExtensions EnableDelayedExpansion
(for /F "tokens=1-8* delims=," %%A in (results.csv) do (
if %%~D EQU 0 (
set var=0
) else if %%~D GEQ %%~G (
set var=%%G
) else set var=%%D
echo %%A,%%B,%%C,%%D,%%E,%%F,%%G,%%H,!var!
))>final.csv
endlocal
There is one more important performance modification in the above code among some other not so important small improvements: the entire for loop is enclosed in round brackets and everything output to standard output stream by the command echo inside the loop is written into the file final.csv.
This code results on execution in opening first the file final.csv for write operations and keeping it open all the time as long as for is processing next the lines read from results.csv with finally flushing the data of final.csv and closing this file after for finished and closed the file results.csv.
The code in question results for each line read from results.csv to open the file final.csv, seek to end of the file, append the line output with command echo and then close the file final.csv. This makes processing thousands or millions of lines in results.csv much slower than the code above although the file caching mechanisms of Windows avoid really writing the opened, changed and closed file final.csv on each line to the hard disk.
Note: The code as posted here works only if there are no empty field values in results.csv which means there is no line in results.csv with ,,. The current directory on starting the execution of the batch file must be the directory containing results.csv as otherwise final.csv will be an empty file.
I'm not quite sure of the purpose of some of your comparisons, so based purely off best guesses, your input file content and your expected output content, would something like this not do what you wanted:
#( For /F "UseBackQ EOL=, Delims=" %%G In ("results.csv"
) Do #For /F "Tokens=4,7 Delims=," %%H In ("%%~G"
) Do #If %%~H Gtr %%~I (Echo %%G,"%%~I") Else Echo %%G,"%%~H"
) 1>"final.csv"
As you can see, there's no need for delayed expansion, or defining variables.

Batch Script - Delete Columns in csv

I do need a batch script who will remove all columns in a csv, except column 1,2 and 5
My Code:
(for /f "tokens=1,2,5 delims=;" %%i in (Input.csv) do echo %%i,%%j,%%k) > Output.csv
Input CSV
1;2;3;4;5;6;7;8;9;10
10160;"Some Name";"Something:0.8";;5;;;;;XY
Expected Output:
1;2;5
10160;"Some Name";5
Real Output
1,2,5
10160,"Some Name",XY
Does anyone have any idea why it keeps the tenth column in the second line instead of the fifth?
SETLOCAL ENABLEDELAYEDEXPANSION
(FOR /f "delims=" %%b IN (Input.csv) DO SET "line=%%b"&SET "line=!line:;;=; ;!"&for /f "tokens=1,2,5 delims=;" %%i in ("!line:;;=; ;!") do echo %%i,%%j,%%k)
The problem is that a sequence of delimiters is considered as a single delimiter, so you need to change each delimiter pair so that it contains a string, and repeat the operation for any remaining delimiter-pairs.
Obviously, you would need to take action to take care of a reported field that now contains a single space, and this will alter any quoted field that contains ;;
Note also that any data containing ! or % is likely to be corrupted and certain other symbols (such as &) may also yield unexpected results. If the data is restricted to alphamerics, spaces, commas, etc. it should be fine.

split csv by year with batch

i'm trying to split a csv file based on the year. The year is allways shown in UTC Format after first delimiter ";".
example for csv file:
ID;Datum;EUR
1;2021-12-12 12:12:12;50
1;2020-12-12 12:12:12;10
2;2020-12-12 12:12:12;20
1;2019-12-12 12:12:12;80
So far I have the following batch code as a solution.
findstr /I /R /C:"^[^;]*;2019-" "test.csv" > "test_year1.csv"
findstr /I /R /C:"^[^;]*;2020-" "test.csv" > "test_year2.csv"
findstr /I /R /C:"^[^;]*;2021-" "test.csv" > "test_year3.csv"
It works but how can i add the first line (Header) to each of this splitfiles?
Also, the code is a bit simple and unwieldy. Is it possible to automate the whole thing a bit and automatically determine the possible year's from inputfile and create a corresponding output file?
thx for help,
SaXe
#ECHO OFF
SETLOCAL
rem The following settings for the source directory, destination directory, filenames, output filename are names
rem that I use for testing and deliberately include names which include spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.
SET "sourcedir=u:\your files"
SET "destdir=u:\your results"
SET "filename1=%sourcedir%\q71435963.txt"
:: remove variables starting #
FOR /F "delims==" %%b In ('set # 2^>Nul') DO SET "%%b="
:: set environment variables #year#yearnumber# from sourcefile
FOR /f "skip=1usebackqtokens=2delims=-;" %%b IN ("%filename1%") DO SET "#year#%%b#=Y"
:: set "line1" to first line of file
FOR /f "usebackqdelims=" %%b IN ("%filename1%") DO SET "line1=%%b"&GOTO buildfiles
:buildfiles
:: for each year found, create a new file & append data
FOR /f "tokens=2delims=#" %%b IN ('set # 2^>nul') DO (
>"%destdir%\test_%%b.csv" ECHO %line1%
>>"%destdir%\test_%%b.csv" findstr /I /R /C:"^[^;]*;%%b-" "%filename1%"
)
GOTO :EOF
The 2^>nul in each of the set commands suppresses error messages that would occur if there are no variables matching the pattern currently set. The ^ "escapes" the >, telling cmd that the > is part of the set, not of the for.
The usebackq option is only required because I chose to add quotes around the source filename.
--------- Fix for - in column 1:
in place of the for /f "skip... line, use
FOR /f "skip=1usebackqtokens=2delims=;" %%c IN ("%filename1%") DO FOR /f "delims=-" %%b IN ("%%c") DO SET "#year#%%b#=Y"
This assigns the string from the second to third occurrence of ; which grabs the second column to %%c, skipping the first line. Then the variable is set as before using the value of %%c, using - as a delimiter and selects the first token (ie. %%cup to the first delimiter) into %%b as before. The default for tokens= is tokens=1.

How to replace commas with semicolons except for commas in Quotes?

I have a csv file with commas used to separate values. I want to replace commas with semicolons via batch, but leave the commas that are inside quotations.
So for example:
012,ABC,"DE,FG",345
must become:
012;ABC;"DE,FG";345
How can I do that via Batch?
If you happen to have the JREPL.BAT regular expression text processing utility (v7.9 or later), then you can use:
jrepl "," ";" /p "([\c\q]+)|\q.*?\q" /prepl "$1?{$0}:$0" /f "test.csv" /o -
Use call jrepl if you put the command within a batch script.
The original file will be overwritten. You can substitute a new file name for - if you don't want to overwrite the original.
JREPL.BAT is pure script (hybrid JScript/batch) that runs natively on any Windows machine from XP onward - no 3rd party .exe file required.
The JREPL solution works by performing the replacement in two steps.
1) The /P option breaks each line into unquoted strings and quoted strings. The /PREPL option passes unquoted strings on to the normal FIND/REPLACE, and unquoted strings are preserved as is.
2) The main FIND/REPLACE substitutes ; for ,
It is possible to reliably accomplish this with pure batch using a variant of a technique developed by jeb at 'Pretty print' windows %PATH% variable - how to split on ';' in CMD shell. Although any pure batch solution will be significantly slower than hybrid solutions like JREPL.BAT, ParseCSV.bat, or a powershell solution.
Here is a batch script derived from jeb's technique - simply pass the name of the CSV file as the one and only argument. The original file will be overwritten. It should be trivial to modify the script to write the output to a new file instead. See jeb's post for an overview of how this seemingly magical technique works.
#echo off
setlocal disableDelayedExpansion
>"%~1.new" (
for /f usebackq^ delims^=^ eol^= %%A in ("%~1") do (
set "ln=%%A"
call :repl
)
)
move /y "%~1.new" "%~1" >nul
exit /b
:repl
set "ln=%ln:"=""%"
set "ln=%ln:^=^^%"
set "ln=%ln:&=^&%"
set "ln=%ln:|=^|%"
set "ln=%ln:<=^<%"
set "ln=%ln:>=^>%"
set "ln=%ln:,=^,^,%"
set ln=%ln:""="%
set "ln=%ln:"=""%"
set "ln=%ln:,,=;%"
set "ln=%ln:^,^,=,%"
set "ln=%ln:""="%"
setlocal enableDelayedExpansion
echo(!ln!
exit /b
The script should be able to process almost any valid CSV file input. The only restrictions are:
Empty lines are stripped from the output (should not be a problem with CSV)
Line lengths are limited to around 8 kb. The exact limit is dependent on how many intermediate substitutions must be performed.
Powershell is probably the better solution but you can use a neat hybrid batch file called ParseCSV.bat. It allows you to specify the input and output delimiters. The input delimiter uses a comma by default. So you only need to specify the output delimiter.
ParseCSV.bat /o:; <"file.csv" >"filenew.csv"
This possible alternative appears to work with the single line example you've provided:
#Echo Off
If Not Exist "file.csv" Exit/B
(For /F "Delims=" %%A In ('FindStr "^" "file.csv"') Do (Set "$="
For %%B In (%%A) Do Call Set "$=%%$%%;%%B"
Call Echo %%$:~1%%))>"filenew.csv"

Batch echo string is capped when passed as argument

I run the batch script on below on a csv file.
#echo off
set "filename=test.csv"
for /f "USEBACKQtokens=*" %%A in ("%filename%") do (
call :myecho "%%A"
)
pause
:myecho
echo %1
GOTO :eof
A typical line in the csv file looks like this:
Text and some more 2/11/2015,A234,Mr. Example Guy,18217.01,"$20,729.89","$4,111.06","$4,453.08"
The echo will actually result in the following output:
Text and some more 2/11/2015,A234,Mr. Example Guy,18217.01,"$20,
Somehow it cuts off at the comma, after the first quotation mark. I don't understand why. Could someone explain, and tell me how I could print the entire line there? Note that if I print directly in the for loop it prints the entire line.
The following line
call :myecho "%%A"
is expanded to
call :myecho "Text and some more 2/11/2015,A234,Mr. Example Guy,18217.01,"$20,729.89","$4,111.06","$4,453.08""
Arguments to CALLed subroutines are delimited by any of <space>, ,, ;, <tab>, =, or <0x255>. Delimiters within quotes are ignored. The commas in $20,729.89 are quoted in the original CSV, but they are not quoted in your CALL because you added your own enclosing quotes. So argument %1 ends at the first unquoted comma in the dollar amount.
If you want to pass a string containing quotes and delimiters to a subroutine, then you should pass them by reference via an environment variable. You should use delayed expansion to expand the variable so that all characters are protected. But delayed expansion should be off within the FOR loop so that ! values are not corrupted when the FOR loop variable is expanded.
#echo off
setlocal disableDelayedExpansion
set "filename=test.csv"
for /f "USEBACKQtokens=*" %%A in ("%filename%") do (
set "var=%%A"
call :myecho var
)
pause
exit /b
:myecho
setlocal enableDelayedExpansion
echo !%1!
exit /b