Batch echo string is capped when passed as argument - csv

I run the batch script on below on a csv file.
#echo off
set "filename=test.csv"
for /f "USEBACKQtokens=*" %%A in ("%filename%") do (
call :myecho "%%A"
)
pause
:myecho
echo %1
GOTO :eof
A typical line in the csv file looks like this:
Text and some more 2/11/2015,A234,Mr. Example Guy,18217.01,"$20,729.89","$4,111.06","$4,453.08"
The echo will actually result in the following output:
Text and some more 2/11/2015,A234,Mr. Example Guy,18217.01,"$20,
Somehow it cuts off at the comma, after the first quotation mark. I don't understand why. Could someone explain, and tell me how I could print the entire line there? Note that if I print directly in the for loop it prints the entire line.

The following line
call :myecho "%%A"
is expanded to
call :myecho "Text and some more 2/11/2015,A234,Mr. Example Guy,18217.01,"$20,729.89","$4,111.06","$4,453.08""
Arguments to CALLed subroutines are delimited by any of <space>, ,, ;, <tab>, =, or <0x255>. Delimiters within quotes are ignored. The commas in $20,729.89 are quoted in the original CSV, but they are not quoted in your CALL because you added your own enclosing quotes. So argument %1 ends at the first unquoted comma in the dollar amount.
If you want to pass a string containing quotes and delimiters to a subroutine, then you should pass them by reference via an environment variable. You should use delayed expansion to expand the variable so that all characters are protected. But delayed expansion should be off within the FOR loop so that ! values are not corrupted when the FOR loop variable is expanded.
#echo off
setlocal disableDelayedExpansion
set "filename=test.csv"
for /f "USEBACKQtokens=*" %%A in ("%filename%") do (
set "var=%%A"
call :myecho var
)
pause
exit /b
:myecho
setlocal enableDelayedExpansion
echo !%1!
exit /b

Related

Batch Script - Delete Columns in csv

I do need a batch script who will remove all columns in a csv, except column 1,2 and 5
My Code:
(for /f "tokens=1,2,5 delims=;" %%i in (Input.csv) do echo %%i,%%j,%%k) > Output.csv
Input CSV
1;2;3;4;5;6;7;8;9;10
10160;"Some Name";"Something:0.8";;5;;;;;XY
Expected Output:
1;2;5
10160;"Some Name";5
Real Output
1,2,5
10160,"Some Name",XY
Does anyone have any idea why it keeps the tenth column in the second line instead of the fifth?
SETLOCAL ENABLEDELAYEDEXPANSION
(FOR /f "delims=" %%b IN (Input.csv) DO SET "line=%%b"&SET "line=!line:;;=; ;!"&for /f "tokens=1,2,5 delims=;" %%i in ("!line:;;=; ;!") do echo %%i,%%j,%%k)
The problem is that a sequence of delimiters is considered as a single delimiter, so you need to change each delimiter pair so that it contains a string, and repeat the operation for any remaining delimiter-pairs.
Obviously, you would need to take action to take care of a reported field that now contains a single space, and this will alter any quoted field that contains ;;
Note also that any data containing ! or % is likely to be corrupted and certain other symbols (such as &) may also yield unexpected results. If the data is restricted to alphamerics, spaces, commas, etc. it should be fine.

split csv by year with batch

i'm trying to split a csv file based on the year. The year is allways shown in UTC Format after first delimiter ";".
example for csv file:
ID;Datum;EUR
1;2021-12-12 12:12:12;50
1;2020-12-12 12:12:12;10
2;2020-12-12 12:12:12;20
1;2019-12-12 12:12:12;80
So far I have the following batch code as a solution.
findstr /I /R /C:"^[^;]*;2019-" "test.csv" > "test_year1.csv"
findstr /I /R /C:"^[^;]*;2020-" "test.csv" > "test_year2.csv"
findstr /I /R /C:"^[^;]*;2021-" "test.csv" > "test_year3.csv"
It works but how can i add the first line (Header) to each of this splitfiles?
Also, the code is a bit simple and unwieldy. Is it possible to automate the whole thing a bit and automatically determine the possible year's from inputfile and create a corresponding output file?
thx for help,
SaXe
#ECHO OFF
SETLOCAL
rem The following settings for the source directory, destination directory, filenames, output filename are names
rem that I use for testing and deliberately include names which include spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.
SET "sourcedir=u:\your files"
SET "destdir=u:\your results"
SET "filename1=%sourcedir%\q71435963.txt"
:: remove variables starting #
FOR /F "delims==" %%b In ('set # 2^>Nul') DO SET "%%b="
:: set environment variables #year#yearnumber# from sourcefile
FOR /f "skip=1usebackqtokens=2delims=-;" %%b IN ("%filename1%") DO SET "#year#%%b#=Y"
:: set "line1" to first line of file
FOR /f "usebackqdelims=" %%b IN ("%filename1%") DO SET "line1=%%b"&GOTO buildfiles
:buildfiles
:: for each year found, create a new file & append data
FOR /f "tokens=2delims=#" %%b IN ('set # 2^>nul') DO (
>"%destdir%\test_%%b.csv" ECHO %line1%
>>"%destdir%\test_%%b.csv" findstr /I /R /C:"^[^;]*;%%b-" "%filename1%"
)
GOTO :EOF
The 2^>nul in each of the set commands suppresses error messages that would occur if there are no variables matching the pattern currently set. The ^ "escapes" the >, telling cmd that the > is part of the set, not of the for.
The usebackq option is only required because I chose to add quotes around the source filename.
--------- Fix for - in column 1:
in place of the for /f "skip... line, use
FOR /f "skip=1usebackqtokens=2delims=;" %%c IN ("%filename1%") DO FOR /f "delims=-" %%b IN ("%%c") DO SET "#year#%%b#=Y"
This assigns the string from the second to third occurrence of ; which grabs the second column to %%c, skipping the first line. Then the variable is set as before using the value of %%c, using - as a delimiter and selects the first token (ie. %%cup to the first delimiter) into %%b as before. The default for tokens= is tokens=1.

Batch file to move second line/row of a csv/text file to another csv/text file (must skip first line of csv/text file)

I need to move second line of a csv/text file to another text/csv file.
I have a solution to delete the second line of csv/text file.
The first line of csv/text file contains header and must be not moved or deleted.
Warning: Path folders have spaces
My code
#echo off
Pushd "D:\Program Files\datasources\"
setlocal disableDelayedexpansion
>archive.new (
break | for /F "tokens=*" %%F in (archive.csv) do #(
echo %%F
exit /b
)
for /F "skip=2 tokens=*" %%F in (archive.csv) do echo %%F
)
DEL /S "archive.csv"
REN "archive.new" "archive.csv"
I need only to move the second line in another file, this code delete the second line of text/csv file.
I think you are over complicating your code. Every line gets preserved in one file or another, so you might as well read every line in your one loop and then choose which file to write the line to based on the line number.
I use SET /A to increment a ln counter, and if the resultant value is 2 then I get a division by 0 error. The && code writes all the success lines (all but 2) to the new archive, and the || code writes the failure (line 2) to the other file.
#echo off
pushd "d:\Program Files\datasources\"
set "src=archive.csv"
set "file2=otherFile.csv"
set ln=0
>"%src%.new" (
for /f usebackq^ delims^=^ eol^= %%A in ("%src%") do (
2>nul set /a "1/(2-(ln+=1))" && (echo(%%A) || (>>"%file2%" echo(%%A)
)
)
move /y "%src%.new" "%src%" >nul
Some additional advanced "tricks" I employed:
Use usebackq so file name can be enclosed in quotes, just in case there are spaces and or poison characters in the name.
Use move instead of del followed by ren
Arcane syntax to set both eol and delims to nothing, thus guaranteeing all lines are preserved exactly (provided they don't exceed the ~8191 character limit)
Use echo( to guarantee correct output, no matter the line (including empty line)
Put redirection in front and enclose echo statements in parentheses to ensure no unwanted trailing white space characters in output

Windows Batch: replace substring in CSV file

This is a popular subject on the Web but I can't find a simple way to do a find-and-replace inside a CSV file. My CSV file looks like this:
"0.219530613504834,43.7737904197643,PR RUE D'ARTOIS"
"0.522235882011867,41.7681203998576,PR DE LA FOSSE AU ROI"
"0.039404145384227,44.7565229712732,PR DES PETITS PRES"
I need to remove double quotes at the beginning and end of each line.
I tried the following script:
setlocal enabledelayedexpansion
for /f "tokens=*" %%a in (D:/data.csv) do (
set temp=%%a
set temp=%temp:^"=%
)
My logic is to store each line in %%a and to replace the double quotes by nothing.
I escape the double quote with ^.
This script is doing nothing. Could you help?
There is a much simpler solution that avoids delayed expansion altogther - The ~ modifier strips enclosing quotes from FOR variable content.
for /f "delims=" %%a in (D:/data.csv) do echo %%~a
Your attempt has several issues:
You need to use delayed expansion (use ! instead of %). Without delayed expansion, the variable gets expanded before the loop is entered.
It's more reliable to use "delims=" than "tokens=*". With "tokens=*" each line is still tokenized, and then all tokens are concatenated separated by a space. But the tokens may have originally been separated by a ;. So if you had "one;two" in the input you'd get "one two" in the output.
You don't need to escape the ".
You need to print temp, not just modify it without using the new value.
setlocal enabledelayedexpansion
for /f "delims=" %%a in (D:/data.csv) do (
set temp=%%a
echo !temp:"=!
)

How to drop all but last cell in CSV using CMD

my goal is to write a script that will monitor process memory usage and run % based comparison on it to determine if there is a memory leak in the said process.
I am using the following command to get the momory usage of the process:
tasklist /fi "imagename eq %PROCESS%" /FO csv | findstr K
SAMPLE:
"cmd.exe","11640","Console","1","3,160 K"
This gives me a CSV file with last cell being the memory usage. I have two problems that I need help with.
Problem 1) How do I drop all but the last cell so that I can then assign the Kb used to a variable for comparison.
Problem 2) How do I get rid of the comma in the number? That kind of makes using comma as delim hard :/
Is there a better command than tasklist for this? I just need the raw number that the program is using, it can be in KB or MB.
Id love to be able to not have dependencies, but if I have to have dependencies I can include them with the batch.
Also is there any way for findstr to not return the entire line?
Thanks for any help! Ive been trying to get this solved for two days now with not much luck.
#ECHO OFF
SETLOCAL
FOR /f "delims=" %%i IN (memcsv.csv) DO CALL :process %%i
GOTO :EOF
:process
SET memsize=%~5
SET memsize=%memsize:,=%
ECHO memsize found = %memsize%
GOTO :eof
This should get your output into a variable called memsize.
It uses a file memcsv.csv as input, but you could replace mmcsv.csv with
'tasklist /fi "imagename eq %PROCESS%" /FO csv ^| findstr Mem'
to operate directly on the output of FINDSTR. Your resultant line would thus be
FOR /f "delims=" %%i IN ('tasklist /fi "imagename eq %PROCESS%" /FO csv ^| findstr Mem') DO CALL :process %%i
which, for ease of legibility could be entered as
FOR /f "delims=" %%i IN (
'tasklist /fi "imagename eq %PROCESS%" /FO csv ^| findstr Mem'
) DO CALL :process %%i
Note that the line-breaks are specific - before and after the single-quote.
Also that the single-quotes are REQUIRED and that there is a caret (^) before the pipe (|) which tells cmd that the pipe is part of the command to be executed, not part of the FOR command
Edit to add explanation of HOW.
The ouput of the tasklist...|findstr... can be used as input to a for/f as if it was a file. All you need do is to surround the command with SINGLE-QUOTES and ensure that redirectors like | < > are "escaped" by a caret.
FOR /F "reads" the "file" line-by-line, assigning (by default) the first "token" in the line to the "metavariable" (the loop-control variable, %%i in the above case). This behaviour canbe modified by the addition of control-clauses to the FOR/F. You may use `tokens=x,y,z" for instance to assign token number x, number y and number z to %%i, %%j, %%k respectively.
TOKENS are counted from 1 and have a value of the line contents up to a (series of) delimiter(s). By default, delimiters are spaces, commas, semicolons and TABs, so a line
TOKEN_ONE TOKEN_2,TOKEN_THREE;Token_FOUR
when seen by
for /f "tokens=1,3,4" %%i in (filecontainingaboveline) do
would set %%i=TOKEN_ONE %%j=TOKEN_THREE %%k=Token_FOUR
Using "delims=" turns OFF the delimiters and hence the ENTIRE line is assigned to the metavariable.
HENCE, in the above code, the entire line is assigned to %%i and delivered to the subroutine :process.
From :process's point-of-view, it has been given the argument ** "cmd.exe","11640","Console","1","3,160 K"** which it interprets as a sequence of 5 parameters separated by commas - and a comma (or any other separator) WITHIN "quotes" is data, not a separator.
Parameter number 5 is accessed by %5 - and that is "3,160 K" - including the quotes and comma.
The variable is set to the value of the fifth parameter - the tilde (~) means "remove enclosing quotes." Hence memsize acquires a value of 3,160 K
The next SET replaces the string after the colon in the nominated variable with the string after the = - replace commas with nothing, and assign the result to the memsize variable.
The goto :eof means 'go to the physical end-of-file.` It is very specific - the colon MUST be present. Reaching end-of-file terminates a subroutine or batch-process.
To remove the last 2 characters of the variable, you could use
SET var=%var:0,~-2%
where var is the variable-name.
SEE
SET /?
from the prompt for documentation.
Also GOTO /? and FOR/? for more details on these commands...