I'm trying to create a .bat file to run on Windows 10.
Basically I have multiple .csv files in a folder.
FileA.csv
abc,abc
def,def
FileB.csv
ghi,ghi
jkl,jkl
I want the files to be merged to FileC.csv formatted like this.
abc,abc
def,def
ghi,ghi
jkl,jkl
If I use:
copy /b *.csv FileC.csv
I get this output in FileC.csv:
abc,abc
def,defghi,ghi
jkl,jkl
I need to add a line break between merged files but can't figure out how. Any help would be appreciated.
type A.csv >> C.csv
echo. >> C.csv
type B.csv >> C.csv
Type will show the contents of a file
>>C.csv will append the output to an existig file
echo. will print an empty line
UPD
FOR %f IN (*.csv) DO type %f >> ..\newfile.csv & echo. >> ..\newfile.csv
this will merge all .csv files in new .csv file in parent folder (if that new file will be same folder, after create first iteration, for also merge that file too)
In a cmd window this will insert breaks and remove empty lines:
(for /f "delims=" %A in ('Type FileA.csv^&Echo:^&Type FileB.csv') Do #Echo %A)>FileC.csv
In a batch
#Echo off
( for /f "delims=" %%A in (
'Type fileA.csv^&Echo:^&Type FileB.csv'
) Do #Echo:%%A
) > FileC.csv
If a space before each line break is acceptable
#echo off
rem Concatenate input text files, inserting a line break between each file
set inpattern=*.csv
set outfile=all.csv
set count=0
rem Delete any existing output file
del /q %outfile%
rem Count the number of input files
for %%f in (%inpattern%) do (call :count "%%f")
rem Concatenate the input files
for %%f in (%inpattern%) do (call :concat "%%f")
goto :EOF
rem End of main routine
rem Subroutines
:count
rem Increment the input file counter
set /a count+=1
goto :EOF
:concat
rem Append this input file to the output file
type %1 >> %outfile%
rem If this is not the last input file, then append a line break to the output file
if %count% gtr 1 echo.>> %outfile%
rem Decrement the input file counter
set /a count-=1
goto :EOF
About this script:
The echo. command inserts a space (\x20 byte) followed by a Windows-style two-byte line break: a carriage return (Cr, \x0D) followed by a line feed (Lf, \x0A).
The lack of a space in the script between echo. and >> is deliberate. If there is a space between echo. and >>, then the echo. "honors" that space, and the output contains two spaces before the line break.
The space before the line break annoys me, but I can live with it.
Otherwise
If you cannot live with a space before the line break, or you need a Unix-style line break (or some other type of line break):
Create a text file, linebreak.txt, that contains only the line break.
Use the following slightly modified script, which appends the linebreak.txt file between the input files, instead of using the echo. command.
#echo off
rem Concatenate input text files, inserting a line break between each file
set inpattern=*.csv
set outfile=all.csv
set linebreak=linebreak.txt
set count=0
rem Delete any existing output file
del /q %outfile%
rem Count the number of input files
for %%f in (%inpattern%) do (call :count "%%f")
rem Concatenate the input files
for %%f in (%inpattern%) do (call :concat "%%f")
goto :EOF
rem End of main routine
rem Subroutines
:count
rem Increment the input file counter
set /a count+=1
goto :EOF
:concat
rem Append this input file to the output file
type %1 >> %outfile%
rem If this is not the last input file, then append a line break to the output file
if %count% gtr 1 type %linebreak% >> %outfile%
rem Decrement the input file counter
set /a count-=1
goto :EOF
Related
i'm trying to split a csv file based on the year. The year is allways shown in UTC Format after first delimiter ";".
example for csv file:
ID;Datum;EUR
1;2021-12-12 12:12:12;50
1;2020-12-12 12:12:12;10
2;2020-12-12 12:12:12;20
1;2019-12-12 12:12:12;80
So far I have the following batch code as a solution.
findstr /I /R /C:"^[^;]*;2019-" "test.csv" > "test_year1.csv"
findstr /I /R /C:"^[^;]*;2020-" "test.csv" > "test_year2.csv"
findstr /I /R /C:"^[^;]*;2021-" "test.csv" > "test_year3.csv"
It works but how can i add the first line (Header) to each of this splitfiles?
Also, the code is a bit simple and unwieldy. Is it possible to automate the whole thing a bit and automatically determine the possible year's from inputfile and create a corresponding output file?
thx for help,
SaXe
#ECHO OFF
SETLOCAL
rem The following settings for the source directory, destination directory, filenames, output filename are names
rem that I use for testing and deliberately include names which include spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.
SET "sourcedir=u:\your files"
SET "destdir=u:\your results"
SET "filename1=%sourcedir%\q71435963.txt"
:: remove variables starting #
FOR /F "delims==" %%b In ('set # 2^>Nul') DO SET "%%b="
:: set environment variables #year#yearnumber# from sourcefile
FOR /f "skip=1usebackqtokens=2delims=-;" %%b IN ("%filename1%") DO SET "#year#%%b#=Y"
:: set "line1" to first line of file
FOR /f "usebackqdelims=" %%b IN ("%filename1%") DO SET "line1=%%b"&GOTO buildfiles
:buildfiles
:: for each year found, create a new file & append data
FOR /f "tokens=2delims=#" %%b IN ('set # 2^>nul') DO (
>"%destdir%\test_%%b.csv" ECHO %line1%
>>"%destdir%\test_%%b.csv" findstr /I /R /C:"^[^;]*;%%b-" "%filename1%"
)
GOTO :EOF
The 2^>nul in each of the set commands suppresses error messages that would occur if there are no variables matching the pattern currently set. The ^ "escapes" the >, telling cmd that the > is part of the set, not of the for.
The usebackq option is only required because I chose to add quotes around the source filename.
--------- Fix for - in column 1:
in place of the for /f "skip... line, use
FOR /f "skip=1usebackqtokens=2delims=;" %%c IN ("%filename1%") DO FOR /f "delims=-" %%b IN ("%%c") DO SET "#year#%%b#=Y"
This assigns the string from the second to third occurrence of ; which grabs the second column to %%c, skipping the first line. Then the variable is set as before using the value of %%c, using - as a delimiter and selects the first token (ie. %%cup to the first delimiter) into %%b as before. The default for tokens= is tokens=1.
I need to move second line of a csv/text file to another text/csv file.
I have a solution to delete the second line of csv/text file.
The first line of csv/text file contains header and must be not moved or deleted.
Warning: Path folders have spaces
My code
#echo off
Pushd "D:\Program Files\datasources\"
setlocal disableDelayedexpansion
>archive.new (
break | for /F "tokens=*" %%F in (archive.csv) do #(
echo %%F
exit /b
)
for /F "skip=2 tokens=*" %%F in (archive.csv) do echo %%F
)
DEL /S "archive.csv"
REN "archive.new" "archive.csv"
I need only to move the second line in another file, this code delete the second line of text/csv file.
I think you are over complicating your code. Every line gets preserved in one file or another, so you might as well read every line in your one loop and then choose which file to write the line to based on the line number.
I use SET /A to increment a ln counter, and if the resultant value is 2 then I get a division by 0 error. The && code writes all the success lines (all but 2) to the new archive, and the || code writes the failure (line 2) to the other file.
#echo off
pushd "d:\Program Files\datasources\"
set "src=archive.csv"
set "file2=otherFile.csv"
set ln=0
>"%src%.new" (
for /f usebackq^ delims^=^ eol^= %%A in ("%src%") do (
2>nul set /a "1/(2-(ln+=1))" && (echo(%%A) || (>>"%file2%" echo(%%A)
)
)
move /y "%src%.new" "%src%" >nul
Some additional advanced "tricks" I employed:
Use usebackq so file name can be enclosed in quotes, just in case there are spaces and or poison characters in the name.
Use move instead of del followed by ren
Arcane syntax to set both eol and delims to nothing, thus guaranteeing all lines are preserved exactly (provided they don't exceed the ~8191 character limit)
Use echo( to guarantee correct output, no matter the line (including empty line)
Put redirection in front and enclose echo statements in parentheses to ensure no unwanted trailing white space characters in output
I have created batch file to combine all csv file in folder. Below is my batch file code.
#ECHO OFF
SET first=y
SET newfile=Summary.csv
for %%F in (*.csv) do IF NOT %%F==%newfile% (
if defined first (
COPY /y "%%F" %newfile% >nul
set "first="
) else (
FOR /f "skip=1delims=" %%i IN (%%F) DO >> %newfile% ECHO %%i
)
)
My question is, how do i add into the code if i want to add the autosum for every column?
Below is my example csv file after i run the batch file.
Name,A4 Used,A3 Used,Others
A,23,9,2
B,61,41,0
C,5,85,7
I need to create an autosum for every column like example below.
Name,A4 Used,A3 Used,Others
A,23,9,2
B,61,41,0
C,5,85,7
Total,89,135,9
Any idea guys?
This task could be done with following commented batch code depending on contents of processed CSV files:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem Exit this batch file if current directory does not contain any CSV file.
if not exist *.csv goto EndBatch
rem The summary CSV file is created first in directory for temporary
rem files to avoid that outer FOR loop below tries to process also
rem the summary file. The summary file is created with header row.
set "NewFile=%TEMP%\Summary.csv"
echo Name,A4 Used,A3 Used,Others>"%NewFile%"
rem Make sure there is no summary CSV file in current directory
rem from a previous execution of this batch file in this directory.
del "Summary.csv" 2>nul
rem Initialize the environment variables for total sum of each column.
set "TotalColumn2=0"
set "TotalColumn3=0"
set "TotalColumn4=0"
rem The outer loop is executed for each CSV file in current directory.
rem The inner loop reads each CSV file line by line. The first line is
rem always skipped. Skipped are also empty lines and lines starting with
rem a semicolon. All other lines are split up into four substrings using
rem comma as separator (delimiter).
for %%I in (*.csv) do (
for /F "usebackq skip=1 tokens=1-4 delims=," %%A in ("%%I") do (
if not "%%D" == "" (
set /A TotalColumn2+=%%B
set /A TotalColumn3+=%%C
set /A TotalColumn4+=%%D
>>"%NewFile%" echo %%A,%%B,%%C,%%D
) else (
del "%NewFile%"
echo ERROR: A line in "%%I" has not four comma separated values.
echo/
pause
goto EndBatch
)
)
)
rem Append to summary file the total sums and move the summary file
rem from temporary files directory to current directory. If that fails
rem unexpected, delete the summary file in temporary files directory.
>>"%NewFile%" echo Total,%TotalColumn2%,%TotalColumn3%,%TotalColumn4%
move "%NewFile%" "Summary.csv" >nul
if errorlevel 1 (
del "%NewFile%"
echo ERROR: Could not move Summary.csv to "%CD%".
echo/
pause
)
:EndBatch
endlocal
Please note the limitations of Windows command interpreter:
Arithmetic expressions can be done only with 32-bit signed integers which means the value range is limited from -2147483648 to 2147483647. There is no support for floating point arithmetic.
Command FOR interprets a sequence of delimiters as one delimiter on splitting up a line into substrings. So a row like D,80,,20 in a CSV file results in loop variable A gets assigned D, loop variable B gets assigned 80, loop variable C gets assigned 20 and loop variable D has nothing assigned. In this case the batch file exits with an error message.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
del /?
echo /?
endlocal /?
for /?
goto /?
if /?
move /?
pause /?
rem /?
set /?
setlocal /?
Read also the Microsoft article about Using Command Redirection Operators.
I have files that look like that:
1989_Footer.gif
1989_Header.gif
260273_Footer.gif
260273_Header.gif
...
and I have a CSV that looks like this:
1989;10773387
260273;10776516
...
I want to rename the files number with the one given in the CSV. The first number in the CSV is the old number and the second one (after the semicolon) is the new number. How can I achive that without touching the text after the underline?
I thought about getting the file name, then search the number in the CSV and replace it with the value in the same line after the semicolon.
#ECHO OFF
SETLOCAL
SET "sourcedir=U:\sourcedir"
SET "filename1=%sourcedir%\q40908252.txt"
FOR /f "usebackqtokens=1*delims=;" %%a IN ("%filename1%") DO (
FOR /f "tokens=1*delims=_" %%r IN ('dir /b /a-d "%sourcedir%\%%a_*"') DO (
ECHO(REN "%sourcedir%\%%r_%%s" "%%b_%%s"
)
)
GOTO :EOF
You would need to change the setting of sourcedir to suit your circumstances.
I used a file named q40908252.txt containing your data for my testing.
The required REN commands are merely ECHOed for testing purposes. After you've verified that the commands are correct, change ECHO(REN to REN to actually rename the files.
Read the csv file, using ; as the delimiter, placing the from-prefix in %%a and the to-prefix in %%b.
Perform a directory list of files named "%%a_anything" in the source directory, tokenise on _ so that the first token (which must be %%a) goes to %%r and the remainder-of-name to %%s, then rename the file with the prefixes switched.
Assumes that the first _ in the filename will never be a multiple-_
My JREN.BAT regular expression renaming utility can simplify the solution. JREN.BAT is pure script (hybrid batch/JScript) that runs natively on any Windows machine from XP onward - no 3rd party exe file required. Full documentation is available from the command line via jren /?, or jren /?? for paged help.
From within a batch script:
#echo off
for /f "delims=; tokens=1,2" %%A in (rename.csv) do call jren "^%%A_" "%%B_" /fm *.gif
From the command line:
for /f "delims=; tokens=1,2" %A in (rename.csv) do #jren "^%A_" "%B_" /fm *.gif
I would probably do it like that (see the explanatory remarks rem):
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem // Define constants here:
set "_CSV=%~dpn0.csv" & rem // (path to the CSV file holding the old and new numbers)
set "_LOC=%~dp0" & rem // (path to the directory containing the files to rename)
set "_PAT=*.gif" & rem // (pattern of the file name part after the first `_`)
rem // Walk through the CSV file line by line and read old and new numbers:
for /F "usebackq tokens=1-2 delims=,;" %%K in ("%_CSV%") do (
rem // Search files whose names begin with the old number and an `_`:
for /F "delims=" %%F in ('dir /B /A:-D "%_LOC%\%%K_%_PAT%"') do (
rem // Store current file name into variable:
set "NAME=%%F"
rem // Toggle delayed expansion to not lose any `!` in the file name:
setlocal EnableDelayedExpansion
rem // Rename current file by replacing the old number by the new one:
ECHO ren "%_LOC%\!NAME!" "%%L_!NAME:*_=!"
endlocal
)
)
endlocal
exit /B
Remove the upper-case ECHO command after having successfully tested the script!
The advantage of this method is that it works even in case the file name part after the first _ begins with an _ as well; so a file name 1989__some_name.gif would become renamed to 10773387__some_name.gif, so the two consecutive __ are maintained.
I need some help and guidance.
I have the script below that looks for csv files in a folder. Grabs the file name and header of the file to an output file.
The scripts works, but the header is getting chopped at the end of the line in the output file. how do I pipe the entire header to the output file?
#ECHO OFF &SETLOCAL
for %%a in (*.csv) do (
set "line="
set /p "line="<"%%~a"
SETLOCAL ENABLEDELAYEDEXPANSION
echo %%a %line%":">>output.txt
echo(!line!>> output.txt
endlocal
)
It depends on how long your csv headers are.
In your code, you are reading the header using redirected input and set /p. This method allows a maximum input of 1022 characters.
You can use a for /f to read the files, but this is also limited to the max command line length, in this case 8192 characters.
#echo off
setlocal enableextensions disabledelayedexpansion
(for /f "tokens=1,2,* delims=:" %%a in ('
findstr /n "^" *.csv ^| findstr /b "[^:]*:1:"
') do (
echo(%%a:
echo( %%c
)) > output.txt
This code simply read input files with a findstr command, numerate output lines and from this output retrieve only the first line for each file. The for /f splits the lines using colons to retrieve the first three tokens in the lines: the file name, the line number and the line contents.
edited to adapt to comments
As pointed, there is a limit and the real data has collided with it.
I don't see a bulletproof pure batch solution, but a hybrid file can handle the problem
#if (#This==#IsBatch) #then
#echo off
setlocal enableextensions disabledelayedexpansion
call :getBatchFileReference _f0
(
for %%a in (*.csv) do (
echo %%a
<"%%a" cscript //nologo //e:Jscript "%_f0%"
)
) > output.txt
goto :eof
:getBatchFileReference returnVar
set "%~1=%~f0"
goto :eof
#end
WScript.StdOut.WriteLine(WScript.StdIn.ReadLine());
Saved as .cmd or .bat, the file contains an initial zone with batch code and a final zone with javascript code.
For each csv file, the batch part will redirect it as input to the javascript part to read the first line and output it.