I try to make a batch script which update header line of each .csv file in folder (and subfolders).
Since I don't want to really modify source files, I create new files with new headers, same content and new extension ".csv.modified"
The script works fine when I have only one .csv (I just remove the /s) but ignore content of other files when > 1.
Note: I have many subfolders and some of them contains whitespaces.
Any idea ?
#echo off
cls
setlocal enabledelayedexpansion
set HEADERS=header1,header2
for /f "delims=" %%i in ('dir /b /s *.csv') do (
set filename=%%~i
echo !filename!
echo.
set cpt=1
set new_filename=!filename!.modified
#copy nul "!new_filename!"
echo creating !new_filename!
echo %HEADERS%>"!new_filename!"
for /f %%a in (%%~i) do (
set line=%%a
if !cpt! gtr 2 (
echo Y
echo !line!>>"!new_filename!"
) else (
echo N
)
echo !cpt! %%a
set /a cpt=!cpt!+1
)
)
endlocal
This should be all you need to achieve that:
#SET "HEADERS=header1,header2"
#FOR /F "DELIMS=" %%A IN ('DIR/B/S/A-D-S-L *.csv') DO #((ECHO %HEADERS%
MORE +1 "%%~A")>"%%~A.modified")
Related
I wrote a batch to merge multiple CSVs from current directory into one taking into account that my CSVs have a header. But the file I get contains a space character at the end of each line.
#echo off
pushd %~dp0
setlocal ENABLEDELAYEDEXPANSION
set cnt=1
for %%i in (*.csv) do (
if !cnt!==1 (
for /f "delims=" %%j in ('type "%%i"') do echo %%j >> merged.csv
) else if %%i NEQ merged.csv (
for /f "skip=1 delims=" %%j in ('type "%%i"') do echo %%j >> merged.csv
)
set /a cnt+=1
echo !cnt!
)
This batch will leave the header in only the 1st csv
If you use a var as a flag and just check if it exists you don't need delayedexpansion.
You also need to exclude merged.csv from being appended (to itself).
#echo off
pushd %~dp0
set "flag="
for /f "delims=" %%i in ('dir /B *.csv ^|find /v "merged.csv" ') do (
if not defined flag (
copy "%%i" merged.csv
set flag=anything
) else more +1 "%%i" >>merged.csv
)
popd
As Stephan commented, you explicitly tell it to add a space, but as another option maybe this will be a shorter version?
#echo off
pushd %~dp0
for %%i in (*.csv) do if "%%i" NEQ "merged.csv" type "%%i" | more +1 >> merged.csv
I'm new to batch files and this is a tricky question. In stores.csv there is a column called 'Image' which stores vertical-line-delimited image URLs as values. There are also additional columns called 'AltImage2', 'AltImage3', etc. How can I split the vertical-line-delimited string into columns that start with 'AltImage' for each row in the CSV? 'AltImage' columns only go to AltImage5, and there may not be five image URLs in a given row. I would also like to keep the first image URL in the 'Image' column if possible.
Example of headers and single row of data:
Company,Title,Image,AltImage2,AltImage3,AltImage4,AltImage5
Testco,U2X40,image1.png|image2.png|image3.png
Desired result after running batch:
Company,Title,Image,AltImage2,AltImage3,AltImage4,AltImage5
Testco,U2X40,image1.png,image2.png,image3.png
So far I've tried this:
for /f "tokens=3 delims=, " %%a in ("stores.csv") do (
echo run command here "%%a"
)
But cannot even echo the values in the Image column.
Here is a solution using Bash script (unfortunately I need batch): How do I split a string on a delimiter in Bash?
#echo off
setlocal
< stores.csv (
rem Read and write the header
set /P "header="
call echo %%header%%
rem Process the rest of lines
for /F "tokens=1-3 delims=|" %%a in ('findstr "^"') do echo %%a,%%b,%%c
)
I think this handles your parsing problem. Pay attention to quotes and the usebackq option.
for /f "skip=1 tokens=3 delims=," %%a in (stores.csv) do for /f "tokens=1-5 delims=|" %%b in ("%%a") do echo %%b %%c %%d %%e %%f
Here's a fuller solution to play with. There may be a more elegant way to handle optional commas. And you'll have to handle directing the output to whichever place is appropriate.
#echo off
setlocal enabledelayedexpansion
echo Company,Title,Image,AltImage2,AltImage3,AltImage4,AltImage5
for /f "skip=1 tokens=3 delims=," %%a in (stores.csv) do (
for /f "tokens=1-5 delims=|" %%b in ("%%a") do (
set line=%%b
if not "%%c"=="" set line=!line!,
set line=!line!%%c
if not "%%d"=="" set line=!line!,
set line=!line!%%d
if not "%%e"=="" set line=!line!,
set line=!line!%%e
if not "%%f"=="" set line=!line!,
set line=!line!%%f
echo !line!
)
)
read the file line by line and replace | with , (you have to escape the | and use delayed expansion:
#echo off
setlocal enabledelayedexpansion
(
for /f "delims=" %%a in (old.csv) do (
set line=%%a
echo !line:^|=,!
)
)>new.csv
I'm currently working on a way to combine 3 csv files and have the following script to do so:
Script
#echo off
ECHO Set working directory
pushd %~dp0
setlocal ENABLEDELAYEDEXPANSION
set cnt=1
for %%i in (*.csv) do (
if !cnt!==1 (
for /f "delims=" %%j in ('type "%%i"') do echo %%j >> combined.csv
) else if %%i NEQ combined.csv (
for /f "skip=1 delims=" %%j in ('type "%%i"') do echo %%j >> combined.csv
)
REM increment count by 1
set /a cnt+=1
)'
This works like a charm and also strips the header of the other 2 csv files in the working dir.
The script now outputs combined.csv, which is nice but I would like the script to output NL2 ddmmyyyy.csv.
The issue I have and can't seem to figure out is how to make the name of the output file incremental and date-based.
Code can parse the date elements from the DATE variable. The format of the date will vary based on locale. Mine is set to YYYY-MM-DD, but yours may be different.
M:>echo %DATE%
2016-01-27
8:23:58.28 \\SWPDCENDWXTK01\D$ M:\DW_Devl\Paul\phs_dmx_config\bin
M:>SET FNDATE=%DATE:~8,2%%DATE:~5,2%%DATE:~0,4%
8:24:08.54 \\SWPDCENDWXTK01\D$ M:\DW_Devl\Paul\phs_dmx_config\bin
M:>ECHO %FNDATE%
27012016
Inside the loop, set a variable and use it for the output.
SET COMBO_FILENAME=NL!cnt!%FNDATE%.csv
ECHO %%j >>!COMBO_FILENAME!
I want to merge two similar CSV files using batch. I have found a file that was working perfectly and now is not. This may have been due to renaming the CSV files that were used.
I want what is demonstrated below:
File 1:
name1,group1,data1
name2,group2,data2
name3,group3,data3
File 2:
name1,group1,data1,time1
name2,group2,data2,time2
Merged file:
name1,group1,data1,time1
name2,group2,data2,time2
name3,group3,data3
(Note that the fourth column was not filled in by name3 and was subsequently not on file 2.)
The following code was modified from: http://www.experts-exchange.com/OS/Microsoft_Operating_Systems/MS_DOS/Q_27694997.html.
#echo off
set "tmpfile=%temp%\importlist.tmp"
set "csvfile=importlist.csv"
copy nul "%tmpfile%" >nul
echo.
echo Processing all CSV files...
set "header="
for %%a in (%1) do (
if not "%%a"=="%csvfile%" (
set /p =Processing %%a...<nul
for /f "tokens=1* usebackq delims=," %%b in ("%%a") do (
if /i "%%b"=="Keyword" (
if not defined header (
set /p =Found header...<nul
set "header=%%b,%%c"
)
) else (
title [%%a] - %%b,%%c
findstr /b /c:"%%b" /i "%tmpfile%">nul || echo %%b,%%c>>"%tmpfile%"
)
)
echo OK
)
)
echo Finished processing all CSV files
echo.
echo Creating %csvfile%
echo %header%>"%csvfile%"
set /p =Sorting data...<nul
sort "%tmpfile%">>"%csvfile%"
echo OK
del "%tmpfile%"
echo Finished!
title Command Prompt
exit /b
The problem is that when executed it just creates a sorted CSV with all the data from the first file and not the second.
I have attempted to get it working by putting quotation marks around the parameter (%1 - "directory*.csv") to no avail.
you might try this
#echo off &setlocal disabledelayedexpansion
for /f "delims=" %%a in (file1.csv) do set "#%%~a=7"
for /f "tokens=1-4delims=," %%a in (file2.csv) do (
set "lx1=#%%~a,%%~b,%%~c"
setlocal enabledelayedexpansion
if defined !lx1! (
endlocal
set "#%%~a,%%~b,%%~c="
) else (
endlocal
)
set "#%%~a,%%~b,%%~c,%%~d=4"
)
(for /f "delims==#" %%a in ('set #') do echo %%~a)>merge.csv
type merge.csv
It doesn't work, if you have = or # in your data.
I have a number of CSV files in directory "C:\Paypal_EMT\data" that I need to process and extract certain data columns based on the first column.
But I'm getting the following error "The system cannot find the file C:\Paypal_EMT\data\STL-20130614.01.005.CSV"
C:\Paypal_EMT\data\STL-20130614.01.005.CSV
C:\Paypal_EMT\data\STL-20130615.01.005.CSV
C:\Paypal_EMT\data\STL-20130616.01.005.CSV
....
And data file looks as follows:
"RH",2013/06/15 02:14:58 -0400,"X","LQ3SUEEWPWKL6",005,
"FH",01
"SH",2013/06/14 00:00:00 -0400,2013/06/14 23:59:59 -0400,"LQ3SUEEWPWKL6",""
"CH","TransactionID","InvoiceID","PayPalReferenceID","PayPalReferenceIDType","TransactionEventCode","TransactionInitiationDate","TransactionCompletionDate","TransactionDebitOrCredit","GrossTransactionAmount","GrossTransactionCurrency","FeeDebitOrCredit","FeeAmount","FeeCurrency","CustomField","ConsumerID","PaymentTrackingID"
....
Here's my code so far:
#echo off
setlocal EnableDelayedExpansion
:: Set path directory where csv files reside in variable
Set _InputPath=C:\Paypal_EMT\data
FOR %%I IN (!_InputPath!\*.CSV) DO (
Set "_OutFile=%%I.sum.rpt"
FOR /F "tokens=1-18* delims=," %%A IN (!I!) DO (
if "%%~A"=="RH" echo %%~B
if "%%~A"=="FH" echo %%~B
if "%%~A"=="SH" echo %%~B
if "%%~A"=="CH" echo %%~B,%%~Q,%%~R
if "%%~A"=="RF" echo %%~B,%%~Q,%%~R
)
) > %_OutFile%
try this:
#echo off
setlocal EnableDelayedExpansion
:: Set path directory where csv files reside in variable
Set "_InputPath=C:\Paypal_EMT\data"
FOR %%x IN ("%_InputPath%\*.CSV") DO (
Set "_OutFile=%%~x.sum.rpt"
(FOR /F "usebackq tokens=1-18* delims=," %%A IN ("%%~x") DO (
if "%%~A"=="RH" echo %%~B
if "%%~A"=="FH" echo %%~B
if "%%~A"=="SH" echo %%~B
if "%%~A"=="CH" echo %%~B,%%~Q,%%~R
if "%%~A"=="RF" echo %%~B,%%~Q,%%~R
))>"!_OutFile!"
)