I'm trying to merge csv files into one text file using a batch file.
My batch file is located in C:\Users\aallen and the CSV files are located in C:\Users\aallen\Test
The batch file will only work when its located in the same location as the csv.
I have tried the following commands with no joy:
1) cd "C:\Users\aallen\Test" copy *csv test.csv
2) copy "C:\Users\aallen\Test" *csv test.csv
What I'm I missing?
Collecting the information from Question and Comments, you want to combine several CSV files into one, but only keep the headerline once.
more +1 is able to show a file, skipping first lines (see more /?), but more +1 *.csv does only skip the first line of the first file and keeps it at all other files (just the opposite of what you need). So you have to process one file after the other with a for loop and check for first file yourself (can be done with a flag-variable (flag here). Redirect the whole loop to your resultfile.
#echo off
set "first=yes"
(for %%a in ("C:\Users\aallen\Test\*.csv") do (
if defined first (
type "%%a"
set "first="
) else (
more +1 "%%a"
)
))>"d:\new location\test.csv"
Note: more at command line prints just one screen and then pauses until you press a key. But when you use it in a batchfile, it doesn't (well, to be honest, it does, but after ~65000 lines. I hope, your files are shorter)
Related
I would like to read all the csv file in the folder and compile it using an awk file. Below is the code that i had wrote:
#echo off
del c_1.csv
setlocal ENABLEDELAYEDEXPANSION
set file2=*.csv
set outputfile=c_1.csv
REM get header:
set /p header=<%outputfile%
for %%i in (*.csv) do (
if not exist %header% (
nawk -f "c_1.awk" *.csv >> c_1.csv
)
if exist %header% (
nawk -f more +1 "c_1.awk" *.csv >> c_1.csv
)
)
echo done!
setlocal
pause
goto:eof
But the header still printed in my output file and it had also printed extra data that is incorrect also. Ur help will be appreciated.Thanks
Will this not do what you want?
nawk "FNR==1 && NR!=1{next;}{print}" *.csv>c_1.csv
Idea taken from here.
EditAs it seems I understood your request wrongly, (I didn't properly read the question and assumed you were concatenating files, but only retaining the header on the first). You appear to be running an awk script, c_1.awk on all csv's in the current directory, if the header of any csv doesn't match the input from outputfile then you're intending to 'compile' the entire file, if it does then you're wanting to bypass that header.
The main problem with your batch-file lies with the fact that if exist doesn't tell you if the content of %header% is empty, for that you'll need If Defined header. That said, as you have already deleted the input file, your set /p command would output an error The system cannot find the file specified. and header will still not be defined.
I think that what you should really do is adjust your awk script such that it takes the header to match as an input parameter. That would be much better than trying to check the content in a different language then run one of two awk commands depending upon that content.
I have to process multiple CSV and TXT files in one awk script. My cmd file on windows looks like: gawk -f script.awk *.csv *.txt > output.file
I'd like to use this cmd file as I don't want to always type into the command prompt whenever I want to run the script. I would like to perform different tasks with the different file types. I have tried some stuff inside the script file like if (match(FILENAME, ".csv")) && (FNR > 1) but none of them were working. I have about 4-5 CSV files and a lot of (like 1000+) TXT files, these are all input files. The content of the CSV files are all in the same schema, one column between quotes. Example:
"Player"
"adigabor"
I want to ignore the first line of all the input CSV files when processing them and add each record w/o the quotes into an array and after that I'd like to process the TXT files which I can do just fine, my problem is that I couldn't perform the different tasks with the different input file extensions in one script.
It would be extremely useful if you told us in what way "none of them were working" so we're not just guessing but here goes anyway:
The main problem with match(FILENAME, ".csv") is it'll match csv preceded by any char anywhere in the file name. To get files that end in literally .csv you want:
match(FILENAME,/\.csv$/)
but you don't need to call a function for that:
FILENAME ~ /\.csv$/
So your script would look like:
FILENAME ~ /\.csv$/ {
if ( FNR > 1 ) {
do CSV stuff
}
next
}
{
do TXT stuff
}
If you still can't do whatever you're trying to do then edit your question to include sample input files (at least one of each small .csv and .txt files) and expected output along with a better explanation of what you are trying to do.
I want to create a .bat file that will present the last row of every .csv files that the file name start with "Togo".
The batch file will be located in the same folder as the .csv files.
To output should be the:
[File Name]
[Last Row Data]
This batch file should always run and test the .csv files every 5 minutes.
SO is not a free code-writing service. Your question is likely to be deleted or closed since you have not shown any attempt to solve your problem.
That having been said, it's difficult to start in batch, so here's a solution.
#ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION
SET "sourcedir=U:\sourcedir\t w o"
FOR /f "delims=" %%a IN (
'dir /b /a-d "%sourcedir%\togo*.csv" '
) DO (
FOR /f "usebackqdelims=" %%q IN ("%sourcedir%\%%a") DO SET "line=%%q"
ECHO %%a !line!
)
GOTO :EOF
The first two lines turn off batch's debugging display (show-the-command, then execute it) and invoke a mode where access to variables that have changed within a "code block" (parenthesised series of commands) is enabled (normally, it's disabled and only the value at the time the if (or whatever) is encountered is available.)
Line 3 sets a variable called sourcedir and assigns it a value. The enclosing quotes ensure that trailing spaces are not included in the value assigned. I've deliberately used a directoryname that includes spaces because that's a common problem and it proves the batch in my test regime. Your directoryname would be different - simply substitute that. The directoryname . means "the current directory"
Lines 4-6 could be combined to one - just stylistics. It means "perform a directory scan, no directory names (/a-d) in basic form (/b) that is, names-only, of the directory whose name is in the variable sourcedir and whose names fit the pattern togo*.csv. Process each resultant line by ignoring the default delimiters and assigning the result (ie the entire line of the directory list, ie the filenames) to the metavariable (loop-control variable) %%a.
The next line reads each line of the file built from the source-directory name and the filename currently in %%a. delims is set to nothing so the entire line will be assigned to the metavariable %%q and the usebackq option tells cmd that the parenthesised string is a quoted-filename, not a string (or a single-quoted command-to-be-executed as in the first for.) the variable line will then be set with each successive line from the file, so ;ine will have the last line from the file when the for...%%q... ends.
The following line show the filename in %%a and the text from the last line of that file in line.
Note the difference - %%x to access the contents of a metavariable, %var% to access the contents of a variable, but !var! to access the changed value (if delayedexpansion has been invoked).
The goto :eof means "go to the physical end-of-file" CMD understands :eof to mean "physical end-of-file"
So - cut-and-paste to a file named whatever.bat and then run by simply entering
*whatever*
In general,
for /?
will yield help for the for command, and this holds for most batch commands. Look on SO for thousands of examples.
You may also examine
timeout
cls
choice
for clues about how to achieve your every 5 minutes ambition. You might want to run this from the task scheduler to get an every 5 minutes display - many ways to achieve the same thing.
I have several thousand .csv files in a folder and am trying to use the cdm to append them. Each file is the same table with top header and bottom notes. For example,
121030_2003.csv
121030_2004.csv
...
121031_2003.csv
121031_2004.csv
...
I tried copy *.csv all.csv from cmd and I would like to add code for the resulting file to have:
the header reported only once at the beginning, and possibly no notes
an additional column, reporting the name of the source file to keep track of it.
I think you can use cat in Linux.
e.g. for appending 1.txt 2.txt 3.txt to an output file 0.txt
cat 1.txt 2.txt 3.txt > 0.txt
csv files are just like txt files so it will be the same in your case, except that 'type' is used in cmd for 'cat'.
I may at various times have .csv files I need to combine. They have the same headers and column layout. I just need a simple way to combine them in Windows 7. The user may not always have excel installed.
A .cmd macro would be great, but the ones I found online don't work.
The best i've got so far is this:
"open a command window ("cmd.exe") and type the following two lines (no brackets)
cd "Desktop\[csv-files]"
type *.csv > my-new-file.csv"
Where the files to be combined are in Desktop\[csv-files].
BUT - it seems to create duplicates (or in some case triplicates) of the combined entries. For instance I have 2 files I tested with 23 and 26 unique entries respectivly. I got out a file with 100 entries and at least one entry repeated 3 times.
Right now the .csv files I am testing are only ~25 entries long, but in time they could be thousands or more.
Sounds like you have an issue with using *.csv and redirecting the output to a .csv file in the same folder. DOS seems to be finding the my-new-file.csv file because of the *.csv and is typing it into itself... You could use a different output filename extension until after the type command finishes, then you could rename the output file... Something like:
cd "Desktop\[csv-files]"
type *.csv > my-new-file.txt
ren my-new-file.txt my-new-file.csv
You can also skip the header of each file after the first, so that you don't end up with file headers throughout the middle of the output file. Try the following:
#echo off
setlocal ENABLEDELAYEDEXPANSION
set cnt=1
cd "Desktop\[csv-files]"
for %%i in (*.csv) do (
if !cnt!==1 (
for /f "delims=" %%j in ('type "%%i"') do echo %%j >> my-new-file.txt
) else (
for /f "skip=1 delims=" %%j in ('type "%%i"') do echo %%j >> my-new-file.txt
)
set /a cnt+=1
)
endlocal
ren my-new-file.txt my-new-file.csv
Explanation:
I used ENABLEDELAYEDEXPANSION to make sure the cnt variable is properly evaluated. When delayed expansion is enabled, you use ! to distinguish variables instead of %. So to evaluate the cnt variable, you use !cnt! instead of %cnt%. Delaying expansion makes it wait to evaluate the value of cnt until the moment that it is used. Sometimes, but not always, if you use %cnt%, it will equal a value from a previous iteration. If you enable delayed expansion and use !cnt!, it will always evaluate the correct current value.
By setting cnt to 1, we can run different code for the 1st .csv file that is processed. The code includes all lines from the 1st .csv file, but skips the first line of all subsequent .csv files.
I used a nested for loop. The outer for cycles through all .csv files in the current folder. The inner for loop executes the type "%%i" command, where %%i is the name of the .csv file. Each line of the file is processed individually as %%j, which is passed to the echo %%j command. echo would normally print the value for %%j to the command prompt window. However, you can redirect the output to a file using > or >>. The > redirector overwrites the output file with the new value. The >> redirector appends the new value to the output file. Since each line of each file, and each file is being processed individually, we must use the >> redirector to push all content into a single file.
When using the for /f command, the output is broken into individual parts using the specified delimiter. The default delimiter is a space. If I didn't include "delims=", then the text This is fun would be broken into the following:
%%j = This
%%k = is
%%l = fun
We want to process the whole line from the .csv file all-at-once. By setting the delimiter to nothing ("delims="), the whole line can be processed using %%j.
For more specific help about how the for command works, type for /? at a command prompt.
endlocal reverts the environment to its state at the point where setlocal was used. Any variables you declared are removed, and extensions are set back to their prior value.