I'm trying to merge csv files into one text file using a batch file.
My batch file is located in C:\Users\aallen and the CSV files are located in C:\Users\aallen\Test
The batch file will only work when its located in the same location as the csv.
I have tried the following commands with no joy:
1) cd "C:\Users\aallen\Test" copy *csv test.csv
2) copy "C:\Users\aallen\Test" *csv test.csv
What I'm I missing?
Collecting the information from Question and Comments, you want to combine several CSV files into one, but only keep the headerline once.
more +1 is able to show a file, skipping first lines (see more /?), but more +1 *.csv does only skip the first line of the first file and keeps it at all other files (just the opposite of what you need). So you have to process one file after the other with a for loop and check for first file yourself (can be done with a flag-variable (flag here). Redirect the whole loop to your resultfile.
#echo off
set "first=yes"
(for %%a in ("C:\Users\aallen\Test\*.csv") do (
if defined first (
type "%%a"
set "first="
) else (
more +1 "%%a"
)
))>"d:\new location\test.csv"
Note: more at command line prints just one screen and then pauses until you press a key. But when you use it in a batchfile, it doesn't (well, to be honest, it does, but after ~65000 lines. I hope, your files are shorter)
I want to create a .bat file that will present the last row of every .csv files that the file name start with "Togo".
The batch file will be located in the same folder as the .csv files.
To output should be the:
[File Name]
[Last Row Data]
This batch file should always run and test the .csv files every 5 minutes.
SO is not a free code-writing service. Your question is likely to be deleted or closed since you have not shown any attempt to solve your problem.
That having been said, it's difficult to start in batch, so here's a solution.
#ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION
SET "sourcedir=U:\sourcedir\t w o"
FOR /f "delims=" %%a IN (
'dir /b /a-d "%sourcedir%\togo*.csv" '
) DO (
FOR /f "usebackqdelims=" %%q IN ("%sourcedir%\%%a") DO SET "line=%%q"
ECHO %%a !line!
)
GOTO :EOF
The first two lines turn off batch's debugging display (show-the-command, then execute it) and invoke a mode where access to variables that have changed within a "code block" (parenthesised series of commands) is enabled (normally, it's disabled and only the value at the time the if (or whatever) is encountered is available.)
Line 3 sets a variable called sourcedir and assigns it a value. The enclosing quotes ensure that trailing spaces are not included in the value assigned. I've deliberately used a directoryname that includes spaces because that's a common problem and it proves the batch in my test regime. Your directoryname would be different - simply substitute that. The directoryname . means "the current directory"
Lines 4-6 could be combined to one - just stylistics. It means "perform a directory scan, no directory names (/a-d) in basic form (/b) that is, names-only, of the directory whose name is in the variable sourcedir and whose names fit the pattern togo*.csv. Process each resultant line by ignoring the default delimiters and assigning the result (ie the entire line of the directory list, ie the filenames) to the metavariable (loop-control variable) %%a.
The next line reads each line of the file built from the source-directory name and the filename currently in %%a. delims is set to nothing so the entire line will be assigned to the metavariable %%q and the usebackq option tells cmd that the parenthesised string is a quoted-filename, not a string (or a single-quoted command-to-be-executed as in the first for.) the variable line will then be set with each successive line from the file, so ;ine will have the last line from the file when the for...%%q... ends.
The following line show the filename in %%a and the text from the last line of that file in line.
Note the difference - %%x to access the contents of a metavariable, %var% to access the contents of a variable, but !var! to access the changed value (if delayedexpansion has been invoked).
The goto :eof means "go to the physical end-of-file" CMD understands :eof to mean "physical end-of-file"
So - cut-and-paste to a file named whatever.bat and then run by simply entering
*whatever*
In general,
for /?
will yield help for the for command, and this holds for most batch commands. Look on SO for thousands of examples.
You may also examine
timeout
cls
choice
for clues about how to achieve your every 5 minutes ambition. You might want to run this from the task scheduler to get an every 5 minutes display - many ways to achieve the same thing.
I'm attempting to automate something at work that requires minimal background process. Normally I'd use C#, but this is problematic, as part of what I'm automating is a test process that scans the system looking for running processes. Command windows are fine, but most anything beyond that get's to be problematic.
As part of kicking off this series of tests, I am trying to find a way to get a batch file to read one line at a time from a .csv file. Each line contains 4 values (separated by commas, of course) which should be put into variables. The first var will be the test number. When starting the batch file, I'll be providing a test number as an arg, and want it to compare the test number received to the test number it read from the file. If it matches, continue testing. If not, move on to the next line and read that data into the variables.
Everything else is really simple, but I can't figure out how to get it to read and parse a .csv file one line at a time in the manner described. Any help would be appreciated.
Just for information, I am well aware of how to do this in C#, and I know there are many other solutions that would be better (easier) than batch files. However, there are specific reasons why employing those solutions is a poor idea in our situation, so we're trying to go old school with the batch files.
#echo off
rem Read theFile.csv and get 4 tokens separated by commas
for /F "tokens=1-4 delims=," %%a in (theFile.csv) do (
rem Tokens read are placed in %%a, %%b, %%c and %%d replaceable parameters
rem If first token is equal to first Batch file argument
if %%a equ %1 (
rem Continue testing here, ie
echo %%a,%%b,%%c,%%d
)
)
The Octave documentation on this subject is both intimidating and sparse.
I did not know where else to document the solution I found, so I am posting here. I apologize if that's inappropriate, but I want to help the next guy.
The following solution is for a simple windows distributable.
Use Case:
A solution is developed in Octave 3.2.4, and needs to be distributed to end-users with few computer skills. Installing and explaining Octave is impossible, the solution must be "one-click" or "brain-dead-simple."
Known Issues:
imread fails in 3.2.4 because file_in_path.m is wrong. You will need to update the file file_in_path.m to the following (just replace it):
function name=file_in_path(p,file)
idx=[1 findstr(p,pathsep) length(p)+1];
for i=1:length(idx)-1
if idx(i+1)-idx(i)<=1
dir=strcat(pwd,"/");
else
dir=p(idx(i)+1:idx(i+1)-1);
end
name = fullfile(dir, file);
fid = fopen(name,"r");
if fid >= 0
fclose(fid);
return
end
end
fid = fopen(file,"r");
if fid >= 0,
fclose(fid);
name=file;
return
end
name=[];
Solution: Create a distributable exe using mkoctfile, and package this exe with the core Octave files, and other .oct and .m files as necessary.
Step 1: Create a stand-alone executable.
You can see code that works here:
http://www.gnu.org/software/octave/doc/interpreter/Standalone-Programs.html
Particularly the file "embedded.cc".
I have simplified that file as follows:
#include <iostream>
#include <octave/oct.h>
#include <octave/octave.h>
#include <octave/parse.h>
int
main (int argc, char *argvc[])
{
string_vector argv (2);
argv(0) = "embedded";
argv(1) = "-q";
octave_main (2, argv.c_str_vec(), 1);
octave_value_list in = octave_value (argvc[1]);
octave_value_list out = feval ("your_custom_m_file", in);
if (!error_state && out.length () > 0)
{
}
else
{
std::cout << "invalid\n";
}
return 0;
}
Build this file with the command
mkoctfile --link-stand-alone embedded.cc -o embedded
It may throw warnings, but as long as it throws no errors, you should be fine. The file embedded.exe will be built, and can be run. The only issue is that it will lack all the goodies that make octave awesome. You will have to provide those.
Step 2: Create a distribution folder
You will need to create a copy of many of the Octave files. I suggest a directory specifically for this. At a minimum, you will need a copy of all or most of the DLLs in \bin. Additionally, place your distributable executable in this directory.
Step 3: Other files whack-a-mole
You will now need to find out what other files will be necessary to run your .m script. You can simplify this step by copying \oct\i686-pc-mingw32*.oct and \share\octave\3.2.4\m\*\*.m to the distribution directory, although this will be overkill, and will not actually prevent the whack-a-mole step.
Now, you must play whack-a-mole or the time-honored tradition of "where my includes be at, yo?"
Open a cmd prompt and navigate to your distribution folder.
Get rid of any useful PATH strings. Your customers won't have them.
Attempt to run the program embedded.exe. You will get an error such as the following:
embedded.exe
error: `max' undefined near line 83 column 22
error: evaluating argument list element number 1
error: evaluating argument list element number 1
error: called from:
error: T:\sms\Development\research\c2\disttest\strcat.m at line 83, column 3
error: T:\sms\Development\research\c2\disttest\file_in_path.m at line 5, column 10
error: T:\sms\Development\research\c2\disttest\imread.m at line 50, column 6
A Search in your Octave installation for "max". It will either be a .oct or a .m file. In this case, it is a .oct file, max.oct. Copy it to your distribution directory.
B You search for something obvious like "min", and get no results. This is because the Loadable Function "min" is in the .oct file "max.oct". Make a copy of max.oct, and rename it to min.oct. It will work now. How do you know where the functions are? I'm not sure. Most of them are in obvious places like "max.oct" for min, and "fft2.oct" for "ifft2.oct". Good luck with all that.
Repeat until your executable runs.
Just to add that if you want to run a script instead of an m function, then the line of the embedded.cc:
octave_value_list out = feval ("your_custom_m_file", in);
should be:
octave_value_list out = feval ("your_custom_m_script");
Also use 'which' to find where the missing functions are packed. For example for the min function:
octave:22> which min
min is a function from the file C:\Octave\Octave3.6.2_gcc4.6.2\lib\octave\3.6.2\oct\i686-pc-mingw32\max.oct
Something I found when linking my custom m file into an Octave standalone:
Needed #include <octave/toplev.h>
Replace return 0; (as above) with clean_up_and_exit(0);
Without these steps my program repeatedly crashed on exit.
Run mkoctfile --link-stand-alone embedded.cc -o embedded
from the octave solution and not from a batch file.
Just saved you half day (-;
In the above solution in bullet 4 B:
B You search for something obvious like "min", and get no results.
This is because the Loadable Function "min" is in the .oct file
"max.oct". Make a copy of max.oct, and rename it to min.oct. It will
work now.
This might not work if some function is being called from #folder function.m and also to avoid unnecessary duplicated files, just add the following code somewhere in your m file outside #folder
autoload ("min", "max.oct");
Likewise, it can be removed via
autoload ("min", "max.oct", "remove");
Ensure that the path to max.oct is provided here.
The above understanding is based on a file PKG_ADD and PKG_DEL in the communications package located at \Octave-4.0.1\lib\octave\packages\communications-1.2.1\i686-w64-mingw32-api-v50+\
Check out Stratego Octave Compiler.
(I've not tested it yet, but plan to do so in the next few days.)
I had that very same requirement (one-click, brain-dead-simple), so I made a setup that contained only curl.exe, the batch file below, an exe which was a .bat in disguise (simply calling the batch file below) and the .vbs script below (not writen by me). And of course my m-file.
This will download Octave 4.2.1 as a portable program (32 bit, otherwise we'dd have to download again if the system turns out to be 32 bit), unpack using the vbs script, move the contents to the same folder as the batch file and run it in GUI mode. Every next time the same script is called, it will only check if octave.bat is still there.
Of course this results in a huge waste of disk space, downloading the 280MB zip, which unpacks to over 1GB (which I make even worse by not deleting the zip afterwards), and you're stuck with a cmd window that is not easy to hide.
But it does offer the simplest solution I could find. It is also less likely to break in the future (either with an update of your own, or an update from Octave). Some glorious day, mkoktfile will actually be easy to use and will solve dependencies on its own, but until that day this remains the least headache-inducing solution I could find. And aspirins are more expensive than someone else's disk space.
::this file will test if the octave portable is downloaded and unpacked
#ECHO OFF
SET my_m_file=your_mfile.m
SET name_of_this_script=run_me.bat
::if the file exists, skip to the actual running.
IF EXIST "octave.bat" goto OctaveIsExtracted
IF EXIST "octave-4.2.1-w32.zip" goto OctaveIsDownloaded
ECHO The runtime (Octave portable 4.2.1) will now be downloaded.
ECHO This may take a long time, as it is about 280MB.
ECHO .
ECHO If this download restarts multiple times, you can manually download the octave-4.2.1-w32.zip from the GNU website. Make sure to unpack the contents.
::if this errors, you can uncomment the line with archive.org (which doesn't report total size during download)
curl http://ftp.gnu.org/gnu/octave/windows/octave-4.2.1-w32.zip > octave-4.2.1-w32.zip
::curl http://web.archive.org/web/20170827205614/https://ftp.gnu.org/gnu/octave/windows/octave-4.2.1-w32.zip > octave-4.2.1-w32.zip
:OctaveIsDownloaded
::check to see if the file size is the correct size to assume a successful download
::if the file size is incorrect, delete the file, restart this script to attempt a new download
::file size should be 293570269 bytes
call :filesize octave-4.2.1-w32.zip
IF /I "%size%" GEQ "293560000" goto OctaveIsDownloadedSuccessfully
del octave-4.2.1-w32.zip
::start new instance and exit and release this one
start %name_of_this_script%
exit
:OctaveIsDownloadedSuccessfully
IF EXIST "octave.bat" goto OctaveIsExtracted
::unzip and move those contents to the current folder
ECHO Unzipping octave portable, this may take a moment.
cscript //B j_unzip.vbs octave-4.2.1-w32.zip
SET src_folder=octave-4.2.1
SET tar_folder=%cd%
for /f %%a IN ('dir "%src_folder%" /b') do move %src_folder%\%%a %tar_folder%
pause
:OctaveIsExtracted
octave.bat %my_m_file%
goto :eof
:filesize
set size=%~z1
exit /b 0
And j_unzip.vbs
' j_unzip.vbs
'
' UnZip a file script
'
' By Justin Godden 2010
'
' It's a mess, I know!!!
'
' Dim ArgObj, var1, var2
Set ArgObj = WScript.Arguments
If (Wscript.Arguments.Count > 0) Then
var1 = ArgObj(0)
Else
var1 = ""
End if
If var1 = "" then
strFileZIP = "example.zip"
Else
strFileZIP = var1
End if
'The location of the zip file.
REM Set WshShell = CreateObject("Wscript.Shell")
REM CurDir = WshShell.ExpandEnvironmentStrings("%%cd%%")
Dim sCurPath
sCurPath = CreateObject("Scripting.FileSystemObject").GetAbsolutePathName(".")
strZipFile = sCurPath & "\" & strFileZIP
'The folder the contents should be extracted to.
outFolder = sCurPath
'original line: outFolder = sCurPath & "\"
WScript.Echo ( "Extracting file " & strFileZIP)
Set objShell = CreateObject( "Shell.Application" )
Set objSource = objShell.NameSpace(strZipFile).Items()
Set objTarget = objShell.NameSpace(outFolder)
intOptions = 256
objTarget.CopyHere objSource, intOptions
WScript.Echo ( "Extracted." )
i have a process that downloads a file from a webbrower. it has the same name always (can't change that) so each file gets downloaded as file([latestnumber])
so in this directory i have:
joe.pdf
joe(1).pdf
joe(2).pdf
etc . . .
I now would like a script to take the "latest file" (joe(2).pdf in this case) and copy it to another directory.
something like GetLatestFile("joe") and copy to "X:\mydirectory"
can anyone think of an easy way to do this.
Do you have a preference as to what language you write your script in?
I wouldn't go by the name of the file, I'd choose whatever scripting language you are going to use, loop through the directory and look at the file attributes for each file to pick out the latest one, then move it to your target directory. This would be fairly trivial in a .NET console application with the classes available in the System.IO namespace. (namely the DirectoryInfo, FileInfo and File classes)
Try this: XCOPY C:\BATCH\*.* C:\UPLOAD /M
Put the code in a text file and rename it as whateveryouwant.bat and execute.
Be sure to edit the source and destination folder to your liking.
Is this what you're looking for ?
So, as it is enough to get the latest filename sorted by date, I suggest something like:
#echo off & setLocal enabledelayedexpansion
for /f "tokens=* delims= " %%a in ('dir /b/a-d/o-d') do (
set N=%%~Fa
goto :done
)
:done
echo !N!
Replace the last echo command for the "copy ..." or whatever you want to do with the newest file.
HTH!
Edit> If the files are not in the current directory, change the "dir" command accordingly
this uses sed, and regular expressions
http://gnuwin32.sourceforge.net/packages/sed.htm
it generates a bat file that does the job.
i've put the bat file in c:\crp so it doesn't become a latest file.
as a demonstration, i've created a latest file latestfile.txt
you can see the line that generates copyit.bat and you can amend it so the files goes exactly where you want.
C:>md c:\crp
C:>copy /y con latestfile.txt
fgfdgd^Z
1 file(s) copied.
C:>dir /o-d/a-d/b | find /N /V "QWERTY" | find "[1]" | sed -e s/[1](.*)/cop
y\d32\1\d32c:\newdir/>c:\crp\copyit.bat
C:>type c:\crp\copyit.bat
copy latestfile.txt c:\newdir