i'm trying to split a csv file based on the year. The year is allways shown in UTC Format after first delimiter ";".
example for csv file:
ID;Datum;EUR
1;2021-12-12 12:12:12;50
1;2020-12-12 12:12:12;10
2;2020-12-12 12:12:12;20
1;2019-12-12 12:12:12;80
So far I have the following batch code as a solution.
findstr /I /R /C:"^[^;]*;2019-" "test.csv" > "test_year1.csv"
findstr /I /R /C:"^[^;]*;2020-" "test.csv" > "test_year2.csv"
findstr /I /R /C:"^[^;]*;2021-" "test.csv" > "test_year3.csv"
It works but how can i add the first line (Header) to each of this splitfiles?
Also, the code is a bit simple and unwieldy. Is it possible to automate the whole thing a bit and automatically determine the possible year's from inputfile and create a corresponding output file?
thx for help,
SaXe
#ECHO OFF
SETLOCAL
rem The following settings for the source directory, destination directory, filenames, output filename are names
rem that I use for testing and deliberately include names which include spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.
SET "sourcedir=u:\your files"
SET "destdir=u:\your results"
SET "filename1=%sourcedir%\q71435963.txt"
:: remove variables starting #
FOR /F "delims==" %%b In ('set # 2^>Nul') DO SET "%%b="
:: set environment variables #year#yearnumber# from sourcefile
FOR /f "skip=1usebackqtokens=2delims=-;" %%b IN ("%filename1%") DO SET "#year#%%b#=Y"
:: set "line1" to first line of file
FOR /f "usebackqdelims=" %%b IN ("%filename1%") DO SET "line1=%%b"&GOTO buildfiles
:buildfiles
:: for each year found, create a new file & append data
FOR /f "tokens=2delims=#" %%b IN ('set # 2^>nul') DO (
>"%destdir%\test_%%b.csv" ECHO %line1%
>>"%destdir%\test_%%b.csv" findstr /I /R /C:"^[^;]*;%%b-" "%filename1%"
)
GOTO :EOF
The 2^>nul in each of the set commands suppresses error messages that would occur if there are no variables matching the pattern currently set. The ^ "escapes" the >, telling cmd that the > is part of the set, not of the for.
The usebackq option is only required because I chose to add quotes around the source filename.
--------- Fix for - in column 1:
in place of the for /f "skip... line, use
FOR /f "skip=1usebackqtokens=2delims=;" %%c IN ("%filename1%") DO FOR /f "delims=-" %%b IN ("%%c") DO SET "#year#%%b#=Y"
This assigns the string from the second to third occurrence of ; which grabs the second column to %%c, skipping the first line. Then the variable is set as before using the value of %%c, using - as a delimiter and selects the first token (ie. %%cup to the first delimiter) into %%b as before. The default for tokens= is tokens=1.
Related
I need to move second line of a csv/text file to another text/csv file.
I have a solution to delete the second line of csv/text file.
The first line of csv/text file contains header and must be not moved or deleted.
Warning: Path folders have spaces
My code
#echo off
Pushd "D:\Program Files\datasources\"
setlocal disableDelayedexpansion
>archive.new (
break | for /F "tokens=*" %%F in (archive.csv) do #(
echo %%F
exit /b
)
for /F "skip=2 tokens=*" %%F in (archive.csv) do echo %%F
)
DEL /S "archive.csv"
REN "archive.new" "archive.csv"
I need only to move the second line in another file, this code delete the second line of text/csv file.
I think you are over complicating your code. Every line gets preserved in one file or another, so you might as well read every line in your one loop and then choose which file to write the line to based on the line number.
I use SET /A to increment a ln counter, and if the resultant value is 2 then I get a division by 0 error. The && code writes all the success lines (all but 2) to the new archive, and the || code writes the failure (line 2) to the other file.
#echo off
pushd "d:\Program Files\datasources\"
set "src=archive.csv"
set "file2=otherFile.csv"
set ln=0
>"%src%.new" (
for /f usebackq^ delims^=^ eol^= %%A in ("%src%") do (
2>nul set /a "1/(2-(ln+=1))" && (echo(%%A) || (>>"%file2%" echo(%%A)
)
)
move /y "%src%.new" "%src%" >nul
Some additional advanced "tricks" I employed:
Use usebackq so file name can be enclosed in quotes, just in case there are spaces and or poison characters in the name.
Use move instead of del followed by ren
Arcane syntax to set both eol and delims to nothing, thus guaranteeing all lines are preserved exactly (provided they don't exceed the ~8191 character limit)
Use echo( to guarantee correct output, no matter the line (including empty line)
Put redirection in front and enclose echo statements in parentheses to ensure no unwanted trailing white space characters in output
I just need to delete the second row from a csv file using a batch file. Its always going to be the second row, though the text could be different.
I keep finding answers, but it seems like they are always more complicated then what I would like to do
#echo off
setlocal EnableDelayedExpansion
rem Read lines from input.csv
< input.csv (
rem Read and copy the first line
set /P "line="
echo(!line!
rem Just read the second line
set /P "line="
rem Copy the rest of lines
findstr "^"
) > output.csv
#echo off & setlocal
>outfile.csv (
for /f "tokens=1* delims=:" %%I in (
'findstr /n "^" infile.csv ^| findstr /v "^2:"'
) do echo(%%J
)
To follow the logic of this script, read it from the inside out. The first findstr will read infile.csv and prepend line numbers to each line. The piped second findstr command will exclude all lines beginning with 2:. The outer for /F will strip the prepended line numbers. The whole thing is dumped to outfile.csv.
I have files that look like that:
1989_Footer.gif
1989_Header.gif
260273_Footer.gif
260273_Header.gif
...
and I have a CSV that looks like this:
1989;10773387
260273;10776516
...
I want to rename the files number with the one given in the CSV. The first number in the CSV is the old number and the second one (after the semicolon) is the new number. How can I achive that without touching the text after the underline?
I thought about getting the file name, then search the number in the CSV and replace it with the value in the same line after the semicolon.
#ECHO OFF
SETLOCAL
SET "sourcedir=U:\sourcedir"
SET "filename1=%sourcedir%\q40908252.txt"
FOR /f "usebackqtokens=1*delims=;" %%a IN ("%filename1%") DO (
FOR /f "tokens=1*delims=_" %%r IN ('dir /b /a-d "%sourcedir%\%%a_*"') DO (
ECHO(REN "%sourcedir%\%%r_%%s" "%%b_%%s"
)
)
GOTO :EOF
You would need to change the setting of sourcedir to suit your circumstances.
I used a file named q40908252.txt containing your data for my testing.
The required REN commands are merely ECHOed for testing purposes. After you've verified that the commands are correct, change ECHO(REN to REN to actually rename the files.
Read the csv file, using ; as the delimiter, placing the from-prefix in %%a and the to-prefix in %%b.
Perform a directory list of files named "%%a_anything" in the source directory, tokenise on _ so that the first token (which must be %%a) goes to %%r and the remainder-of-name to %%s, then rename the file with the prefixes switched.
Assumes that the first _ in the filename will never be a multiple-_
My JREN.BAT regular expression renaming utility can simplify the solution. JREN.BAT is pure script (hybrid batch/JScript) that runs natively on any Windows machine from XP onward - no 3rd party exe file required. Full documentation is available from the command line via jren /?, or jren /?? for paged help.
From within a batch script:
#echo off
for /f "delims=; tokens=1,2" %%A in (rename.csv) do call jren "^%%A_" "%%B_" /fm *.gif
From the command line:
for /f "delims=; tokens=1,2" %A in (rename.csv) do #jren "^%A_" "%B_" /fm *.gif
I would probably do it like that (see the explanatory remarks rem):
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem // Define constants here:
set "_CSV=%~dpn0.csv" & rem // (path to the CSV file holding the old and new numbers)
set "_LOC=%~dp0" & rem // (path to the directory containing the files to rename)
set "_PAT=*.gif" & rem // (pattern of the file name part after the first `_`)
rem // Walk through the CSV file line by line and read old and new numbers:
for /F "usebackq tokens=1-2 delims=,;" %%K in ("%_CSV%") do (
rem // Search files whose names begin with the old number and an `_`:
for /F "delims=" %%F in ('dir /B /A:-D "%_LOC%\%%K_%_PAT%"') do (
rem // Store current file name into variable:
set "NAME=%%F"
rem // Toggle delayed expansion to not lose any `!` in the file name:
setlocal EnableDelayedExpansion
rem // Rename current file by replacing the old number by the new one:
ECHO ren "%_LOC%\!NAME!" "%%L_!NAME:*_=!"
endlocal
)
)
endlocal
exit /B
Remove the upper-case ECHO command after having successfully tested the script!
The advantage of this method is that it works even in case the file name part after the first _ begins with an _ as well; so a file name 1989__some_name.gif would become renamed to 10773387__some_name.gif, so the two consecutive __ are maintained.
I need some help and guidance.
I have the script below that looks for csv files in a folder. Grabs the file name and header of the file to an output file.
The scripts works, but the header is getting chopped at the end of the line in the output file. how do I pipe the entire header to the output file?
#ECHO OFF &SETLOCAL
for %%a in (*.csv) do (
set "line="
set /p "line="<"%%~a"
SETLOCAL ENABLEDELAYEDEXPANSION
echo %%a %line%":">>output.txt
echo(!line!>> output.txt
endlocal
)
It depends on how long your csv headers are.
In your code, you are reading the header using redirected input and set /p. This method allows a maximum input of 1022 characters.
You can use a for /f to read the files, but this is also limited to the max command line length, in this case 8192 characters.
#echo off
setlocal enableextensions disabledelayedexpansion
(for /f "tokens=1,2,* delims=:" %%a in ('
findstr /n "^" *.csv ^| findstr /b "[^:]*:1:"
') do (
echo(%%a:
echo( %%c
)) > output.txt
This code simply read input files with a findstr command, numerate output lines and from this output retrieve only the first line for each file. The for /f splits the lines using colons to retrieve the first three tokens in the lines: the file name, the line number and the line contents.
edited to adapt to comments
As pointed, there is a limit and the real data has collided with it.
I don't see a bulletproof pure batch solution, but a hybrid file can handle the problem
#if (#This==#IsBatch) #then
#echo off
setlocal enableextensions disabledelayedexpansion
call :getBatchFileReference _f0
(
for %%a in (*.csv) do (
echo %%a
<"%%a" cscript //nologo //e:Jscript "%_f0%"
)
) > output.txt
goto :eof
:getBatchFileReference returnVar
set "%~1=%~f0"
goto :eof
#end
WScript.StdOut.WriteLine(WScript.StdIn.ReadLine());
Saved as .cmd or .bat, the file contains an initial zone with batch code and a final zone with javascript code.
For each csv file, the batch part will redirect it as input to the javascript part to read the first line and output it.
I have read this thread, which helped, but doesn't answer my specific question. I'm hoping someone can help.
I am trying to export a CSV file with TWO COLUMNS of data. Column 1 is the actual filename (with extension), and Column 2 would be the immediate Folder Name (without any path info) of the file location. Question I have is, is that possible? Next question is, how can I export this as a CSV file with two columns of information?
This is a good starting point except this only has the filename (doesn't have the second column that shows the immediate folder name), and this doesn't seem to return to the next line for each filename. Instead this is simply separating with commas and not returning to new lines.
Can you advise if this is possible and offer some ideas?
#echo off
<nul (
for /f "eol=: delims=" %%F in ('dir /b /o:n') do set /p =""%%F","
) >fileList.csv
Thanks everyone!
If by the "Immediate folder name" you mean the name of the containing directory but without the path to that directory, then:
#ECHO OFF
SETLOCAL
PUSHD "%~1"
FOR /f "delims=" %%i IN ("%cd%") DO SET directory=%%~nxi
(
FOR /f "delims=" %%i IN ('dir /b /a-d /on') DO (
SETLOCAL enabledelayedexpansion
ECHO "%%i","!directory!"
endlocal
)
)>filelist.csv
POPD
The pathname of the directory required should be supplied as the first parameter, quoted if necessary.
Essentially, change to the directory in question, find and save the name of the leaf directory, then execute a directory scan returning the filenames. Quote both and output with a comma between. The inner setlocal is to allow at least some silly directory names.
edit 20130422-1421Z
#ECHO OFF
SETLOCAL
PUSHD "%~1"
FOR /f "delims=" %%i IN ("%cd%") DO SET directory=%%~nxi
(
FOR /f "delims=" %%i IN ('dir /b /a-d /on') DO (
SET fdate=%%~ti
SETLOCAL enabledelayedexpansion
ECHO "%%i","!directory!","!fdate:~0,10!"
endlocal
)
)>filelist.csv
POPD
Edited to show date as third element. Quotes retained - remove at will.
If date AND TIME are required, remove the SET fdate line and replace the "!fdate:~0,10!" with "%%~ti
Date and time format - to be certain, need to know the format you are using.
If you're doing a recursive directory search, filename with extension only can be obtained within your for /f loop from %%F by using %%~nxF. That's easy.
The trickier part is scraping the last folder from the path (%%~pF). There's actually an easy trick to that as well though. Use another for loop to get %%~nxI of %%~dpF. Yes, the filename.ext of a full path is the trailing directory.
This only works if the directory does not end in a trailing backslash, though. Since the result of %%~dpF does end in a trailing backslash, you can work around it simply by adding a single dot to the end. So instead of "c:\users\me\Desktop\" you get the ~nx of "c:\users\me\Desktop\." with a trailing dot.
Enough explanation. Here's how it's done.
#echo off
for /f "delims=" %%F in ('dir /s /b /o:n') do (
for %%I in ("%%~dpF.") do echo "%%~nxF","%%~nxI"
) >filelist.csv