Windows Batch - Extract values for ffmpeg processing - json

I have dozens of json files, and I am trying to find two values in each of them and assign the results to two separate variables for ffmpeg processing.
An example json file looks like this:
{
"year": "2018",
"track": "12",
... other data omitted
}
I wish to extract 2018 and 12 so that I can use them in the following ffmpeg command:
ffmpeg -i "same_file_name_as_json.m4a" -metadata:s:a:0 year=2018 --metadata:s:a:0 track=12 -acodec libmp3lame "same_file_name_as_json.mp3"
Is it possible to write a single batch file to achieve the desired result? Any help would be greatly appreciated as I am a complete novice at findstr and setting variables. Thank you.
EDITED:
set "year=" & set "track="
for %%i in (*.json) do (
for /f "usebackq tokens=1,2 delims={:}, " %%a in ("%%i") do (
set "%%~a=%%~b"
if defined year if defined track goto :CONT
)
:CONT
C:\ffmpeg -i "%%~ni.m4a" -metadata:s:a:0 year=%year% -metadata:s:a:0 track=%track% -acodec libmp3lame "%%~ni.mp3"
)
pause

Windows batch scripting does not understand the JSON file format, so it is better to use a language natively supports it. It is not the best idea to treat JSON as "normal" text, because only a slight change (for instance, added, deleted, or moved line-breaks) that do not violate the JSON format can still make big troubles then.
That said, given that the JSON file exactly appears as you have shown it and it features Unix- or DOS/Windows-style line-breaks (that is, a carriage-return character followed by a line-feed character), this code could work for you:
for /F "usebackq tokens=1,2 delims={:}, " %%M in ("file.json") do set "%%~M=%%~N"
echo year = %year%
echo track = %track%
If you have got a huge JSON file you do not want to unnecessarily fully process, you could use this code instead:
set "year=" & set "track="
for /F "usebackq tokens=1,2 delims={:}, " %%M in ("file.json") do (
set "%%~M=%%~N"
if defined year if defined track goto :CONT
)
:CONT
echo year = %year%
echo track = %track%
If the (non-array) values you want to extract may also contain one of the defined delimiters ({, :, }, ,, SPACE), you could extend the code to this, given that the values do not contain the characters *, ?, <, >:
set "year=" & set "track="
for /F "usebackq tokens=1,* delims={:}, " %%M in ("file.json") do (
for %%K in (%%N) do set "%%~M=%%~K"
if defined year if defined track goto :CONT
)
:CONT
echo year = %year%
echo track = %track%
To prevent the script from assigning unwanted superfluous variables, you may try this:
for /F "usebackq tokens=1,2 delims={:}, " %%M in ("file.json") do (
if "%%~M"=="year" (set "%%~M=%%~N") else if "%%~M"=="track" set "%%~M=%%~N"
)
echo year = %year%
echo track = %track%
Or this, which prepreocesses the data by the findstr command and filters out the desired lines:
for /F "tokens=1,2 delims={:}, " %%M in ('
findstr /R /C:"^ *\"year\" *:" /C:"^ *\"track\" *:" "file.json"
') do set "%%~M=%%~N"
echo year = %year%
echo track = %track%
Based on your edit, let me suggest to use the last of the above methods, because there is no goto :CONT, which cannot be used within loops as it breaks the block context, and it does not assign additional unwanted variables. Since variables are written and read within the loop body, you have to enable and apply delayed variable expansion. I would do all that the following way:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem /* Iterate over the `*.json` files in the current working directory (`%CD%`);
rem to use the parent directory of this script, use `%~dp0*.json` instead: */
for %%I in ("*.json") do (
rem // Store name of current JSON file in variable:
set "name=%%~nI"
rem // Clear variables for later check for availability:
set "year=" & set "track="
rem // Process the current JSON file:
for /F "tokens=1,2 delims={:}, " %%M in ('
findstr /R /C:"^ *\"year\" *:" /C:"^ *\"track\" *:" "%%~I"
') do (
rem // Assign year and track variables:
set "%%~M=%%~N"
rem // Check of both year and track are available:
if defined year if defined track (
rem // Toggle delayed expansion to avoid troubles with `!`:
setlocal EnableDelayedExpansion
rem // Eventually execute `ffmpeg` tool using all the derived data:
ffmpeg -i "!name!.m4a" -metadata:s:a:0 year=!year! -metadata:s:a:0 track=!track! -acodec libmp3lame "!name!.mp3"
endlocal
)
)
)
endlocal
exit /B

I have dozens of json files...
Windows' cmd doesn't support JSON, so you'd have to resort to PowerShell, or use an external tool that does. You might find xidel interesting.
To extract the value for "year" and "track":
xidel -s input.json -e "$json/(year,track)"
#or
xidel -s input.json -e "$json/year,$json/track"
2018
12
To export to a variable %year% and %track%:
FOR /F "delims=" %A IN ('xidel -s input.json -e "$json/(year:=year,track:=track)" --output-format^=cmd') DO %A
#or
FOR /F "delims=" %A IN ('xidel -s input.json -e "year:=$json/year,track:=$json/track" --output-format^=cmd') DO %A
You don't however need variables to create the strings (ffmpeg commands) you want. xidel can do that too.
You could use a FOR-loop to iterate over all your JSON-files...
FOR %A IN (*.json) DO #xidel -s %A -e "$json/concat('ffmpeg -i \"%~nA.m4a\" -metadata:s:a:0 year=',year,' --metadata:s:a:0 track=',track,' -acodec libmp3lame \"%~nA.mp3\"')"
ffmpeg -i "name-of-json-file.m4a" -metadata:s:a:0 year=2018 --metadata:s:a:0 track=12 -acodec libmp3lame "name-of-json-file.mp3"
...but to call xidel for each and every JSON-file is very inefficient. xidel can do this much more efficiently.
xidel's equivalent for FOR %A IN (*.json) DO #ECHO %A is xidel -se "file:list(.,false(),'*.json')"
Then you can use the following query to process all your JSON-files at once:
xidel -se "for $x in file:list(.,false(),'*.json') return json-doc($x)/concat('ffmpeg -i \"',replace($x,'json','m4a'),'\" -metadata:s:a:0 year=',year,' --metadata:s:a:0 track=',track,' -acodec libmp3lame \"',replace($x,'json','mp3'),'\"')"
Prettified command/query:
xidel -se ^"^
for $x in file:list(.,false(),'*.json') return^
json-doc($x)/concat(^
'ffmpeg -i \^"',^
replace($x,'json','m4a'),^
'\^" -metadata:s:a:0 year=',^
year,^
' --metadata:s:a:0 track=',^
track,^
' -acodec libmp3lame \^"',^
replace($x,'json','mp3'),^
'\^"'^
)^
"

Related

Modify For loop to go from certain string forward and delete a selection of lines using batch

Ok so i am writing a batch file in which i delete every line containing "," after finding string:
"plugins": {
Is it possible to make this condition in for loop ?
now i know you can avoid quotes using ^ but i just cant make it work.
what i do right now is the following:
#echo OFF
::removePlugins
setlocal enabledelayedexpansion
echo. 2>tempPackage.json
SET #FOUND=0
for /F "delims=" %%G in (./package.json) do (
echo %%G|find "," > nul
if not errorlevel 1 (SET #FOUND=1)
if !#FOUND!==1(
#echo ON
SET #FOUND=0
ECHO: >> tempPackage.json
#echo OFF
)
ECHO %%G>>tempPackage.json
)
move "./tempPackage.json" ".package.json"
So in this case i would only make a new line in every line that contains ",".
So how does one write a for loop that only goes from this string forward and not make new line but delete it?.
the expected result after runing the batch would be :
{
"scripts"{ still the same scripts},
"dependencies"{ still the same dependencies},
"cordova": {
"platforms": [],
"plugins": {}
}
}
}
I tryed to use the code from #Stephan , like this:
#echo OFF
::removePlugins
setlocal enabledelayedexpansion
SET #findPlugins=0
find /i /c "plugins" ./package.json >NUL
if not errorlevel 1 (SET #findPlugins=1)
IF !#findPlugins!==1 (
SET "#FOUND=1"
>tempPackage.json (
for /F "delims=" %%G in (./package.json) do (
echo %%G|findstr /b "}" >nul && SET "#FOUND=1"
if defined #FOUND ECHO %%G
echo %%G|findstr /b "\"plugins\": {" >nul && SET "#FOUND="
)
)
type tempPackage.json
)
And it just rewrote the json to temp and didnt delete anything...what am i doing wrong ?
Keep in mind, batch can't interpret .json files and handles them as pure text. So any batch solution will highly depend on the exact format of the file. Any change in the format (a stray space might be enough) may cause trash.
That said: use a flag that changes at (each) line that starts with "plugins": and changes back when hitting the line starting with } (end of the block) and write the line dependent on the flag:
#echo OFF
::removePlugins
setlocal
SET "#FLAG=1"
>tempPackage.json (
for /F "delims=" %%G in (./package.json) do (
echo %%G|findstr /e "[^{]}" >nul && SET "#FLAG=1"
if defined #FLAG ECHO %%G
echo %%G|findstr "\"plugins\":" >nul && SET "#FLAG="
)
)
type tempPackage.json
This uses a regex to capture everything up to the "plugins" line.
powershell -NoLogo -NoProfile -Command ^
Get-Content -Raw .\pi.txt ^| ^
ForEach-Object { if ($_ -match '(?sm)(.*\"plugins\": {).*') { $Matches[1] + \"`n`n}\" ^| Out-File -FilePath '.package.json'}}

Getting variable from jq.exe output in Windows batch reading JSON

I managed having a JSON file created by ffprobe which contains basic info about a video stream in a MKV container. By jq-win64.exe "[.format.duration]" %%~ni.mkv.json the duration of the movie is read correctly from the file and jq echos ["1:36:55.184000"]. Now I want to store this value in a global variable of my script for further processing. I tried several approaches but each of them led to errors and/or left %duration% empty. I tried e.g.
for %%i in (*.mkv) do (
SETLOCAL ENABLEDELAYEDEXPANSION
for /F "tokens=* USEBACKQ" %%F IN ('_tools\jq\jq-win64.exe "[.format.duration]" %%~ni.mkv.json') DO (SET duration=%%F)
echo Duration is: %duration%
ENDLOCAL
)
but could not manage to echo %duration%. I think it can't be that hard, most likely I don't do the syntax right on Windows batch. Any ideas? Here is the JSON file as well:
{
"format": {
"filename": "TestFile_1080p_26Mbs_8bit_BT709.mkv",
"nb_streams": 1,
"nb_programs": 0,
"format_name": "matroska,webm",
"format_long_name": "Matroska / WebM",
"start_time": "0:00:00.000000",
"duration": "1:36:55.184000",
"size": "17.586597 Gibyte",
"bit_rate": "25.978148 Mbit/s",
"probe_score": 100,
"tags": {
"title": "TestFile",
"encoder": "libmakemkv v1.14.4 (1.3.5/1.4.7) win(x64-release)",
"creation_time": "2019-08-17T21:01:18.000000Z"
}
}
}
Here's a batch-file solution based upon my understanding after the comments:
For /F Tokens^=2Delims^=^" %%F In (
'_tools\jq\jq-win64.exe "[.format.duration]" "%%~ni.mkv.json" 2^>NUL')Do (
Set "duration=%%F"
SetLocal EnableDelayedExpansion
Echo( !duration!
EndLocal
)
If all you want is the duration, then there's no need for intermediate JSONs, because FFprobe can also tell you that:
ffprobe.exe -v 0 -i <input> -show_entries format=duration -of compact=p=0:nk=1
1:36:55.184000
Create a variable:
FOR /F "delims=" %%A IN (
'ffprobe.exe -v 0 -i <input> -show_entries format=duration -of compact=p=0:nk=1'
) DO SET duration=%%A
SET duration=1:36:55.184000
If you still want to parse FFprobe's JSON, then there's no need to create json-files either, as you can simply pipe it to JQ instead:
ffprobe.exe -v 0 -i <input> -show_format -of json | jq.exe -r .format.duration
1:36:55.184000
Create a variable:
FOR /F "delims=" %%A IN (
'ffprobe.exe -v 0 -i <input> -show_format -of json ^| jq.exe -r .format.duration'
) DO SET duration=%%A
SET duration=1:36:55.184000

How to read JSON data in batch script

I have a JSON file, named "Config.json", that looks like this:
{ "RunEnvironment": "DEV"}
In a batch file under the same directory, I want to read the value of the "RunEnvironment" element.
My batch script would look like:
if [jsonElement] == 'DEV' (
:: do something
)
Can anyone show me how to do this?
In PowerShell for example, you could do:
> If((Get-Content '.\Config.json'|ConvertFrom-Json).RunEnvironment -eq 'DEV'){"is DEV:Whatever"}
is DEV:Whatever
To be on topic on cmd line
> for /f "tokens=1,2 delims=:{} " %A in (Config.json) do #If "%~B"=="Dev" #Echo (%~A = %~B)
(RunEnvironment=DEV)
In a batch file
#Echo off
for /f "tokens=1,2 delims=:{} " %%A in (Config.json) do (
If "%%~B"=="Dev" Echo (%%~A=%%~B^)
)
The Echo (%%~A=%%~B^) could be replaced with whatever you plan to do.
If all you need to do is perform a specific command only if the value of RunEnvironment is DEV then this should be all you require, (from a batch-file or the command-line):
"%__APPDIR__%FindStr.exe" /IRC:"\"RunEnvironment\":\ \ *\"DEV\"" "Config.json">NUL 2>&1&&Echo Your command here
You'd simply replace Echo Your command here with your intended command.
Yo can do in this way
#echo off
set string={ "RunEnvironment": "DEV" }
for /f "tokens=3,5" %%a in ('echo %string%') do set d=%%~a
if "%d%" == "DEV" echo %d%
pause & goto :EOF
In a batch file:
#ECHO off
SET "FilenameForJsonFile=Config.json"
SET "FilenameForRunEnvironment=RunEnvironment.txt"
powershell -Command "Select-String -Pattern 'RunEnvironment\"\: \".*?\"' .\%FilenameForJsonFile% ^| ForEach-Object { $_.Matches.Value.substring(18,$_.Matches.Value.Length-19) }>%FilenameForRunEnvironment%
FOR /f "delims=" %%x IN (%FilenameForRunEnvironment%) DO SET JsonElement=%%x
IF %JsonElement%==DEV (ECHO development environment) ELSE (ECHO abc123 environment)

parse csv to make more readable

I am using PsInfo from sysinternals to return drive information in the form of a csv.
When I run this command.
psinfo -c -d volume
I get the following output.
PCName,,C:,Fixed,NTFS,,930.97 GB,705.81 GB,75.8%,D:,CD-ROM,,,,,0.0%,G:,Fixed,NTFS,My Book,1862.98 GB,889.71 GB,47.8%
My goal here is to parse that output to achieve this format:
PCNAME,,
,,C:,Fixed,NTFS,,930.97 GB,705.81 GB,75.8%
,,G:,Fixed,NTFS,My Book,1862.98 GB,889.71 GB,47.8%
So when the output is read by a spreadsheet viewer it appears readable and organized.
I've tried using regex to match the drive letter bit I don't know how to correctly capture the output.
Edit: (forgot to actually post my code...)
This is what I have so far.
ECHO OFF
For /f "tokens=1*" %%x in ('net view ^| find "\\"') do (psinfo %%x filter -d -c >> out.csv & echo. >> out.csv)
::used to remove blanmk lines from output
#jrepl "^[ \t]*(.*?)[ \t]*$" "$1?$1:false" /jmatch /f out.csv /o -
pause
#ECHO OFF
SETLOCAL enabledelayedexpansion
FOR /f "delims=" %%a IN ('psinfo -c -d volume') DO SET "var=%%a"
SET "var2=!var:%%=\!"
SET "retained="
:again
REM FOR /f "tokens=delims==:" %%a IN () DO
FOR /f "tokens=1*delims=:" %%a IN ("%var2%") DO (
CALL :FORMAT "%%a"
SET "var2=%%b"
IF NOT "%%b"=="" GOTO again
)
GOTO :EOF
:FORMAT
SET "data=%~1"
IF "%data:~-1%"=="\" (SET "data=%retained%:%data%") ELSE (SET "data=%retained%:%data:~,-2%")
IF DEFINED retained (
ECHO ,,!data:\=%%!
) ELSE (
ECHO %data:~1%,
)
SET "retained=%~1"
SET retained=%retained:~-1%
GOTO :eof
substitutes \ for % for ease of processing.
This worked for me. Given this example you should be able to play around with the format of the output you desire.
#echo off
FOR /F "tokens=1* delims=," %%G IN ('psinfo -c -d volume') DO (
SET "pcname=%%G"
SET "driveinfo=%%H"
)
set driveinfo="%driveinfo:,=","%"
:LOOP
FOR /F "tokens=1-7* delims=," %%G IN ("%driveinfo%") DO (
echo "%pcname%",%%G,%%H,%%I,%%J,%%K,%%L,%%M
SET "driveinfo=%%N"
)
IF DEFINED driveinfo GOTO LOOP
Output
"COCO","C:","Fixed","NTFS","","930.97 GB","705.81 GB","75.8%"
"COCO","G:","Fixed","NTFS","My Book","1862.98 GB","889.71 GB","47.8%"
And here is the code changed to replicate your exact needed Output.
#echo off
FOR /F "tokens=1* delims=," %%G IN ('psinfo -c -d volume') DO (
SET "pcname=%%G"
SET "driveinfo=%%H"
)
set driveinfo="%driveinfo:,=","%"
echo %pcname%,,
:LOOP
FOR /F "tokens=1-7* delims=," %%G IN ("%driveinfo%") DO (
echo ,,%%~G,%%~H,%%~I,%%~J,%%~K,%%~L,%%~M
SET "driveinfo=%%N"
)
IF DEFINED driveinfo GOTO LOOP
And here is doing it in a couple of less lines.
#echo off
setlocal enabledelayedexpansion
FOR /F "tokens=1* delims=," %%G IN ('psinfo -c -d volume') DO (
echo %%G,,
SET "driveinfo=%%H"
)
set "drive2=echo ,,!driveinfo:%%=%%&echo,,!"
%drive2:~0,-1%
You can get the same information with Powershell, eg with the Get-Volume command. This returns, everything except the ration eg:
DriveLetter FileSystemLabel FileSystem DriveType HealthStatus OperationalStatus SizeRemaining Size
----------- --------------- ---------- --------- ------------ ----------------- ------------- ----
D CD-ROM Healthy Unknown 0 B 0 B
C Win7 NTFS Fixed Healthy OK 43.59 GB 238.03 GB
H Projects NTFS Fixed Healthy OK 96.73 GB 375.24 GB
Get-Volume returns volume information objects. The shell just displays the most common ones.
You can select only the properties you want, and add more columns with the select command:
Get-Volume | where {$_.DriveType -ne 'CD-ROM'}
| select DriveLetter, FileSystemLabel,FileSystem,SizeRemaining, Size,
#{Name="Ratio";Expression={"{0:P}" -f ($_.SizeRemaining/$_.Size)}}
|Format-Table
DriveLetter FileSystemLabel FileSystem SizeRemaining Size Ratio
----------- --------------- ---------- ------------- ---- -----
C Win7 NTFS 46793633792 255584587776 18,31%
H Projects NTFS 103853056000 402911129600 25,78%
I'm cheating a bit here, by filtering out the CD ROM with where
If the results are too long to fit in the window, Powershell displays them as a list of name/values. The final Format-Table is used to force Powershell to format the results as a table
I can store the table to a file with a normal redirection :
Get-Volume | where {$_.DriveType -ne 'CD-ROM'}
| select DriveLetter, FileSystemLabel,FileSystem,SizeRemaining, Size,
#{Name="Rate";Expression={"{0:P}" -f ($_.SizeRemaining/$_.Size)}}
|Format-Table > somefile.txt
or I can export the results to a real CSV with Export-CSV
Get-Volume | where {$_.DriveType -ne 'CD-ROM'}
| select DriveLetter, FileSystemLabel,FileSystem,SizeRemaining, Size,
#{Name="Rate";Expression={"{0:P}" -f ($_.SizeRemaining/$_.Size)}}
| Export-CSV somefile.csv
The following code snippet reformats the output of psinfo -c -d volume as you wish. Since it seems that you want it filtered by drive type (Fixed) according to your sample data, I implemented such a feature; to not filter anything, simply change line set "FILTER=Fixed" to set "FILTER=":
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem // Define constants here:
set "FILTER=Fixed"
(set LF=^
%= empty line =%
)
for /F "tokens=1,* delims=, eol=," %%K in ('psinfo -c -d volume') do (
echo(%%K,,
set "LINE=%%L"
setlocal EnableDelayedExpansion
set ^"LINE=!LINE:%%,=%%^%LF%%LF%!^"
for /F "delims= eol=," %%E in ("!LINE!") do (
endlocal
if defined FILTER (
for /F "tokens=2 delims=, eol=," %%F in ("%%E") do (
if "%%F"=="%FILTER%" (
echo(,,%%E
)
)
) else (
echo(,,%%E
)
setlocal EnableDelayedExpansion
)
endlocal
)
endlocal
exit /B

Ping output to csv via batch

I got most of the following batch file from this site and have modified it for my needs:
#echo off
set/p host=host Address:
set logfile=Log_%host%.log
set csfile=pings_%host%.csv
echo Target Host = %host% >%logfile%
netsh interface show interface >>%logfile%
nslookup myip.opendns.com resolver1.opendns.com >>%logfile%
for /f "tokens=*" %%A in ('ping %host% -n 1 ') do (echo %%A>>%logfile% && GOTO Ping)
:Ping
for /f "tokens=* skip=2" %%A in ('ping %host% -n 1 ') do (
echo %date% %time:~0,2%:%time:~3,2%:%time:~6,2% %%A>>%logfile%
echo %date%,%time:~0,2%:%time:~3,2%:%time:~6,2%,%%A>>%csfile%
echo %date% %time:~0,2%:%time:~3,2%:%time:~6,2% %%A
timeout 1 >NUL
GOTO Ping)
The output to the .log file is exactly how I want it. The csv file output looks like this:
02/16/2016 Tue, 8:02:03,Reply from 173.194.115.33: bytes=32 time=16ms TTL=56
I would like to see something like this at the very least:
02/16/2016 Tue, 8:02:03,Reply from 173.194.115.33:, bytes=32, time=16ms, TTL=56
Optimally, I'd like the csv to look like this:
Date,Time,IP,Bytes,Time,TTL
02/16/2016 Tue,8:02:03,173.194.115.33,32,16ms,56
02/16/2016 Tue,8:02:04,173.194.115.33,32,17ms,56
...
I'd like to stay away from PowerShell if possible.
Thanks,
Joe
#ECHO OFF
SETLOCAL
SET "destdir=U:\destdir"
set "host=google.com"
set "csfile=%destdir%\pings_%host%.csv"
set "flagfile=%destdir%\flagfile.flg"
:: Ensure flagfile exists
ECHO.>"%flagfile%"
ECHO(Date,Time,IP,Bytes,Time,TTL>"%csfile%"
:PING
IF NOT EXIST "%flagfile%" GOTO :EOF
for /f "tokens=* skip=2" %%A in ('ping %host% -n 1 ') do (
FOR /f "tokens=1,4,6,8,10delims=:= " %%P IN ("%time:~0,2% %%A") DO (
echo %date%,%%P:%time:~3,2%:%time:~6,2%,%%Q,%%R,%%S,%%T>>%csfile%
)
echo %date% %time:~0,2%:%time:~3,2%:%time:~6,2% %%A
timeout 1 >NUL
GOTO Ping)
GOTO :EOF
Since you have the logfile processing already sorted, this would generate the csvfile. I've set up filenames suit my system and pinged Google for convenience.
This processes the line
" 8 Reply from 173.194.115.33:, bytes=32, time=16ms, TTL=56"
Assuming "reply from" (no doubt that could be gated if required) usng separators of :=and Space It's then simply a matter of outputting the date data followed by the first token, the minutes and seconds with punctuation and a comma-separated list of the remaining required tokens.
I've added a flagfile which gets created on the first entry. If you delete the flagfile, the batch stops, so you don't have to CtrlC to kill the process.
You can use a FOR loop to parse out only the values you care about.
FOR /F "usebackq tokens=1-10 delims==:< " %%A IN (`ping google.com -n 1`) DO (
REM Only process the response line.
IF "%%A"=="Reply" (
ECHO %%C,%%E,%%G,%%I
)
)
This snippet puts the the following data into CSV format:
IP, bytes, time, TTL
Sample output:
74.125.138.138,32,11ms,43
You should be able to adapt this to your script fairly easily.