How to transcode MP4 video with SRT subtitle on AWS Elemental MediaConvert - json

I have a MP4 video with SRT captions and I need to transcode them with media convert. In media convert I set automatic ABR and I specified the SRT origin path.
At the moment, I have tested the following:
I set SRT file in one output and video/audio in another
I set SRT, video and audio in the same output
For the first test, the job finish successfully, but on the S3 bucket there isnt any .SRT file. For the second test, the job fails with "aption destination type [SRT] requires a raw muxer." message
This is my JSON for the first test
{
"Queue": "arn:aws:mediaconvert:us-east-1:{{ACCOUNT-NUMBER}}:queues/Default",
"UserMetadata": {},
"Role": "arn:aws:iam::{{ACCOUNT-NUMBER}}:role/{{MY-ROLE-NAME}}",
"Settings": {
"TimecodeConfig": {
"Source": "ZEROBASED"
},
"OutputGroups": [
{
"Name": "DASH ISO",
"Outputs": [
{
"ContainerSettings": {
"Container": "MPD"
},
"VideoDescription": {
"ScalingBehavior": "DEFAULT",
"TimecodeInsertion": "DISABLED",
"AntiAlias": "ENABLED",
"Sharpness": 50,
"CodecSettings": {
"Codec": "H_264",
"H264Settings": {
"InterlaceMode": "PROGRESSIVE",
"ScanTypeConversionMode": "INTERLACED",
"NumberReferenceFrames": 3,
"Syntax": "DEFAULT",
"Softness": 0,
"GopClosedCadence": 1,
"GopSize": 90,
"Slices": 1,
"GopBReference": "DISABLED",
"SlowPal": "DISABLED",
"EntropyEncoding": "CABAC",
"FramerateControl": "INITIALIZE_FROM_SOURCE",
"RateControlMode": "QVBR",
"CodecProfile": "MAIN",
"Telecine": "NONE",
"MinIInterval": 0,
"AdaptiveQuantization": "AUTO",
"CodecLevel": "AUTO",
"FieldEncoding": "PAFF",
"SceneChangeDetect": "ENABLED",
"QualityTuningLevel": "MULTI_PASS_HQ",
"FramerateConversionAlgorithm": "DUPLICATE_DROP",
"UnregisteredSeiTimecode": "DISABLED",
"GopSizeUnits": "FRAMES",
"ParControl": "INITIALIZE_FROM_SOURCE",
"NumberBFramesBetweenReferenceFrames": 2,
"RepeatPps": "DISABLED",
"DynamicSubGop": "STATIC"
}
},
"AfdSignaling": "NONE",
"DropFrameTimecode": "ENABLED",
"RespondToAfd": "NONE",
"ColorMetadata": "INSERT"
},
"AudioDescriptions": [
{
"AudioTypeControl": "FOLLOW_INPUT",
"AudioSourceName": "Audio Selector 1",
"CodecSettings": {
"Codec": "AAC",
"AacSettings": {
"AudioDescriptionBroadcasterMix": "NORMAL",
"Bitrate": 96000,
"RateControlMode": "CBR",
"CodecProfile": "LC",
"CodingMode": "CODING_MODE_2_0",
"RawFormat": "NONE",
"SampleRate": 48000,
"Specification": "MPEG4"
}
},
"StreamName": "latino",
"LanguageCodeControl": "FOLLOW_INPUT",
"LanguageCode": "SPA"
}
]
},
{
"ContainerSettings": {
"Container": "MPD"
},
"CaptionDescriptions": [
{
"CaptionSelectorName": "Captions Selector 1",
"DestinationSettings": {
"DestinationType": "SRT"
},
"LanguageCode": "SPA",
"LanguageDescription": "latino"
}
]
}
],
"OutputGroupSettings": {
"Type": "DASH_ISO_GROUP_SETTINGS",
"DashIsoGroupSettings": {
"SegmentLength": 30,
"MinFinalSegmentLength": 0,
"Destination": "s3://{{BUCKET-NAME}}/streaming15/dash-iso/",
"FragmentLength": 2,
"SegmentControl": "SINGLE_FILE",
"MpdProfile": "ON_DEMAND_PROFILE",
"HbbtvCompliance": "NONE"
}
},
"AutomatedEncodingSettings": {
"AbrSettings": {
"MaxAbrBitrate": 8000000,
"MinAbrBitrate": 600000
}
}
}
],
"AdAvailOffset": 0,
"Inputs": [
{
"AudioSelectors": {
"Audio Selector 1": {
"Offset": 0,
"DefaultSelection": "DEFAULT",
"ProgramSelection": 1
}
},
"VideoSelector": {
"ColorSpace": "FOLLOW",
"Rotate": "DEGREE_0",
"AlphaBehavior": "DISCARD"
},
"FilterEnable": "AUTO",
"PsiControl": "USE_PSI",
"FilterStrength": 0,
"DeblockFilter": "DISABLED",
"DenoiseFilter": "DISABLED",
"InputScanType": "AUTO",
"TimecodeSource": "ZEROBASED",
"CaptionSelectors": {
"Captions Selector 1": {
"SourceSettings": {
"SourceType": "SRT",
"FileSourceSettings": {
"SourceFile": "s3://{{BUCKET-NAME}}/PROMO_CAP_01.srt"
}
}
}
},
"FileInput": "s3://{{BUCKET-NAME}}/PROMO_CAP_01.mp4"
}
]
},
"AccelerationSettings": {
"Mode": "DISABLED"
},
"StatusUpdateInterval": "SECONDS_60",
"Priority": 0
}
What I am missing?

According to the AWS Elemental MediaConvert user guide, SRT is not a supported output for a DASH-ISO output group when the input caption type is SRT.
Here's a link to that guide (reference page 176):
https://docs.aws.amazon.com/mediaconvert/latest/ug/mediaconvert-guide.pdf
The supported caption outputs for SRT input in DASH-ISO are:
Burn in
IMSC (as sidecar .fmp4)
IMSC (as sidecar .xml)
TTML (as sidecar .fmp4)
TTML (as sidecar .ttml)
Additionally, there is a gap in the documentation. SRT->DASH-ISO+WebVTT is supported, even though it is not listed. The documentation will be corrected, but I wanted to share that with you in case it helps.
If you must send SRT to the output destination, then you could create a separate output group where the caption is in a track with no container (see pages 192-196 in the document).

Related

Autodesk Forge Viewer: Connect Fragment Material to Revit Model Data

EDIT: TLDR there is currently no way to know material names on the fragment level.
I want to read the materials from fragments of a node and change their materials according to a map that uses the Revit material names as keys.
I have the following "Materials and Finishes" properties from a node in the model (retrieved via Viewer3D):
And I have the following THREE materials from the fragments of that node:
Is there a way to set the names of the THREE materials to match the model data (or use them at all)?
Ideally I would be able to match these THREE materials with the following materials extracted from this node:
The SVF file format (generated by the Model Derivative service and loaded by Forge Viewer) does not preserve material names unfortunately. The fragments are simply associated with a specific material based on its index in the list.
The "Materials and Finishes" data is basically just a property specific to the original file (in this case a Revit model), and it may not be available in other file formats.
EDIT: I tried looking into the Materials.json.gz file, and unfortunately the names are not included there, either:
{
"name": "LMVTK Simple Materials",
"version": "1.0",
"scene": {
"SceneUnit": 8215,
"YIsUp": 0
},
"materials": {
"0": {
"version": 2,
"userassets": ["0"],
"materials": {
"0": {
"tag": "",
"proteinType": "",
"definition": "SimplePhong",
"properties": {
"integers": {
"mode": 4
},
"booleans": {
"color_by_object": false,
"generic_is_metal": false,
"generic_backface_cull": true
},
"scalars": {
"generic_transparency": {
"units": "",
"values": [0]
}
},
"colors": {
"generic_diffuse": {
"values": [{
"r": 0,
"g": 1,
"b": 0,
"a": 1
}]
}
}
},
"transparent": false,
"textures": {
}
}
}
},
"1": {
"version": 2,
"userassets": ["0"],
"materials": {
"0": {
"tag": "",
"proteinType": "",
"definition": "SimplePhong",
"properties": {
"integers": {
"mode": 4
},
"booleans": {
"color_by_object": false,
"generic_is_metal": false,
"generic_backface_cull": true
},
"scalars": {
"generic_transparency": {
"units": "",
"values": [0]
}
},
"colors": {
"generic_diffuse": {
"values": [{
"r": 0.400000,
"g": 0.400000,
"b": 0.400000,
"a": 1
}]
}
}
},
"transparent": false,
"textures": {
}
}
}
}
...
}

Extracting data from nested JSON with python 3

Looking for a way to extract data from nested json.
The data was extracted from windows scheduler as xml and converted to json.
It represents OS scheduled tasks, there are about 100 tasks and my goal is to extract the information from json to an external DB.
This is what my json file looks like :
{
"Tasks": {
"Task": [
{
"RegistrationInfo": {
"Author": "Adobe Systems Incorporated",
"Description": "This task keeps your Adobe Reader and Acrobat applications up to date with the latest enhancements and security fixes",
"URI": "\\Adobe Acrobat Update Task"
},
"Principals": {
"Principal": {
"GroupId": "S-1-5-4",
"_id": "Author"
}
},
"Settings": {
"DisallowStartIfOnBatteries": "true",
"StopIfGoingOnBatteries": "true",
"MultipleInstancesPolicy": "IgnoreNew",
"StartWhenAvailable": "true",
"IdleSettings": {
"Duration": "PT10M",
"WaitTimeout": "PT1H",
"StopOnIdleEnd": "true",
"RestartOnIdle": "false"
}
},
"Triggers": {
"LogonTrigger": {
"StartBoundary": "2013-08-01T12:05:00",
"EndBoundary": "2027-05-02T08:00:00",
"Delay": "PT12M",
"Repetition": {
"Interval": "PT3H30M"
},
"_id": "TriggerUserLoggon"
},
"CalendarTrigger": {
"StartBoundary": "2013-01-01T09:00:00",
"EndBoundary": "2027-05-02T12:05:00",
"ScheduleByDay": {
"DaysInterval": "1"
},
"_id": "TriggerDaily"
}
},
"Actions": {
"Exec": {
"Command": "C:\\Program Files (x86)\\Common Files\\Adobe\\ARM\\1.0\\AdobeARM.exe"
},
"_Context": "Author"
},
"_xmlns": "http://schemas.microsoft.com/windows/2004/02/mit/task",
"_version": "1.2"
},
{
"RegistrationInfo": {
"Author": "Adobe",
"Description": "This task keeps your Adobe Flash NPAPI Player installation up to date with the latest enhancements and security fixes. If this task is disabled or removed, Adobe Flash Player will be unable to automatically secure your machine with the latest security fixes.",
"URI": "\\Adobe Flash Player NPAPI Notifier"
},
"Principals": {
"Principal": {
"UserId": "S-1-5-21-2755204513-1269241785-1912306605-1001",
"LogonType": "InteractiveToken",
"_id": "Author"
}
},
"Settings": {
"DisallowStartIfOnBatteries": "false",
"StopIfGoingOnBatteries": "true",
"MultipleInstancesPolicy": "IgnoreNew",
"StartWhenAvailable": "true",
"RunOnlyIfNetworkAvailable": "true",
"IdleSettings": {
"Duration": "PT10M",
"WaitTimeout": "PT1H",
"StopOnIdleEnd": "true",
"RestartOnIdle": "false"
}
},
"Triggers": {
"CalendarTrigger": {
"StartBoundary": "1999-12-31T16:14:00-08:00",
"Repetition": {
"Interval": "PT1H",
"Duration": "P1D"
},
"ScheduleByDay": {
"DaysInterval": "7"
},
"_id": "NotificationTrigger"
}
},
"Actions": {
"Exec": {
"Command": "C:\\Windows\\SysWOW64\\Macromed\\Flash\\FlashUtil32_32_0_0_330_Plugin.exe",
"Arguments": "-check plugin"
},
"_Context": "Author"
},
"_xmlns": "http://schemas.microsoft.com/windows/2004/02/mit/task",
"_version": "1.2"
}
The code below works for just one task, but i need to get the data from every task , i guess i need to find a way to insert the number of tasks to the value of index and loop them one by one ,but nothing i try works.
import json
index=1
with open('name.json', 'r') as f:
array = json.load(f)
ts=(array['Tasks'])
print('Author is',ts['Task'][index]['RegistrationInfo']['Author'])
print('Description is' ,ts['Task'][index]['RegistrationInfo']['Description'])
print('URI is',ts['Task'][index]['RegistrationInfo']['URI'])
print('user ID is', ts['Task'][index]['Principals']['Principal']['UserId'])
print('Logon Type is', ts['Task'][index]['Principals']['Principal']['LogonType'])
print(' ID is', ts['Task'][index]['Principals']['Principal']['_id'])
print(' DisallowStartIfOnBatteries is ', ts['Task'][index]['Settings']['DisallowStartIfOnBatteries'])
print(' StopIfGoingOnBatteries is ', ts['Task'][index]['Settings']['StopIfGoingOnBatteries'])
print(' MultipleInstancesPolicy is ', ts['Task'][index]['Settings']['MultipleInstancesPolicy'])
print(' StartWhenAvailable is ', ts['Task'][index]['Settings']['StartWhenAvailable'])
print(' RunOnlyIfNetworkAvailable is ', ts['Task'][index]['Settings']['RunOnlyIfNetworkAvailable'])

Issue while creating a job for AWS Elemental media convert

I am having some issue while creating a job for AWS Elemental media convert.
I have followed the following sequence.
1.) Create a new job
2.) Add input and configurations
3.) Add File output group and configure destination settings
4.) Under Output change Container to No Container
5.) Under Output remove Audio
6.) Under Output -> Video change Codec to JPEG to Frame Capture
7.) Configure frame rate (rate which captures will be produced (more notes and examples below))
8.) Configure max capture settings
I got the following error:
Job_contains_the_following_error:
/outputGroups: Should not match the schema
Here is my job JSON:
{
"Settings": {
"AdAvailOffset": 0,
"Inputs": [
{
"FilterEnable": "AUTO",
"PsiControl": "USE_PSI",
"FilterStrength": 0,
"DeblockFilter": "DISABLED",
"DenoiseFilter": "DISABLED",
"TimecodeSource": "EMBEDDED",
"VideoSelector": {
"ColorSpace": "FOLLOW",
"Rotate": "DEGREE_0"
},
"AudioSelectors": {
"Audio Selector 1": {
"Offset": 0,
"DefaultSelection": "DEFAULT",
"ProgramSelection": 1
}
},
"FileInput": "s3://field-live-user-data/udariyan.mp4"
}
],
"OutputGroups": [
{
"Name": "File Group",
"OutputGroupSettings": {
"Type": "FILE_GROUP_SETTINGS",
"FileGroupSettings": {
"Destination": "s3://field-live-user-data/"
}
},
"Outputs": [
{
"VideoDescription": {
"ScalingBehavior": "DEFAULT",
"TimecodeInsertion": "DISABLED",
"AntiAlias": "ENABLED",
"Sharpness": 50,
"CodecSettings": {
"Codec": "FRAME_CAPTURE",
"FrameCaptureSettings": {
"FramerateNumerator": 30,
"FramerateDenominator": 100,
"MaxCaptures": 2,
"Quality": 80
}
},
"DropFrameTimecode": "ENABLED",
"ColorMetadata": "INSERT",
"Width": 1280,
"Height": 720
},
"ContainerSettings": {
"Container": "RAW"
},
"Extension": "jpg"
}
],
"CustomName": "customGroup"
}
]
},
"Queue": "arn:aws:mediaconvert:us-east-1:469030323850:queues/Default",
"Role": "arn:aws:iam::469030323850:role/myMediaConverter"
}
Currently, you can't have a job template with frame capture only:
AWS Dev forums on this topic

How Do I Display Non-Lit Material + Texture Using LMV Toolkit Specification or a Autodesk Material Library Material?

I'm unable to successfully display a model+texture map WITHOUT the viewer applying lighting effects to it.
I am using a localised version of the viewer to investigate the problem but I welcome an Autodesk Material Library setting solution if it exists.
This is an example of how I want to see the material, i.e. no specular, no reflections (ignore the fact this example is in three.js): https://stemkoski.github.io/Three.js/Texture-Repeat.html
This is an example of my problem: https://myhub.autodesk360.com/ue29c31db/g/shares/SHabee1QT1a327cf2b7a7879b97973545818?viewState=NoIgbgDAdAjCA0IBGMAsBmATAMwKYBMBaCAQwHYBjQ1fATlUNt13UO1pIwDYZMAOTCVogAukA
I have attempted many different "Autodesk Material Library" settings, including ramping up "Self Illumination" however, the texture either fails to load and/or, that glossy shine persists.
Could the Materials.json be tweaked to fix this problem?
This is my Materials.json
{
"name": "LMVTK Simple Materials",
"version": "1.0",
"scene": {
"SceneUnit": 8214,
"YIsUp": 2
},
"materials": {
"0": {
"version": 2,
"userassets": ["0"],
"materials": {
"0": {
"tag": "0",
"proteinType": "",
"definition": "SimplePhong",
"properties": {
"integers": {
"mode": 4
},
"booleans": {
"color_by_object": false,
"generic_is_metal": false,
"generic_backface_cull": false
},
"scalars": {
"generic_transparency": {
"units": "",
"values": [0]
}
},
"colors": {
"generic_diffuse": {
"values": [{
"r": 1,
"g": 1,
"b": 1,
"a": 1
}]
}
}
},
"transparent": false,
"textures": {
"generic_diffuse": {
"connections": ["1_generic_diffuse"]
}
}
},
"1_generic_diffuse": {
"tag": "0",
"definition": "UnifiedBitmap",
"properties": {
"scalars": {
"unifiedbitmap_RGBAmount": {
"units": "",
"values": [1]
}
},
"uris": {
"unifiedbitmap_Bitmap": {
"values": ["image0.jpg"]
}
},
"booleans": {
"texture_URepeat": true,
"texture_VRepeat": true,
"unifiedbitmap_Invert": false
},
"integers": {
"texture_MapChannel": 1
}
}
}
}
}
}
}
I recommend trying this approach. Literally bypass the LMV materials (which are all effected by lighting) and use a custom THREE material that is not effected by lighting. You may possibly need to create a custom shader too.
Start with this...
https://forge.autodesk.com/cloud_and_mobile/2016/02/custom-transparent-meshes-with-view-data-api.html
Let me know if that fixes the problem, and if not I can deep dive into it further.
Best, Michael

how to convert this array in one?

I am playing around with jq at https://jqplay.org/
The JSON I currently have is :
{
"license": {
"type": "permissive",
"url": "http://en.wikipedia.org/wiki/MIT_License",
"name": "MIT"
},
"lib": "libxml2",
"vuln-count": {
"exact": 2,
"total": 3,
"historical": 1
},
"vulns": [
{
"exact": false,
"timestamp-objects": [],
"vuln": {
"published-epoch": "1072839600",
"cvss": 9.3,
"summary": "libxml2, possibly before 2.5.0.\"",
"published": "2003-12-31T03:00:00",
"cve": "CVE-2003-1564",
"modified": "2008-10-24T01:30:02",
"modified-epoch": "1224811802"
}
},
{
"exact": true,
"timestamp-objects": [
"libxml2-2.8.0-1.x86_64.cpio:/usr/lib64/libxml2.so.2.8.0"
],
"vuln": {
"published-epoch": "1356061574",
"cvss": 5,
"summary": "libxml2 before 2.8.0 computes hash values.",
"published": "2012-12-21T03:46:14",
"cve": "CVE-2012-0841",
"modified": "2014-01-28T02:42:55",
"modified-epoch": "1390876975"
}
},
{
"exact": true,
"timestamp-objects": [
"libxml2-2.8.0-1.x86_64.cpio:/usr/lib64/libxml2.so.2.8.0"
],
"vuln": {
"published-epoch": "1346432101",
"cvss": 6.8,
"summary": "libxml2 2.9.0-rc1 and earlier.",
"published": "2012-08-31T16:55:01",
"cve": "CVE-2012-2871",
"modified": "2014-01-28T02:45:36",
"modified-epoch": "1390877136"
}
}
]
}
my current filter is
if .["vuln-count"].exact >0
then {num_of_vulns: .["vuln-count"].exact, lib: .lib, license: .license.type, vuln: .vulns[]|select (.exact==true)|.vuln.cve}
else empty end
And the result is
{
"num_of_vulns": 2,
"lib": "libxml2",
"license": "permissive",
"vuln": "CVE-2012-0841"
}
{
"num_of_vulns": 2,
"lib": "libxml2",
"license": "permissive",
"vuln": "CVE-2012-2871"
}
What is the filter to use in order to get the following output?
{
"num_of_vulns": 2,
"lib": "libxml2",
"license": "permissive",
"vulns": ["CVE-2012-0841", "CVE-2012-2871"]
}
Thanks to Santiago, here is the working filter:
if .["vuln-count"].exact >0 then {
num_of_vulns: ."vuln-count".exact,
lib: .lib,
license: .license.type,
vulns: .vulns | map(select(.exact).vuln.cve)
} else empty end
Your expected result is not meaningful as a JSON! On a JSON object, the property keys must be unique. When property keys are repeated, JSON parsers will usually only keep the last one's value, disregarding the rest.
You can, however, obtain something like this:
{
"num_of_vulns": 2,
"lib": "libxml2",
"license": "permissive",
"vulns": ["CVE-2012-0841", "CVE-2012-2871"]
}
Here's a sample script that does so (I omitted your wrapping if for empty vulns, but it could be added around it):
{
num_of_vulns: ."vuln-count".exact,
lib: .lib,
license: .license.type,
vulns: .vulns | map(select(.exact).vuln.cve)
}