The support of HTML5 video has evolved a lot over the years. I am trying to understand whether the <video> element still needs to have three sources: MP4, WEBM, and OGG.
There are a lot of answers throughout StackOverflow with deeply conflicting information - some of which say that you just need MP4 now, others say, MP4 and WEBM are enough, and then finally many say that you need all three (although many of those article are ~10 years old).
W3 suggests that either MP4 or WEBM alone would have universal support (Even though I found a 2011 article from Google saying that they would be removing support for MP4/H.264). Wikipedia paints a more complicated picture (as well as listing that Google Chrome does indeed support MP4/H.264). Azure Media services ONLY seems to allow output in MP4, which would suggest to me that MP4 must have widespread compatibility.
Also see Example 1, Example 2, Example 3.
Is there any definitive information on what video types to include in an HTML5 video player to achieve widespread compatibility?
Background: I am building a Content Management Platform that allows uploading videos. When a new video is uploaded, a conversion process kicks off to convert the video into the required formats. This takes time and CPU/Memory, so if it is possible I would like to convert uploaded videos into as few formats as possible.
p.s. This question HAS been asked before, however, the fundamentals of playing video on the web continually evolve and most of the answers out there have become irrelevant.
https://caniuse.com/?search=mp4
https://caniuse.com/?search=webm
I used to also include .ogg a few years ago but I see no need anymore. Still render both of these formats just for safe keeping.
You don’t need anything other than mp4 for up to date browsers, but if you want to support older open source browsers as well you can add a ogg or webm file
Related
I need to play MPEG-1 files dynamically from a browser. Uploading them to YouTube or converting the videos is not an option.
How can I do this?
I've seen this, but the answers do not apply to MPEG-1. Is there a way to play mpeg videos in HTML5?
The video tag is not working for the file with Chrome:
<video class="fullscreen" autoplay>
<source src="video/test2.mpeg">
</video>
It just displays a black box and stuttering noise/sound. I can verify that the video is not corrupt because I can play it with VLC. I only need this to work on one specific browser (it does not have to be cross-compatible). Plugins are OK too, as long as I do not have to convert the video. Though I'd rather avoid them.
Some browsers don't support older formats in <video> on purpose, to limit number of crappy, legacy, and potentially insecurely implemented video formats on the web.
The only combination that has a decent chance of working is H.264 (MP4) and WebM (or Ogg Theora), so you must convert the video and for good browser support you will have to generate at least these two formats.
The good news is that modern codecs are much more efficient than MPEG-1, so you'll get much smaller files.
Other options may be:
Give users a direct link to the video so they can download it and play in an external player like VLC.
Embed video using oldschool <object> element, and hope some browsers still have legacy plugins that can play videos (but e.g. Chrome has recently removed support for all plugins except a couple written specifically for Chrome's own API).
Use Emscripten (asm.js) to compile an MPEG-1 decoder to JavaScript and decode the video yourself to <canvas>. JS these days is fast enough to pull that off (although it will quickly drain battery of mobile devices, and a poor video codec combined with an extra download of a JS decoder will be a massive waste of bandwidth).
You can play MPEG-1 videos using JSMPEG: https://jsmpeg.com/
Sample code here: https://github.com/phoboslab/jsmpeg/blob/master/view-stream.html
It seems clear that officially the html5 audio tag is intended for use with the ogg format.
Does the html5 audio tag encompass .mid (MIDI) unofficially?
Updates:
This is a somewhat old question but it is still highly ranked on Google so here are some updates to the answers above - standards and tools that may have not been available when the question was asked but may be useful to people who find this question today:
Web MIDI API (this is currently a W3C Editor's Draft from December 26, 2012 - ie. today - so it may change)
MIDI.js - playing MIDI with JavaScript
jasmid - MIDI synthesis with JavaScript
See also those questions:
Generate live MIDI from Javascript
generating MIDI in javascript
In spite of the MIME type "audio/midi" a MIDI file is not audio ... it is a musical performance only, there is no sound. In other words, a MIDI file stores which keys a piano player hit, when he or she hit them, and how hard he or she hit them, but no actual piano sounds. You have to put the musical performance through a sampler that has a piano sound in it, just like you have to put a piano roll through a player piano. Apple QuickTime 7 has such a sampler built-in, but I don't know if QuickTime X does. Anyway, you would have to use an embed tag, it doesn't work through the audio tag.
It seems clear that officially the html5 audio tag is intended
for use with the ogg format.
No. HTML5 is a markup standard. It doesn't address audio standardization at all. In other words, it tells you how to encode markup, not how to encode audio. Audio standardization is done by ISO's Moving Picture Experts Group for the past 20 years now. The current audio standard is MPEG-4 AAC (audio/mp4), and the previous standard is MPEG-2 MP3 (audio/mp3). If you are standardizing your markup to the W3C HTML5 specification, then you should standardize your audio to the ISO MPEG-4 specification. Otherwise, just ignore all standards and use whatever markup and audio you like, and make content that only you can see. All PC's and mobiles support both HTML5 and MPEG-4 out-of-the-box. It is only the alternate browsers that block access to the user's hardware MPEG-4 player, so you should work around those by offering the MP4 audio file to those users as a simple link they can click and open in a helper app, like iTunes (which is on about 90% of all PC's). Or you can invoke a FlashPlayer in those alternate browsers and load the same standard MP4 in there.
As for Ogg, it plays in some alternate browsers, but it doesn't play in the user's RSS reader, Podcast reader, native apps that offer Web views, and in many other contexts that the user may see your web page. Standardized audio plays in all of those contexts. So if you are offering Ogg, you have to offer it as a second choice after MP4, because the Ogg is only going to be needed a small minority of the time.
It doesn't appear to to support raw MIDI files, in Chrome 10 at least.
However, the W3C says it supports XMF which can have MIDI embedded. Not sure about browser support though.
To quote from the official Web MIDI API page (http://www.w3.org/TR/webmidi):
To some users, "MIDI" has become synonymous with Standard MIDI Files and General MIDI. That is not the intent of this API; the use case of simply playing back a .SMF file is not within the purview of this specification (it could be considered a different format to be supported by the HTML5 <audio> element, for example).
In other words, W3 Org says browser support for simple MIDI files should be through the HTML5 tag. Sadly, as far as I know, no browser yet supports MIDI via the <audio> tg. But I'll bet that the first one which does is going to get a huge chunk of market share. For example, the site at http://www.hymntime.com/tch has thousands of MIDI files that still have to use the old <object> tag to play. And even then, it still requires a plugin. Try explaining to a non-technical user what a "plugin" is or how to install one!
Browsers are not required to support any specific codecs or encodings by the HTML5 spec. There is some discussion around supporting MIDI-like sound behaviour in the future, but not much - these two posts on the mailing list last year didn't get any direct responses.
I know that QuickTime does (or used to?) support MIDI, and Safari will play any audio that QuickTime supports, so it may work already in Safari but I haven't tried it.
It seems clear that officially the html5 audio tag is intended for use with the ogg format.
Does the html5 audio tag encompass .mid (MIDI) unofficially?
Updates:
This is a somewhat old question but it is still highly ranked on Google so here are some updates to the answers above - standards and tools that may have not been available when the question was asked but may be useful to people who find this question today:
Web MIDI API (this is currently a W3C Editor's Draft from December 26, 2012 - ie. today - so it may change)
MIDI.js - playing MIDI with JavaScript
jasmid - MIDI synthesis with JavaScript
See also those questions:
Generate live MIDI from Javascript
generating MIDI in javascript
In spite of the MIME type "audio/midi" a MIDI file is not audio ... it is a musical performance only, there is no sound. In other words, a MIDI file stores which keys a piano player hit, when he or she hit them, and how hard he or she hit them, but no actual piano sounds. You have to put the musical performance through a sampler that has a piano sound in it, just like you have to put a piano roll through a player piano. Apple QuickTime 7 has such a sampler built-in, but I don't know if QuickTime X does. Anyway, you would have to use an embed tag, it doesn't work through the audio tag.
It seems clear that officially the html5 audio tag is intended
for use with the ogg format.
No. HTML5 is a markup standard. It doesn't address audio standardization at all. In other words, it tells you how to encode markup, not how to encode audio. Audio standardization is done by ISO's Moving Picture Experts Group for the past 20 years now. The current audio standard is MPEG-4 AAC (audio/mp4), and the previous standard is MPEG-2 MP3 (audio/mp3). If you are standardizing your markup to the W3C HTML5 specification, then you should standardize your audio to the ISO MPEG-4 specification. Otherwise, just ignore all standards and use whatever markup and audio you like, and make content that only you can see. All PC's and mobiles support both HTML5 and MPEG-4 out-of-the-box. It is only the alternate browsers that block access to the user's hardware MPEG-4 player, so you should work around those by offering the MP4 audio file to those users as a simple link they can click and open in a helper app, like iTunes (which is on about 90% of all PC's). Or you can invoke a FlashPlayer in those alternate browsers and load the same standard MP4 in there.
As for Ogg, it plays in some alternate browsers, but it doesn't play in the user's RSS reader, Podcast reader, native apps that offer Web views, and in many other contexts that the user may see your web page. Standardized audio plays in all of those contexts. So if you are offering Ogg, you have to offer it as a second choice after MP4, because the Ogg is only going to be needed a small minority of the time.
It doesn't appear to to support raw MIDI files, in Chrome 10 at least.
However, the W3C says it supports XMF which can have MIDI embedded. Not sure about browser support though.
To quote from the official Web MIDI API page (http://www.w3.org/TR/webmidi):
To some users, "MIDI" has become synonymous with Standard MIDI Files and General MIDI. That is not the intent of this API; the use case of simply playing back a .SMF file is not within the purview of this specification (it could be considered a different format to be supported by the HTML5 <audio> element, for example).
In other words, W3 Org says browser support for simple MIDI files should be through the HTML5 tag. Sadly, as far as I know, no browser yet supports MIDI via the <audio> tg. But I'll bet that the first one which does is going to get a huge chunk of market share. For example, the site at http://www.hymntime.com/tch has thousands of MIDI files that still have to use the old <object> tag to play. And even then, it still requires a plugin. Try explaining to a non-technical user what a "plugin" is or how to install one!
Browsers are not required to support any specific codecs or encodings by the HTML5 spec. There is some discussion around supporting MIDI-like sound behaviour in the future, but not much - these two posts on the mailing list last year didn't get any direct responses.
I know that QuickTime does (or used to?) support MIDI, and Safari will play any audio that QuickTime supports, so it may work already in Safari but I haven't tried it.
I am developing an HTML5 player on Windows, but I have some real difficulties to make a real cross-browser and platform HTML5 video player:
It seems that Firefox and safari have different format support on their video tag, .ogg for Firefox, mp4 based for Safari.
Meanwhile Firefox supports the .ogg format without further plugins, Apple Safari needs the Quicktime plug-in for playing mp4 video in the tag video. This doesn't sound good.
Apple hates Flash for its CPU consumption and extra plug-in, but for itself requires a Quicktime plug-in for playing video on HTML5. Why don't these browsers use a standard?
Quite simple.
Video is historically a minefield with regards to patents. People from the opensource world do not have the money to pay MPEG-LA consortium which controls H264. On the other hand, the codec proposed by Google and implemented by Firefox and Opera is felt as risky by company such as Apple, Nokia, or Microsoft. The issue is still not yet solved, but there is progress on discussions.
There was a very good summarizing post about it. I will try to find it again.
A good article on how to use the video element in your document
An excellent summary article about the issues
Actually, the HTML5 standards body was going to standardize on OGG (with the ability to use other codecs, as previous HTML versions would have said).
Most of the major players who already had a license with MPEG-4 said "screw you guys, we've already paid for a site license (and are part of the MPEG-4 consortium) so we're going to screw over the standards body".
I want so serve some videos on my site. They are available as .MP4 files gotten from a FlipShare camera.
Now I tried converting them to WMV (which succeeded, but when embedded in html in a <object id='mediaPlayer' width='320' height='285' classid='CLSID:22d6f312-b0f6-11d0-94ab-0080c74c7e95' etc' tag, users have to install an addon and the user experience of 10 Windows Media Players on the site isn't just so good.)
So then I looked at youtube and wondered how they do it, but I can't figure out what format they convert the video to.
So my question is: What format do I have to convert my video to, to show it in a player which does not need to be installed in the users browser. What tool do i need and what is the html code to embed such a video?
As you can see: starting from scratch.
PS: I often hear: This or that file extension is just a container, there can be anything inside. If you're using this in your answer, can you explain this to me, because I never understood this. For me a .cs is a c# file and a .doc a Word file, and not 'a container'.
This isn't an easy question. The basic answer is that you need to use a format that the user's computer already supports. There is no one answer to this question. YouTube encodes videos as MP4 and embeds them in the page with a Flash-based movie player, and Flash is pretty widely supported, but you'll notice that Flash isn't available on a lot of mobile platforms — so anybody using an iPhone is shut out if you go the Flash route.
HTML5 introduces the video tag, which is meant to solve this problem once and for all, but there's still a hiccup even there — most HTML5-enabled browsers support h.264-encoded video, but Mozilla supports Ogg Theora instead. YouTube is currently experimenting with a <video>-based player, so this does seem like the future.
I believe the current best practice to support the most people possible is to encode as h.264, try to serve as a <video> element, and have a Flash-based player as a fallback if that doesn't work (which can play the same h.264 file).
I'd say the most popular solution at the moment - utilized by YouTube and other major video portals - is H.264 encoded Flash-based Video. Flash can play Video since... I think Version 8 or 9, and has since gained significant market share.
My personal favourite Flash player is LongTail Player, but it isn't free for commercial use.
Here's a SO question with a list of Flash based players including open source ones.
Flash won't play on iPhone and iPad, though.
If you want to support computers with Flash Player 9 (I've seen some around, but I don't have any hard numbers) you will need to encode FLV files (which use a codec named Sorenson I think).
The upcoming alternative is HTML 5 Video but suport for that in browsers is nowhere near a major market share.
This question requires a re-answer now that it's no longer 2010 and HTML5 videos (as utilized by most video hosting sites) and chunk-based videos (sent as responses to periodic XHR requests - as utilized by Youtube) are the norm. While there is no best way to add video to your site, Flash is definitely nowhere close to even being good as of the time of writing of this answer.
The simple un-researched answer is: Just use a video tag and it'll work out!
This is simple and intuitive, and should work fine in many of your use-cases.
The researched answer is: Unfortunately, upload the video on Youtube and embed it on your site.
The pros and cons of embedding on Youtube over just having a video tag:
The pros:
Allows you to offer your video in a multitude of qualities.
Very bandwidth efficient. Youtube is bandwidth efficient for your users since it reencodes videos, and is also bandwidth efficient for you since you'll no longer be serving your videos off your own hosting.
Offers features like closed captions, automatic subs, playing at multiple speeds, full screen player, etc.
The cons:
It's a very heavily monetized service, chances are they'll want to put ads on your video if any of its content isn't 100% originally yours.
It has very strict laws/terms and conditions that you need to adhere to, at least in my opinion.
It tracks your users. If your application requires privacy and you can't rely on your users to protect themselves, then Youtube isn't ideal.
Other alternatives that mix the pros and cons of those two options are:
Using a Javascript video library to get HTML5 video along with some of the pros of using youtube, but none of its cons.
Using FFMPEG on the server side, for bandwidth issues.
Using some CDN that supports video to deliver the video, for reliability and bandwidth issues. My current favorites are ones that rely on service workers and the bittorent protocol, to stream from users to each other, but whether that suits you or not depends on your application.
Using AWS storage services to store the video, and AWS gateway/CDN services to serve it, which might be a great solution cost-wise and efficiency-wise if you don't want youtube but don't want to store videos on whatever infrastructure is serving your website.
Sources of this answer: Personal experience. As much as I didn't want to answer from experience, this question really needed a new answer! Feel free to edit it with something more concrete.
Converting your video MPEG-4 with H.264 will get you 97% coverage on current browsers across desktop and mobile, although some Android devices don't support hardware acceleration for this format. To address that you could also serve WebM with VP9 codec.
I wrote up a summary of browser support that might be useful: https://stuartk.com/posts/whats-the-best-html-video-format-to-serve/