I have thousands of mp3 files in a database and a website that lets people listen to them. I'm using a flash player but want to move to html5 audio player. Does this mean that I need to make ogg and wave versions of all my audio files? What is a good approach to making these files more accessible?
In short, yes you need to support multiple formats. (Assuming you care about decent browser support.)
If you are lacking disk space, don't get a lot of traffic, and don't mind some delay before the data gets to the user, you can convert these on the fly. Just write some code so that on request, it checks the conversion cache to see if you have already converted the file. If not, convert it on the fly (with something like FFMPEG) and write the data to disk at the same time you are writing it to the client.
As Imre pointed out, browser support is changing all the time. Look up what codecs are supported from time to time.
Related
I am trying to build a web application which will need to have audio streaming functionality implemented in some way. Just to give you guys some context: It is designed to be a purely auditive experience/game/idkhowtocallit with lots of different sound assets varying in length and thus file size. The sound assets to be provided will consist of ambient sounds, spoken bits of conversation, but also long music sets (up to a couple of hours). Why I think I won't be able to just host these audio files on some server or CDN and serve them from there is, because the sound assets will need to be fetched and played dynamically (depending on user interaction) and as instantly as possible.
Most importantly, consuming larger files (like the music sets and long ambient loops) as a whole doesn't seem to be client-friendly at all to me (used data consumption on mobile networks and client-side memory usage).
Also, without any buffering or streaming mechanism, the client won't be able to start playing these files before they are downloaded completely, right? Which would add the issue of high latencies.
I've tried to do some online research on how to properly implement a good infrastructure to stream bigger audio files to clients on the server side and found HLS and MPEG-DASH. I have some experience with consuming HLS players with web players and if I understand it correctly, I would use some sort of one-time transformation process (on or after file upload) to split up the files into chunks and create the playlist and then just serve these files via HTTP. From what I understand the process should be more or less the same for MPEG-DASH. My issue with these two techniques is that I couldn't really find any documentation on how to implement JavaScript/TypeScript clients (particularly using the Web Audio API) without reinventing the wheel. My best guess would be to use something like hls.js and bind the HLS streams to freshly created audio elements and use these elements to create AudioSources in my Web Audio Graph. How far off am I? I'm trying to get at least an idea of a best practice.
To sum up what I would really appreciate to get some clarity about:
Would HLS or MPEG-DASH really be the way to go or am I missing a more basic chunked file streaming mechanism with good libraries?
How - theoretically - would I go about limiting the amount of chunks downloaded in advance on the client side to save client-side resources, which is one of my biggest concerns?
I was looking into hosting services as well, but figured that most of them are specialized in hosting podcasts (fewer but very large files). Has anyone an opinion about whether I could use these services to host and stream possibly 1000s of files with sizes ranging from very small to rather large?
Thank you so much in advance to everyone who will be bothered with helping me out. Really appreciate it.
Why I think I won't be able to just host these audio files on some server or CDN and serve them from there is, because the sound assets will need to be fetched and played dynamically (depending on user interaction) and as instantly as possible.
Your long running ambient sounds can stream, using a normal HTMLAudioElement. When you play them, there may be a little lag time before they start since they have to begin streaming, but note that the browser will generally prefetch the metadata and maybe even the beginning of the media data.
For short sounds where latency is critical (like one-shot user interaction sound effects), load those into buffers with the Web Audio API for playback. You won't be able to stream them, but they'll play as instantly as you can get.
Most importantly, consuming larger files (like the music sets and long ambient loops) as a whole doesn't seem to be client-friendly at all to me (used data consumption on mobile networks and client-side memory usage).
If you want to play the audio, you naturally have to download that audio. You can't play something you haven't loaded in some way. If you use an audio element, you won't be downloading much more than what is being played. And, that downloading is mostly going to occur on-demand.
Also, without any buffering or streaming mechanism, the client won't be able to start playing these files before they are downloaded completely, right? Which would add the issue of high latencies.
If you use an audio element, the browser takes care of all the buffering and what not for you. You don't have to worry about it.
I've tried to do some online research on how to properly implement a good infrastructure to stream bigger audio files to clients on the server side and found HLS and MPEG-DASH.
If you're only streaming a single bitrate (which for audio is usually fine) and you're not streaming live content, then there's no point to HLS or DASH here.
Would HLS or MPEG-DASH really be the way to go or am I missing a more basic chunked file streaming mechanism with good libraries?
The browser will make ranged HTTP requests to get the data it needs out of the regular static media file. You don't need to do anything special to stream it. Just make sure your server is configured to handle ranged requests... most any should be able to do this right out of the box.
How - theoretically - would I go about limiting the amount of chunks downloaded in advance on the client side to save client-side resources, which is one of my biggest concerns?
The browser does this for you if you use an audio element. Additionally, data saving settings and the detected connectivity speed may impact whether or not the browser pre-fetches. The point is, you don't have to worry about this. You'll only be using what you need.
Just make sure you're compressing your media as efficiently as you can for the required audio quality. Use a good codec like Opus or AAC.
I was looking into hosting services as well, but figured that most of them are specialized in hosting podcasts (fewer but very large files). Has anyone an opinion about whether I could use these services to host and stream possibly 1000s of files with sizes ranging from very small to rather large?
Most any regular HTTP CDN will work just fine.
One final note for you... beware of iOS and Safari. Thanks to Apple's restrictive policies, all browsers under iOS are effectively Safari. Safari is incapable of playing more than one audio element at a time. If you use the Web Audio API you have more flexibility, but the Web Audio API has no real provision for streaming. You can use a media element source node, but this breaks lock screen metadata and outright doesn't work on some older versions of iOS. TL;DR; Safari is all but useless for audio on the web, and Apple's business practices have broken any alternatives.
I'm trying to add a few videos to my website using HTML5. My videos are all 1080, but I want to give people the option to watch in a lower quality if needed. Can I do this without having to upload multiple videos (1 for each quality) without the usage of a server-side language?
I've been extensively searching for this. Haven't find anyone say that it can't be done, but no one said it can either. I am using Blogger as my host, which is why I am can't use server-side languages.
Thank you.
without the usage of a server-side language?
Yes, of course. The client can choose what version of the video to download.
Can I do this without having to upload multiple videos (1 for each quality)
Not practically, no. You need to transcode that video and upload those different versions.
Haven't find anyone say that it can't be done
A couple things to consider... first is that a video file can contain many streams. I don't know what your aversion is to multiple files, but yes it is possible to have several bitrates of video in a single container. A single MP4, for example, could easily contain a 768 kbps video, a 2 Mbps video, and an 8 Mbps video, while having a single 256 kbps audio track.
To play such a file, a client (implemented with Media Source Extensions and the Fetch API) would need to know how to parse the container and make ranged requests for specific chunks out of the file. To my knowledge, no such client exists as there's little point to it when you can simply use DASH and/or HLS. The browser certainly doesn't do this work for you.
Some video codecs, like H.264, support the concept of scaling. The idea here is that rather than having multiple encodings, there's just one where additional data enhances the previous video that was sent. There is significant overhead with this mechanism, and even more work you'd have to do. Not only does your code now need to understand the container, but now it has to handle the codec in use as well... and it needs to do it efficiently.
To summarize, is it possible to use one file? Technically, yes. Is there any benefit? None. Is there anything off-the-shelf for this? No.
Edit: I see now your comment that the issue is one of storage space. You should really put that information in your question so you can get a useful answer.
It's common for YouTube and others to transcode things ahead of time. This is particularly useful for videos that get a ton of traffic, as the segments can be stored on the CDN, with nodes closer to the clients. Yes, it's also possible to transcode on-demand as well. You need fast hardware for this.
No.
I can't fathom how this could ever be possible. Do you have an angle in mind?
Clients can either download all or part(s) of a file. But to do this you would have to somehow download only select pixels of each frame. Even if you had knowledge of which byte-ranges of each frame were which pixels, the overhead involved in requesting each byte-range would be greater than the size of the full 1080p video.
What is your aversion to hosting multiple qualities? Is it about storage space, or complexity/time of conversion?
I have an AVI file that I would like to be played inside Flowplayer. I understand it uses HTML5 which requires movie files to be converted to MP4/OGV, so I was wondering if there was a framework that exists which will convert an AVI file to an MP4 file in real-time (and without necessarily being stored on the server)
...the more I think about this, the more I'm beginning to think this isn't possible. Please prove me wrong.
The video can be transcoded in (sort of) realtime by hardware or even software, but I'ts never a practical aproach since you are spending lot of processing power for each client for each video. This is madness. It's adviceable to cache pages... so videos are needed to be cached.
A simple way is to upload/place the video in a folder in the server, then trigger a transcoding (using FFMPEG) to a file which is the file to be served.
This may be too broad a question, but how is soundcloud actually programmed?
To be more specific,
What language was used to program it?
How does it display the frequency data?
If user uploads a file in a format different from MP3, is it converted MP3 or gets played as is? If former, how does the conversion work?
How does it "graphically" appear on a browser as it does? Is it also HTML 5 thing which I don't know anything about?
I'm a big fan of the soundcloud and couldn't stop wondering how all of these work!
Please help me out :)
SoundCloud developer here,
The API and the current website are built with Rails. For information about the architecture/infrastructure and how it evolved over the last 5 years, check out Evolution of SoundCloud's Architecture. The "next" version of the website (still in private beta) is built entirely with Javascript, and just uses the API to get its data. There's a lot more detail available in Building The Next SoundCloud.
I'm not sure exactly what language/libraries are used to process the audio, but many audio libraries do provide frequency data, and we just extract that.
Users can upload AIFF, WAVE (WAV), FLAC, OGG, MP2, MP3, AAC, AMR or WMA files. The originals are kept exactly as is for the download option, but for streaming on the site, they're converted to 128kbps MP3 files. Again, I'm not sure of the software/libraries, but I'm pretty sure it'd be ffmpeg.
For displaying the waveforms, on the back-end when the audio files are processed when they're uploaded, the waveform data is saved into a PNG file. On the current version of the website, we simply load that file. On Next, the png is processed to get the original data back out, and then it is drawn to a canvas at the exact dimensions needed (which keeps the image crisp). We're currently experimenting with getting waveform data in a JSON format to speed up this process.
I am copying the following info posted by David Noël somewhere else in 2010.
Web tier: Varnish, nginx, haproxy, thin
Data Management: Cassandra, MongoDB, mySQL master/slave cluster, memcached
Web framework: Ruby on Rails
CDN: Akamai and Edgecast
Transcoding/storage: AWS EC2/S3
I'm recording a mono audio stream using a PIC at 8-bit 8Khz and streaming it raw to another microprocessor that houses a web server. I'm currently buffering the data and turning it into a wav file that gets played in the browser. What I'd like to be able to do is continuously stream the audio as it's being recorded without putting a lot of encoding overhead on the second processor. I've been searching but most searches turn up just streaming from a stored file, but since the file size isn't known ahead of time I'm not sure how to do this without the overhead of mp3 encoding.
You may find that simply creating a WAV file (or other raw format) that keeps growing will, in most players/browser plugins, cause the file to act as a live stream. This is, I believe, basically how like Ogg streaming and similar works. Because the player begins playing before it is done downloading anyway, it keeps playing and downloading until the end of the file, but the file has no end so it just keeps going.
Vlc media player can stream flv and many other formats.