So that your browser vendor can force their philosophical decisions about file formats on their users. That's basically it.
Security is supposedly an issue, but "clicking on a video file in your browser to play it" is no easier than clicking "open file" in a download prompt -- if there's a vulnerability in your media player, attackers don't need <video> to get to it.
I think you may be slightly biased here :). A security exploit that requires clicking is a lot less serious than one that happens without user intervention. Since <video> and <object>/<embed> can play automatically, it makes sense to limit the tags' exposure to unknown code from the operating system's installed codecs.
Are you sure every codec on your system is kept up to date against the latest exploits? Every single one?
Which is updated more often... Your browser or your codecs?
Browsers are taking over certain functions because the OS side (both OS X and Windows) are a mess. I imagine it makes browsers easier to test too not having even more dependencies over the quirks of whatever dot release of the OS you are running.
Better yet, why do my codecs have access to anything on my computer other than their input stream and output stream. They probably want to run on their own core anyway, isolate them and forget about them.
(Ok, I'll give you an H.264 or MPEG-2 being tied into my video hardware for great performance and low power consumption, but the rest of them…)
I'm going to only cover the desktop/laptop space. This approach actually makes a lot of sense in the smartphone/tablet market, especially with all the multi-core chips coming out.
It's true that the bare minimum that a codec needs to do is take an input bytestream and output it to the various output(video and audio) bytestreams. So, for a hardware accelerated video, this is pretty simple-all you need to do is pipe your input stream to the hardware, and that will handle all the magic of getting it directly out to the screens and speakers. However, for non-native hardware accelerated codecs(i.e. not H.264 or MPEG-2), this process involves a lot more hardware and software to actually get something to the screen.
The non-native hardware accelerated codecs need access to(at a bare minimum) the sound and video system APIs. Unless the codec developers are certifiably insane, the codec will be using DirectX(for Windows) or OpenGL(for everything else) to handle the actual writing to the screen, especially if you want to use the video hardware for faster decoding. While it's possible for browsers to wrap those API calls, it just makes a lot more effort on all developers involved, and doesn't really get you much security at the expense of both complexity and performance. So it makes sense for any codec to have access to the lower level APIs for displaying things to the screen.
Sandboxing the process should be possible for the file inputs, but I'm not sure if it's really possible for file outputs in the cases where the codec is using a system-level API.
EDIT: After reading what I wrote, this is a good argument for the browsers themselves wrapping the codecs and placing them into a sandbox. Unfortunately, that will never happen, because of the legal minefield that almost all have right now...
Actually, the codecs themselves usually aren't responsible for displaying to the screen via DirectX, OpenGL, etc. In the case of <video>, the browser needs to be able to mix the video into the web page, which means the browser needs the video data in its own memory, or as a pixmap/texture in video card memory.
But, most codecs are going to be written in C and assembly language for speed, which brings the potential for buffer overflows and other low-level exploits. Plus, that video data does eventually make it to the kernel and then the video hardware (often via a separate overlay interface like DirectShow, Xv, or VDPAU, though that is probably not the case with web browsers), so a vulnerability at any point along the chain is a serious issue.
So they don't have to deal with inconsistent behaviors on different platforms. You can pretty much mess up your os installation anyway you would like, but bundling ffmpeg, libtheora or libvpx and you no longer need to worry about different platforms or configs
What precludes bundled libraries from having support for hardware acceleration? Firefox 4 accelerates full-screen WebM videos on Windows and Mac, for instance.
You need platform-specific APIs to give you direct access to the relevant hardware, which you then need to be able to use efficiently.
I'd rather see applications just feed the bytestream to the OS to decode using the best means available, even if it means support for new codecs is a bit slow. (Although an OS-level codec plugin architecture like QuickTime Components can get you software decoding for new or obscure codecs.)
If my GPU has a high-quality, high-speed H.264 decoder, I shouldn't have to worry about which browsers will use it. They should all default to the OS-provided decoding and playback methods unless there are sound technical reasons not to. If I want to use a new codec that's not supported by the fixed-function decoder, I shouldn't have to worry about which web browsers have bothered to include the necessary OpenCL code to decode on my GPU. I should just be able to install one codec that hooks in to the OS's media framework, and all browsers should use that first.
You need platform-specific APIs to give you direct access to the relevant hardware, which you then need to be able to use efficiently.
Yes, and Firefox already has platform-specific Direct3D and OpenGL backends. Remember that a browser today needs to be able to accelerate not just video but all content and compositing, so they already have to put in that investment. The investment in video is just an additional delta.
I'd rather see applications just feed the bytestream to the OS to decode using the best means available, even if it means support for new codecs is a bit slow. (Although an OS-level codec plugin architecture like QuickTime Components can get you software decoding for new or obscure codecs.)
Hm. "I'd rather see applications just feed the HTML to the OS to render using the best means available, even if it means support for new HTML features is a bit slow. (Although a engine-level plugin architecture like ActiveX or NPAPI can get you some sort of rendering for new or obscure features.)" How is your assertion materially different?
If my GPU has a high-quality, high-speed H.264 decoder, I shouldn't have to worry about which browsers will use it. They should all default to the OS-provided decoding and playback methods unless there are sound technical reasons not to. If I want to use a new codec that's not supported by the fixed-function decoder, I shouldn't have to worry about which web browsers have bothered to include the necessary OpenCL code to decode on my GPU. I should just be able to install one codec that hooks in to the OS's media framework, and all browsers should use that first.
HTML is a web technology, but video codecs are needed by a wider range of applications, including playing optical media, games, and presentation software.
The non-political arguments the mozillazine article makes against using the OS's media frameworks can be solved by using ffmpeg/etc. as fallbacks, except for the one about having to support multiple rendering paths (and even that could be solved by just packaging ffmpeg in the OS-supported codec format).
If a web browser maker wants to make it hard to use H.264 even when the user already has a fully-licensed decoder as part of their operating system, then they are putting their political goals above the users' needs.
If I want my browser to supplant my operating system, I'll use ChromeOS. Otherwise, I want my browser to play nice and not bring along political baggage.
I don't want my browser to include it's own unoptimized codecs, it's own half-assed OpenGL implementation, it's own font rendering code, it's own widget library, it's own PDF viewer, and it's own device drivers. I want my browser to include a kick-ass [X]HTML+CSS+JS engine, and delegate the rest to the operating system like any sane application.
That doesn't really matter. There's basically no usability difference between:
"You need to install plugin X to view this video."
"You need to install QuickTime Component X to view this video."
"You need to install third-party Firefox codec X to view this video."
Except that the third piece of software likely won't exist, and Mozilla might not even let it exist.
If browser vendors make it hard to install and use a codec for patent-encumbered formats like H.264, that will only prolong the use of even worse plugins like Flash for the purpose of playing H.264 videos.
HTML is a web technology, but video codecs are needed by a wider range of applications, including playing optical media, games, and presentation software.
But inside the browser it's a web technology. That's the whole point of the video tag.
When a browser's raison d'être is "political" as you put it, then of course any such decision is going to have a moral component. It isn't just about short-term support -- it's about a long-term vision for what the web should be like, and more importantly shouldn't be like. H.264 isn't compatible with that vision.
they are putting their political goals above the users' needs.
But those goals are the users' long-term needs. The minor loss if people can't play H.264 video right now is offset by the much greater victory if the web stays free and open.
I don't want my browser to include it's own unoptimized codecs, it's own half-assed OpenGL implementation, it's own font rendering code, it's own widget library, it's own PDF viewer, and it's own device drivers.
Stop making ridiculous blanket statements. This is much more nuanced than that, and involves balancing the costs of including custom code with its benefits.
Is there any valid technical reason for browsers to handle media files in any way other than wrapping OS provided codecs?