Ben Waggoner

Demo of Silverlight 2 scaling quality improvements

benwaggoner — Thu, 02 Oct 2008 21:03:00 GMT

My colleague Donald Karlov did a A/B compare for me showing the quality improvement for scaling between Silverlight 2 Beta 2 and the new Silverlight 2 RC0. Check this detail of a shot from an encode I did of Big Buck Bunny, scaled to 2.5x (640x360 to 1600x900 – a pretty common scaling ratio for full-screen playback on a smaller LCD or a laptop).

Silverlight 1.0 through Silverlight 2 Beta 2

Silverlight 2 RC0 and later

Note the improvements in detail, and the elimination of some stray pixels at the edge of the butterfly wing. Most shots won’t be as dramatically improved, but all will be helped at least somewhat.

And even with the better quality, playback CPU load dropped by 10-15%, due to better perf in the scaler and decoder.

Silverlight 2 RC0 is out

benwaggoner — Wed, 01 Oct 2008 15:01:00 GMT

The first public release canditate (RCo) for Silverlight 2 is now available for download by developers. While anyone can install it, the main goal is for developers to make sure they aren't hit by any breaking changes for Silverlight 2 RTM so they can fix any issues. No one will be auto-updated to this release. Users will be auto-updated to Silverlight 2's final release when it's posted.

All Silverlight 1.0 projects should work fine, but there may be some Silverlight 2 Beta projects that require updates.

For the media folks, there's a couple of new things I want to mention.

There's a new scaling algorithm that's a lot faster and much higher quality - a nice compromise when you can get it!
Additional optimizations have been done for the VC-1 video decoder, so playback will be faster. The gains are biggest for content using B-frames.

Since the default Expression Encoder templates have "always on" scaling and its presets use B-frames by default, the above will provide a nice performance boost for existing content. The above are "always on" features - you don't need to update anything to take advantage of them. So expect smoother frame rates on lower-end machines.

Further information:

EDIT: Added ScottGu link

First MediaStreamSource example is up

benwaggoner — Wed, 01 Oct 2008 14:04:00 GMT

My colleague Larry Olson has posted the first public demo project for MediaStreamSource. This new Silverlight 2 feature enables file parsers and network protocols to be implemented in managed code inside of Silverlight, thus enabling support beyond the built-in native formats. Since MSS passes off the compressed media samples to the native decoders, it should offer nearly the same performance as a native code implementation (parsers aren't that expensive). But since it runs inside of the Silverlight managed code sandbox, support for new formats and protocols can be just added into player's .XAP package file, and will be completely transparent to the end user.

MediaStreamSource is one of the biggest enabling features in Silverlight 2. At IBC, people came up with all kinds of crazy, awesome, and even useful ideas of what to do with it beyond anything I'd ever imagined. Really, the only big limitation is that you can't use UDP packets, as they're not supported by the Silverlight networking stack.

Some people have had trouble finding the MSDN docs about MSS, so I'll list them here as well:

MediaStreamSource Class
MediaStreamSource Members

Expression Encoder Service Pack 1 preview

benwaggoner — Wed, 24 Sep 2008 03:19:00 GMT

James Clarke has a blog post up describing some of the new features in the forthcoming Expression Encoder 2 Service Pack 1.

It's got some great new stuff, and we're demoing it here at Streaming Media West this week.

Read the whole post - there's lots of good stuff in there. A couple of my favories are

A new Silverlight 2 base player using .NET. Among other things, this will enable players that display the video at 100% scale by default, improving quality and performance.
New A/B compare modes. Awesome stuff for high-touch encoding and codec tweaking
Some even more VC-1 advanced options for High Codec Nerditry.
The first public release of Microsoft H.264 compression technology.

Anyway, a lot of good stuff for a SP1. I'm looking forward to it.

Ben's IBC Schedule

benwaggoner — Wed, 10 Sep 2008 16:54:17 GMT

So, if you're coming to IBC, come swing by the Topaz Lounge and say hello. I'll be demoing on the below schedule, alternating between the Silverlight Formats pod and floating between pods answering questions. And yes, we'll be showing H.264 playback in Silverlight at the Formats pod.

The Topaz Lounge is a little out of the way, but that means it's not nearly as deafening as the show floor. Last year we had a complementary coffee/soda bar set up.

Friday, September 12th

11:45am - 3:00pm: Silverlight Formats

Saturday, September 13th

9:15 am - 1:30pm: Floating

Sunday, September 14th

9:15 am - 1:30pm: Silverlight Formats

Monday, September 15th

1:30pm - 6:00pm: Silverlight Formats

Tuesday, September 16th

9:15 am - 12:30pm: Floating

H.264 and AAC support coming in Silverlight

benwaggoner — Tue, 09 Sep 2008 19:09:00 GMT

So, our big IBC press announcement went out this morning. Lots of blog-worthy stuff in there, but as a compression nerd, it's the codec stuff I'm going to talk about first.

The big news is that, yes, we're going to add support for MPEG-4 to Silverlight, in the version coming after the fast-approaching fall release of Silverlight 2. Specifically, this will be H.264 for video and AAC for audio.

Why?

So, why MPEG-4 support in Silverlight? It's pretty straightforward, really. We have customers with libraries of H.264 content they wanted to publish to Silverlight, but didn't want to reencode to VC-1. Silverlight's strengths go far beyond media playback, and customers wanted the choice to deploy a wide variety of existing content within Silverlight. Silverlight aspires to provide as much choice as feasible as to how Silverlight can be authored and delivered. H.264 support is something we'd considered for past versions, but there were higher priority features we needed to deliver first. Silverlight 2 provides us a very rich base for delivering web apps, so we can start spreading our wings a bit.

What?

We've got more technical details we'll be sharing at IBC and later, but today, I'll just quote the details from Scott Guthrie's interview:

PressPass: Will you be showing or announcing anything new at the IBC conference this week?

Guthrie: Yes. At IBC 2008 we will be demonstrating a technology preview of H.264 video and Advanced Audio Coding (AAC) audio playback support in Silverlight, and H.264 authoring using Microsoft Expression Encoder and Windows Server 2008 for delivery. Until now, Silverlight has supported the SMPTE VC-1 and Windows Media formats, as well as MP3 for audio, enabling customers to take advantage of broad support across the Windows Media ecosystem, including third-party tools, service providers and content delivery networks.

We’ve always wanted Silverlight to support a variety of formats, so today we’re announcing that H.264 and AAC support will be available in a future version of Silverlight, which will offer content owners greater flexibility and choice to deliver video and audio.

PressPass: Historically, people have associated Microsoft with VC-1. Does this signal a change in direction?

Guthrie: No. Although we have been working with VC-1 for some time, it’s not widely recognized that Microsoft has also been an active participant in the standardization of H.264/MPEG AVC for many years, and we’ve included H.264 support in several Microsoft products. Microsoft’s Gary Sullivan was the chairman of the Joint Video Team (JVT), which developed the H.264 standard, and he recently accepted an Emmy Award on behalf of the JVT.

PressPass: Does this mean that Silverlight is moving away from Windows Media?

Guthrie: Not at all. This is about offering our customers more choice. Media producers and distributors around the world have enjoyed the high quality, flexibility and affordability of Windows Media formats for over a decade. As a testament to its pervasiveness, Windows Media can be found in almost every conceivable media scenario from desktop home video to feature films and TV broadcasts.

Again, don't read this as as a big change around our strategy for media formats. We've long-supported WMV and MPEG-4 side-by-side in products like Xbox and Zune. As I posted last week, Microsoft has been contributing to H.264 since its inception.

This doesn't represent any change in our support for Windows Media. Windows Media continues to work well for today's Silverlight customers. I expect (not a goal, just a prediction) that the majority of Silverlight content will remain in WMV well after we release MPEG-4 support.

In particular, VC-1 will likely remain the codec of choice for HD for some time. Comparing VC-1 Advanced Profile to H.264 High Profile with all the bells and whistles turned on, VC-1 only needs about half as many MIPS per pixel for playback. This won't matter as much for lower resolution content, or podcasting stuff that's in the simpler Baseline profile, but makes for a big reduction in system requirements for 720p and higher.

Where/When can I learn more?

If you're at IBC, swing by! If you want a head start on using MPEG-4 in Silverlight, feel free contact me directly. And we'll have plenty of more technical info to share down the road a bit.

Technical Emmy for H.264/MPEG-4 AVC

benwaggoner — Thu, 28 Aug 2008 19:05:00 GMT

I have many ways of reminding my wife I'm an hopeless video nerd. Among them is that I only pay attention to the Emmy Awards she's watching when the recap of the technical awards comes on.

So, I was really excited that our own Dr. Gary Sullivan, chairman of the Joint Video Team, was on hand last week to receive a 2008 Primetime Emmy^® Engineering Award for developing H.264/MPEG-4 AVC High Profile. In the words of the Academy:

Joint Video Team Standards Committee (JVT) for the development of the High Profile for H.264 / MPEG-4 AVC. The High Profile enables high definition images in the H.264 video coding system used today to deliver HD video over satellite and cable TV as well as Blu-ray Disc. The JVT is a standardization team comprised of members from the International Standardization Organization (ISO), the International Electrotechnical Commission (IEC), and the International Telecommunication Union (ITU).

The award recognized Broadcom, NTT DoCoMo, Dolby Laboratories, Fraunhofer Heinrich Hertz Institute, Microsoft, Motorola, Panasonic, Sony, and Thomson as contributing companies, as well as standards organizations ISO, IEC, and ITU.

Gary has long represented Microsoft as as chair and co-chair of the Joint Video Team that coordinated the H.264/MPEG-4 AVC effort between ITU-T VCEG and ISO/IEC MPEG, amongst his many other efforts like DXVA; his page on Wikipedia has more details.

I suppose some people might be surprised at Microsoft's inclusion here, but they shouldn't be. While Microsoft has a long heritage of making our own ground-up codecs like VC-1, we've also long been involved in standards-based codecs, including the original MPEG-4 Part 2 as well as MPEG-4 Part 10/H.264. And we've got a number of products that incorporate H.264 today, including Mediaroom, the Zune 4/8 and 80, and Xbox 360. H.264 are and VC-1 are both great codecs, with somewhat different sweet spots, and we use either or both as appropriate in our products, platforms, and services.

The picture above has, left to right: Malcolm Johnson (director of the ITU standardization bureau), Ajay Luthra (JVT vice-chair from Motorola), Gary Sullivan (JVT chair from Microsoft), Thomas Wiegand (JVT vice-chair from Fraunhofer HHI), and Scott Jameson (chair of ISO/IEC JTC 1)

ANSI

EE Times

Dolby

Minoru "Mick" Etoh's blog (who represented DoCoMo at the ceremony)

Alex Zambelli's blog is reborn with Olympics info roundup

benwaggoner — Fri, 15 Aug 2008 19:55:00 GMT

Yikes! No posts since early July. I've been a bad, bad blogger. Fortunately, compression wizard Alex Zambelli's blog has relaunched to take up the slack!

He's started off with a series of Olympics posts.

NBC Olympics 24/7 - a great roundup of information and details

NBC Olympics video without Silverlight? - showing how to play back the Olympics without Silverlight on Windows (and why you'd want to use the Silverlight version if given a choice).

Why no full screen mode in the NBC Olympics player?, citing this silverlight.net forum discussion including information from our own Tom Taylor.

Hopefully he'll inspire me to get back at it...

Silverlight Skinning in Flip4Mac

benwaggoner — Thu, 03 Jul 2008 22:54:00 GMT

Much has been made of our making the Mac a first-class citizen of Silverlight. We build the Mac and Windows versions together, and release them simultaneously. We've intentionally structured Silverlight to minimize dependencies on the underlying hardware and OS, and carry around all our codecs inside the runtime.

What I haven't talked a whole lot about in too long is our support for authoring Silverlight video experiences on the Mac. While our own Expression Encoder 2 is Windows-only, we work with partners to provide WMV and VC-1 encoding on other platform. Telestream has been a great partner here, with their Flip4Mac QuickTime component and their Episode stand-alone compression tool (originally known as Compression Master, and which came along with their acquisition of Popwire).

Flip4Mac is a testament to both the portability of the Windows Media technologies and the flexibility of Apple's QuickTime architecture. It works as a QuickTime component, enabling QuickTime to play Windows Media content like any other supported format, with full access to all of QuickTime's features like hardware accelerated full-screen playback (all supported in the free version). For example, in the QuickTime Pro Properties window:

Looks just like any other QuickTime file. Even the ASF metadata shows up.

Flip4Mac also supports content authoring in the paid version, as well as import into content creation apps like Final Cut Pro. It's got pretty deep control, even exposing features like B-Frames that'd take a registry key with Windows Media Encoder.

So, how about a quick walkthrough through the Flip4Mac encoding settings?

Flip4Mac is accessed like any other exportable format from QuickTime (like MPEG-4). It also comes with a bunch of presets for typical scenarios.

It has the normal set of basic video export features you'd expect, nicely Macified. I appreciate the "Size: Current" so I don't have to always type in the frame size of the source when I'm not scaling.

The Advanced button reveals some deeper options, including a complexity control for speed/quality tradeoffs and B-Frame distance. You can also set Flip4Mac to deinterlace interlaced sources, or pass interlacing on through to the final encode. Since it can both encode and decode interlaced VC-1, Flip4Mac can be a great way to transfer editable 480i video sources over slow connections. It can match DV quality in 20% the data rate.

For Silverlight audio, Flip4Mac supports both WMA 9 Standard and 9 Professional. It doesn't encode the 32-96 Kbps WMA 10 Pro low bitrate modes that Silverlight 2 will be able to decode, but does have the full variety of WMA encoding modes.

And the coolest new feature of Flip4Mac: built-in Silverlight templates! This lets you build a complete Silverlight player (using Expression Encoder templates we provided) directly from any app that support QuickTime export.

Low Latency webcasting with Windows Media and Siverlight

benwaggoner — Sun, 22 Jun 2008 05:25:00 GMT

So, Streaming Media is doing a special "Europe edition" of Streaming Media, and I'm doing an article about webcasting for it. I've getting a bunch of questions about how to deliver low-latency live streaming to Silverlight, and so with their permission, I'm excerpting this section on that topic.

EDIT: A correction was made; turning off Fast Cache in Windows Media Services does not have any effect on webcasting latency.

While by default, Windows Media can offer 15-20 second end-to-end delay, it's possible to drive it down to 2-3 seconds with best practices on a good network, and we're looking at what we can do to push it to below even that. The critical thing is to tune the encoder, server, and player latency together.

There's an older but more detailed "Reducing Broadcast Delay" document on this topic over at the Windows Media portal that may be worth perusing.

Low Latency Webcasting

Latency is the measure of how much time goes between when video enters the encoder and leave the user display. Latency is something that doesn’t matter at all in some markets, and matters a lot in others.

The reason we have latency is buffering, and the reason we have buffering is for quality and reliability. By having the server wait several seconds after a video stream is received before sending it out, it’s able to support more peaks and valleys in the data rate, and makes it possible for a dropped packet to be detected and resent before its needed. In the same way, buffering in the player lets it average out data rates and recover dropped packets as well.

Large buffers were extremely important in the modem era, and are still useful in many kinds of networks today. The defaults are good for delivering high quality content over a variety of networks. But when minimizing the latency is important, and reliable networks are available, the end to end delay can be reduced substantially.

Since total end-to-end latency is the sum of the encoder latency, server latency, and player latency, plus how long it takes the packets to travel between each of those, improving latency requires tweaks to the encoder, server, and player in parallel.

Encoder

On the encoder side, reducing buffer size reduces startup latency. WME and Expression Encoder both enable you to go down to one second.. Using Lookahead or Lookahead Rate Control will increase latency beyond the buffer value (typically about another half a second for LRC).

For audio, Windows Media includes the “WMA Low Delay” audio codecs from 192-64 Kbps which provide lower latency than the normal WMA modes. If you’re targeting Silverlight 1.0 or WMP 10 or earlier, you’ll want to use that for low latency. If you can require WMP 11 or Silverlight 2, you can use the lower delay yet WMA 10 Pro codec at 32-96 Kbps.

Server

WMS features like Advanced Fast Start can dramatically reduce latency for on-demand content, but don’t apply to live webcasting as the server has to wait for content to arrive from the encoder it in real time, and so can only play out at real time.

Server buffering can be turned off entirely in WMS for Windows Server 2003 and 2008, yielding a significant drop in latency.

EDIT:

The previous version of this post suggested that turning off Fast Cache would help webcasting latency. On further research, this turns out not to be the case. Disabling Fast Cache will slow down on-demand startup time, but will have no effect one way or the other for live content.

Player

Normally the player isn’t under control of the streamer. By default WMP dynamically picks an optimum buffer size based on its measurements of network and stream performance. However, it’s possible to lower the buffer size in the player’s options. This can help reduce latency when watching streams with a good connection, but could produce pauses in the video when watching video from the general Internet.

Silverlight makes the player buffer size a controllable parameter, so an optimal setting can be applied for the content, and even adjusted on the fly. This is controlled by the BufferingTime parameter in a Silverlight MediaElement.

Announcing Expression Encoder Customer Panel

benwaggoner — Sat, 21 Jun 2008 06:29:00 GMT

The Silverlight and Expression teams are very customer focused, and we like to release products on a fast cadence. We've got a lot of methods of customer outreach, and I'm happy to share a new one, the Expression Encoder Customer Panel.

David Sayed has all the details over on his blog:

Expression Encoder Customer Panel

Now that Expression Encoder 2 has been released, the team is hard at work on the next version of the product.
We want to be as responsive to community feedback as possible, and would like to set up a panel of customers with whom we can have ongoing conversations.

What is the Customer Panel?

The Customer Panel is a way for us to build strong relationships with users of our product. We'll use it as a way to understand your needs, workflows as well as how you use the product. We'll also use it to float future feature ideas and get feedback on them

What is the time requirement?

The Customer Panel is an ongoing project. The overall impact should be minimal and if at any point you no longer want to participate, just tell us. Typically we'll aim to reach out the Panel every couple of months or so. There may some times when we reach out more frequently.

How is this different to a Technology Adoption Program (TAP)?

TAPs are for specific releases and focus on getting you pre-releases of the software to play with. We'll use the same framework for the Customer Panel, but it is longer lived and goes beyond a particular release. This is an opportunity to have lasting impact on product direction.

OK sign me up!

If you want to take part, please send me an email by clicking good article about NBC Universal''s upcoming Olympics broadcasts in Silverlight.

Read the article yourself, but here's a few choice quotes from it:

"NBCU plans to offer 3,600 hours of live programming from Beijing. That’s 212 live hours for each of the 17 days of the Olympics... In addition to the sheer volume of live content to be delivered—three times what was offered in 2004— what’s notable is that most of NBCU’s live programming—2,200 hours—will be delivered online at NBCOlympics.com."

"The 2008 Summer Games in Beijing will mark the arrival of streaming as a viable alternative to the Olympics’ television broadcast. This summer, NBCOlympics.com will offer 4,400 hours of on-demand streaming in addition to its 2,200 hours of live programming, making the Beijing Olympics the most ambitious streaming media project in history."

"To help meet that challenge, the NBCOlympics.com player offers a “metadata overlay” feature, which allows the player to display transparent data and navigation tools over the video window. This enables users to access statistics and other data without covering up, pausing, or leaving the primary video display. For example, play-by-play announcers’ dialogue can be keyed into an XML data stream, then rendered as a timecoded, scrolling text caption that transparently overlays the bottom of the video display. The player also enables the TiVo-like experience of pausing, rewinding, and replaying content, and these two features together allow viewers to use either the timecode or the play-by-play captioning to rewind to a specific point in the on-screen action and replay it."

"A slew of DRC-Stream software and encoder boards from Canada-based Digital Rapids are being deployed in Beijing to populate NBCOlympics.com’s encoding farm, but other than committing to VC-1, NBCOlympics.com has yet to confirm encoding bitrates, frame rates, or frame sizes. (Without offering more specifics, Miller says NBCOlympics.com will be streaming through a managed bitrate solution to optimize the user’s connection, with a target maximum bitrate of 650KB/sec.) Digital Rapids is also supplying software to enable transcoding from other digital media formats into VC-1.Miller promises hundreds of hours of online HD video..."

Note that there's already a bunch of content up at NBCOlympics.com if you want to get an early taste of what's in store on 8/8/08.

Silverlight 2 Beta 2 is out!

benwaggoner — Sat, 07 Jun 2008 14:39:00 GMT

After months of hard work, Silverlight 2 Beta 2 and its SDK are now available for download!

The best overall description of what's in it is in this post from Scott Guthrie.

Media Features in Silverlight 2 Beta 2

Of course, my focus is on the media features. New to Beta 2 is:

Hooks to enable adaptive streaming

Beta2 adds client support for "adaptive streaming" - which enables managed code to be able to pull in media files and streams from arbitrary URLs, and then reassemble them and pass them off to the video and audio decoders. This can enable a whole lot of different ways to address how to make media available, including supporting seamless streaming switching between content encoded at different bitrates. And since this is all running inside managed code, a CDN or content provider can tune the heuristics used to get the optimal content from the optimum server.

While the API is in place, we don't have any full end-to-end demos for how this works at the moment; you'll be seeing a variety of ways to use this technology down the road. Since our API is so flexible, I imagine customers and partners will find all kinds of fun things to do with it.

Content Protection

Beta2 includes client sideWindows Media DRM 10 and PlayReady DRM support. Both work cross browser and cross platform. Note that WMDRM10 will require a PlayReady license server (available in the coming weeks); existing content will work with Silverlight, but with the new license server.

Server Side Playlists

Beta2 adds improved support for server side playlists on Windows Media Services (previous releases only supported client-side playlists). That's still a feature in development (we are still a beta), so if you have server-side playlists that aren't working as you expect, it'd be great if you could put a link in comments, or email them directly to me so we can see what's going on. Note that web playlists from the IIS 7.0 Media Pack have always been fully supported.

Quality and Performance improvements

While there's no API change here, Beta 2 includes a variety of improvements to both performance and quality (particularly scaling quality) during media playback. Full-screen playback in particular is improved.

Windows Media Audio 10 Professional

And don't forget that Silverlight 2 adds support for the WMA 10 Pro codec. This isn't changed from Beta 1, but it's a big improvement compared to the stock WMA in Silverlight 1. WMA 10 Pro in the 32-96 Kbps range is what we call the "LBR" or Low Bit Rate mode, which is up to 2x as efficient as classic WMA. Audio is quite understandable at 32 Kbps, danceable at 48, an flawless at 64-96. Expression Encoder 2 supports the new modes, and Windows Media Player 11 can play them back as well.

For Silverlight 2-only projects, WMA 10 Pro is the codec of choice for any kind of bandwidth constrained content. And, of course, it's supported in Expression Encoder 2.

Also, note that, like WMA, WMA 10 Pro also supports 2-pass VBR encoding, which can help quality further when doing progressive download. 48 Kbps average 96 Kbps peak can sound pretty great.

I'll try to get up some A/B comparison demos in the next few days.

Expression Encoder SDK is out!

benwaggoner — Thu, 05 Jun 2008 22:01:00 GMT

One of the neat features of Expression Encoder 2 we've been talking about is its .NET object model interface for automation. I'm happy to report that it's now available for download!

Here it is

Sample projects

It includes a number of sample projects (quoting from the help file). These are a good jumping off point for various scenarios.

Simple encode

The simple encode sample is a console application that demonstrates how to encode a file from a console application when showing progress. To use it, just run the built application (Simple.exe) at a command prompt.

Asynchronous encoding

The WPFEncode application demonstrates how to encode a file from a UI application when displaying the progress of the encoding process. To use the application, click the Browse button to point to a file that you want to encode and then click the Encode button to encode it. When the file is encoding, a bar displays the encoding progress.

When you press the Encode button, the application creates a thread to perform the encode to make sure that the UI thread is not blocked. The progress events are called from a non-UI thread. Consequently, we use the Windows Presentation Foundation Dispatcher to marshal the updates to the UI thread.

If you try to close the application when the encoding is still occurring, the application prompts you to confirm that you want to stop the process. If you confirm, the application stops the encoding thread but delays closing the application until the thread actually stops.

MediaInfo

The MediaInfo sample is a console application that demonstrates how you can use the MediaItem class to extract media and metadata information from a media file. To use it, just run the built application (MediaInfo.exe) at a command prompt with the full path of a media file as the parameter, as shown in the following example:

Publishing plug-in

Publishing plug-ins enables you to take video and Microsoft Silverlight template assets that you have created as part of an encoding job and then do something with encoded assets. For example, you can upload the media asset by using FTP, WEBDAV, or METAWEBLOG API, publishing the asset to an asset management system or web service, or creating a data CD.

The following code sample demonstrates how to use a publishing plug-in to add the output of an encode job to a Zip file by using the Open Packaging Convention components found in Microsoft .NET 3.0.

A publishing plug-in is required to derive from the PublishPlugin class, which in turn derives from the EncoderPlugin class:

Template plug-in

The template plug-in lets you customize template settings and interact with the job through the IPluginHost interface. The plug-in also lets you add or remove files from the template output when the template is published.

The following code sample demonstrates how to add the default template parameter control to Microsoft Expression Encoder with only several lines of XAML. It also implements a custom script command creation feature that shows how the template plug-in can interact with the current job by adding a script command at the current play head position of the selected MediaItem.

Within Expression Encoder, every template receives a template plug-in. If the template doesn’t specify one, a default one will be assigned. A template plug-in derives from the TemplatePlugin class, which in turn derives from the EncoderPlugin class.

What a difference a half-decade makes! Live VC-1 today and at launch

benwaggoner — Thu, 05 Jun 2008 02:11:00 GMT

A recent conversation over at the Streaming Media Advanced list sparked a rant from me about the importance of comparing implementations of codecs, not just codecs.

To that end, I thought I'd do a demo to show how much improvement there's been in Windows Media since the launch of Windows Media 9 Series back in 2003.

A recent conversation over at the Streaming Media Advanced list sparked a rant from me about the importance of comparing implementations of codecs, not just codecs.

To that end, I thought I'd do a demo to show how much improvement there's been in Windows Media since the launch of Windows Media 9 Series back in 2003.

Below, you'll find two streams, encoded with the same settings but with tools from different eras. The streams are both

Live encoding from a preprocessed file (so that preprocessing differences don't matter)
And yes, it's the "Lady Washington" footage again.
640x360
29.97 fps
600 Kbps video
64 Kbps 44.1 stereo WMA audio
5 second buffer
Keyframe every 5 seconds

The difference is that the first is encoded with the original Windows Media Video 9 codec (ala just Windows Media Player 9 installed, like a stock Windows XP SP2 machine), and the second with the VC-1 Encoder SDK implementation in Expression Encoder 2. Pretty dramatic differences, I hope!

	Live Windows Media 2003	Live Windows Media 2008
Silverlight embedded page	2003 in Silverlight	2008 in Silverlight
Direct link to WMV (right-click to download)	2003 direct WMV link	2008 direct WMV link

...reminding me that I really need to blog how to make a Silverlight dual media player that can play two versions of the same clip in sync.

So, what makes this big difference? There's been a huge amount of work and three major releases (Format SDK 9.5 and 11, and VC-1 Encoder SDK) since then, so I won't give a complete list, but a few of the highlights.

4-way threading instead of 2-way threading, doubling performance on modern machines.
Lots of SSE2 and SSE3 optimizations to improve performance.
"Adaptive Complexity" that dynamically adjusts the complexity of the encoder, to make sure it's always using all available CPU power, without ever dropping frames. This compares to the old default live complexity of 1 (out of a 0-4 range).
Using B-frames (supported in the old decoder, but not used in the original encoder) which improve compression efficiency and enable efficient encoding of flash frames.
Lookahead Rate Control, where the codec buffers a few frames into the future, so it knows when it needs to start saving some bits for an upcoming keyframe, or when it's save to use a lot of bits on a few challenging frames.

A lot of the above are about performance. With offline encoding, better performance just makes for faster encoding. But for live encoding, it helps quality, because it allows the codec to do more math per pixel to find the optimum way to encode that file.

So, how big a difference is this? Below is a graph showing the Quantization Parameter (QP) for the two encodes. QP is a measure of how encoded each frame is, with higher values more highly compressed. In VC-1, the range is 0-31. A good rule of thumb is that QP of much below 8 looks pretty good, and QP of 8 or above...won't. Now, a live SD encode at 600 Kbps is pretty darn aggressive, so there's plenty of spots where both encodes certainly show artifacts. And quality varies a lot throughout the file as the the complexity of the video goes up and down, as this is a Constant Bitrate (CBR) encode. Note the relatively low QPs near the end of the file, where the easy credits scroll comes in.

But still, the modern VC-1 implementation (in red) with all of the above is dramatically better. While the old encoder (in blue) spikes all the way up to the maximum QP of 31, the new one is typically several QP lower, and maxes out at a QP of 20 instead.

That said, man, my Excel-fu sure has declined over the years. Hard to believe I used to teach classes on making good-looking Excel charts back in the early 90's. Anyway, just remember that lower is better, and red is our current stuff.

Enterprise Deployment resources for Silverlight

benwaggoner — Mon, 02 Jun 2008 16:58:00 GMT

Most of the examples here for Silverlight media have been focused on consumer content, and that's been the bulk of our broader marketing story so far. But we are Microsoft, and so enterprise is a huge portion of our bread and butter, and we want to see Silverlight be as successful inside enterprises as it's becoming in the consumer world.

Silverlight has a lot to offer the enterprise. Among the usual Silverlight scenarios, we've seen growing interest in using Silverlight for doing corporate training, since it offers such rich mechanisms to mix video and audio with quizzes and other content.

I've had a bunch of questions about how we're going to support Silverlight deployment in the enterprise. My colleague Tim Sneath has had a couple of great blog posts I want to crib from. In particular, don't miss the link to the Silverlight Enterprise Deployment Guide.

Deploying Silverlight in the Enterprise

As more and more Silverlight sites become available, enterprise administrators are starting to ask for guidance on how Silverlight should be deployed in a corporate environment. Obviously, at one level it's as straightforward as executing the installer, but there are typically a whole ton of questions that need answering:

What files and registry keys does Silverlight install?

Will Silverlight break any existing applications?

What are the command-line switches to configure installation?

How do I deploy Silverlight through SMS or System Center?

How is Silverlight serviced and updated?

How do you configure Silverlight settings via Group Policy?

Fortunately, one of my colleagues, David Tesar has kindly come to the rescue and written a very comprehensive white paper that answers all these questions plus many more. The Silverlight Enterprise Deployment Guide is available for download now. We gave the white paper a trial run ourselves when Microsoft IT used SMS to deploy Silverlight internally. Download it and send it to your systems administrator today - it's worth their time to read.

Making Silverlight Easier for Systems Administrators to Install

...we've made a few changes to the installation process to enable systems administrators to deploy Silverlight into enterprise environments more easily.
...Silverlight is available as an optional update for Windows XP and Windows Server 2003 via the Windows Server Update Services tool (and, inevitably, Microsoft Update, since WSUS relies on Microsoft Update as its source for updates).

Many customers have requested help with deploying Silverlight internally, and this change will hopefully be welcomed by enterprises who want to distribute Silverlight within their organizations using their established management tools. By adding it as an optional update, enterprises can control the roll-out of Silverlight within their organizations and schedule its installation as a background task so that the perceptible impact is minimal, and ensure that end-users can view Silverlight content without requiring administrative rights to install the plug-in.

I'm really glad this is now available: the need for administrator rights to install a web plug-in is not unique to Silverlight, of course, but it's been raised a number of times as a potential blocker to enterprise adoption. Having this available will solve that problem and make it far easier for enterprise administrators to control deployment of Silverlight to their users.

Direct links to files on Silverlight Streaming (and new All-Stars clip)

benwaggoner — Mon, 02 Jun 2008 02:15:00 GMT

After spending more of my Saturday than I care to remember pulling up Scotch Broom at my parent's weekend place in Sandy, OR, I'm happy to return to some good compression news that's not so nearly hard on the knees.

I've had a lot of requests for a way to directly access the WMV files in my demos I host up on Silverlight Streaming. They've activated "HTTP endpoint for videos hosted in Silverlight Streaming."

URLs come out as :
http://silverlight.services.live.com/accountId/fileSet/fileName.wmv, where:

accountId is your Silverlight Streaming account ID; check the Manage Account page to find it
fileSet is the folder name where your video file is stored; you can see the list of folders in your Silverlight Streaming account in the Manage Videos page
fileName.wmv is the video file name

For a demo, I've finally gotten around to fixing the audio sync for the Streaming Media All-Stars demo, in prepration for the article Streaming Media Magazine will be publishing about the workflow.

So, by example, here's the Silverlight link:
http://silverlight.services.live.com/invoke/31260/SMAllStarsFix/iframe.html

And the link to the file directly:
http://silverlight.services.live.com/31260/SMAllStarsFix/smAllStars_audiofix.wmv

In other news, they've also got a new "Robust uploader for Silverlight Streaming videos" for Silverlight 2 Beta users. It allows pauses and resumes of video uploads. I don't use the uploader myself much (my whole reason for existing being largely based on doing my own encodes). However, one thing a lot of folks don't know about the uploader is that it won't recompress files that meet the basic Silverlight Streaming specs (something like 640x480 and 1.4 Mbps or less).

Videos from the "Secret Compressionist's Ball"

benwaggoner — Thu, 29 May 2008 03:39:00 GMT

I've always called my annual compressionists party something like the "Ben Waggoner Compressionist's Party" but David Sayed from the Expression Encoder team suggests the much better name of the "Secret Compressionist's Ball"

Better yet, he's got a couple of video interviews from the party! You'll see me in the background in my mildly famous purple shirt in a few shots.

First, Tony Houghton from Promoscape, talking about Silverlight.

Second, Bruce Lidl from MainConcept talking about their VC-1 codec implementation they're now licensing out, including support on Mac and Linux.

The clips are embedded at smaller than they're encoded; double-click to see them in their full glory.

Ben at Streaming Media East

benwaggoner — Mon, 19 May 2008 15:37:00 GMT

I'm at Streaming Media East this week in New York.

On Monday I'm doing a three-hour workshop on "Deploying On-Demand and Live Media Experiences with Microsoft Silverlight."

My handouts are available here.

I'll also be at the Microsoft booth on Tuesday and Wednesday.

Tuesday:
10 am - 12 pm
2 pm - 6 pm

Wednesday:
10 am - 12 pm
2 pm - 4 pm

Expression Suite 2 and Expression Encoder 2 now shipping!

benwaggoner — Thu, 01 May 2008 22:43:00 GMT

May 1st is a happy day for me! I went over to www.microsoft.com/expression, and was delighted to find that the Expression Studio 2 is now shipping, including the final version of Expression Encoder 2. There's a 30-day trial available for download. Boxed copies of Studio are available today at CompUSA and Office Max, with other retailers coming. There will also be the Microsoft Expression Professional Subscription for $999/year (academic and volume pricing coming soon), which includes Expression Studio along with Visual Studio, Office, Visio, XP, Vista, Virtual PC, and Parallels Desktop (to run all of the above on Mac).

As I've said before, I feel that Expression Encoder 2 is the best compression tool at fulfilling the needs of its market since Terran Interactive's Media Cleaner Pro 3.1 (so long ago I can't even find anything on the web about it!). Here are some of my favorite things about it from my obsessive compression nerd's perspective.

First desktop priced VC-1 Encoder SDK based product ($199).
And the VC-1 implementation gives you great quality by default, and gives you an advanced mode that accesses the useful features, while handling the more esoteric ones behind the scenes. For example, the rule for WMV PowerToy was use Static Motion Vector Cost if using P-frame DQuant, otherwise use Dynamic. So Expression Encoder 2 just has a DQuant setting, and uses the right Motion Vector Cost based on that.
High quality scaling and deinterlacing modes
WMA Pro support, targeting Silverlight 2's support for that. So danceable music at 48 Kbps.
"Smart by default" - you can use the same preset and feed it a PAL 16:9 interlaced file and a NTSC 4:3 progressive file, and it'll do the right thing without having to do a single click: you'd get a 640x360 deinterlaced 25p file and a 640x480 29.97 file.
Which means you don't have a huge combination of settings to deal with all the video formats. Instead, the presets are scenario based; you just need one "1 Mbps streaming" option that'll handle all the video file variants.
Great multi-clip templates, so you can make a video gallery, with each clip having its own optional thumbnail-based chapter navigation (hmm, that deserves a nice demo).
Easy creation of thumbnail-based chapter navigation!
Captioning and metadata support, including ISAN.
Available soon, an integrated publishing module to go straight to Silverlight Streaming.
Support for a wide variety of source formats, including QuickTime, AVI, Windows Media, VOB (DVD), many other flavors of MPEG-2, MPEG-4, and AVISynth.
Automation available via both command-line and a .NET object model.
Simple editing features, like in/out points, preroll/postroll videos, and image and animated overlays.
GPU acceleration for previews
A great A/B compare feature, enabling short sections of the file to be encoded with a variety of compression settings, and then compared (playing in real time! zoomed in!, with an A/B split screen slider!) to both the source and each other.
An unbelievably cool live encoding module, which supports multiple cameras with live switching, streaming from files, including looping, live metadata insertion, and big quality improvements from lookahead rate control and dynamic complexity. I owe you a blog post describing that as well.

So, in summary, I guess I'd say I like it.

I've had a some blog posts highlighting different projects done with Expression Encoder 2 which you can follow along with.

Hands on with high-touch encoding: Streaming Media All-Stars Redo

Encoding screen recordings for Silverlight in VC-1 with Expression Encoder 2

"What Happens in Vegas" - 720p Movie trailer at 2 Mbps via Silverlight Streaming

I've got other information about Expression Encoder and how it fits into Silverlight at

My NAB Presentations

Silverlight Media technologies overview in Expression newsletter

And, of course I'll be covering Expression Encoder 2 as one of the products we cover in my

My June 23-27th class at Stanford

Hands on with high-touch encoding: Streaming Media All-Stars Redo

benwaggoner — Wed, 30 Apr 2008 06:47:00 GMT

Introduction

As Expression Encoder 2 approaches its immenent release, I've been using it for more and more real-world projects. This recent one was particularly chewy fun, and I thought it would make a good tutorial for a high-touch workflow.

As you may remember from a few weeks ago, I was one of the inaugural class of Streaming Media's Streaming Media All-Stars. There was a fun video montage of all of us on baseball cards being announce by ballpark-style narration. Good stuff, but the FLV compression wasn't quite up to my standards for this rare intersection of compression obsession and personal vanity. So I contacted Streaming Media and asked if I could take my own whack at it.

I'll have an expanded version of this post as an article in an upcoming issue of Streaming Media Magazine. If you don't get it, you can sign up for a free subscription.

The Source

One thing I noticed in the original is that the background graphics and a few of the animations were interlaced, as you can see in the last "before" image at the very bottom of the page.

While deinterlacing it may have been possible, the heavyweight motion-adaptive deinterlacers available for technologies like AVISynth can be finicky to configure, and extremely slow. And in the end, nothing beats getting the source fixed in the first place. Compression is the art of getting output that's as close to the original as possible with the bits you have available; often getting access to higher quality sources can provide a much bigger improvement to final quality than all the codec tweaking in the world.

So, I contacted the post house, and they fixed the background interlacing (it was just a matter of properly flagging the source as interlaced in After Effects) and re-rendered it for me as a lossless RGB PNG codec QuickTime .mov file. However, there were two shots that snuck through where one layer was still interlaced. I didn't want to wait for another disc, so I dived into After Effects (in the end, all difficult preprocessing jobs seem to wind up in After Effects). I used the "Reduce Interlace Filter" with a softness of 1 to blend the two fields together. Traditional deinterlace methods messed up the text on the cards too much. However, the softness increase from that filter wound up causing a slight visual discontinuity when it kicked in. So, I broke out the two shots with interlacing into layers, and then used a five-frame cross-dissolve transition from the original progressive frames to the start of the interlaced shot which hid the slight loss of focus (masked in part by the motion). Both interlaced shots ended on a hard cut, so I was able to switch back to the original video without a transition.

I then rendered the new version out from After Effects in 32-bit float (to reduce the risk of introducing banding via an 8-bit to 8-bit conversion) into the Lagarith codec in YV12 mode, which uses the native 8-bit 4:2:0 colorspace of VC-1 and other codecs. This means that Expression Encoder doesn't need to do any color space conversion, making compression slightly faster.

Markers

The other notable issue with the original clip was "keyframe popping"; when an obvious "jump" in the video happens at the keyframe rate of the video. Watch the original FLV, and you'll see it during any of the longer static shots. Since the whole section with the cards is one single long shot over 3 minutes long without any hard cuts, there wasn't a place for natural keyframes (automatically inserted at a hard cut) to go. Thus keyframe transitions would happen while the cards were otherwise static, making even a slight change visible.

I also wanted to show off the Expression Encoder templates a bit by doing thumbnail navigation. In EEv2, I'm able to graphically set markers on particular frames, and set them to be keyframes and/or thumbnails. A thumbnail becomes an image file which, with the supported templates, automatically gets included in the menus for navigation (think a chapter on a DVD). Normally you also want to make the chapter points keyframes, since keyframes support immediate random access, as no other frames need to be decoded before displaying a keyframe.

This was an opportunity to kill two birds with one stone; if I set the markers on the first static frame of every card, it'd be nice high quality image that all the later frames that reference that I-frame can be based on, propagating its quality forward. If I set my keyframe spacing long enough, there wouldn't be any other keyframes in that interval to cause keyframe popping, and so the static card would be very consistent.

So, I set a marker for each person, flagged to be both a thumbnail and a keyframe. The audio doesn't always sync up exactly so that the person's name begins after their card is down, so sometimes the first name is cut off. This would have been easy to fix by just delaying the audio a second.

You can also use non-thumbnail keyframe markers; these become keyframes without showing up in navigation. I stuck a few of those in as well in the intro/outro sections, on the first full frames after the logo gets built. Since the sponsor pays the bills (Ripcode in this case), I always want to make sure that logos remain nice and crisp.

Setting keyframes has been around in compression projects for ages now; I did a lot of this in Premiere 4.0 for Cinepak encodes in the pre-Media Cleaner days, since Cinepak was prone to keyframe popping issues. Modern codecs like VC-1 do a much better job of finding good natural keyframes, and also to reduce popping issues. The Silverlight version would have looked a lot better than the Flash even if I hadn't set them, but they did get a further boost in quality. But don't think this is something you should be doing in every case; this clip is unusual in having minutes without cuts with a mix of static and moving elements, at an extremely low bitrate.

Encoding Settings

Now, what encoding settings do we want to use?

Video

Frame Rate: Source. We want to capture all the motion in the source perfectly (29.97 frames per second in this case).
Keyframe Interval: 15 seconds. The longest gap between markers in the source is a hair left than 15 seconds, so this will prevent keyframe popping between cards.
Profile: VC-1 Advanced Profile. So we can use DQuant, as discussed below.
Mode: VBR Peak Constrained. This is a progressive download project, so VBR Peak Constrained gives us optimum quality.
Bitrate (average): 488 Kbps. Matching the original FLV's actual bitrate (400 Kbps was requested, but VP6 overshot by over 20%).
Peak bitrate: 896 Kbps. So video + audio + overhead (9 Kbps in this case) max bitrate is a consumer broadband friendly 1000 Kbps total.
Peak Buffer Size: 15 seconds. So the buffer duration can contain an entire Group of Pictures (a keyframe and frames that reference it).
Width and Height: 640x480. Same as source. The original project had both 320x240 and 640x480, but they used the same data rate, so I'm doing just the 640x480 and Silverlight embed can be set to the desired size.

Audio

Codec: WMA. We're targeting Silverlight 1.0 compatibility, so WMA Pro isn't an option
Mode: VBR. Always better quality for progressive download.
Bitrate: 48 Kbps. Matching the data rate of the FLV source. Also this is minimum bitrate for WMA VBR. I always try to use at least 48 Kbps for WMA progressive for that reason; it's a massive quality jump from 32 Kbps CBR for typical content.
Sample Rate: 44.1 KHz. Same as source. Also, 44.1 is the native audio rendering mode for Silverlight, and so offers the same quality and better performance versus 48 KHz.
Bits per Sample :16. The only option for WMA
Channels: Stereo. VBR audio requires stereo. I'd use mono if I needed to do 32 Kbps, since there isn't stereo separation important to the experience here.
Audio peak bitrate: 96 Kbps. Again, so total peak comes out as 1000 Kbps. The audio isn't that difficult or variable, so higher likely wouldn't sound any different.
Audio peak buffer size: 1.5 Kbps. The default is almost always fine.

Advanced Codec Settings

Video Complexity: Best (5). It's a short clip at a reasonable frame size. Complexity 3 probably would have been just as good, but the encode only takes about 12 minutes at 5, so I didn't bother doing anything less than the max (I love my new 8-core workstation!).

Perceptual Optimizations

Adaptive Dead Zone: Conservative. The normal default. It softens out edges that might ring or get too blocky, but not by too much. I tried both Off and Aggressive, and Conservative definitely looked the best, as usual.
DQuant: I-Frames Only. There aren't many I-frames (mainly the few dozen we set manually, and perhaps a few more natural ones), but they contain the important visual data of the faces on the cards, so we want them to be as high quality as possible. DQuant spends too many bits on smooth parts of the image to use on every frame, but upping the bitrates on a few dozen I-frames won't hurt quality much, and improves the quality of the static parts of the card we wind up staring at for those many seconds.

Filters

In-Loop: On. Always on unless using Simple Profile; it helps reduce artifacts and improve quality, particularly at these aggressive bitrates
Overlap: On. Further hides artifacts, which are a challenge with motion graphics at such a low bitrate
Denoise: Off. The source doesn't have a hint of noise. If there were a lot of textures, Denoise can help to soften them some for easier encoding, but there's not much texture either.
Noise Edge Removal: Off. This is really only useful for noisy edges of analog captures, and even then we're better off cropping. It obviously doesn't apply here.

Group of Pictures

B-Frame Number: 2. Normally we use 1 for film and video sources, but for this kind of motion graphics, 2 is more efficient. This gives us an IBBP pattern, so each B-frame is adjacent to a P-frame. 3 B-frame is less efficient in this case, probably since with the IBBBP pattern the middle B-frame is two frames away from a reference frame (only I and P frames can be reference frames), and the P-frames are too far apart and so require more bits to store the change over four frames instead of three since the previous I or P frame. Using 2 also gives us better random access than 1, since worse-case random access time is based on the maximum number of P-frames between I-frames. With 15 seconds between keyframes at 30 fps, that gives us 450 frames per GOP maximum. With 2 B-frames, that'll gives us 149 P-frames per GOP, the same (and thus the same random access) as if we had a 5 second GOP without P-frames (the old Windows Media Encoder default).
Scene Change Detection: On. This will give us natural keyframes where need them. The codec seems to do a good job of putting them in the right place. I've never changed this in EE.
Adaptive GOP: On. Always have this on.
Closed GOP: Off. This is required to be on for CBR encodes in EE, but slightly reduces quality with VBR encoding. In particular, it can increase keyframe popping, since an Open GOP pattern starts with B-frames before the first keyframe/I-frame, you get BBIBBPBBP..., with the B-frames able to reference the last P-frame of the previous GOP. This helps smooth over changes between GOPs, since you have the leading B-frame(s) to spread the change over.

Motion Estimation

Chroma Search: Full True Chroma. Motion Graphics is a canonical time we want chroma search. The encode is so fast, there's no reason to not go for the full meal deal and do Full True Chroma.
Match Method: SAD. For this kind of content with very simple, flat areas, the Sum of Absolute Differences Motion Match is actually both higher quality and faster than either Hadamard or my normal video/film default of Adaptive.
Search Range: Adaptive. The smallest range works for most of the frames, but there's some very fast motion when the cards zoom in which need the bigger range. Adaptive it is.

Output

The Output pane has some of my favorite usability features of Expression Encoder, letting us apply rich templates and automatic publishing.

First, the Template. I picked the "Clean" template, which has a nice subtle overlay control, and a popup navigation via the thumbnails we made above when you mouse over the top of window. It also supports going full screen with a double-click. One thing I like about Clean is that the video fills the frame exactly, without having to account for the control bar or other elements. So I can embed at exactly 640x480 for a 640x480 clip.

The publish mode (I've got the optional Silverlight Streaming publishing plugin installed) lets me automatically or manually upload the final project to our Silverlight Streaming service. This is a great way to test or deliver Silverlight projects. You can sign up for a free account with 10 GB of storage and 5 TB/month of bandwidth.

Before/After

So, how much did all this help? Here's a couple of the more pronounced before/after shots. All the below are inserted as 100% scale PNG, so there's no scaling or further compression to complicate the comparison. Note that the FLV came out darker for some reason. I'm not sure what the cause of that was; the VC-1 brightness matches the source. Perhaps something to do with the Mac/Windows gamma difference on the platform the FLV was encoded on? This actually makes VC-1's job relatively harder, since the motion graphics are easier to see.

And you can see the actual clips in action here:

Before: FLV VP6 in Flash

After: WMV VC-1 in Silverlight

Detail improvements

I grabbed a frame right after the transition that really shows the detail difference between VP6 and VC-1 here; it's especially striking in the texture of the shirt. The VP6 gets sharper after a keyframe pop, but this is how it starts. VC-1 quality in the card is maintained perfectly throughout.

FLV VP6:

WMV VC-1:

Deinterlacing improvements

In this frame (man, do I look like a stiff!), you can see the effect of my blend deinterlace to hide the fields. Notice the ringing artifacts in the original frame. Encoding fields as progressive is extremely challenging for codecs, since you have high motion 1-pixel high horizontal lines, combing high frequency and high detail. I normally don't like doing a blend, since those double-images are also hard to encode, but it was only for a very short duration in this clip, and the deinterlacing filters I had handy had a lot of trouble preserving the text perfectly.

FLV VP6:

WMV VC-1:

Silverlight Media technologies overview in Expression newsletter

benwaggoner — Mon, 28 Apr 2008 16:47:00 GMT

In this month's Expression Newsletter, I've got an article giving an overvew of Media Technologies in Silverlight.

There's not much in there that hasn't been discussed elsewhere, but it's a nice one-stop overview of what's available in 1.0 and coming in 2.

"Silverlight to shine in NBC's Olympic's coverage" on News.com

benwaggoner — Fri, 25 Apr 2008 16:16:00 GMT

News.com has a nice article by Victoria Ho up today about how Silverlight is going to be used for NBC's Olympic's coverage. It's based on an interview with Johnathan D. Miller, NBC's Senior Vice President, Sports Programming.

Some bits indicating the scope of what we're working on here:

NBC intends to broadcast and archive 17 days of the Olympics on the site, resulting in 2,200 hours of video which users may watch live and call up on-demand for free.

Miller sees this as a complementary repository to what the broadcaster will air on TV. "We're talking about doing 25 sports online that we aren't really doing on broadcast. There will be significantly more content online than on air," said Miller, adding that he has observed strong demand from fans for on-demand content.

...
Since the prototype, the team has focused on tweaking the user experience of the player, including the navigation interface, MBR (managed bit rate)--which matches the quality of the video to the user's connection speed--and metadata tagging of video streams to make searching and archiving easier.
...

On NBC's choice of Silverlight as a platform over competitors, Miller noted NBC's existing partnership with Microsoft in MSNBC, but added that Microsoft is a "holistic partner", being able to offer its services as a "media and technology company, not just a technology partner".

NBC is also working with a Los Angeles-based digital media agency, Schematic, to develop the user interface. Schematic CTO Matthew Rechs told CIO magazine last month that some 25 people were working on the project at the agency.

It's great stuff, and certainly the biggest, coolest project I've ever been involved with (not that I'm doing that much of the heavy lifting personally at this point). It's amazing how far things have come since the 1998 World Cup when I was struggling to get 320x240 web video clips to look good enough for Nike's web site for the event while still offering decent buffering times on 56K modems.

Sample Encoder Test Clips

benwaggoner — Wed, 23 Apr 2008 02:24:00 GMT

I've been working for what seems like forever on a good 1080p24 film-sourced content with clear rights that can be used for codec testing by the broader community.

Since I've finally realized that a big purpose of a blog is an auxilliary memory for myself, and to have a place to point people towards things I get tired of typing over and over, here's two test clips showing what's available.

Both of these are from a shoot on the Lady Washington shoot a bunch of interepid Microsoft employees and and a hand-picked crew did before my time at the company. It was shot on a mix of 35mm and 1080i. There's also a nice if not incredibly challenging 5.1 orchestral soundtrack with it. Being of a tall ship on the water, it's got lot of good imagery to stress gradient preservation (sky) fine random details (water) and sharp diagonal lines (rigging).

Both of these comps are 960x540 QHD ("Quarter HD") @ 3 Mbps CBR, for relatively easy playback off Silverlight Streaming. I'll get around to doing full-rez encodes and the source clips down the road (we'll see if Microsoft.com lets me host 20 GB of downloads)...

Clip 1: 1080i
This is from 1080i30, obviously. And just as obviously, this one was edited and color-corrected by an actual professional, not me. A good test for HD interlaced compression (note the QHD sample is actually deinterlaced to 540p30). I wouldn't do anything else to it to make it a test clip. The usage rights for the clips are on the final frame.

Clip 2: 1080p24
This is from 35mm film sources. And it was thrown together in a few hours, sans any color correction, by myself (not an editor!) for a rush job to get some DLNA test bitstreams together. There's a thread over at Doom9 discussing what we should do to this to make it a better codec test clip for feature film content.