Dynamic Flash metadata injection with PHP

So, between the summer lull in industry news and my own globe hopping, the blog has been pretty sparse lately. I think things will now begin to normalize, and I figured I’d start it off with a bit of info about a project I’ve been working on.

First, some history. Streaming flash video normally requires an expensive Flash Media Server license. Otherwise, you’re limited to progressive download from http. Some very clever folks have, in the past, figured out how to do streaming over http using php. They accomplish this by adding a bunch of metadata in the flv file which cross references keyframe timestamps with byte offsets. So, if you have a keyframe 50 seconds into your video, they’d include where in the physical file data to find that. This allows you to seek in the video file without having to decode the whole flash video stream – you just need to read the metadata and do an fseek to an appropriate offset.

It gets slightly more complex, in that you need to also rebuild a proper flash header to that the file still has the right format, but for the most part that’s not too difficult either.

There are a number of projects out there to make this pretty easy. Start by taking a look at the phpstream project. These solutions all require you to do the metadata injection with a separate tool, either the closed-source and Windows-only flvmdi, or the open-source, ruby-based flvtool2. Neither of these is optimal if you’re looking to either do injection dynamically, or integrate injection into an existing automation workflow.

Luckily, there is another project, someone neglected, called flv4php. It implements many of the necessary FLV-parsing routines in php, allowing you to build your metadata array directly within php and write it back to the FLV file, or store it separately. You can even do this at runtime, if you’re so inclined. I’d recommend against that particular approach if you’re dealing with long flash videos, as there is a significant amount of processing overhead involved.

If you browse around the source, you’ll find a php4 and php5 version of flv4php. The php4 version has many more features, but the php5 version has sample code for implementing metadata injection. Take a look at the test2.php code to get started. However, for long files, replace the line that says

$ts = number_format($tag->timestamp/1000, 3);

with:

$ts = $tag->timestamp/1000;

To prevent php from adding commas to your timestamps and thus breaking flash.

The test2.php code writes out a .meta file, containing the metadata for your video. That leaves the issue of how to read that data back in and inject it appropriately. That code is below.

To actually view streaming FLVs, you can use JeroenWijering’s free flv player. Take a look at the source if you’re curious how it all works on the client side.

Anyways, here’s how to playback a .meta file in conjuction with an FLV file. $streamPos is the offset within the video that we’re seeking to (it comes from Flash as a byte offset already). A bunch of this was stolen from the php4 tree of flv4php.

$fp = fopen( $targetFile, ‘r’ );

header(‘Content-type: flv-application/octet-streamn’);

header(‘Content-Disposition: attachment; filename=”‘ . $fakeName . ‘”‘);

header(“Pragma: public”);

header(“Cache-Control: must-revalidate, post-check=0, pre-check=0”);

fseek($fp, 0);

$FLV_HEADER_SIZE = 9;

$hdr = fread( $fp, $FLV_HEADER_SIZE );

fseek($fp, 0);

$bodyOfs = (ord($hdr[5]) << 24) + (ord($hdr[6]) << 16) + (ord($hdr[7]) << 8) + (ord($hdr[8]));

echo fread($fp, $bodyOfs + 4);

$metadataFile = $targetFile . “.meta”;

$metad= file_get_contents($metadataFile);

echo $metad;

$chunkSize = 4096;

$skippedOrigMeta = empty($origMetaSize);

if($streamPos == 0) {

}

else {

fseek($fp, $streamPos);

}

while (! feof($fp))

{

// if the original metadata is present and not yet skipped…

if (! $skippedOrigMeta)

{

$pos = ftell($fp);

// check if we are going to output it in this loop step

if ( $pos <= $origMetaOfs && $pos + $chunkSize > $origMetaOfs )

{

// output the bytes just before the original metadata tag

if ($origMetaOfs – $pos > 0)

echo fread($fp, $origMetaOfs – $pos);

// position the file pointer just after the metadata tag

fseek($fp, $origMetaOfs + $origMetaSize);

$skippedOrigMeta = true;

continue;

}

}

echo fread($fp, $chunkSize);

}

fclose($fp);

Sony HDV “video walkman”

There are many folks with fond memories of the Sony DSR-V10, a small clamshell DVCam deck with an integrated monitor. They were great for in-the-field viewing, and also for use with lipstick cams and other devices without integrated recording. Well, there’s now an HDV successor, the GV-HD700E. Notice however, that it’ll only record to HDV when you feed it a signal over firewire – no component inputs (just like the HVR-1500). Lame, Sony.

Azden four channel field mixer

Anyone who has shopped for a field mixer knows that, more than in many other areas, cheap = crap. Azden’s new FMX-42 field mixer looks to be a pretty good mix of features and value. You get four channels in, two channels out, nice VU meters, battery or wall port and a case to carry it. I also appreciate that it has both a normal XLR output and a stereo mini output for use with lower end cameras. At $750 MSRP ($650 street), it’s a nice alternative to the venerable Shure FP33.

iPhone vs Dash

So, I’m not entirely dead. Just mostly.

I’m finally feeling halfway competent enough to write up my thoughts on the iPhone. In doing so, I figured I’d compare it to my previous phone, the T-Mobile Dash (HTC Excalibur).

Img 8119

Let me begin by pointing out my inherent bias in this discussion – the iPhone was always going to be the God phone in my eyes. That said, it’s not perfect.

Lets start with form factor. The iPhone and the Dash are very similar in size and weight. The Dash is a bit wider, the iPhone is a bit taller. The Dash is a fair svelte device for a smartphone (pre-iPhone), and I’m a huge fan of the rubberized plastic used on the case. The iPhone is far more likely to take an accidental tumble than the Dash. Luckily you’d never be so careless as to put your iPhone in harm’s way.

Img 8127



Taken strictly as a phone, the Dash is also the winner. Much of this is (I believe) down to the fact that my Dash was a T-mobile device, whereas the iPhone is an AT&T device. I’ve had more dropped calls in a week with my iPhone than in 6 months with my Dash. The Dash was by far the best performing phone I’ve used though, so it’d be tough to match. (Other recent phones I’ve used with T-mobile – Nokia 3230, Nokia 3650, Motorola Razr, Siemens S55, Samsung S105). It’s also worth noting that I live in a cave, where any RF reception is tenuous at best.

What about internet functionality? There’s no comparison. The iPhone is by far the best mobile internet experience I’ve ever had. Having a true and proper web browser in your pocket really is a game-changer. Websites render properly and quickly, and navigation is easy, despite the small screen. Similarly, the Email client is fast and easy to use. The Dash webbrowser was slow and problematic, often locking the phone or refusing to render a page. Similarly, the email client always took ages to pull in my mail, and would often fail to complete the process, leaving my mailbox in funny half-present states.

In terms of net connectivity, the iPhone is the first phone I’ve had with proper, usable Wifi. It’s not something you have to think about – any time you make a data request and the iPhone sees a nearby access point, it asks if you want to join. If you say yes, it connects quickly and without drama. If you say no, it uses EDGE. While EDGE is a bit of a bummer compared to the 3G networks, the iPhone is definitely the fastest EDGE device I’ve used. The iPhone also remembers Wifi access points you’ve visited in the past, and automatically joins them next time you’re in range. Nice.

The other smartphone tools are pretty much par for the course. The calendar is functional, the notes tool is a notes tool, and the contact management is “alright,” – no better than other phones. The touch interface makes it all fun, and I have no complaints about the functionality. A todo feature is sorely missing though, and one would hope it would find its way into a software update.

In terms of media playback, there isn’t any comparison between the two. The Dash had a semi-functional version of Windows Media Player which would display tiny little videos and stuttering audio. The iPhone gives you all the functionality of an iPod, with the addition of a beautiful touch interface and high-res screen.

All in all, the iPhone has met my expectations of what an Apple phone should be. It’s beautiful, intuitive, and does a few things far better than anyone has done before. And, like any Apple device, it is also to some extent defined by its limitations. No iChat, no video capture, no third party applications. It’s a good enough device that I can ignore the limitations for now. Because Apple has the ability to push software updates easily and universally, unlike any phone before, I hope the next few months will reveal an ever-improving iPhone that continues to delight and amaze.

Drobo Rocks

I have made it back from Beijing (woot) and am slowly getting caught up. I want to begin with a review of my Drobo.

IMAGE_257

It really is as simple as they say. I unboxed it, plugged in three fresh 500 gig SATA drives, turned it on and formatted it. That was it. No setup, configuration, or anything else. As an aside, the drives were only $99 each, on sale at NewEgg. If you’re in the market, keep an eye on the “cheap drive deals” forum at DroboSpace.

Next, I started copying data to the Drobo. While doing so, I popped in a 300 gig drive in the fourth slot. There was no excitement whatsoever – the copy continued, the green light came on and the Volume was suddenly a little bit bigger.

Picture 2-2

So, setup is as simple as could be.

Performance is probably the biggest downside of Drobo. If you need crazy thruput, this isn’t the device for you. I’m seeing about 16 Megabytes/second, compared to about 60 megabytes/second on the internal disk. That’s a pretty big difference, but I’m just using the device for archival purposes.

Some folks are also confused by the way it reports free space. The DroboDashboard application (shown above) reports the true disk space of Drobo. However, the OS will report it as a 2 terabyte volume, no matter how large the actual drives are. This is how they’re able to grow and shrink the volume dynamically – they trick the OS a bit. What it means is that you can’t trust the “Free space” report in Finder. There are also lights on the front of Drobo that tell you how much space is available. Not a big downside for me, but some folks might not be so keen.

Another issue that some folks have is noise. Drobo does have an 80mm fan in the back, which runs at varying speeds depending on workload and ambient temperature. For me, it seems pretty quiet, and I certainly can’t hear it over my air conditioner. If you’re one of those silent-workspace types though (still running a Cube?), you won’t be thrilled.

Overall, I’m very impressed. Build quality is nice, with a bunch of high end touches, like magnets to hold the front faceplate on. For a first product from a new company, it all feels very polished.

So, if you’re a person with lots of data to store, without a need for high disk bandwidth, and you want to be able to grow over time, Drobo will make you very happy.

Drobo is taking over the world!

If anyone wants to learn how to market a product in the 21st century, look no further than Drobo. First off, a bit about why it’s so cool (and why I’m going to buy one) and then I’ll explain how amazing their marketing has been.

Drobo is an external storage solution which uses RAID-like technology to protect your data. What makes it cool is that it can be dynamically expanded, without reformatting, and supports mixing and matching of various disk sizes. So the deal is that you connect it over USB2, pop in whatever SATA disks you have lying around, and it creates a single large volume for you. Any one of those disks can fail catastrophically without you losing access to any of your data. When that happens, you just pop in a new disk and it keeps humming along. Similarly, when you start to run low on storage space on the array, it’ll turn on a light next to the smallest drive in your array, queueing you that it’s time to upgrade. Just pop the smallest disk out and pop a larger disk in. Magically, you’ve got more space in your volume.

I’m excited about it for two reasons. One, I’ve got five external disks hooked up to my Powermac at home, along with two internals. Around 1.8tb of space overall. About 1.5tb of that is totally unbacked-up. That’s a bit scary. It’s also annoying to have data spread across seven different volumes. So, I’m looking to get a Drobo populated with two 500 gig disks and two of my existing 250 gig disks. It’ll give me just about a terabyte of usable, redundant storage, which is a pretty good start.

So, it’s obviously a pretty cool product. But why the buzz? Because this company is very clever.

Their marketing began (as best as I can tell) with a video on YouTube, demonstrating the ways the array reacts to having a drive removed, replaced, etc. From there, they got samples into the hands of the influential tech bloggers and podcasters – DL.TV, Leo Laporte, Engadget, Scoble, etc. Suddenly every blog I read and podcast I listen to is talking about this device like it’s the second coming. This morning I got an email from MacConnection telling me how awesome it is. The marketing must be working because the device is apparently pretty hard to find in stock at the moment.

Bright guys.

Clever Kids at Kodak

Kodak has announced a new sort of filter to go on CMOS or CCD sensors to replace the existing “bayer pattern.” For those who don’t know, a CCD or CMOS chip is inherently only sensitive to variations in brightness, not to colors themselves. In order to get around that, you either need a three chip camera (with each chip filtered to just received red/green/blue light) or you need a bayer pattern on a single chip. Essentially a bayer pattern is just a grid of very small filters, so each pixel on the sensor is only receiving one of the three primary colors. A true RGB image can later be reconstructed by interpolating the pixels.

The new Kodak filter ads a “clear” filter to the existing bayer pattern, which allows for much greater light sensitivity. In essence, it’s like adding a “luma” pixel to the RGB pixels. Pretty clever!