It’s been almost a year exactly since my last update. I guess I’ve been pretty preoccupied. I wanted to do a post about an age old problem very often encountered in Flash – compositing video so that overlaying elements sync perfectly with a video. In essence, getting the true current frame of an FLV or F4V. Specifically the problem I want to talk about tackling is having an image be composited as if it were in the video. I think a great example of this is a site we (as in Firstborn, not me) did for 5 React a year or so ago. There are a couple of solutions to solve this problem that work almost perfectly 99% of the time. In general, they usually involve an XML file that contains corner points for each frame of how an image should be transformed to match the video. For live video (no CG) tracking is set up in After Effects and then the cue points are exported. While After Effects doesn’t have a native export cue points as XML option, the keyframe data can be copied & pasted out as plain text, and then you can just run a simple RegEx to convert it to XML. This is all pretty simple and isn’t the problem, the issue lies with not being able to get the exact frame that a video is displaying.
Just a quick note that I will be
regularly contributing to Creativity Online’s CATScan blog. It’s a collection of post from various people about digital media and advertising. I’ll be posting one article per month along with Dave Snyder, who is one of the Creative Directors at Firstborn. My first article is titled ‘It Doesn’t Hurt to Twitter Up Close and Personal’ and is about how businesses can use twitter to intimately reach their audience. My first article is linked here: It Doesn’t Hurt to Twitter Up Close and Personal.
Just a quick update here, we recently relaunched the new Doritos.com [edit: no longer online], my latest project. I was lead developer on this project with much help from Lucas Motta and Michael Roushey on the flash forefront, and as always Francis Turmel as our technical Sensei. The project was done with the world renowned ad agency Goodby, Silverstein & Partners. We put a lot of hours into this site, but I think it was worth it. My job as lead developer was doing the overall structure and framework of the site. I also developed some helpful debugging utils along the way, which I hope to post on here rather soon.
I probably should have mentioned this way earlier, but I’ll be speaking along with Mathieu Badimon at OFFF Paris on behalf of Firstborn. Our presentation will be on Friday, June 25th at 12:30. I have to say, this is quite an honor to speak at this conference. If you’re interested, there should be more information on the OFFF Paris 2010 website.
So I realize this is pretty last minute, but I’ll be speaking at FITC Toronto this Monday with Mathieu Badimon. We’ll be speaking on Firstborn’s behalf. Our presentation is titled “Tools & Prototypes: Crafting a Unique Digital Experience.” It’s basically focusing on how for each project we make, we usually need to make demos to test techniques or make tools to aid in development. It’s more of a creative presentation than a technical one, as we really won’t be talking about code directly. So if you’re up at FITC this weekend, we’ll be presenting Monday at 12:10.
Damn, it’s been a long time since I’ve had a site a launch. But I’m happy to announce my latest project at Firstborn has gone live: SoBe.com. I was the lead flash developer for this project. The current SoBe campaign also featured a facebook app with contest and a mobile version of the site as well (which were not developed by me, but Phil and Miller respectively).
There are two sides to the site. The product ‘flavors’ section features videos of real people stopped and interviewed on street about what they think of SoBe lifewater (which we, firstborn, filmed). We used streaming video from FMS provided by Akamai in order to provide responsive seekable video. This was actually a challenge, as streamed video has some slight programmatic behavior differences than progressive download. For instance, streamed video will not dispatch any Net Events for when it has reached the end of the stream. Another issue I ran into was that due to security restrictions, the BitmapData.draw function is prohibited unless specifically enabled on the FMS side. The other side of the site, ‘world’, was pretty straight forward, except that since we will be doing updates for the rest of the year the site needed to be extremely dynamic. This section of the site also features actress Ashley Greene in the nude, painted in two skinsuits to promote SoBe lifewater’s two new flavors.
Quick update on my current experiment building a 3D engine in Flash – I’ve added light and shaders. Shaders are special materials that are effected by the direction each face is pointing (called the normal) and another vector, in this case, a light direction. During the render process, each face calculates its brightness thus giving the illusion that light is reflection off of the mesh.
Upon researching, I found on a few occasions that the way to calculate lighting is to simply add the normalized normal and normalized light vector. This, however, seems to be slightly incorrect applicable to this situation. First, the result is another Point3D, where instead I am looking for a brightness value. Upon consulting coworker Mathieu Badimon, he suggested using the dot product of the normal and light vector. This returns a single number value, and when used to calculate brightness seems to work perfectly.
Next step? Finally tackling bitmap distortion in 3D, allowing for bitmap materials. (Which opens the doors to different types of shades as well, such as bump maps and dot3 normal mapping.) Once I’ve tackled that, I plan to completely restructure the engine, as I have a better idea now of how it can work while still sticking to my original plan of excluding a view3D. In the process of restructuring, I plan to write up the core math equations used in the most simplified way that I can, as finding some of these equations in a non-language specific explanation or simplified manor can be difficult.
Check out the new demo here. Click and drag the white dot to move the light source.
A few weeks ago I had to create a simple X-axis based parallax ‘engine’ for a project. Just basic horizontal movement, which is scaled based on an abstract z value. Well, somehow that turned into a full 3D engine. So I am continuing to build on what I had started in order to create my own Flash 3D engine, not because I think I can build a better Papervision, Away3D, or Five3D, (because I’m sure I can’t) but rather in order to learn. Since I end up using 3D engines fairly often now, I figure it’s good experience for me to build my own and see how it can work from the ground up.
So for the last few weeks I’ve been experimenting with Away3D. Away3D is a free 3D engine for Flash, very similar to Papervision3D. As a previous user of Papervision, I noticed a lot of similarities, but there are a few differences, both good and bad, to take note of. There are different features between the two, though for the most part they’re pretty similar.
I’m a little behind here, but here is my latest project from Firstborn: AirTran EveryFlight.com, where I was the front-end developer for the project. We worked with Cramer-Krasselt in order to bring this site to life. AirTran is a small domestic airliner, and they wanted to run a promotion to see what people wanted on every flight. So we built a site to collect ideas, then let people vote on them and see how each one ranked on a state and national level. The site only ran live for about 4 weeks until the winner was announced which happened to be Wi-Fi. You can see the old phase 1 site persevered here on Firstborn’s portfolio. It was a really cool project to work on, especially once it launched to see the suggestions come piling in. Our back end developer for the project, Wes Grubbs, built the database and admin tools, and we used AMF php, which provided to be really simple and easy to use. And as always, I also got to do some nifty features on the site.
So the first challenge was that we wanted the user to be able to write in their suggestion on one of the cards and actually have a pencil do the writing. There were two important parts to this process. The first is getting each letter traced out. The basic methodology behind this is that we will create for each letter a set of points on a path (anchors and control points for curves) and then use that path to mask out the actual letter. By drawing that path over time, we can create the illusion of the letter being drawn in as it is being reveled. The second part is having a pencil follow the letter, which is simple since everything is done by numbers and the drawing API, so from the current path being drawn we can grab the ‘end’ of it and just have an object track that point, thus having it follow the path being drawn in and the illusion that the pencil is drawing the letter. To create the pencil, I used FIVe3D. Mathieu had created a pencil for the M&M’s Join the Hunt site, so to avoid having it look too similar I didn’t bother using his model/code for reference, I just built out my own (plus it’s more fun that way).
In order to get the paths for each letter, I created a tracing tool. The tool allowed me to add points and make a path, as well as play with the curves and thickness of the line. Additionally, I added the feature that would allow the pencil to be “picked up” over certain points, meaning it wouldn’t be drawing for that segment (shift+click an anchor point). That is important for drawing letters that have different individual strokes, such as F, where when the pencil was moving I wouldn’t want to unmask a section of that letter as the pencil crossed it to get to another location. Once each letter is traced, it can be saved out as XML which is imported to the site and parsed into usable information.
Another thing that was kind of interesting, although at this time I don’t have a demo for, was creating the rollover for the stack information. One of our newly hired 3D/Motion Designers, Brett Swanson, first modeled the stack to resemble the initial comps our designer Verena did. In order to make the cross-section, I first made a black and white image of the stack. The black being one side of the paper, and white being the other. Where the two colors met is the corner. So in Flash when the user rolls over the sack, I look across the horizontal strip of pixels from that mouse point and look for the area where white meets black, and I know that’s the corner. I can then do some masking, then calculate the rest of the points around it to create the cross-section plane, which is not 3D in anyway. As for the animation of the card being pulled out, Brett created a static animation that we imported as a PNG sequence, then position it to where the mouse click is, and scale it accordingly to be the correct size. Due to perspective, it doesn’t always match up. But since it is such a quick animation, we can fake the perspective by just slightly rotating the animation a little so that looks like the perspective matches up. As the animation plays, it rotates back to zero and scales back to the default size, and then is swapped out with the flipping animation which is dynamically done in five3D. As I always say, it’s all smoke and mirrors.
(images after the jump)