Experiments

Experiments with Machine Learning & Lex: Teaching a Machine to Identify Color

Amazon has (somewhat) recently added some new services under the Artificial Intelligence offerings, one of them being a Machine Learning service. I wanted to play around with their predictive analysis service so I decided to make a really simple proof of concept.

Predictive analysis, in a nutshell, is basically looking through a large dataset of various input values that each contain an outcome. That outcome may be a true or false conditional (Binary Classification), a numerical value (Regression), or identifying a label (Multiclass Classification). This data is used to generate a model that makes a correlation between the input variables and the outcome, which can then be fed new input values to predict what the outcome will be. The catch, of course, is that you need to have this large set of training data to work with.

Since I didn’t have any data available, I wanted to see what I could possibly generate on my own. I decided on trying to make a model that could guess the name of a color based on the input value. The end result would look something like the following (once integrated into slack):

Continue reading “Experiments with Machine Learning & Lex: Teaching a Machine to Identify Color”

Doing Stupid Things with Google Spreadsheets

I use Google Spreadsheets pretty often at work, but for more mundane typical scenarios such as budgets, time and scope estimates, and so on. However, as I played with formulas and some of the built-in features, I was curious to what other non-productive things I could create. This lead me to a project I recently completed that uses the spreadsheet grid to recreate images. This was recently featured in the 2nd annual Firstborn Art Show (All These Things 2016) under the title Pixcell Images.

You can check out the entry here: Pixcell Images by Eric Decker. There are about 20 images that rotate every 15 seconds.
There’s also a version that accepts and shows user-submitted images via MMS, which you can text to at 585-672-6394 (try it out while it’s still active!)

The concept is pretty simple — I display small images (up to 32 x 32) in a spreadsheet by coloring three cells for each pixel’s red, green, and blue value. Basically, more or less how a typical monitor or screen works. A separate external script processes the images and sends the data to the Google Spreadsheet, but the spreadsheet processes and displays the “image” all on it’s own without any additional scripting.

Pixcell: Mario Kart Toad — Toad sprite from the original Mario Kart displayed in a Google Spreadsheet.

It was a fun process, so here are some of the interesting steps it took to achieve the final results.

Continue reading “Doing Stupid Things with Google Spreadsheets”

Gravity Wells

gravity-wells-1

Dug up a project I had been working on a while ago that I want to revisit – a game prototype called Gravity Wells. The idea of the game is that’s it’s a somewhat typical 2.5D space shooter, except that objects you encounter have weight that deforms the 2D plane that the ship and obstacles sit on. The gravity wells that objects – enemy ships, asteroids, space debris, etc – create affect other objects and even the projectiles that the ship shoots. I had another thought that maybe the ship doesn’t even shoot, but instead has some sort of gravity manipulation device that requires you to fling objects into each other. Extremely heavy objects, a la black holes, could present some interesting options as well. This started as a quick Flash prototype for proof of concept but would be executed with webGL.

NanoTween

Recently I was working on preloader for a project and needed to use a tweening engine. (We have an internally developed one at Firstborn that is based off of the syntax of Tweener). I wanted to see if I could write my own that would be super small. I wasn’t as concerned with performance – meaning being able to tween thousands of objects at once, a highly unlikely scenario anyways. I wanted to keep compiled file size down, as well as number of classes. So I created a NanoTween class that is a super simple tweening utility. When compiled on it’s own, it only consist of about 1.3 Kb.

Continue reading “NanoTween”

Time Displacement Experiment (part II)

I made some updates to my time displacement experiments. I lumped them all into one nice little app, as well as give some options to toggle the effect. I also added the option to choose video source, so it doesn’t have to be with a webcam anymore. I added the option to tint what area is being effected, as well as experimented with frame blending to try to lessen hard edges. Like the previous post, I’m still using Pixel Bender to apply the displacement.

experiments.eric-decker.com/time_displacement

Continue reading “Time Displacement Experiment (part II)”

More Time Displacement Experiments: Now with Pixel Bender

I decided to try using Pixel Bender to do the time displacement in the recent experiments I’ve been doing, as I was hoping it would perform better. It actually seems to, but there still is a significant lag. However, I’m pretty sure this is because of having to make a huge sprite sheet first.

I basically make a huge sprite sheet of all the frames I have saved. Since Flash 10 lets you have bigger bitmaps (16,777,215 pixels total, max dimension at 8,191 src) I can create a large image that tiles all the frames I currently have saved. I can have a max of 12 frames across (at 640) and 4 frames down (at 480). So after I create/update this giant bitmap, I send it to a pixel bender filter that is currently applied to a gray scale image. The Pixel Bender filter then looks at each pixel, evaluates the brightness and interpolates it to what frame it needs to update from, then replaces the current pixel with the appropriate one from the selected frame.

These experiments are still a little rough, but I’ll post ’em anyways. No videos yet, I need to find some time to do screen recordings. The ticker is obviously by Mr. Doob. You’ll need a webcam for all of these, sorry.

brightness: Displacement based off of brightness of current frame.
movement: Displacement based off of movement (difference between frames)
painting: Displacement based off area pained by user, as in previous post (inspired by Smudge City).
noise: Displacement based off of perlin noise.
filmstrip: Displacement based off of titled blocks.

In any of these experiments, you can right-click to show the source that is controlling the displacement.

Video Time Painting Experiment

I started playing around with doing some experiments with video and time delay. When showing the previously mentioned JPEG video experiment to my coworker Roushey, he mentioned how just the effect of the time delay was interesting. I was thinking about neat things to do with, and the first idea I had was to offset each frame as one row or column of pixels. So in a 640 x 480 video, the left-most column of pixels would be from the current frame, and the last rightmost column would be from 639 frames ago. The experiment is live here, and there’s a video of it on YouTube.

Continue reading “Video Time Painting Experiment”

‘Encoding’ Video as JPEG real time in AS3

A few days ago I was thinking if there was a way to reproduce the effect of JPEG compression real time in flash. I figured the best way would be to literally encode the image with the native AS3 JPEGEncoder class. The problem there was that the returned ByteArray could not be interpreted as an image anymore. However, the Loader class does allow for loading a ByteArray and converting it to a Bitmap. And luckily, the loadBytes method let us load the ByteArray right from Flash (as opposed to having to save out a file and load it externally).

Continue reading “‘Encoding’ Video as JPEG real time in AS3”

Webcam Snapshot Tool

Every once and a while, I find that I need to just take a quick snapshot with my webcam at work. Since I’m on a PC, so I don’t have Apple’s pre-installed Photobooth app, or any other simple program (that I’m at least aware of). I also didn’t want to install any bulky 3rd party software that might come with the webcam that I’m using, since all I need in order for it to work is the driver, which more or less Windows installed all by itself. Photoshop has an option to capture from webcam, but the util is pretty limited as it only lets me capture a very small thumbnail size image.

Continue reading “Webcam Snapshot Tool”

Video Processing Tests

Recently I did some playing around with video and thresholds, mostly just for fun. I first experimented with just changing the threshold of the video, essentially converting it to black and white. Then I tried comparing the current frame threshold and the previous frame threshold to get the difference. The result looks like an outline/edge effect. It’s an interesting way to visualize movement. Then, just for kicks, I decided to display the previous comparison and the current one, colorizing the current difference as red and the previous as cyan. The result is a very pseudo-3D outline effect. This fake 3D technique can also be applied to to straight up video, showing the previous frame as cyan and the current as red and screening the two. Since this all based on movement and the idea that objects closer to the camera appear to move greater distances due to their perspective, it’s very easy for it to display incorrectly.

Check out the first demo. You’ll need a webcam. Click on the video to change the mode, and the two scroll bars control the threshold level and amount of blur applied to the video before any image processing.

video-test-threshold video-test-3d-outlines Continue reading “Video Processing Tests”