Anatomical Travelogue Research » Image processing

Color Reduction

Sven — Sat, 24 Dec 2011 16:39:37 +0000

The main purpose of the Color Reduction tool is to maximize the contrast in the color channels of an image. This can improve the input for color segmentation tools as well as the results of color to grayscale conversions. We implemented two different algorithms in order to determine color space transformations that optimize the contrast: Principle Component Analysis (PCA) and Locality Preserving Projections (LPP).

Both methods can be applied to transform or reduce 2-4 input channels to 1-4 output channels, for a single image or a range of images. A secondary image input can be used to supply a weighting mask that determines the influence of each pixel during the color reduction algorithm. Afterwards, the output channels can be normalized and optionally, the color transformation can be encoded in a Fusion 3D LUT.

Download Color Reduction 1.0

Description of methods

The results of PCA and LPP are not necessarily the same since the optimization is based on different criteria. PCA determines the principle components that account for most of the contrast in color space, e.g. if the contrast in the red and green channels is much higher then the contrast in the blue channel, the first channel returned by PCA will be a linear combination of the red and green channel.

In addition, LPP considers the proximity of colors in the image space, e.g. if many yellow pixels are neighbors of green pixels in image space, green and yellow pixels may get the same color in the first channel of LPP. In constrast, red and yellow pixels may get different colors although their distance in the color space is the same (see details on the effect of the LPP weighting factor below).

Comparison of PCA and LPP

The following video shows the effect of PCA and LPP for color to grayscale conversion. The results are compared to simply using the lightness channel of the LAB color space. For better visibility, only the 2D histogram for the red and blue color channels is shown.

In the first step, the color space of the input image is projected onto a line. The orientation of the line and the direction of projection are different for all three methods and result in a different distribution of contrast. In the second step, the line is mapped to the grayscale range and normalized.

While PCA will always find the axis of highest contrast in the color space and is independent of any parameters, the results of LPP depend on a weighting factor.

Effect of the LPP weighting factor

The image below helps to understand the effect of the LPP weighting factor. Internally, the weighting factor is applied to the color space distances of neighboring pixels, so that a higher factor will lower the chance of pixels with different colors to be recognized as neighbors. The example shows the input image which contains dark green, light green, yellow and orange pixels and a simplified red-green histogram.

Although the color space distance in vertical direction is smaller than the distance in horizontal direction, LPP will recognize the neighborhood between light green and yellow pixels as well as the neighborhood between dark green and orange pixels in image space. Therefore, LPP with a low weighting factor will project light green and yellow to the same color as well as dark green and orange to another color. However, after increasing the weighting factor, the color space distances will become more important than the spatial neighborhoods again and the result will be the same as for PCA.

RGB to LAB

Chad — Fri, 18 Nov 2011 20:40:34 +0000

Download RGBtoLAB 1.03

Ok, this Fuse started off as a really tiny one. Fusion already has a CSConvert() that does L a b. So all I have to do is…

if mode == 0 then

out:CSConvert("RGB", "LAB")

else

out:CSConvert("LAB", "RGB")

end

Simple and fast. But the results were a bit… unexpected. So we added a little tweak which has no basis in known color science, it just makes is so that a linear RGB input returns what I think is a linear L output.

So why doesn’t ColorSpace have an Lab button? I don’t know. You can see LAB color in action in the Wand tool, though.

Other options include output a LUT Image only, as if it were a Source tool. Same as the new option on ColorMatrixTransform. You can also choose whether to affect the canvas color or not.

Signed Distance

Arun — Wed, 11 May 2011 15:23:59 +0000

The signed distance operator can be quite useful when doing image/voxel data processing based on a pixel’s distance from an iso-surface. This plugin calculates just that. The plugin is implemented using Open CL, so the Open CL program should also be copied into the same directory as the Fuse plugin.

Download Signed Distance Fuse

The tool has inputs to control the position of the iso-surface (Zeropass Threshold) and the number of rasterizing passes (Number of Passes). The output can be a distance value or a 2D – vector pointing towards the closest point in the iso-surface.

Bilateral Filter

Sven — Fri, 10 Dec 2010 20:57:45 +0000

Smoothing an image without losing all the edges, that’s what the Bilateral Filter is made for. It basically performs a Gaussian blur not only in the image dimensions, but also on the color intensities. Therefore it will smooth homogeneous areas while at the same time respecting the image’s edges. The following example shows the effect very well.

The current version is for Fusion 6.1 in 64 bit only. It provides the standard bilateral filter as well as the cross bilateral filter for grayscale and color images. DoD and RoI can be used to limit processing to a desired section of the image. Since the exact filter is very slow when applying large filter kernels, calculation in OpenCL with an optional approximation method is implemented as well. First tests on supported graphic cards show a speed up by a factor of 3-5 for OpenCL with default parameters. On gray scale images with large filter sizes, the approximation method can be more than a hundred times faster.

Download Bilateral Filter 1.2

Fun with OpenCL: Unit Vector

Chad — Fri, 29 Oct 2010 14:35:46 +0000

Finally took the plunge and converted one of our icky Lua Fuses into shiny OpenCL.

Download Unit Vector OpenCL Fuse

It’s a very simple function, normalizing a vector and returning that in the RGB while putting the original length into the A. In the case of the 4D normalize, you don’t get the length. I have no idea what took me so long to get around to this, it only took an hour or two to flesh out, but I got hung up by the fact that my video card, a Quadro 5000, does not, at this time, have OpenCL drivers that support float3. Float4? Fine. Float2? No problem. Float3? Fail. So I went ahead and implemented the 3D normalize manually, and used the fast_normalize OpenCL function to make a 4D normalize.

If you want to make your own OpenCL Fuse, make sure you turn on “Verbose console messages” in the preferences. Would have saved me a lot of frustration had I done that from the beginning. If you want to know what your machine’s OpenCL capabilities are, there is a “Debug: List OpenCL Device Info” when you shift-click on the cache meter.

I also copy-pasted my old Lua Fuse in there, which is great for testing, but also as a fallback in case OpenCL isn’t available. Since even with the OpenCL, you have to do certain tasks no matter what (reading in parameters, outputting an image, etc) the speedup from using OpenCL will increase as the image size increases. So while a 16×16 image might not see any improvement, a 4096×4096 image will see a huge boost.

When I get a valid OpenCL 1.1 driver, I might revisit this to see if I can increase the speed, but we’ll have to see. Couple other notes, currently the OpenCL modes are limited by the maximum image size supported by your device. I didn’t implement any chunking to get around that, but you can use the Lua mode. I didn’t implement ROIDS, but I’ll get around to that soon. With the OpenCL, it’s silly fast, so that’s not the issue, but it breaks a nice DoD chain if you have one. For now use SetDomain. Finally, there is no workgroup size management. This is a REALLY simple kernel (which is why I chose to do it first) so it’s unlikely you’ll hit the 5000ms limit, but I suppose the risk is still there. In my machine a 2048×1556 image takes about 6.7ms.

So what’s next? Any tool that we can make with these dead-simple OpenCL functions would be prime candidates, but really, OpenCL opens up the possibility, like we saw with CUDA and KrazyKey, to make tools of very high computational complexity if they are still highly parallel. When the next revision of the SDK comes out, we’ll try adding OpenCL to some of our C++ plugins, too. There’s also the mysterious “gl” OpenGL interop which piques my interest.

Alpha Divide/Multiply

Chad — Thu, 21 Oct 2010 22:04:23 +0000

Stefan Ihringer pointed out something on the VFXPedia that finally got me to put together a Fuse I’ve been putting off. Since Joe Laffey never got around to updating his plugins, it makes sense to go ahead and toss this up.

Download AlphaDivideMultiply Fuse 1.01

Just a simple RGB/A or RGB*A, plus a “Solid” and “Clear” so you don’t have to use the annoying Bol or Mat as much. Nothing groundbreaking, but it should save a few clicks per day.

UPDATE: I made a ViewShader for this in case you want to keep this operation in the viewer and not in the flow.

MIP

Chad — Mon, 18 Oct 2010 17:34:25 +0000

Not multum in parvo, but Maximum (or Minimum) Intensity Projection.

Download MIP Fuse 1.13

This simple Fuse just takes a range over time and generates an image with the highest or lowest pixels. Great for normalizing. It does so for each channel, so it can be useful for color normalization too, or for finding the bounding box of a position pass. It can also be useful for making “summary” thumbnails of footage.

There is also an option to output the maximum (or minimum) pixel value from the entire sample, in X, Y, and Time. This is a number output suitable for connection to other number inputs.

Download MIP Sample Comp A01

Fragment fun

Chad — Wed, 23 Jun 2010 22:37:18 +0000

New shader(s) coming soon

Color Blindness

Chad — Sat, 20 Feb 2010 05:56:55 +0000

A sizable segment of the population suffers from color blindness, enough so that it’s worth considering the implications on color palettes and usability. This tool allows you to simulate the ways that various color vision deficiencies will affect you imagery. I’ve noticed that some of the images we create probably won’t read very well to some people, and this easily lets us check if we’ve created something that could be ambiguous.

ColorBlind fuse

It’s not a perfect simulation, since it only does a linear color transform, not accounting for local color frequency, and it doesn’t account for any output specific color shifts. Nothing unusual when it comes to color, of course. Typical things to consider, colors may look different on an LCD screen compared to web offset printing.

You can use this as a viewer macro, though I may at some point make a proper Cg viewshader.

Download ColorBlind Fuse 1.0

Color Matrix Transform

Chad — Fri, 23 Oct 2009 00:55:51 +0000

Fusion 6 added a Color Matrix tool that lets you enter your own matrix by hand, but the biggest problem with it is the lack of any methods to modify it with. You can’t even assign controllers to it.

Fuses, however, let you use handy methods to modify a matrix. I’ve used some of them to create an RGB equivalent of the 3D Transform tool. It has a similar UI, just as 3TT does, but this modifies RGB, not XYZ or UVW.

Color Matrix Transform fuse

Why would you want to do this? Imagine that your RGB isn’t color, but UVW coordinates from a 3D render pass. Now you can do texture transformations by using this tool to modify the mapping. Also, if you have comps where you’re converting XYZ to RGB, you can use this to post-edit the tranformation, so you can apply spatial transformations in the form of XYZ -> RGB -> ColorMatrixTransform -> XYZ.

In the future I might add an input and output similar to the Color Matrix tool so you can insert a matrix found elsewhere, as well as generate an output matrix you can read off, but for the time being, input matrices can processed by the Color Matrix tool just prior to the ColorMatrixTranform tool, just remember to use 32bit float colors.

If you’re a bit confused as to what the tool is doing, try viewing the results in the 3D Histogram SubV. Really helps you get your bearings.

Download ColorMatrix Transform 1.1