We were recently commissioned to do some visualizations for a virtual colonoscopy procedure.   Also known as a colonography.   If you’re unfamiliar with the process, Wikipedia has enough information to give you a general overview. While there is a lot of information on the internet about the scanning process itself and what the patient will experience, there isn’t much about the ways in which the CT data can be analyzed once it is acquired.  So here’s a brief overview of what we at Anatomical Travelogue did with it.

Screengrab from the realtime colonoscopy demo

The CT

Here we have an axial projection of the CT, (commonly known as CAT, for computed axial tomography).  The video shows a series moving through the Z axis from the anus to the top of the diaphragm / base of heart.  I’ve gone ahead and adjusted the brightness and gamma so it is easier to see on a normal computer monitor.

What we can see here is that the patient was lying supine, and the contrast enhanced fluids in the colon are collected on the bottom surfaces of the colon.  Sometimes the patient is scanned twice, once supine, once prone to make sure that all surfaces are imaged, but we found that the contrast between the walls of the colon and the fluid was sufficient to make out any “submerged” surface details to the same resolution as in the upper “air exposed” surfaces.

Segmentation

Ok, so the next steps are to remove the fluid from the colon, and to segment the colon lumen.  The values in the CT are sufficient for this, the lighter values are the fluid, the darker values are air, and the middle greys are the colon itself.  We have to isolate the colon from the bones, lungs, small intestines, etc. which share similar values, but they are spatially much different, so we can exclude them that way. The image below shows the result of this, a single axial slice of the CT on the left, the fluid removed in the center, and the segmentation mask of the colon lumen on the right.

Segmentation results for colon lumen. Left: Original CT; Center: Fluid removed; Right: Lumen segment

Normals

The CT dataset can now be processed to get some additional visualization data out of it.  We clean up some of the noise in the scan, and then extract gradients from the data which will be used as normals at the threshold interface of the colon lumen.   Essentially, we’re determining where a voxel is “facing”.  This vector is encoded as color, where red represents the X axis, green represents the Y axis, and blue represents the Z axis.

Encoding normal direction to color

The image below shows the normals we calculated, as well as those values multiplied by the CT values.

Left: Normals as RGB (not yet normalized); Right: Multiplied by the CT intensities

And this video shows these images stacked on top of one another with a little perspective added to show the depth.

And here’s a video with just the normals on the colon segment itself.

Isosurface

From the lumen segmentation, we extract a polygonal isosurface.  The resulting mesh is scaled to the correct real-world size according to the CT scan itself.

Marching cubes extraction of the lumen, scaled to real world units

Rationalization

The isosurface is then rationalized to a curve network.  Along the length of the colon, longitudinal lines are created as well as cross-sectional lines.

Rationalized curve network of colon lumen

Centerline Finding

The curves generated from the rationalization can then be averaged together to find the centerline of the colon, which will be used as a camera path and as the base of our unwrapping process.

Centerline path (thickened to show more clearly)

We normalized the spline and took some measurements off it.  In this case, the colon measured 1866 mm.

Unwrapping

So now we have all the information we need to do a basic unwrapping.  We create a new voxel array XYZ where X is the length along the centerline spline, Y is the complete rotation of the vector perpendicular to tangent of the centerline curve, and Z is the distance from the centerline curve.

How the voxels are sampled

Or put another way, it looks something like this:

Conceptually what is happening

This lookup was applied twice, once to the CT normal (gradient) volume and once to a unit cube of the dataset.   We also created a 1D array where the length along the spline intersected the dataset unit cube.

Here is a video showing the result from the CT normals.  The black voxels are outside the bounds of the dataset.  I’ve resized it such that each voxel is 3.5 mm in size.

And here is a still image where the density and normal values from the CT have been accumulated to generate a “surface” based on the unwrapping of the colon.  This isn’t an isosurface, we’re still looking at voxels at this point, but as a 2D projection, it creates a nice “map” of the colon.

Unwrapped normals, accumulated at the density of the CT. Each voxel is 1mm in X.

By applying a depth cue to the voxels based on distance from centerline, however, we make it a bit easier to read.   The colors show where the surface is facing, so that red is dextral (on the right), blue is superior, and green is posterior.   Something to notice… where the horizontal bands of color shift downward or upward, that is where the colon bends, or at a rectal valve.  Conversely, where you see the bands stretch out is the colon is relatively straight.  The darkening helps to differentiate protruding features, like polyps, from recessed features, like the appendix or diverticula.

With depth cue. Can you find anything suspicious?

By the same technique, we accumulate the samples from the unit cube.   This gives us a position lookup between the unwrapped dataset and the original CT.

Unwrapping samples from the unit cube of the dataset

It is now possible to create a vector between any point along the centerline and any unoccluded surface point of the colon.  But since we still have the voxel data, we can explore the occluded and subsurface details, if present.

Rendering

From here we can construct some renders that show the data relative to a point in space somewhere along the centerline.

Here is a volume rendering showing the path through the colon as a bright light.

We also created a raytraced image of the polygonal isosurface from inside, sampled along the length of the centerline.  This render covers a complete 360°
view from each millimeter along the centerline path.

This video shows the spherical fisheye view on top.  We can convert this to a latitude/longitude view, as shown on the bottom.

Because the view is a full 360°, you can see everything visible from that one point.   This lets us tumble around in a more normal “human eye perspective” to look around.

This is similar to how a traditional endoscope works, but a LOT easier to control.  If we wanted to be more “authentic” we could have put a vignette around the images.

Putting it all together

Once you have all the assets created, it becomes a question of packaging and delivery.  The unwrapped normal map with depth cueing can be printed out on a piece of paper, or viewed on a small cellphone screen.

Cut into strips, ready for printout to include in patient file

Another way to view the colon on paper

We can combine the other renders together into a linear video, where the number of frames is something fixed, like length in millimeters of the colon, and compare that to the unwrapped map X axis.   This could be burned to a DVD or posted on YouTube.

Or we can create a complete interactive system where the user can look for polyps in ANY of the views, the CT, the exterior view, the endoscope view, or the map, and the various views will all stay synced together.  This could be operated from a local workstation, or processed on a remote server and viewed on a webpage.   An automated version of this could be rendered out to a video as well.

A user interface for synchronizing the various views of the colon

Download Full size user interface movie (7.3MB x264 AVI)

You can see more of our colon explorations including other datasets at www.thevisualmd.com.