A plug in for Sony Vegas Movie Studio to correct defective pixels and dust spots in video.
As a professional video shooter you might never have heard about stuck, hot or dead pixels. However a new kind of 'actors' are entering the video scene. With the release of the latest DSLR cameras such as the Canon 5D mk2 or the Nikon D90 the line between video cameras and photo cameras is fading. Capturing HD video will become a common feature on DSLR cameras and regarding video capabilities, DSLRs have 2 major assets. A huge set high quality lenses is available. In-lens stabilization and high optical quality used to be a solid base for ensuring outstanding pictures and now they can be used to shoot high quality video too. A second asset of a modern DSLR is its sensor. State of the art sensors ensure very low noise, high quality pictures in low light situations. The current DSLRs feature a first implementation of video (HD) capturing. A photographer will call it a gadget... a video shooter will laugh it away. But one thing is for sure. DSLRs will change the the video scene fundamentally.
Sounds promising... But... Of course there is a 'but'...
Hot, Dead and Stuck Pixels - An important criteria in the DSLR competition is the sensor. In the very competitive digital camera market, the sensors in modern DSLRs are pushing technology to its limits in its strive to have more pixels with less noise. This result in outstanding performance but comes at a price of small defects during manufacturing. It is generally considered normal that some pixels out the the millions are defective.
The defects come in 3 kinds. The pixel of the sensor captures the light by storing it as an electrical charge in a small capacitor. During the exposure time it is possible that electrical charge for powering the sensor is leaking into the capacitor. During readout, this charge is read and interpreted as light, causing this pixel in the photo being lighter then it actually should be. This kind of defect is called hot pixel. Depending on the level of leakage, hot pixels show up at longer exposure time (since the sensor is powered during a longer time, more charge is leaking into the capacitor).
The extreme of a hot pixel is a stuck pixel. A stuck pixel is always lit at maximum at any exposure time.
Another kind of defect is a pixel that is not sensible to light anymore. The pixels doesn't charge the capacitor during exposure and shows up as a black dot in your photo. This is called a dead pixel.
Dust on the sensor - Despite of all the cleaning features that are now standard for DSLR cameras, every DSLR owner at a certain moment will be faced with dust on the sensor. These small particles of dust prevent the light reaching the sensor and hence result in small black or grey spots in the picture that is taken. For a photographer, this shouldn't be a problem since cleaning the sensor is a relatively small and routine task and with some minor post processing the spots are easily removed in the pictures that were taking when there was still dust on the sensor.
Dust or host/stuck/dead pixels are generally not considered as a problem for a photographer. Modern DSLRs (e.g. the Canon 5D mk2) have built in defective pixel remapping. Most RAW processing software (eg Lightroom and Bibble Labs) have a feature to automatically fix defective pixels. When the picture was converted to JPEG it gets a little bit more complicated since the strong discontinuity of a hot pixel causes JPEG compression artifacts in the surrounding pixels. However sufficient tools exist to fix these spots with a minimum of effort.
It's more tricky when using your DSLR to shoot video. Since this is a relatively new feature on DSLRs the defective pixel problem or the dust spots in the captured video is also new. There are not much tools yet available to fix this problem. To make it even worse... For reasons not known to me Canon does not even use the defective pixel remapping data that is available in the camera during video capture.
I wrote StuckVideoPixelRemover to solve my hot pixel issue.
My first experience with hot pixels was when my Canon 10D was about 2 years old. Suddenly 4 defective pixels showed up in my photos. When borrowing my sisters camera to shoot some pictures in slightly more difficult circumstance, I noticed the same issue in her brand new camera. At that time they didn't bother me very much since they were automatically corrected by the RAW processing software.
Things changed when I bought my Canon 5D mk2 some weeks ago. His good performance in low light situations and its capability of capturing 1080p full HD video convinced me spending somewhat more money. Mixed feelings after the first tests. The overall picture quality was outstanding but soon I discovered that the sensor suffered from more then 1000 hot pixels. Soon I learned that hot pixels are almost unavoidable in full frame sensors and that especially the 5D mk2 is notorious for this issue. Since the camera was taking cloudless night pictures with the lens cover installed, the dealer agreed to exchange for a new camera. The new one counts 1 stuck pixels when taking photos and 18 others (grouped in 1 big spot) when shooting video. Of course the stuck pixels in video mode are also stuck in picture mode, but in picture mode the camera remaps the stuck pixels using is stuck pixel remapping data.
After having browsed the Internet I assessed my chances on getting a faulty less sensor very little. So I started looking for a way of living with the issue. As mentioned earlier... for photos the issue is easily solved. For video however I couldn't find any tool to correct it.
My first problem was finding a a video processing software that could handle the Canon 5D clips. A trial version of Pinnacle could not open them. Corel Video Studio seemed fine at first sight but I had issues rendering to the resolution and quality that I wanted. My third attempt has become my favorite. Currently I'm using the trial version of Sony Vega Movie Studio Platinum. It reads the Canon QuickTime format very well. The video processing is very user friendly (although I would like to see the Pinnacle Storyboard view which doesn't seem to be available in Vegas). The rendering to final video formats is very flexible and fulfills all my need.
Most important of all however... Sony has an easy to use SDK package available to write video filter plug ins. It didn't take much time to get my first version ready. The defective pixels of my camera were statically defined in the plug in. I was so pleased with the results that I decided the spend some extra time... make it a little bit more flexible... add some robustness... and hopefully now it can you some other would-be video shooter out.
StuckVideoPixelRemover is a plug in that simple read a camera profile that contains a list of all the defective pixels. For each defective pixel you can define by what other pixels it should be replace. During the final rendering of your video by Sony Vega Movie Studio, each frame will be processed by this plug in and the defective pixels will be fixed.
|
Not processed with StuckVideoPixelRemover |
Processed with StuckVideoPixelRemover |
| Download full sized clip | Download full sized clip |
My apologies... installation must be done manually.
I didn't create a tool yet to assist in the camera profile generation. If there is sufficient interest for this I still might do this in the future. However since the camera profiles are simply stored in the Registry and are totally independent of the plug in itself anyone with some programming experience can do this. Also defining the profile for your camera is a small task that only needs to be done once for each video mode your camera might support.
The camera profiles are stored in the Windows Registry in the key HKLM/SOFTWARE/Shardana/Stuck Video Pixel Remover/Camera List. The sub keys in this key are the camera profiles. The name of the sub key is the name of the camera profile. This camera profile sub key has 2 value 'ResX' and 'ResY', being the camera resolution. The sub key 'StuckPixels' stores a list ofdefective pixels, together with the corrective pixels that should be used to correct a defective pixel. A defective pixel is defined as a value in the StuckPixels key. The value name defines the defective pixel. The value data defines 3 (exactly 3) pixels that will be used to correct. The defective pixel will be replaced by the average of the 3 corrective pixels. Of course you can put 3 times the same pixel if you want to have a defective pixel replaced by exactly 1 pixel. Just have a look at the image below and it will all become clear.
How to get the coordinates of your dead and corrective pixels?
Well that is the tricky part as long as I haven't developed a tool for this. This is how I did it:
I know... it's cumbersome. If there is sufficient interest I'll develop a tool for it.
When installed correctly, the filter 'Stuck Video Pixel Remover' shows up in Sony Vegas Movie Studio in the filter list on the Video FX tab. Add this filter to the clip you want to process for defective pixels.
Important note: make sure that the 'Stuck Video Pixel Remover' is the first filter in the filter chain. Any filter processed before the pixel correction propagates and distributes the defective pixels and will make the correction applied later on less effective.
Camera - Select the camera profile that is applicable for the video clip you want to process. The profiles are read from the registry (see previous section to learn how to create and add profiles). When the profile is selected, the defective pixels are listed. In case an error occurred during the profile reading, the Camera fields automatically resets again, hence it is not possible to select an invalid profile. Most common errors in the profile definitions are missing camera resolution fields or pixel definitions that are not in the scope of the camera resolution.
If resolution does not match... - If the resolution of the video frames that are passed by the host software (Sony Vegas) to StuckVideoPixelRemover do not match with the resolution that is specified in the selected camera profile you have 2 options
Don't apply any correction - The video will not be touched and just passed through to the next filter.
Interpolate to new resolution - StuckVideoPixelRemover will recalculate the defective and corrective pixel coordinates to match with the resolution in the clip to be processed.
Interpolate to new resolution is the recommended option. The filter is applied much more then you would think at first sight:
The preview window. The resolution of the preview window is a Preference setting you can set independent from the original clip resolution and the final rendering resolution. Each time you are previewing, the original video clip is resized to the preview resolution, processed by the applied filters and showed in the preview window.
The final rendering happens in the following order: original clip is resized to the final resolution that is defined in the rendering template that was selected for the project. This resized video is processed by the selected filters. The output of the filters is coded by the selected encoding format.
The best results are when the video clip is processed by the filter on its original resolution. This can be done by creating a rendering template that renders to this original resolution.
Donate - Any contribution to support further development of StuckVideoPixelRemover is greatly appreciated. This button will forward you to a PayPal form.
Questions, suggestions, bug reports can be sent to info@stuckvideopixelremover.com
© Shardana Comm.V