Continuing our series, Effects of Compression on Video Imagery, let’s explore deblocking filters.
Compression Encoders and Macroblocks
Compression encoders analyse each incoming video frame and divide them into encoding zones called macroblocks.
The number of pixels included in a block varies with the image detail. As well as how the encoding side analyses a frame and the sequence of frames. In the widely known sample image to the below, the size and shape of the macroblocks are defined by the image content. The finer the detail to be captured. Therefore, the smaller the number of pixels used in the macroblock. The smallest macroblock of the H.264 standard is 8×8. Each macroblock is converted to a matrix of numbers that define the average intensity of the block of pixels. Thus a representation of the gradient and direction of the gradient of the block. This data is what is used to reconstruct the image at the decode end.
A decoded compressed image without the application of a deblocking filter is shown here. A section of the decoded image is magnified where the reconstructed macroblocks can clearly be seen. The vector direction of the gradient of reconstructed block is also specified by the encoder. A consequence of using average intensity and colour gradient is that there are mismatches at the boundaries of the blocks. A powerful technique to hide the transitions between reconstructed macroblocks interpolates pixel values at the macroblock boundaries in an attempt to smooth out the transitions. This is deblock filtering. The decoder has the major responsibility to deblock filter. There are many deblock filter algorithms and several parameters that affect the filter profile and number of boundary pixels the deblock filter uses. There is not a “standard” deblock algorithm, however. Therefore, each decoder may produce a different result.
As can be seen in the last magnified view, the deblocking filter does not eliminate blocks; however, they are far less noticeable since the boundaries are smudged.
In order to really see the impact of macroblocks on an image, it is helpful to examine Monochrome Frames. Monochrome frames are easier to “see” as the eye is far more sensitive to changes in intensity than changes in colour.
In the sequence below, the images are the original, a decoded version without deblocking and the decoded and deblocked image. The loss of resolution is quite evident.
In this example, the macroblocks are large, but illustrate the point. Notice the sheet music on the piano. After encode/decode the music scale and notes are obliterated. This example suggests that that the effective resolution of the image is reduced and real information is lost.
In evidence, surveillance and analysis applications, these effects may not be acceptable. High resolution imagery is often needed to identify distant objects. Such as people, trucks, friend, foe, text. Or very small objects (fragments, rivets, seams, insects, etc.). As can be seen, if video is used in real time or later for analysis the use of compression may work against this goal. If the object of interest in any given scene falls below the dimensions of the macroblock, the detail needed to identify, classify, or otherwise characterize a small element in a frame may be lost. It may be lost forever, at least in a single frame. Over a series of frames, it may be possible to evaluate a small object as it crosses macroblock boundaries. However, a critical event, position, or identifier may only appear in one or two frames.
The point is that when capturing scenes that require discovery of fine details, extra care must be taken to scale the image on the sensor. Therefore, to ensure you have the detail needed when the video must be transported or saved in a compressed form. Scaling will result in reduced field of view of the observation area which may further hamper event capture.
Alternatively, one can transport or record uncompressed video and thereby simplifying the video capture. In order to transport in real time, the bandwidth will likely eliminate radio links and non-fiber Ethernet. If recording, devices that capture uncompressed video will ensure that everything the sensor sees is recorded, played back and archived in the video clip.
The EON Instrumentation model 6520 records uncompressed HD-SDI video. With the 6520 you can capture clips, play them back, playback in slow motion, single step frame to frame and loop on a subclip. Each frame presented is the full resolution and correct colour just as the camera collected it. The actual video samples can also be downloaded via Ethernet to a file using our 6520 DownloadVideo© GUI.
This GUI can also transcode the video to an uncompressed AVI clip, a compressed MP4 or Windows Media Video clip. Audio (if any) as well as imagery are captured, downloaded and transcoded. The video clip may also contain KLV metadata that during the time of recording can sample an array of information at the start each frame with time precision.
If the devil is in the details, the details are important. Single frame shots are an integral part of analysis or evidence retention. The 6520 Inserter-Recorder can be furnished with a removable 1Terabyte SSD. This capacity can hold a single 1080 clip of 151,500 frames. At 60 fps, 151K frames is 42 minutes of uncompressed 1080 video. However, it can hold 151K 1080 frames that may be comprised of many clips, up to 512 different clips can be captured to the SSD. The clips may even be different SD-SDI and HD-SDI formats.
For more information, please contact us.