This Technical Brief has been provided by our valued supplier ITS.
The ITS 6520 records uncompressed HD-SDI video, play it back, view it frame by frame and play a subclip, but the actual video samples can be downloaded via Ethernet to a file using our 6520 Download Video GUI.
Once an HD video clip is downloaded to a PC, the DownloadVideo© GUI permits you to play it using an included executable version of the open source FFplay function. The FFplay is used by nearly every VLC based player software package. The 6520 Download Video kit also includes FFmpeg which is open source transcoding software that is also an underlying element of VLC players.
The DownloadVideo© GUI combines these external tools to render a software package that can download video from a 6520 via Ethernet, parse it and format it to a raw video file that may be played by our GUI and transcode it to AVI, MP4 or Windows Media Player formats.
Using the 6520 Inserter-Recorder and the DownloadVideo© GUI, ITS conducted a brief study about the effects of compression on a still shot taken from an HD video clip.
A 15 second 1080/30 clip was recorded on the 6520 then downloaded to a PC via an Ethernet over our company LAN. The DownloadVideo© GUI then transcoded this source file and generated and uncompressed video packaged in the AVI format.
In order to play this AVI file, the PC used must be able to process frame to frame imagery at an average disk transfer rate of 186 MB/sec. A SATA III disk drive has a max transfer rate of 6 Gb/sec which translates to about 600 MB/sec without access latency, seek delay and intervention by the operating system. A mass storage device of this class is needed to play uncompressed video clips. We played and captured screenshots using a PC having a Pentium i7 4 core processor and 32 GB of RAM running Windows 7 professional 64-bit. This is likely the minimum horsepower needed to play this video to neither skip frames nor slow the frame rate. The image below is a screenshot of one frame of AVI playback.
The source uncompressed file was also transcoded to an MP4 (H.264) file.
The next image is a screenshot of a frame of MP4 playback . The H.264 was being played by VLC which as we noted earlier uses FFplay to display video clips on a PC.
Both images are within a frame or two of the same uncompressed HD digital video clip source.
Even at this level, one can see several artifacts of compression. Note the colour shift. The uncompressed image is much closer to the real colour.
This is a common side effect when a compressor changes sampling from 4:2:2 as delivered by the camera to 4:2:0 sampling. This sampling change saves bits per pixel, but increases colour aliasing and changes the colour resolution from effectively 20 bits/pixel to 15 bits/pixel. Fewer shades even at the same pixel count reduce effective resolution of the reproduction. A 4:2:0 format shares one 20 bit colour sample with 4 pixels instead of 2 (4:2:2 sampling). Since the yellow component of a pixel colour is derived from the overall brightness (luma) and its related colour (red and blue) components, the colour of such four pixels will be somewhat different, particularly at edges and shade transitions. Colour errors are therefore an expected side effect of this type of compression.
If the time (46:32) in the lower left corner of each image is observed, the resolution loss is clearly evident. Observing the upper left corner, one can see that the stroke width yellow text (ITS overlay) is wider in the compressed image. While both are clearly readable, it is evident that resolution is compromised.
Looking around the frame, take a close look at the serial cables hanging from the shelf, the chassis in the lower left and the banks for BNC connectors and the VME board on the upper right corner. Also note that on the boxes on the lower right, appear to have text on them. The area circled in yellow will be studied in more detail below.
The images below are a zooms in of the compressed frame and note that the BNC connectors are now just gray blobs, the text is replaced by a gray bar , and the serial cables are much less defined.
As you zoom-in even closer, it becomes apparent that the image is a reconstruction comprised of macroblocks. Deblocking filters can hide the block edges at the expense of edge detail. The original pixels are not present in this image anywhere. None of this is very noticeable when things are in motion displaying a video clip. That is the magic of MPEG compression.
The scheme is designed to take advantage of human perception, bandwidth and sensitivity. Great while watching a movie but for analysis, the degradation of frame images may negative impact what can be seen. The “devil is in the detail” and details are lost.
As you zoom in more and more, the loss of detail becomes more and more apparent. The inserted zoom-in clearly shows that the text on the boxes is totally obliterated. This information is forever lost in the compressed image data.
As the elements in the image get near or below the size of an available macroblock (8×8 is the smallest) detail is replaced by a macroblock and gradient of colour. This effect is particularly noticeable in the BNC Connectors.
In contrast, the image at the right is a greater zoom-in of the uncompressed AVI frame. In this zoom, the serial cables are still clearly serial cables; the BNC connectors begin to reveal the centre conductor. The shade tones are continuous rather than a mosaic of macroblocks.
The evidence is clear, compression does compromise image quality and loses informative content. Compression does not restore the camera samples; it uses luma and colour value differentials to represent a group of pixels (16×16) that a decoder attempts to reproduce.
Similarly, in the zoom-in insert of the boxes, it is obvious that text is present. Image analysis may be able to reconstruct what the text is (the word KEYBOARD).
Using the crude tools (contrast, brightness, sharpens) begins to reveal a few of the letters that appear on the box.
We recorded a second clip where an object is in motion to see the effects of compression on moving elements. The image below is a screenshot of an uncompressed (AVI playback) frame of this clip. The moving object is signage about the ITS complete HD-SDI instrumentation solution. The frame has been cropped to show a close up of the time stamp, an area of the signage in motion and still space behind it.
The same raw clip was also transcoded to an MP4 file (compressed). While not exactly the same frame it is within 4 frames of the uncompressed sample. The cropping is the same area of the frame and is shown below.
As with the prior clip, a colour shift is seen due to the change in sampling resolution (4:2:2 to 4:2:0). The loss of bits/pixel (20 to 15) can be noticed in the still area and the shadows in the background.
What is new is an observable expansion of blur of the moving object in the compressed image. As one may recall, MPEG compression employs differential frames and prediction algorithms to code video and later decode it. It also uses macro-blocks that can effect edges as things are in motion. Lastly deblocking filters soften edges. One can visualise how these encoding and decoding tools could appear to expand blur.
Expansion of blur will obscure fine detail even sooner than with fixed images.
The AVI image represents the blur of the original image capture as the camera achieved it. The amount of blur in a frame is proportional to the speed of the moving object relative to the camera, the type of shutter used and the image capture needed by the camera sensor.
The expanded blur and detail loss is entirely introduced by the compression of encoding and decode blocking filters.
Zooming in a bit further, the loss of detail is even more apparent (Compare the images above). The blurring seems to be twice as wide as the camera image itself. One may ask “Is this what H.264 will do each time?”
Many factors will come into play to yield an answer. From encoder to encoder, the only element specified by a standard is the output stream and how it is coded. The image processing performed ahead of that is up to the encoder developer. Therefore it is likely that these effects will change from encoder to encoder. As the bit rate output is restricted by transport bandwidth and traffic, image quality will decline.
A part of the decline will be more blur. The nature of the deblocking filters used and their relative passbands is also decoder unique and may result in different blurring effects. So the answer; it depends.
The 6520 Inserter-Recorder captures uncompressed clips. These video clips may also contain KLV metadata. The DownloadVideo© GUI can receive recorded clips from the 6520. These clips are comprised of the 4:2:2 image samples delivered by the camera to the recording. The storage required for such clips is large, very large. Each 1080 frame (p or i) consumes 6.2 MB of storage space. The 15 second clip used for this study contains 455 1080 frames; more than 2,800 MB. Storage needs add up fast.
The compressed (MP4) file of this clip is only 4.8 MB; 580:1 reduction in storage needs. As one might expect, since there are objects moving in the second clip, the net compression is less, (1,274 MB to 3.1 MB; 424:1). The video processing data rate also is reduced when attempting to play the file. The MP4 file is perfect for general viewing and network distribution over a 100 Mb Ethernet network.
If the devil is in the details, the details are important. Single frame shots are an integral part of analysis or evidence retention. Compressing during recording will limit utility in these types of applications. For analysis, the image quality compromise may not be worth the storage savings.
The 6520 Inserter-Recorder can be furnished with a removable 1Terabyte SSD. This capacity can hold a single 1080 clip of 151,500 frames. At 60 fps, 151K frames is 42 minutes of uncompressed 1080 video. However, it can hold 151K 1080 frames that may be comprised of many clips, up to 512 different clips can be captured to the SSD. The clips may even be different SD-SDI and HD-SDI formats.
For further information on the ITS 6520 Inserter-Recorder, contact us.