Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi c om Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology ng th an co ng Digital Video cu u du o Tien Pham Van, Dr rer nat Hanoi University of Science and Technology CuuDuongThanCong.com https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology c om Agenda Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi cu u du o ng th an co ng • Introduction to video basics • Video data presentation and rendering • Video compression and communication CuuDuongThanCong.com https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi cu u du o ng th an co ng c om Introduction to Video Basics CuuDuongThanCong.com https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi Types of Video Signals an – – The primaries can either be RGB or a luminance-chrominance transformation of them (e.g., YIQ, YUV) Best color reproduction Requires more bandwidth and good synchronization of the three components ng – c om Component video each primary is sent as a separate video signal co • th Composite video color (chrominance) and luminance signals are mixed into a single carrier wave Some interference between the two signals is inevitable • cu u – du o ng • S-Video (Separated video, e.g., in S-VHS) a compromise between component analog video and the composite video It uses two lines, one for luminance and another for composite chrominance signal CuuDuongThanCong.com https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Analog Video Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi c om Analog video is represented as a continuous (time varying) signal; Digital video is represented as a sequence of digital images PAL (SECAM) Video 625 scan lines per frame, 25 frames per second (40 msec/frame) Interlaced, each frame is divided into fields, 312.5 lines/field Color representation: Uses YUV color model co an du o ng th 525 scan lines per frame, 30 fps (33.37 msec/frame) Interlaced, each frame is divided into fields, 262.5 lines/field 20 lines reserved for control information at the beginning of each field So a maximum of 485 lines of visible data ng NTSC Video cu u • Laserdisc and S-VHS have actual resolution of ~420 lines • Ordinary TV ~320 lines • Each line takes 63.5 microseconds to scan Color representation: • Uses YIQ color model CuuDuongThanCong.com https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi Frame Rate and Interlacing The basic idea is quite simple, single still frames are presented at a high enough rate so that persistence of vision integrates these still frames into motion co – c om Persistence of vision: The human eye retains an image for a fraction of a second after it views the image This property is essential to all visual display technologies ng • an Motion pictures originally set the frame rate at 16 frames per second This was rapidly found to be unacceptable and the frame rate was increased to 24 frames per second In Europe, this was changed to 25 frames per second, as the European power line frequency is 50 Hz When NTSC television standards were introduced, the frame rate was set at 30 Hz (1/2 the 60 Hz line frequency) Movies filmed at 24 frames per second are simply converted to 30 frames per second on television broadcasting • cu u du o ng th • CuuDuongThanCong.com https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi Frame Rate and Interlacing c om Unfortunately, there is no easy way to "put a shutter" in front of a television broadcast! Therefore, to arrange for two "flashes" per frame, the flashes are created by interlacing • cu u – du o ng th an co • For some reason, the brighter the still image presented to the viewer, the shorter the persistence of vision So, bright pictures require more frequent repetition If the space between pictures is longer than the period of persistence of vision then the image flickers Large bright theater projectors avoid this problem by placing rotating shutters in front of the image in order to increase the repetition rate by a factor of (to 48) or three (to 72) without changing the actual images ng • With interlacing, the number of "flashes" per frame is two, and the field rate is double the frame rate Thus, NTSC systems have a field rate of 59.94 Hz and PAL/SECAM systems a field rate of 50 Hz CuuDuongThanCong.com https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi c om Scanning Video cu u du o ng th an co ng • Video is obtained via raster scanning, which transforms a 3-D signal p(x, y, t) into a one-dimensional signal s(t) which t (time) can be transmitted FrameK • Progressive scanning: left-to-right and top-to-bottom – Samples in time: frames/sec Frame2 – Samples along y: lines Frame1 – Samples along x: pixels (only for digital video) • We perceive the images as continuous, not discrete: human visual system performs the interpolation ! • How many frames, lines, and pixels ? Progressive scanning CuuDuongThanCong.com https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi Interlaced Scanning an th ng A frame du o co ng c om • If the frame rate is too slow - > flickering and jagged movements • Tradeoff between spatial and temporal resolution – Slow moving objects with high spatial resolution – Fast moving objects with high frame rate • Interlaced scanning: scan all even lines, then scan all odd lines • A frame is divided into fields (sampled at different time) M Even field cu u Odd field CuuDuongThanCong.com https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi th an co ng R G cu u du o ng • Three basic colors R: Red G: Green B: Blue A picture consists of three images c om RGB Color Model CuuDuongThanCong.com B https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Bài tập Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi cu u du o ng th an co ng c om Cho ảnh màu kích thước 288x352 pixel, lấy mẫu 4:2:2, giá trị điểm ảnh nhận 0…255 (8bit/mẫu) Mã hóa JPEG cho ảnh trên, biết tỷ số nén cho ảnh chói Y 10 lần, tỷ số nén cho tín hiệu hiệu màu Cb,Cr 20 lần Tính tỷ số nén ảnh? 19 CuuDuongThanCong.com https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi Sample Quantization – Pixel Resolution du o ng th an co ng c om • Pixel resolution depends quantization levels/bits • Usually, bits for each luma/chroma sample when no compression 8bits/1byte per pixel for gray image, 24bits/3bytes for true color image cu u Luminace (gray) picture Num Level Bit (a) (Monochrome) (b) (c) (d) 16 (e) 32 (f) 64 CuuDuongThanCong.com https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Digital Video Analog TV is a continuous signal Digital TV uses discrete numeric values – Signal is sampled, and samples are quantized – 160 352 Image represented by pixel array 800 1280 ng QSIF (19Kp) 1152 co 120 • 720 Sub-sampling to reduce image resolution or size c om • • Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi SIF (82Kp) du o 486 ng 601 (300Kp) th an 240 cu 600 u SVGA (500Kp) ATV (1Mp) 720 Workstation (1Mp) 900 HDTV (2Mp) 1080 CuuDuongThanCong.com https://fb.com/tailieudientucntt 1920 Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Freq MHz 1050 16:9 P HDTV Eur, ana 1250 16:9 2.4 P HDTV NHK 1125 3.3 I 20 NTSC© 525 4:3 I 4.2 525 4:3 P 4.2 625 4:3 I 5.5 625 4:3 4.3 P 5.5 SECAM© 625 4:3 I SECAM 625 4:3 4.3 P 2.5H ng HDTV USA, ana Opt View dist co Aspect Ratio 16:9 ng th Name c om P/I an Lines du o HDTV Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi NTSC cu u PAL© PAL © : Conventional CuuDuongThanCong.com 22 https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi Computer Video Format – IRIS video board VINO takes NTSC video signal and after digitization can achieve frame resolution of 640x480 pixels, bits/pixel and fps SunVideo digitizer captures NTSC video signal in the form of an RGB signal with frame resolution of 320x240 pixels, bits/pixel and 30 fps ng – c om Depends on the i/p and o/p devices (digitizers) for motion video medium Digitizers differ in frame resolution, quantization and frame rate co • • – The Color Graphics Adapter (CGA): an Computer video controller standards The Enhanced Graphics Adapter (EGA): ng – th 320 x 240 pixels x bits/pixel = 16,000 bytes (storage capacity per image) – du o 640 x 350 pixels x bits/pixel = 112,000 bytes The Video Graphics Array (VGA): 640 x 480 pixels x bits/pixel = 307,200 bytes u The 8514/A Display Adapter Mode: cu – 1024 x 768 pixels x bits/pixel = 786,432 bytes – The Extended Graphics Array (XGA): 1024x768 at 256 colors or 640x480 at 65,000 colors – The Super VGA (SVGS): Upto 1024x768 pixels x 24 bits/pixel = 2,359,296 bytes 23 CuuDuongThanCong.com https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi cu u du o ng th an co ng c om Video Compression 24 CuuDuongThanCong.com https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi c om Introduction (1/2) an co ng • Why video compression technique is important ? • One movie video without compression cu u du o ng th – 720 x 480 pixels per frame – 30 frames per second – Total 90 minutes – Full color – The full data quantity = 167.96 G bytes !! 25 CuuDuongThanCong.com https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi c om Introduction (2/2) co ng • What is the difference between video compression and image compression? an – Temporal Redundancy th • Coding method to remove redundancy du o ng – Intraframe Coding • Remove spatial redundancy cu u – Interframe Coding • Remove temporal redundancy 26 CuuDuongThanCong.com https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi c om Desired Features Better compression Improved quality Interactivity and Manipulation of Content Error Resilience Processing of content in the compressed domain • Identification and selective coding/decoding of the object of interest • Facilitate Search / Indexing (MPEG-7) cu u du o ng th an co ng • • • • • CuuDuongThanCong.com https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi ng c om Time table VC-1/VC-2 H.264 co H.26L H.265 an H.263 MPEG4 du o ng th H.261 MPEG2/H.262 cu u MPEG1 JPEG Year 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 … 2010 28 CuuDuongThanCong.com https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi c om Evolution of Video Compression Standards MPEG ITU-T MPEG-1 Video-CD an co ng H.261 Video Telephony du o ng th H.262/MPEG-2 Digital TV/DVD MPEG-4 Visual Object-based Coding cu u H.263 Video Conferencing H.264 MPEG-4 AVC CuuDuongThanCong.com https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Where used? Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi – MPEG-1 c om Video-CD Usually mpg or mpeg files are MPEG-1 DAB Digital Radio is MP2 (MPEG-1 Layer 2) MP3 files (MPEG-1 Layer 3) ng • • • • co – MPEG-2: th an • vob, m2v, rarely mpg files • Anything to with DVD High Quality AVI files Video Phones DivX Some advanced audio players support MPEG-4 Advanced Audio Coding (AAC) cu • • • • du o – MPEG-4: u • Digital TV (DVB) ng – Camcorders, DVD players, DVD recorders CuuDuongThanCong.com https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Where used? Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi –H.263/+/++ co ng c om • NetMeeting and similar video-chat • Network streaming application, video phone… – H.264 cu – VC-1, VC-2 u du o ng th an • Video Conferencing: over different networks • Multimedia Streaming: live and on-demand • Multimedia Messaging Services (MMS) • Blu-ray, Digital Video Broadcasting, iPod Video, HD DVD • Video on Internet, • HDTV broadcast, UHDTV CuuDuongThanCong.com https://fb.com/tailieudientucntt Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi R-D Performance of MPEG Codecs 50 c om 48 ng 46 co an 42 th 40 du o ng 38 36 32 350 u 34 cu PSNR (Y) 44 450 550 650 750 850 950 Bit rate (kbps) MPEG-1 CuuDuongThanCong.com MPEG-2 MPEG-4 H.264 https://fb.com/tailieudientucntt 1050 Pham Van Tien, Dr rer nat , Embedded Networking Research Group Faculty of Elec and Telecom, Hanoi University of Science and Technology c om Questions Email: tienpv-fet@mail.hut.edu.vn C9-411 Dai Co Viet str 1, Hanoi cu u du o ng th an co ng • What are video/audio codecs ? Name some popular codecs that your media players support What are disadvantages of using specific codecs ? • What is container format? Name some examples • Codecs and Formats CuuDuongThanCong.com https://fb.com/tailieudientucntt ... HDTV Eur, ana 1250 16:9 2.4 P HDTV NHK 1125 3. 3 I 20 NTSC© 525 4 :3 I 4.2 525 4 :3 P 4.2 625 4 :3 I 5.5 625 4 :3 4 .3 P 5.5 SECAM© 625 4 :3 I SECAM 625 4 :3 4 .3 P 2.5H ng HDTV USA, ana Opt View dist co... Evolution of Video Compression Standards MPEG ITU-T MPEG-1 Video- CD an co ng H.261 Video Telephony du o ng th H.262/MPEG-2 Digital TV/DVD MPEG-4 Visual Object-based Coding cu u H.2 63 Video Conferencing... Computer Video Format – IRIS video board VINO takes NTSC video signal and after digitization can achieve frame resolution of 640x480 pixels, bits/pixel and fps SunVideo digitizer captures NTSC video