Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

The Good, the Bad, and the Ugly: Neural Networks Straight from JPEG

Abstract: 

Over the past decade, convolutional neural networks (CNNs) have achieved state-of-the-art performance in many computer vision tasks. They can learn robust representations of image data by processing RGB pixels. Since image data are often stored in a compressed format, from which JPEG is the most widespread, a preliminary decoding process is demanded. Recently, the design of CNNs for processing JPEG compressed data has gained attention from the research community. They process DCT coefficients instead of RGB pixels, saving computation for decoding JPEG images, however, at the cost of increasing the computational complexity of the network. In this paper, we examine how spatial resolution and JPEG quality impacts on the performance of a state-of-the-art CNN designed to operate directly on the JPEG compressed domain. To alleviate its computational complexity, we propose a Frequency Band Selection (FBS) technique to select the most relevant DCT coefficients before feeding them to the network. Experiments were conducted on a subset of the ImageNet dataset considering both fine- and coarse-grained image classification tasks. Results show that such networks are resilient to JPEG quality but are susceptible to spatial resolution. Also, our FBS can reduce the computational complexity of the network while retaining a similar accuracy.

up
0 users have voted:

Paper Details

Authors:
Samuel Felipe dos Santos , Nicu Sebe , and Jurandy Almeida
Submitted On:
6 November 2020 - 9:30am
Short Link:
Type:
Presentation Slides
Event:
Presenter's Name:
Samuel Felipe dos Santos
Paper Code:
2821
Document Year:
2020
Cite

Document Files

Presentation slides

(14)

Subscribe

[1] Samuel Felipe dos Santos , Nicu Sebe , and Jurandy Almeida, "The Good, the Bad, and the Ugly: Neural Networks Straight from JPEG", IEEE SigPort, 2020. [Online]. Available: http://sigport.org/5546. Accessed: Nov. 29, 2020.
@article{5546-20,
url = {http://sigport.org/5546},
author = {Samuel Felipe dos Santos ; Nicu Sebe ; and Jurandy Almeida },
publisher = {IEEE SigPort},
title = {The Good, the Bad, and the Ugly: Neural Networks Straight from JPEG},
year = {2020} }
TY - EJOUR
T1 - The Good, the Bad, and the Ugly: Neural Networks Straight from JPEG
AU - Samuel Felipe dos Santos ; Nicu Sebe ; and Jurandy Almeida
PY - 2020
PB - IEEE SigPort
UR - http://sigport.org/5546
ER -
Samuel Felipe dos Santos , Nicu Sebe , and Jurandy Almeida. (2020). The Good, the Bad, and the Ugly: Neural Networks Straight from JPEG. IEEE SigPort. http://sigport.org/5546
Samuel Felipe dos Santos , Nicu Sebe , and Jurandy Almeida, 2020. The Good, the Bad, and the Ugly: Neural Networks Straight from JPEG. Available at: http://sigport.org/5546.
Samuel Felipe dos Santos , Nicu Sebe , and Jurandy Almeida. (2020). "The Good, the Bad, and the Ugly: Neural Networks Straight from JPEG." Web.
1. Samuel Felipe dos Santos , Nicu Sebe , and Jurandy Almeida. The Good, the Bad, and the Ugly: Neural Networks Straight from JPEG [Internet]. IEEE SigPort; 2020. Available from : http://sigport.org/5546