Sorry, you need to enable JavaScript to visit this website.

Supplementary Material for Effective relationship between characteristics of training data and learning progress on knowledge distillation

Error message

  • The specified file temporary://filehDuOw4 could not be copied, because the destination directory is not properly configured. This may be caused by a problem with file or directory permissions. More information is available in the system log.
  • The specified file temporary://fileH5PeNX could not be copied, because the destination directory is not properly configured. This may be caused by a problem with file or directory permissions. More information is available in the system log.
  • The specified file temporary://fileamKyJA could not be copied, because the destination directory is not properly configured. This may be caused by a problem with file or directory permissions. More information is available in the system log.
  • The specified file temporary://filect5MBk could not be copied, because the destination directory is not properly configured. This may be caused by a problem with file or directory permissions. More information is available in the system log.
  • The specified file temporary://fileRfY9sV could not be copied, because the destination directory is not properly configured. This may be caused by a problem with file or directory permissions. More information is available in the system log.
  • The specified file temporary://filesrvbfA could not be copied, because the destination directory is not properly configured. This may be caused by a problem with file or directory permissions. More information is available in the system log.
  • The specified file temporary://fileMk1Tjw could not be copied, because the destination directory is not properly configured. This may be caused by a problem with file or directory permissions. More information is available in the system log.
  • The specified file temporary://filel9gUCW could not be copied, because the destination directory is not properly configured. This may be caused by a problem with file or directory permissions. More information is available in the system log.
  • The specified file temporary://file07F6Jo could not be copied, because the destination directory is not properly configured. This may be caused by a problem with file or directory permissions. More information is available in the system log.
DOI:
10.60864/7d6c-ry63
Citation Author(s):
Anonymous
Submitted by:
Anonymous Submission
Last updated:
6 February 2025 - 4:02am
Document Type:
Supplementary material
 

In image recognition, knowledge distillation is a valuable approach to train a compact model with high accuracy by exploiting outputs of a highly accurate large model as correct labels. In knowledge distillation, studies have shown the usefulness of data with high entropy output generated by image mix data augmentation techniques. Other strategies such as curriculum learning have also been proposed to improve model generalization by the control of the difficulty of training data over the learning process. In this paper, we explore the relationship between the learning process and data characteristics, focusing on the entropy of output distribution and learning difficulty in knowledge distillation. To validate this relationship, we propose a method to readily generate data that yields high entropy output and adjust the degree of learning difficulty by controlling bounds and a sampling range for mix ratios in mixing images between classes. In evaluation experiments using multiple datasets, our proposed method outperformed conventional approaches in terms of accuracy. This supplementary material gives details of experimental results.

up
0 users have voted: