Sorry, you need to enable JavaScript to visit this website.

Beyond the Limit of Weight-Sharing: Pioneering Space-Evolving NAS with Large Language Models

DOI:
10.60864/xg1v-d042
Citation Author(s):
Submitted by:
Xiu Su
Last updated:
6 June 2024 - 10:28am
Document Type:
Presentation Slides
 

Large language models (LLMs) offer impressive performance across diverse fields, but their increasing complexity raises both design costs and the need for specialized expertise. These challenges are intensified for Neural Architecture Search (NAS) methods reliant on weight-sharing techniques. This paper introduces GNAS, a new NAS method that boosts the search process with the aid of LLMs for efficient model discovery. With insights from existing architectures, GNAS swiftly identifies superior models that can adapt to changing resource constraints. We provide a mathematical framework to facilitate the transfer of knowledge across different model sizes, thereby improving search efficiency. Our experiments conducted on ImageNet, NAS-Bench-Macro, and Channel-Bench-Macro confirm the effectiveness of GNAS across both CNN and Transformer architectures.

up
0 users have voted:

Comments

This is the presentation slides of the paper titled "BEYOND THE LIMIT OF WEIGHT-SHARING: PIONEERING SPACE-EVOLVING NAS WITH LARGE LANGUAGE MODELS".