Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.
5555
Yan, J.; Liu, J.; Xu, H.; Wang, Z.; Qiao, C.
Peaches: Personalized Federated Learning with Neural Architecture Search in Edge Computing Journal Article
In: IEEE Transactions on Mobile Computing, no. 01, pp. 1-17, 5555, ISSN: 1558-0660.
@article{10460163,
title = {Peaches: Personalized Federated Learning with Neural Architecture Search in Edge Computing},
author = {J. Yan and J. Liu and H. Xu and Z. Wang and C. Qiao},
doi = {10.1109/TMC.2024.3373506},
issn = {1558-0660},
year = {5555},
date = {5555-03-01},
urldate = {5555-03-01},
journal = {IEEE Transactions on Mobile Computing},
number = {01},
pages = {1-17},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
abstract = {In edge computing (EC), federated learning (FL) enables numerous distributed devices (or workers) to collaboratively train AI models without exposing their local data. Most works of FL adopt a predefined architecture on all participating workers for model training. However, since workers' local data distributions vary heavily in EC, the predefined architecture may not be the optimal choice for every worker. It is also unrealistic to manually design a high-performance architecture for each worker, which requires intense human expertise and effort. In order to tackle this challenge, neural architecture search (NAS) has been applied in FL to automate the architecture design process. Unfortunately, the existing federated NAS frameworks often suffer from the difficulties of system heterogeneity and resource limitation. To remedy this problem, we present a novel framework, termed Peaches, to achieve efficient searching and training in the resource-constrained EC system. Specifically, the local model of each worker is stacked by base cell and personal cell, where the base cell is shared by all workers to capture the common knowledge and the personal cell is customized for each worker to fit the local data. We determine the number of base cells, shared by all workers, according to the bandwidth budget on the parameters server. Besides, to relieve the data and system heterogeneity, we find the optimal number of personal cells for each worker based on its computing capability. In addition, we gradually prune the search space during training to mitigate the resource consumption. We evaluate the performance of Peaches through extensive experiments, and the results show that Peaches can achieve an average accuracy improvement of about 6.29% and up to 3.97× speed up compared with the baselines.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2024
Venske, Sandra Mara Scós; de Almeida, Carolina Paula; Delgado, Myriam Regattieri
Metaheuristics and machine learning: an approach with reinforcement learning assisting neural architecture search Journal Article
In: Journal of Heuristics, 2024.
@article{Venske-jh24a,
title = {Metaheuristics and machine learning: an approach with reinforcement learning assisting neural architecture search},
author = {
Sandra Mara Scós Venske and Carolina Paula de Almeida and Myriam Regattieri Delgado
},
url = {https://link.springer.com/article/10.1007/s10732-024-09526-1},
doi = {https://doi.org/10.1007/s10732-024-09526-1},
year = {2024},
date = {2024-05-16},
urldate = {2024-05-16},
journal = { Journal of Heuristics},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Guillaume, Lacharme; Hubert, Cardot; Christophe, Lente; Nicolas, Monmarche
The limitations of differentiable architecture search Journal Article
In: Pattern Analysis and Applications, 2024.
@article{Guillaume-paa24a,
title = {The limitations of differentiable architecture search},
author = {
Lacharme Guillaume and Cardot Hubert and Lente Christophe and Monmarche Nicolas
},
url = {https://link.springer.com/article/10.1007/s10044-024-01260-5},
year = {2024},
date = {2024-04-12},
urldate = {2024-04-12},
journal = {Pattern Analysis and Applications},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Wang, Yili; Chen, Jiamin; Li, Qiutong; He, Changlong; Gao, Jianliang
Graph neural architecture search with heterogeneous message-passing mechanisms Journal Article
In: Knowledge and Information Systems , 2024.
@article{Wang-kis24a,
title = {Graph neural architecture search with heterogeneous message-passing mechanisms},
author = {Yili Wang and Jiamin Chen and Qiutong Li and Changlong He and Jianliang Gao
},
url = {https://link.springer.com/article/10.1007/s10115-024-02090-x},
year = {2024},
date = {2024-04-12},
urldate = {2024-04-12},
journal = {Knowledge and Information Systems },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Ma, Benteng; Zhang, Jing; Xia, Yong; Tao, Dacheng
VNAS: Variational Neural Architecture Search Journal Article
In: International Journal of Computer Vision , 2024.
@article{nokey,
title = {VNAS: Variational Neural Architecture Search},
author = {
Benteng Ma and Jing Zhang and Yong Xia and Dacheng Tao
},
url = {https://link.springer.com/article/10.1007/s11263-024-02014-w},
year = {2024},
date = {2024-04-01},
urldate = {2024-04-01},
journal = { International Journal of Computer Vision },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Garavagno, Andrea Mattia; Ragusa, Edoardo; Frisoli, Antonio; Gastaldo, Paolo
An Affordable Hardware-Aware Neural Architecture Search for Deploying Convolutional Neural Networks on Ultra-Low-Power Computing Platforms Journal Article
In: IEEE Sensors Letters, 2024.
@article{Garavagno-sensorsletter24a,
title = {An Affordable Hardware-Aware Neural Architecture Search for Deploying Convolutional Neural Networks on Ultra-Low-Power Computing Platforms},
author = {Andrea Mattia Garavagno and Edoardo Ragusa and Antonio Frisoli and Paolo Gastaldo},
url = {https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10496186&tag=1},
year = {2024},
date = {2024-04-01},
urldate = {2024-04-01},
journal = {IEEE Sensors Letters},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Nasrullah, Nasrullah; Sang, Jun; Alam, Mohammad S.; Mateen, Muhammad; Cai, Bin; Hu, Haibo
Automated Lung Nodule Detection and Classification Using Deep Learning Combined with Multiple Strategies Journal Article
In: Sensors, 2024.
@article{Nasrullah,
title = {Automated Lung Nodule Detection and Classification Using Deep Learning Combined with Multiple Strategies},
author = {Nasrullah Nasrullah and Jun Sang and Mohammad S. Alam and Muhammad Mateen and Bin Cai and Haibo Hu},
url = {https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6749467/},
doi = {10.3390/s19173722},
year = {2024},
date = {2024-04-01},
urldate = {2024-04-01},
journal = {Sensors},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Berezsky, O. M.; Liashchynskyi, P. B.
METHOD OF GENERATIVE-ADVERSARIAL NETWORKS SEARCHING ARCHITECTURES FOR BIOMEDICAL IMAGES SYNTHESIS Journal Article
In: Radio Electronics, Computer Science, Control, no. 1, pp. 104, 2024.
@article{Berezsky_Liashchynskyi_2024,
title = {METHOD OF GENERATIVE-ADVERSARIAL NETWORKS SEARCHING ARCHITECTURES FOR BIOMEDICAL IMAGES SYNTHESIS},
author = {O. M. Berezsky and P. B. Liashchynskyi},
url = {http://ric.zntu.edu.ua/article/view/300976},
doi = {10.15588/1607-3274-2024-1-10},
year = {2024},
date = {2024-04-01},
urldate = {2024-04-01},
journal = {Radio Electronics, Computer Science, Control},
number = {1},
pages = {104},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Elghazi, Khalid; Ramchoun, Hassan; Masrour, Tawfik
Enhancing CNN structure and learning through NSGA-II-based multi-objective optimization Journal Article
In: Evolving Systems , 2024.
@article{Elghazi-es24a,
title = {Enhancing CNN structure and learning through NSGA-II-based multi-objective optimization},
author = {
Khalid Elghazi and Hassan Ramchoun and Tawfik Masrour
},
url = {https://link.springer.com/article/10.1007/s12530-024-09574-9},
year = {2024},
date = {2024-04-01},
journal = {Evolving Systems },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Liang, Jiayu; Cao, Hanqi; Lu, Yaxin; Su, Mingming
Architecture search of accurate and lightweight CNNs using genetic algorithm Journal Article
In: Genetic Programming and Evolvable Machines , 2024.
@article{Linag-gpem24a,
title = {Architecture search of accurate and lightweight CNNs using genetic algorithm},
author = {
Jiayu Liang and Hanqi Cao and Yaxin Lu and Mingming Su
},
url = {https://link.springer.com/article/10.1007/s10710-024-09484-4},
year = {2024},
date = {2024-04-01},
journal = { Genetic Programming and Evolvable Machines },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Wang, Xianbao; Liu, Pengfei; Xiang, Sheng; Weng, Yangkai; Yao, Minghai
Search on dual-space: discretization accuracy-based architecture search for person re-identification Journal Article
In: The Visual Computer , 2024.
@article{Wang-vc24a,
title = {Search on dual-space: discretization accuracy-based architecture search for person re-identification},
author = {
Xianbao Wang and Pengfei Liu and Sheng Xiang and Yangkai Weng and Minghai Yao
},
url = {https://link.springer.com/article/10.1007/s00371-024-03308-3},
year = {2024},
date = {2024-03-28},
journal = {The Visual Computer },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Kapoor, Rahul; Pillay, Nelishia
A genetic programming approach to the automated design of CNN models for image classification and video shorts creation Journal Article
In: Genetic Programming and Evolvable Machines, 2024.
@article{nokey,
title = {A genetic programming approach to the automated design of CNN models for image classification and video shorts creation},
author = {
Rahul Kapoor and Nelishia Pillay
},
url = {https://link.springer.com/article/10.1007/s10710-024-09483-5},
year = {2024},
date = {2024-03-14},
journal = {Genetic Programming and Evolvable Machines},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
jin, Cong; Huang, Jinjie; Chen, Yuanjian
Neural architecture search via progressive partial connection with attention mechanism Journal Article
In: scientific reports , 2024.
@article{jin,
title = {Neural architecture search via progressive partial connection with attention mechanism},
author = {Cong jin and Jinjie Huang and Yuanjian Chen},
url = {https://www.nature.com/articles/s41598-024-57236-2},
year = {2024},
date = {2024-03-01},
urldate = {2024-03-01},
journal = { scientific reports },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
CHEN, BOYU
Neural Architecture Search for Convolutional and Transformer Deep Neural Networks PhD Thesis
2024.
@phdthesis{CHEN-phd24a,
title = {Neural Architecture Search for Convolutional and Transformer Deep Neural Networks},
author = {BOYU CHEN},
url = {https://scholar.google.de/scholar_url?url=https://ses.library.usyd.edu.au/bitstream/handle/2123/32334/chen_b_thesis.pdf%3Fsequence%3D1&hl=de&sa=X&d=15530204168501603894&ei=b9PvZbWAGsfTy9YPt6iP6AU&scisig=AFWwaeZNQdCufcbiu0EUFQdEEyKz&oi=scholaralrt&hist=mvciDDAAAAAJ:2945779489622371749:AFWwaeYSwjSBxI9k5p1JRsFqGwve&html=&pos=1&folt=kw},
year = {2024},
date = {2024-03-01},
urldate = {2024-03-01},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
Luo, Zhirui
DEEP LEARNING METHODS FOR SMART METER DATA ANALYTICS AND APPLICATIONS PhD Thesis
2024.
@phdthesis{luo-phd23a,
title = {DEEP LEARNING METHODS FOR SMART METER DATA ANALYTICS AND APPLICATIONS},
author = {Zhirui Luo},
url = {https://www.proquest.com/docview/2915480009?pq-origsite=gscholar&fromopenview=true&sourcetype=Dissertations%20&%20Theses},
year = {2024},
date = {2024-03-01},
urldate = {2024-03-01},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
(Ed.)
PerFedRLNAS: One-for-All Personalized Federated Neural Architecture Search Collection
2024.
@collection{Yao-aaai24a,
title = {PerFedRLNAS: One-for-All Personalized Federated Neural Architecture Search},
author = {Dixi Yao and Baochun Li},
url = {https://iqua.ece.toronto.edu/papers/dixiyao-aaai24.pdf},
year = {2024},
date = {2024-03-01},
urldate = {2024-03-01},
booktitle = {AAAI 2024},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
Shen, Yanting; Lu, Lei; Zhu, Tingting; Wang, XInshao; Clifton, Lei; chen, Zhengming; Clarke, Robert; Clifton, David A.
AutoNet-Generated Deep Layer-Wise Convex Networks for ECG Classification Journal Article
In: IEEE Trans Pattern Anal Mach Intell , 2024.
@article{Shen-tpami24a,
title = {AutoNet-Generated Deep Layer-Wise Convex Networks for ECG Classification},
author = {Yanting Shen and Lei Lu and Tingting Zhu and XInshao Wang and Lei Clifton and Zhengming chen and Robert Clarke and David A. Clifton},
url = {https://pubmed.ncbi.nlm.nih.gov/38512733/},
doi = {10.1109/TPAMI.2024.3378843 },
year = {2024},
date = {2024-02-01},
urldate = {2024-02-01},
journal = { IEEE Trans Pattern Anal Mach Intell },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Liang, Zixuan; Sun, Yanan
Evolutionary Neural Architecture Search for Multivariate Time Series Forecastin Journal Article
In: Proceedings of Machine Learning Research , 2024.
@article{Liang-jmlr24a,
title = {Evolutionary Neural Architecture Search for Multivariate Time Series Forecastin},
author = {Zixuan Liang and Yanan Sun},
url = {https://proceedings.mlr.press/v222/liang24a/liang24a.pdf},
year = {2024},
date = {2024-02-01},
urldate = {2024-02-01},
journal = {Proceedings of Machine Learning Research },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Cunha, Leandro; Zhang, Li; Sowan, Bilal; Lim, Chee Peng; Kong, Yinghui
Video deepfake detection using Particle Swarm Optimization improved deep neural networks Journal Article
In: Neural Computing and Applications , 2024.
@article{Cunha-nea24a,
title = {Video deepfake detection using Particle Swarm Optimization improved deep neural networks},
author = {
Leandro Cunha and Li Zhang and Bilal Sowan and Chee Peng Lim and Yinghui Kong
},
url = {https://link.springer.com/article/10.1007/s00521-024-09536-x},
year = {2024},
date = {2024-02-01},
urldate = {2024-02-01},
journal = {Neural Computing and Applications },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Meng, Fanfei; Wang, Chen-Ao; Brown, Alexander
Evolution and Efficiency in Neural Architecture Search: Bridging the Gap Between Expert Design and Automated Optimization Journal Article
In: 2024.
@article{Meng_2024,
title = {Evolution and Efficiency in Neural Architecture Search: Bridging the Gap Between Expert Design and Automated Optimization},
author = {Fanfei Meng and Chen-Ao Wang and Alexander Brown},
url = {http://dx.doi.org/10.36227/techrxiv.170792628.85027690/v1},
doi = {10.36227/techrxiv.170792628.85027690/v1},
year = {2024},
date = {2024-02-01},
urldate = {2024-02-01},
publisher = {Institute of Electrical and Electronics Engineers (IEEE)},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Xu, Hongshang; Dong, Bei; Liu, Xiaochang; Wu, Xiaojun
Deep Neural Network Architecture Search via Decomposition-Based Multi-Objective Stochastic Fractal Search Proceedings Article
In: Intelligent Automation & Soft Computing, 2024.
@inproceedings{Xu-IASC24a,
title = {Deep Neural Network Architecture Search via Decomposition-Based Multi-Objective Stochastic Fractal Search},
author = {Hongshang Xu and Bei Dong and Xiaochang Liu and Xiaojun Wu},
url = {https://cdn.techscience.cn/files/iasc/2023/TSP_IASC-38-2/TSP_IASC_41177/TSP_IASC_41177.pdf},
doi = {10.32604/iasc.2023.041177},
year = {2024},
date = {2024-02-01},
urldate = {2024-02-01},
booktitle = {Intelligent Automation & Soft Computing},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Dou, Huanzhang; Zhang, Pengyi; Zhao, Yuhan; Jin, Lu; Li, Xi
CLASH: Complementary Learning with Neural Architecture Search for Gait Recognition Bachelor Thesis
2024.
@bachelorthesis{Dou-tip24a,
title = { CLASH: Complementary Learning with Neural Architecture Search for Gait Recognition },
author = {Huanzhang Dou and Pengyi Zhang and Yuhan Zhao and Lu Jin and Xi Li
},
url = {https://pubmed.ncbi.nlm.nih.gov/38363666/},
doi = { 10.1109/TIP.2024.3360870 },
year = {2024},
date = {2024-02-01},
urldate = {2024-02-01},
journal = { IEEE Trans Image Process },
keywords = {},
pubstate = {published},
tppubtype = {bachelorthesis}
}
Ma, L.; Kang, H.; Yu, G.; Li, Q.; He, Q.
Single-Domain Generalized Predictor for Neural Architecture Search System Journal Article
In: IEEE Transactions on Computers, no. 01, pp. 1-14, 2024, ISSN: 1557-9956.
@article{10438213,
title = {Single-Domain Generalized Predictor for Neural Architecture Search System},
author = {L. Ma and H. Kang and G. Yu and Q. Li and Q. He},
doi = {10.1109/TC.2024.3365949},
issn = {1557-9956},
year = {2024},
date = {2024-02-01},
urldate = {2024-02-01},
journal = {IEEE Transactions on Computers},
number = {01},
pages = {1-14},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
abstract = {Performance predictors are used to reduce architecture evaluation costs in neural architecture search, which however suffers from a large amount of budget consumption in annotating substantial architectures trained from scratch. Hence, how to leverage existing annotated architectures to train a generalized predictor to find the optimal architecture on unseen target search spaces becomes a new research topic. To solve this issue, we propose a Single-Domain Generalized Predictor (SDGP), which aims to make the predictor only trained on a single source search space but perform well on target search spaces. In meta-learning, we firstly adopt feature extractor in learning the domain-invariant features of the architectures. Then, a neural predictor is trained to map the architectures to the accuracy of the candidate architectures over the target domain simulated on the source search space. Moreover, a novel multi-head attention driven regularizer is designed to regulate the predictor to further improve the generalization ability of the predictor for the feature extractor. A series of experimental results have shown that the proposed predictor outperforms the state-of-the-art predictors in generalization and achieves significant performance gains in finding the optimal architectures with test error 2.40% on CIFAR-10 and 23.20% on ImageNet1k within 0.01 GPU days.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Gong, Tao; Ma, Yongjie; Xu, Yang; Song, Changwei
Efficient evolutionary neural architecture search based on hybrid search space Journal Article
In: International Journal of Machine Learning and Cybernetics, 2024.
@article{Gong-ijmlc24a,
title = {Efficient evolutionary neural architecture search based on hybrid search space},
author = {Tao Gong and Yongjie Ma and Yang Xu and Changwei Song},
url = {https://link.springer.com/article/10.1007/s13042-023-02094-z},
year = {2024},
date = {2024-02-01},
urldate = {2024-02-01},
journal = {International Journal of Machine Learning and Cybernetics},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
(Ed.)
Personalized Federated Learning via Knowledge Sharing-based Model Structure Adaption Collection
2024.
@collection{wang-ijcnn23a,
title = {Personalized Federated Learning via Knowledge Sharing-based Model Structure Adaption},
author = {Xiaochan Wang and Zhi Wang},
url = {http://zwang.inflexionlab.org/publications/FedKMA_IJCNN2023.pdf},
year = {2024},
date = {2024-02-01},
urldate = {2024-02-01},
booktitle = {IJCNN 2023},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
Bouali, Yassamine Lala; Ahmed, Olfa Ben; Bradai, Abbas; Mazouzi, Smaine
Towards Efficient Driver Distraction Detection with DARTS-Optimized Lightweight Models Proceedings Article
In: ICAART 2024, Rome, Italy, 2024.
@inproceedings{lalabouali:hal-04447892,
title = {Towards Efficient Driver Distraction Detection with DARTS-Optimized Lightweight Models},
author = {Yassamine Lala Bouali and Olfa Ben Ahmed and Abbas Bradai and Smaine Mazouzi},
url = {https://hal.science/hal-04447892},
year = {2024},
date = {2024-02-01},
urldate = {2024-02-01},
booktitle = {ICAART 2024},
address = {Rome, Italy},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Yang, Yi; Wei, Jiaxuan; Yu, Yhixuan; Zhang, Ruisheng
Multi-label neural architecture search for chest radiography image classification Journal Article
In: Multimedia Systems, 2024.
@article{nokey,
title = {Multi-label neural architecture search for chest radiography image classification},
author = {Yi Yang and Jiaxuan Wei and Yhixuan Yu and Ruisheng Zhang},
url = {https://link.springer.com/article/10.1007/s00530-023-01215-6},
year = {2024},
date = {2024-01-13},
urldate = {2024-01-13},
journal = { Multimedia Systems},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
LI, PEIXIA
Deep Neural Networks for Visual Object Tracking: An Investigation of Performance Optimization PhD Thesis
2024.
@phdthesis{li-phd23a,
title = {Deep Neural Networks for Visual Object Tracking: An Investigation of Performance Optimization},
author = {PEIXIA LI},
url = {https://scholar.google.de/scholar_url?url=https://ses.library.usyd.edu.au/bitstream/handle/2123/32052/li_p_thesis.pdf%3Fsequence%3D1%26isAllowed%3Dy&hl=de&sa=X&d=18223269356716447880&ei=EsGfZemVKsKAy9YP4bqs-Ac&scisig=AFWwaeYD2Xl7kLWLITGKadrDAlTa&oi=scholaralrt&hist=mvciDDAAAAAJ:2945779489622371749:AFWwaeYSwjSBxI9k5p1JRsFqGwve&html=&pos=2&folt=kw},
year = {2024},
date = {2024-01-12},
urldate = {2024-01-12},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
Dong, Minjing
Boosting Adversarial Robustness via Neural Architecture Search and Design PhD Thesis
2024.
@phdthesis{dong-phd23a,
title = {Boosting Adversarial Robustness via Neural Architecture Search and Design},
author = {Minjing Dong },
url = {https://ses.library.usyd.edu.au/bitstream/handle/2123/32060/dong_md_thesis.pdf?sequence=1&isAllowed=y},
year = {2024},
date = {2024-01-12},
urldate = {2024-01-12},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
(Ed.)
ENHANCING GAN PERFORMANCE THROUGH NEURAL ARCHITECTURE SEARCH AND TENSOR DECOMPOSITION Collection
2024.
@collection{Pulakurthi-icassp24a,
title = {ENHANCING GAN PERFORMANCE THROUGH NEURAL ARCHITECTURE SEARCH AND TENSOR DECOMPOSITION},
author = {Prasanna Reddy Pulakurthi and Mahsa Mozaffari and Majid Rabbani and Jamison Heard and Raghuveer Rao},
url = {https://mahsamozaffari.com/wp-content/uploads/2023/11/ICASSP_2024.pdf},
year = {2024},
date = {2024-01-02},
urldate = {2024-01-02},
booktitle = {ICASSP 2024},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
Lu, Aojun; Feng, Tao; Yuan, Hangjie; Song, Xiaotian; Sun, Yanan
Revisiting Neural Networks for Continual Learning: An Architectural Perspective Technical Report
2024.
@techreport{lu2024revisiting,
title = {Revisiting Neural Networks for Continual Learning: An Architectural Perspective},
author = {Aojun Lu and Tao Feng and Hangjie Yuan and Xiaotian Song and Yanan Sun},
url = {https://arxiv.org/abs/2404.14829},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
Broni-Bediako, Clifford; Xia, Junshi; Yokoya, Naoto
Unsupervised Domain Adaptation Architecture Search with Self-Training for Land Cover Mapping Technical Report
2024.
@techreport{bronibediako2024unsupervised,
title = {Unsupervised Domain Adaptation Architecture Search with Self-Training for Land Cover Mapping},
author = {Clifford Broni-Bediako and Junshi Xia and Naoto Yokoya},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
Wang, Yanghu; Zhou, Zheng; Yang, Laihao; Gao, Robert X.; Yan, Ruqiang
Wavelet-driven differentiable architecture search for planetary gear fault diagnosis Journal Article
In: Journal of Manufacturing Systems, vol. 74, pp. 587-593, 2024, ISSN: 0278-6125.
@article{WANG2024587,
title = {Wavelet-driven differentiable architecture search for planetary gear fault diagnosis},
author = {Yanghu Wang and Zheng Zhou and Laihao Yang and Robert X. Gao and Ruqiang Yan},
url = {https://www.sciencedirect.com/science/article/pii/S0278612524000839},
doi = {https://doi.org/10.1016/j.jmsy.2024.04.017},
issn = {0278-6125},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Journal of Manufacturing Systems},
volume = {74},
pages = {587-593},
abstract = {With the advancement of artificial intelligence and the accumulation of industrial big data, intelligent diagnosis methods based on deep learning have become the mainstream for diagnosing mechanical faults in manufacturing systems. Despite this, developing high-performance neural network models for specific tasks necessitates a substantial amount of expert knowledge. This consumes a considerable amount of time in the trial-and-error process, thereby constraining the progress in neural network development. Neural architecture search (NAS) offers a solution to address this problem. However, the feature extraction operators in the existing NAS search space are primarily imported from other fields, leading to domain bias and a lack of interpretability. To address these challenges, we present a NAS method driven by wavelets. To be specific, the wavelet operators that can extract fault related features from vibration signals is added to the search space, and the gradient optimization strategy is utilized to search the optimal architecture from the hypernet. The effectiveness of the proposed method is validated through a dataset specific to planetary gears. Upon comparison with other models, it is evident that the proposed method exhibits superior performance across all added noise levels.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Terin, Rodrigo Carmo; Arenas, Zochil González; Santana, Roberto
Identifying phase transitions in physical systems with neural networks: a neural architecture search perspective Technical Report
2024.
@techreport{terin2024identifying,
title = {Identifying phase transitions in physical systems with neural networks: a neural architecture search perspective},
author = {Rodrigo Carmo Terin and Zochil González Arenas and Roberto Santana},
url = {https://arxiv.org/abs/2404.15118},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
Sinha, Nilotpal; Rostami, Peyman; Shabayek, Abd El Rahman; Kacem, Anis; Aouada, Djamila
Multi-Objective Hardware Aware Neural Architecture Search using Hardware Cost Diversity Technical Report
2024.
@techreport{sinha2024multiobjective,
title = {Multi-Objective Hardware Aware Neural Architecture Search using Hardware Cost Diversity},
author = {Nilotpal Sinha and Peyman Rostami and Abd El Rahman Shabayek and Anis Kacem and Djamila Aouada},
url = {https://arxiv.org/abs/2404.12403},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
Phan, Quan Minh; Luong, Ngoc Hoang
Parameter-less Pareto local search for multi-objective neural architecture search with the Interleaved Multi-start Scheme Journal Article
In: Swarm and Evolutionary Computation, vol. 87, pp. 101573, 2024, ISSN: 2210-6502.
@article{PHAN2024101573,
title = {Parameter-less Pareto local search for multi-objective neural architecture search with the Interleaved Multi-start Scheme},
author = {Quan Minh Phan and Ngoc Hoang Luong},
url = {https://www.sciencedirect.com/science/article/pii/S2210650224001111},
doi = {https://doi.org/10.1016/j.swevo.2024.101573},
issn = {2210-6502},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Swarm and Evolutionary Computation},
volume = {87},
pages = {101573},
abstract = {With the emerging deployment of deep neural networks, such as in mobile devices and autonomous cars, there is a growing demand for neural architecture search (NAS) to automatically design powerful network architectures. It is more reasonable to formulate NAS as a multi-objective optimization problem. In addition to prediction performance, multi-objective NAS (MONAS) problems take into account other criteria like the number of parameters and inference latency. Multi-objective evolutionary algorithms (MOEAs) are the preferred approach for tackling MONAS due to their effectiveness in dealing with multi-objective optimization problems. Recently, local search-based NAS algorithms have demonstrated their efficiency over MOEAs for MONAS problems. However, their performance has been only verified on bi-objective NAS problems. In this article, we propose a local search algorithm for multi-objective NAS (LOMONAS), an efficient local search framework for solving not only bi-objective NAS problems but also NAS problems having more than two objectives. We additionally present a parameter-less version of LOMONAS, namely IMS-LOMONAS, by combining LOMONAS with the Interleaved Multi-start Scheme (IMS) to help NAS practitioners avoid manual control parameter settings. Experimental results from a series of benchmark problems in the CEC’23 Competition demonstrate the competitiveness of LOMONAS and IMS-LOMONAS compared to MOEAs in tackling MONAS within both small-scale and large-scale search spaces. Source code is available at: https://github.com/ELO-Lab/IMS-LOMONAS.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Neumeyer, Simon; Stier, Julian; Granitzer, Michael
Efficient NAS with FaDE on Hierarchical Spaces Proceedings Article
In: Miliou, Ioanna; Piatkowski, Nico; Papapetrou, Panagiotis (Ed.): Advances in Intelligent Data Analysis XXII, pp. 158–170, Springer Nature Switzerland, Cham, 2024, ISBN: 978-3-031-58553-1.
@inproceedings{10.1007/978-3-031-58553-1_13,
title = {Efficient NAS with FaDE on Hierarchical Spaces},
author = {Simon Neumeyer and Julian Stier and Michael Granitzer},
editor = {Ioanna Miliou and Nico Piatkowski and Panagiotis Papapetrou},
url = {https://link.springer.com/chapter/10.1007/978-3-031-58553-1_13},
isbn = {978-3-031-58553-1},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {Advances in Intelligent Data Analysis XXII},
pages = {158–170},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Neural architecture search (NAS) is a challenging problem. Hierarchical search spaces allow for cheap evaluations of neural network sub modules to serve as surrogate for architecture evaluations. Yet, sometimes the hierarchy is too restrictive or the surrogate fails to generalize. We present FaDE which uses differentiable architecture search to obtain relative performance predictions on finite regions of a hierarchical NAS space. The relative nature of these ranks calls for a memory-less, batch-wise outer search algorithm for which we use an evolutionary algorithm with pseudo-gradient descent. FaDE is especially suited on deep hierarchical, respectively multi-cell search spaces, which it can explore by linear instead of exponential cost and therefore eliminates the need for a proxy search space.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Lou, Wenqi; Gong, Lei; Wang, Chao; Qian, Jiaming; Wang, Xuan; Li, Changlong; Zhou, Xuehai
Unleashing Network/Accelerator Co-Exploration Potential on FPGAs: A Deeper Joint Search Journal Article
In: IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, pp. 1-1, 2024.
@article{10505904,
title = {Unleashing Network/Accelerator Co-Exploration Potential on FPGAs: A Deeper Joint Search},
author = {Wenqi Lou and Lei Gong and Chao Wang and Jiaming Qian and Xuan Wang and Changlong Li and Xuehai Zhou},
doi = {10.1109/TCAD.2024.3391688},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Muñoz, J. Pablo; Yuan, Jinjie; Jain, Nilesh
Shears: Unstructured Sparsity with Neural Low-rank Adapter Search Technical Report
2024.
@techreport{muñoz2024shears,
title = {Shears: Unstructured Sparsity with Neural Low-rank Adapter Search},
author = {J. Pablo Muñoz and Jinjie Yuan and Nilesh Jain},
url = {https://arxiv.org/abs/2404.10934},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
Liu, Yue; Yu, Ziyi; Liu, Zitu; Tian, Wenjie
Efficient Neural Architecture Design via Capturing Architecture-Performance Joint Distribution Proceedings Article
In: Dasgupta, Sanjoy; Mandt, Stephan; Li, Yingzhen (Ed.): International Conference on Artificial Intelligence and Statistics, 2-4 May 2024, Palau de Congressos, Valencia, Spain, pp. 1738–1746, PMLR, 2024.
@inproceedings{DBLP:conf/aistats/LiuYLT24,
title = {Efficient Neural Architecture Design via Capturing Architecture-Performance Joint Distribution},
author = {Yue Liu and Ziyi Yu and Zitu Liu and Wenjie Tian},
editor = {Sanjoy Dasgupta and Stephan Mandt and Yingzhen Li},
url = {https://proceedings.mlr.press/v238/liu24b.html},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {International Conference on Artificial Intelligence and Statistics,
2-4 May 2024, Palau de Congressos, Valencia, Spain},
volume = {238},
pages = {1738–1746},
publisher = {PMLR},
series = {Proceedings of Machine Learning Research},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Xiang, Xunzhi; Jing, Kun; Xu, Jungang
A Neural Architecture Predictor based on GNN-Enhanced Transformer Proceedings Article
In: Dasgupta, Sanjoy; Mandt, Stephan; Li, Yingzhen (Ed.): International Conference on Artificial Intelligence and Statistics, 2-4 May 2024, Palau de Congressos, Valencia, Spain, pp. 1729–1737, PMLR, 2024.
@inproceedings{DBLP:conf/aistats/XiangJX24,
title = {A Neural Architecture Predictor based on GNN-Enhanced Transformer},
author = {Xunzhi Xiang and Kun Jing and Jungang Xu},
editor = {Sanjoy Dasgupta and Stephan Mandt and Yingzhen Li},
url = {https://proceedings.mlr.press/v238/xiang24a.html},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {International Conference on Artificial Intelligence and Statistics,
2-4 May 2024, Palau de Congressos, Valencia, Spain},
volume = {238},
pages = {1729–1737},
publisher = {PMLR},
series = {Proceedings of Machine Learning Research},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Qin, Danfeng; Leichner, Chas; Delakis, Manolis; Fornoni, Marco; Luo, Shixin; Yang, Fan; Wang, Weijun; Banbury, Colby; Ye, Chengxi; Akin, Berkin; Aggarwal, Vaibhav; Zhu, Tenghui; Moro, Daniele; Howard, Andrew
MobileNetV4 – Universal Models for the Mobile Ecosystem Technical Report
2024.
@techreport{qin2024mobilenetv4,
title = {MobileNetV4 – Universal Models for the Mobile Ecosystem},
author = {Danfeng Qin and Chas Leichner and Manolis Delakis and Marco Fornoni and Shixin Luo and Fan Yang and Weijun Wang and Colby Banbury and Chengxi Ye and Berkin Akin and Vaibhav Aggarwal and Tenghui Zhu and Daniele Moro and Andrew Howard},
url = {https://arxiv.org/abs/2404.10518},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
Jawahar, Ganesh
Methods for design of efficient on-device natural language processing architectures PhD Thesis
University of British Columbia, 2024.
@phdthesis{Jawahar_2024,
title = {Methods for design of efficient on-device natural language processing architectures},
author = {Ganesh Jawahar},
url = {https://open.library.ubc.ca/collections/ubctheses/24/items/1.0441384},
doi = {http://dx.doi.org/10.14288/1.0441384},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
school = {University of British Columbia},
series = {Electronic Theses and Dissertations (ETDs) 2008+},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
Berezsky, Oleh; Liashchynskyi, Petro; Pitsun, Oleh; Izonin, Ivan
Synthesis of Convolutional Neural Network architectures for biomedical image classification Journal Article
In: Biomedical Signal Processing and Control, vol. 95, pp. 106325, 2024, ISSN: 1746-8094.
@article{BEREZSKY2024106325,
title = {Synthesis of Convolutional Neural Network architectures for biomedical image classification},
author = {Oleh Berezsky and Petro Liashchynskyi and Oleh Pitsun and Ivan Izonin},
url = {https://www.sciencedirect.com/science/article/pii/S1746809424003835},
doi = {https://doi.org/10.1016/j.bspc.2024.106325},
issn = {1746-8094},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Biomedical Signal Processing and Control},
volume = {95},
pages = {106325},
abstract = {Convolutional Neural Networks (CNNs) are frequently used for image classification. This is crucial for the biomedical image classification used for automatic diagnosis in oncology. Designing optimal convolutional neural network architectures is a routine procedure that requires expert knowledge of computer vision and biomedical image features. To address this issue, we developed an automatic method for finding optimal CNN architectures. Our two-step method includes a genetic algorithm-based micro- and macro-search. Micro-search aims to find the optimal cell architecture based on the number of nodes and a set of predefined operations between nodes. Macro-search identifies the optimal number of cells and the operations between them to obtain the final optimal architecture. We obtained several optimal CNN architectures using the developed method of automatic architecture search. We conducted several computer experiments using cytological image classification as an example. The studies’ findings demonstrated that cytological image classification accuracy is higher compared to the classification accuracy of known CNN architectures (VGG-16, AlexNet, LeNet-5, ResNet-18, ResNet-50, MobileNetV3). The method is efficient because the search time for optimal architectures is short. Additionally, the method of optimal architecture search can be used for the synthesis of architectures used for the classification of other classes of biomedical images.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Balaji, Adarsha; Hadidi, Ramyad; Kollmer, Gregory; Fouda, Mohammed E.; Balaprakash, Prasanna
Network architecture search of X-ray based scientific applications Technical Report
2024.
@techreport{balaji2024network,
title = {Network architecture search of X-ray based scientific applications},
author = {Adarsha Balaji and Ramyad Hadidi and Gregory Kollmer and Mohammed E. Fouda and Prasanna Balaprakash},
url = {https://arxiv.org/abs/2404.10689},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
Song, Changwei; Ma, Yongjie; Xu, Yang; Chen, Hong
Multi-population evolutionary neural architecture search with stacked generalization Journal Article
In: Neurocomputing, vol. 587, pp. 127664, 2024, ISSN: 0925-2312.
@article{SONG2024127664,
title = {Multi-population evolutionary neural architecture search with stacked generalization},
author = {Changwei Song and Yongjie Ma and Yang Xu and Hong Chen},
url = {https://www.sciencedirect.com/science/article/pii/S0925231224004351},
doi = {https://doi.org/10.1016/j.neucom.2024.127664},
issn = {0925-2312},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Neurocomputing},
volume = {587},
pages = {127664},
abstract = {In recent years, neural architecture search (NAS) algorithms based on Evolutionary Computation (EC) have demonstrated immense potential in the automated design of deep neural network architectures, garnering widespread attention in the field of deep learning. Most EC-based NAS algorithms select the best individual based on overall fitness score. However, some eliminated suboptimal individuals may only perform weakly in overall classification performance, but perform well on certain classes. To search valuable suboptimal individuals and prevent them from being eliminated, we propose a multi-population evolutionary NAS algorithm with stacked generalization (MPE-NAS). Each population evolves based on the classification accuracy of different classes. After completing the evolution process, the stacked generalization approach is utilized to fuse the searched architectures. Moreover, an integrated performance predictor based on k-nearest neighbor (KNN) regression, random forest (RF) and support vector machine (SVM) is proposed to alleviate computational cost during architecture performance evaluation. On the CIFAR benchmark dataset, the proposed algorithm is examined and compared with the most advanced algorithms, and its effectiveness is confirmed based on experiments. In addition, the proposed multi-population evolutionary (MPE) search strategy is applied to others EC-based NAS algorithms, and achieves the performance improvement without increasing computational resources.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Ye, Yin; Chen, Yaxiong; Xiong, Shengwu
Field detection of pests based on adaptive feature fusion and evolutionary neural architecture search Journal Article
In: Computers and Electronics in Agriculture, vol. 221, pp. 108936, 2024, ISSN: 0168-1699.
@article{YE2024108936,
title = {Field detection of pests based on adaptive feature fusion and evolutionary neural architecture search},
author = {Yin Ye and Yaxiong Chen and Shengwu Xiong},
url = {https://www.sciencedirect.com/science/article/pii/S0168169924003272},
doi = {https://doi.org/10.1016/j.compag.2024.108936},
issn = {0168-1699},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Computers and Electronics in Agriculture},
volume = {221},
pages = {108936},
abstract = {Accurate detection of pests is vital in smart agriculture as it is among the main factors that profoundly influence the yield and quality of crops. In the actual field, pests frequently manifest as small objects, thereby presenting a considerable obstacle to effectively detect pests in the field. For the problem of ineffective utilization of plant context information and inadequate design of neural architecture in field pest detection, we propose the pest detection model (PestNAS) based on adaptive feature fusion and evolutionary neural architecture search. It consists of the adaptive feature fusion module: plant context information is extracted, and the adaptive fusion of pest-related features and plant auxiliary features is designed to effectively utilize plant information; the evolutionary search space module: the novel search space that includes resolution and receptive field enhancement operations is designed with evolution to improve pest representation; the GA-Adam search algorithm: the Adam with genetic algorithm is designed to optimize the objective function of neural architecture search and obtain the relatively better neural architecture for pest detection. The ablation experiments verify the effectiveness of each module in the PestNAS. The comparison experiments reveal that the PestNAS can achieve higher detection accuracy than the other ten neural architecture search models on eleven field pests.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Wang, Shaowei; Zhang, Lingling; Qin, Tao; Liu, Jun; Li, Yifei; Wang, Qianying; Zheng, Qinghua
Multi-view cognition with path search for one-shot part labeling Journal Article
In: Computer Vision and Image Understanding, vol. 244, pp. 104015, 2024, ISSN: 1077-3142.
@article{WANG2024104015,
title = {Multi-view cognition with path search for one-shot part labeling},
author = {Shaowei Wang and Lingling Zhang and Tao Qin and Jun Liu and Yifei Li and Qianying Wang and Qinghua Zheng},
url = {https://www.sciencedirect.com/science/article/pii/S1077314224000961},
doi = {https://doi.org/10.1016/j.cviu.2024.104015},
issn = {1077-3142},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Computer Vision and Image Understanding},
volume = {244},
pages = {104015},
abstract = {The diagram is an abstract form of visual expression in the field of education, which is often used to express complex phenomena and convey logic relationships. In recent years, tasks such as diagram classification and textbook question answering have attracted attention and become a new benchmark for evaluating the complex reasoning ability of models. However, due to the lack of large corpora and the abstract and sparse visual expressions, it is difficult for research methods on natural images to achieve good results on diagrams. In order to solve the above challenges, the researchers consider using the one-shot setting for limited samples challenge and using part labeling to enhance the learning of relational structures. By definition, the one-shot part labeling task is to label multiple parts of an object in the query diagram given only a single support diagram of that category. Under this setting, we propose the Automated Search Multi-view Matching Network (Auto-MMN) which simulating human cognitive methods and process of set-to-set matching problem. We define three views operations based on the attention mechanism and multiplex graph, including the learning of global visual features (global–local view), the interaction between neighboring parts (local–local view), and the comparison of counterparts (cross-local view). We propose a novel learning path search technology to adaptively plan paths for the above three views, which can also increase the generalization performance of the model. We evaluate the Auto-MMN on three different datasets, that is, image-to-image, diagram-to-diagram, and image-to-diagram part labeling scenarios. Extensive experiments show that our model significantly outperforms other baselines on different scenarios and both the multi-view operations and the learning path search produce excellent results. We open source the core code in https://github.com/WayneWong97/Auto-MMN.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Ahmad, Afzal; Du, Linfeng; Xie, Zhiyao; Zhang, Wei
Accel-NASBench: Sustainable Benchmarking for Accelerator-Aware NAS Technical Report
2024.
@techreport{ahmad2024accelnasbench,
title = {Accel-NASBench: Sustainable Benchmarking for Accelerator-Aware NAS},
author = {Afzal Ahmad and Linfeng Du and Zhiyao Xie and Wei Zhang},
url = {https://arxiv.org/abs/2404.08005},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
Lyu, Bo; Yang, Yin; Cao, Yuting; Shi, Tuo; Chen, Yiran; Huang, Tingwen; Wen, Shiping
A memristive all-inclusive hypernetwork for parallel analog deployment of full search space architectures Journal Article
In: Neural Networks, vol. 175, pp. 106312, 2024, ISSN: 0893-6080.
@article{LYU2024106312,
title = {A memristive all-inclusive hypernetwork for parallel analog deployment of full search space architectures},
author = {Bo Lyu and Yin Yang and Yuting Cao and Tuo Shi and Yiran Chen and Tingwen Huang and Shiping Wen},
url = {https://www.sciencedirect.com/science/article/pii/S0893608024002363},
doi = {https://doi.org/10.1016/j.neunet.2024.106312},
issn = {0893-6080},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Neural Networks},
volume = {175},
pages = {106312},
abstract = {In recent years, there has been a significant advancement in memristor-based neural networks, positioning them as a pivotal processing-in-memory deployment architecture for a wide array of deep learning applications. Within this realm of progress, the emerging parallel analog memristive platforms are prominent for their ability to generate multiple feature maps in a single processing cycle. However, a notable limitation is that they are specifically tailored for neural networks with fixed structures. As an orthogonal direction, recent research reveals that neural architecture should be specialized for tasks and deployment platforms. Building upon this, the neural architecture search (NAS) methods effectively explore promising architectures in a large design space. However, these NAS-based architectures are generally heterogeneous and diversified, making it challenging for deployment on current single-prototype, customized, parallel analog memristive hardware circuits. Therefore, investigating memristive analog deployment that overrides the full search space is a promising and challenging problem. Inspired by this, and beginning with the DARTS search space, we study the memristive hardware design of primitive operations and propose the memristive all-inclusive hypernetwork that covers 2×1025 network architectures. Our computational simulation results on 3 representative architectures (DARTS-V1, DARTS-V2, PDARTS) show that our memristive all-inclusive hypernetwork achieves promising results on the CIFAR10 dataset (89.2% of PDARTS with 8-bit quantization precision), and is compatible with all architectures in the DARTS full-space. The hardware performance simulation indicates that the memristive all-inclusive hypernetwork costs slightly more resource consumption (nearly the same in power, 22%∼25% increase in Latency, 1.5× in Area) relative to the individual deployment, which is reasonable and may reach a tolerable trade-off deployment scheme for industrial scenarios.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}