Encoder-Side Neuron Identification and Amplification for Acoustic Perception in Large Audio-Language Models
IAAN (Identifying and Amplifying Acoustic Neurons) is a training-free, label-free method that scores feed-forward neurons in the audio encoder by contrasting activation on real waveform vs noise reference, then amplifies top-scoring neurons at inference. On ten non-semantic speech attributes, IAAN improves average accuracy by 25.7 points on Audio-Flamingo-3, 21.4 on Qwen2.5-Omni, and 9.7 on Kimi-Audio.
Development
- First ReportEncoder-Side Neuron Identification and Amplification for Acoustic Perception in Large Audio-Language ModelsarXiv cs.AI
- Current AssessmentThis method offers a low-cost inference-time enhancement for audio models, potentially accelerating deployment of more perceptive voice assistants and emotion-aware systems.Agent Pulse · analysis
IAAN is a training-free method that identifies and amplifies acoustic neurons in the audio encoder of large audio-language models, improving fine-grained non-semantic speech attribute accuracy without retraining.
IAAN demonstrates that neuron-level intervention in the audio encoder can significantly improve acoustic perception, suggesting that current LALMs underutilize encoder representations for non-semantic tasks.
This method offers a low-cost inference-time enhancement for audio models, potentially accelerating deployment of more perceptive voice assistants and emotion-aware systems.
IAAN enables rapid improvement of existing audio models without retraining, reducing cost and time for companies to enhance user experience in voice-based products.
Future work may extend IAAN to other modalities or integrate it with fine-tuning; next signal: adoption in production audio models or extension to video/audio joint encoders.