BROWSE
Detail
Conference Paper
From Continuous Pretraining to Domain-Adaptive Reranking via Task Vector Adaptation
- Authors
- Sanghyun Cho, Myeongjin Lee, Jong-Hun Shin, Jeong Heo, Kiyoung Lee, Soojong Lim
- Issue Date
- 2026-07
- Citation
- International Conference on Research and Development in Information Retrieval (SIGIR) 2026, pp.3652-3657
- Publisher
- ACM
- Language
- English
- Type
- Conference Paper
- Abstract
- Large language model (LLM)-based rerankers have demonstrated strong performance in information retrieval tasks, but adapting them to specialized domains remains challenging due to the substantial cost and effort required to construct high-quality domain-specific training datasets. To address this limitation, we leverage task vectors derived from continuous pretraining as a mechanism for transferring domain knowledge. However, existing task vector integration methods are highly sensitive to scaling factors and can lead to unstable performance across domains and model scales. In this work, we propose a fine-grained task vector adaptation method that learns parameter-wise scaling coefficients for the task vector. These coefficients are optimized using a language modeling objective while keeping all model parameters fixed, enabling effective integration of domain-specific knowledge without degrading reranking capabilities. Experiments on a general-domain benchmark and five specialized domains across two model scales demonstrate that our method provides stable improvements across most domains and avoids the degradation observed with fixed task vector scaling.
- Keyword
- task vector arithmetic, domain-adaptive reranking, adaptive merging
- KSP Keywords
- Domain-specific knowledge, Fine grained(FG), General-Domain, High-quality, Information retrieval, Integration method, Model parameter, domain knowledge, highly sensitive, language modeling, scaling factor
This work is distributed under the term of Creative Commons License (CCL)
(CC BY)
(CC BY)
