Hybrid Active Learning for Low-Resource LM Fine-tuning

Figure 4: Average per-iteration acquisition time over 5 random runs. Unlike other approaches, TYROGUE’s runtime does not increase with the size of the datasets, thereby significantly reducing acquisition latency.

We identified two key designs that can improve the effectiveness and efficiency of sample acquisition: random sampling reduces the unlabeled pool being considered for acquisition, and decouples the diversity and uncertainty objectives in hybrid acquisition. Based on an investigation of existing methods, we propose a novel active learning method: TYROGUE.