This thesis revolutionizes Active Learning (AL) in Deep Learning, tackling the scarcity of labeled
data. Introducing the Bayesian Estimate of Mean Proper Scores (BEMPS) and novel acquisition
functions, it enhances AL across various domains. BEMPS is adapted for image classification
with Monte-Carlo dropout and dialogue act classification, showing remarkable outcomes. The
work also features a Deep AUC Maximization approach for Named Entity Recognition,
leveraging AUC-Optimized Latent Sigmoid Embedding for precise uncertainty estimation. In
its culmination, the thesis presents BESRA for multi-label text classification, extensively tested
and proven on specialized medical datasets, marking a significant leap in AL's efficiency and
applicability.