Active learning reduces annotation costs by prioritizing the most informative samples, however, methods that are using fixed batch sizes can be inefficient as they fail to adjust to variations in data and resource limitations. This challenge becomes even more critical in resource-constraint environments, such as IoT and edge devices, where both computational power and the availability of labeled data for training are limited. We propose Adaptive Batch Size using Reinforced Active Learning, a novel method that employs a dual-strategy design to dynamically optimize batch sizes. Firstly, the proposed method identifies performance benchmarks, including target precision and annotation budget thresholds, by exploring various batch sizes during an initialization phase. These benchmarks guide the second phase, where a reinforcement learning agent adjusts batch sizes in real-time, ensuring efficient resource usage while maintaining robust model performance. Evaluated on CIFAR-10, CIFAR-100, and MNIST datasets, the proposed method demonstrates significant annotation cost reductions and efficient learning, even in scenarios with constrained computational resources and limited labeled data.