This work proposes a learning-based energy management policy that takes into consideration the trade-off between the depth-of-discharge (DoD) and the lifetime of batteries. The impact of DoD on the energy management policy is often neglected in the past due to the inability to model its effect on the marginal cost per battery usage. In this work, a novel battery cost evaluation method that takes into consideration the DoD of each battery usage is proposed, and is utilized to devise the day-ahead energy management policy using reinforcement learning and linear value-function approximations. The policy determines the amount of energy to purchase for the next day in the day ahead market. A next-state policy iteration (NSPI) scheme with linear predictions of the next-day system parameters is used to learn the energy management policy. Simulations are provided based on real load profiles, pricing data, and renewable energy arrival statistics. The consideration of the battery cost due to DoD provides a more accurate evaluation of the actual energy cost and leads to an improved energy management policy.