End-to-end encryption challenges mobile network operators to assess the quality of the HTTP Adaptive Streaming (HAS), where the quality assessment is coarse-grained, e.g., detecting if there exist stalling during the whole playback. Targeting on this issue, this paper proposes an attention-based hybrid RNN-HMM model, which integrates HMM with attention mechanism to predict the player states. The model is trained and evaluated based on the download speed and player state sequences of encrypted video sessions collected from YouTube. Experiment results show that the proposed model is able to recognize the player states with 86.53%~94.35% accuracy, and thus achieves to assess video quality in a fine-grained manner, where how long the stalling lasts and when the stalling occurs can be evaluated effectively from the download speed sequence even when encryption is employed.