Maximum Entropy Exploration Without the Rollouts. - Jacob Adamczyk; Adam Kamoski; Rahul V. Kulkarni (CoRR, 2026)