I am in love with this place. Historic Babylon is a hidden swimming hole. Best time to go is in the spring and early summer. There is no shade what so ever so make sure you pack a umbrella or some kind of shade.
The turnoff is on the north side of Leeds. There is a sign that says Historic Babylon. There are a couple of hikes and cool places out that way but Babylon is at the very end of the rocky and sandy road. I would take a 4 wheel drive to be on the safe side.