Such places are found along the way and are usually free of cost. This is one example in Arima Onsen. The place was closed in fact, thus the water is missing. And the colour comes from the water composition.
This is another example of ashiyu in Nagasaki. I shyly stole a glance with the camera. Some ladies were relaxing there and I gingerly took this photo from a wrong angle, in order to inconvenience them as little as possible.