Make a turn at the farm Seljavllir (clearly marked), drive as far as the road takes you, park your car and walk for ten minutes or so. Bask in your surroundings. Thank your stars. Ponder life.
Remember to bring a towel. And should you eventually grow bored of pondering life in a pool, the surrounding area offers lots of opportunities for general enjoyment, given that you're not averse to nature.
Note that the Seljavallalaug swimming pool is not regularly maintained, it has no showering facilities (and very modest changing booths) and it’ll often have a bunch of algae in it. It was built in 1924 and re-done in 1998 by locals. You venture there of your own accord, and on your own responsibility. Be respectful of the locals, the pool itself and of nature in general. And bring a towel.
Take a long, hard look at the accompanying photo. You'll want to be there. And for good reason. Seljavallalaug is just as beautiful, calming, transcendent and awe-inspiring as it looks. It's accessible, too. Located just a couple of hours away from Reykjavík, you'll have to venture South on Route I, past Selfoss and Hvolsvöllur and make sure to slow down once you're nearing Skógarfoss.