If you were an atheist and are not now, you never were an atheist...
I disagree. First there's the type of atheism was all have when we are born. Then there are those that arrived to adult atheism via faulty reasoning. Sort of doing the math wrong but arriving to a correct answer. Then there are those that have damaged their brain though chemicals or accident. Fourth, those that abandon rationality in extreme emotional distress.
A common example might be someone who lived a dissolute lifestyle who never really paid attention to priests and church, viewing them as yet another authority figure of no value, hit rock bottom, then 'find Jesus'
If you are referring to the majority of those here, who arrived at atheism via rational consideration, then yes, you are right(except type 3 and 4). I could no longer stop being an atheist than I could stop believing a stage magician is using tricks and not real magic.
In fact I suspect that when dying I might abandon rationality and atheism, because I would be scared beyond the ability to think logically. However, that would not make me right, just scared stupid.