I started programming and working with databases because when I had to do route tasks that required repeating a sequence of steps to be performed manually, I’d inevitably screw it up. Directions were hard to keep in my “buffer”, so I started asking for landmarks instead. I think the hard part about aphantasia or other neurodivergent traits is that many of the deficiencies/divergencies have cobbled together mental workarounds, so the symptoms are harder to recognize.
If the processing you have is what you’ve had your entire life, it’s pretty hard to describe what is missing. My usual answer was, “Maybe I’m just not good at this.”