No that’s not how it works. Handling a larger address space (e.g., 32-bit vs 64-bit) maybe could affect speed between same sized modules on a very old CPU but I’m not sure that’s even the case by any noticeable margin.
The RA in RAM stands for random access; there is no seeking necessary.
Technically at a very low level size probably affects speed, but not to any degree you’d notice. RAM speed is actually positively correlated with size, but that’s more because newer memory modules are both generally both bigger and faster.
You clearly know more than me, but wouldn’t everything from 4GB to 1TB have the same number of walks? And one more walk gets you up to 256TB?