We can't go super fast because our engines aren't strong enough
Space offers nearly no resistance, so even the weakest engine could eventually accelerate a massive object to incredible speeds (NASA is experimenting with “slow and steady” ion thrusters). The problem, however, is fuel. To accelerate for so long you need to keep the engine running on something. Once you hit max speed though…you could turn off the engine and cruise to the end of the universe. Which brings us to the second problem…stopping. You would need reverse thrust to slow down, and that could require just as much fuel and time as it took to speed up!