Even if someone steals a list of encrypted passwords, they have no way to decrypt them without trying their "guesses" against the target, and in reality that means going through the IO port you are trying to log into.
This is incorrect.
The encrypted passwords are "hashes", they look like large strings of characters such as "ASFa87w23598asgha9fg987y432523sdljfsdi". If you give me your actual password, everyone knows how to convert a plaintext (unencrypted) password into a hash. In fact that is how they check that your password is correct, they convert your password into the hash and then compare with the correct hash.
The reason this works is it is a one way function . It is very easy (and fast) to turn a password into a hash. It is very difficult (and time consuming) to turn a hash into the plaintext password.
But once I have the hash, and I have a GPU array (or any other way of trying millions of passwords quickly) then I can do all of the work on my own computer without needing to access the target computer or the network.
I simply, on my own computer, turn each of the hundreds of millions of potential passwords into a hash, and then compare each of these hashes with your password hash (which I have). Once I find a match I have your password.
So yes. If I have your encrypted password and a lot of computing power, I can crack your password without any problem with network speed.