Mountain Lion and SSH

For the most part upgrading to Mountain Lion has been surprisingly smooth.
However, I did run into a problem with SSH. After upgrading to Mountain Lion I noticed that connecting to servers at work via SSH was no longer working. Instead of connection I would just get:

Read from socket failed: Connection reset by peer

If you are [...]