How to SSH into an EC2 instance or a VPS for the first time

13 Sept 2026 · Termalin team sshawsvpscloudtutorial

You clicked “launch,” the dashboard says the instance is running, and now there’s a public IP address and a downloaded key file and no obvious next move. This is the first wall almost everyone hits with a cloud server, and it’s made of four small things that each have one right answer: the address, the login user, the key, and its permissions. Get those lined up and you’re at a shell in under a minute. Miss one and you get an error that sounds scarier than it is. Here’s the whole path, first try to shell.

Step 1 — gather the four things you need

The address. Your provider’s console shows it. On AWS it’s the instance’s Public IPv4 address or Public IPv4 DNS (ec2-3-120-….compute.amazonaws.com); on DigitalOcean, Hetzner, Linode or Vultr it’s the droplet/server’s public IP.

The login user — this trips up more people than anything else. You almost never log in as a name you chose. The image decides it:

AWS EC2: ec2-user on Amazon Linux, ubuntu on Ubuntu AMIs, admin on Debian, centos/rocky/fedora on those. There is no root login.
DigitalOcean / Hetzner / Vultr: usually root (you attach your SSH key when you create the server).
Google Cloud: the username tied to your key or OS Login.
Azure: the admin username you set at creation.

When in doubt, the provider’s “connect” panel names the user.

The key. Cloud providers use key auth, not passwords. Either you downloaded a .pem at launch (AWS), or you pasted your public key into the create screen and hold the matching private key locally (most VPS hosts). Either way you already have the private key you need — you do not download it again, and there’s no password to look up.

Step 2 — fix the key’s permissions (the famous gotcha)

SSH refuses to use a private key that other users on your machine could read. The first ssh attempt on a freshly downloaded .pem very often ends in:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for 'mykey.pem' are too open.

On macOS or Linux the fix is one command:

chmod 400 mykey.pem

On Windows it’s fiddlier — OpenSSH there checks Windows ACLs, not Unix bits, so you either strip inherited permissions with icacls or let a client manage the key for you. This is one of several rough edges that a proper SSH client just… doesn’t have, because it stores keys correctly to begin with.

Step 3 — connect

Put the three pieces together — key, user, address:

ssh -i mykey.pem [email protected]

The first time, SSH shows the server’s fingerprint and asks whether to trust it — that’s trust-on-first-use, and typing yes pins the key so a later change (which could mean a man-in-the-middle) gets flagged. If you land at a shell prompt, you’re in. If not, it’s almost certainly one of the next two errors.

Step 4 — the two errors you’ll actually hit

Connection timed out (or refused). This is a network problem, not a login problem — SSH never even reached a willing server. On a cloud box the usual cause is the firewall in front of the instance: AWS security groups, or the provider’s cloud firewall, don’t allow inbound port 22 from your IP by default on every setup. Open 22 to your address (not 0.0.0.0/0 — see hardening below) and try again. The full decision tree is in connection refused vs timed out.

Permission denied (publickey). SSH reached the server, but the login was rejected. On a first connection this is nearly always one of two things: the wrong user (you tried root@ on an Ubuntu EC2 box that wants ubuntu@), or the wrong key (not the one whose public half is on the server). Add -v to watch which key SSH offers and how the server answers; permission denied (publickey), decoded walks through every cause.

Step 5 — make it repeatable

You do not want to retype that ssh -i … long-user@long-dns line for the life of the server. Two ways to fix it:

A ~/.ssh/config entry turns the whole thing into a nickname:

Host prod
    HostName ec2-3-120-45-67.eu-central-1.compute.amazonaws.com
    User ubuntu
    IdentityFile ~/.ssh/mykey.pem

Now ssh prod is the whole command. The SSH config file guide covers the rest — jump hosts, per-host keys, keepalives.

Or save it in a client that keeps the host, user, key and port together as one entry, shows you the server’s live CPU/disk/memory the moment you connect, and syncs the whole list to your other machines. Same idea as the config file, minus the file.

A word on hardening, before you forget

The box is on the public internet now, and SSH scanners find new ones within minutes. Two habits worth building on day one:

Scope the firewall. Allow 22 from your IP or your VPN, not from the entire internet. If the server sits in a private subnet, reach it through a bastion with ProxyJump rather than giving it a public address at all.
Keys only, no passwords. Cloud images usually ship this way already (PasswordAuthentication no); keep it that way. A key you can revoke per device beats a password every session shares.

Letting an AI agent help run it

Once you can reach the box, the 2026 temptation is to stop SSHing in yourself for every little check and instead ask an assistant — Claude Code, Cursor, Codex — to look at why a service died or a disk filled up. That’s genuinely useful, and it’s also the exact moment people hand an agent the .pem and a raw shell, which is the one move you can’t take back: a key in an agent’s environment is your whole server, forever, wherever that context ends up.

The better arrangement is to let the agent go through something that holds the key for it, limits what it can run, and records what it did — connect the agent over MCP instead of pasting a credential. The server gets a capable helper; the helper never gets the keys to it.

Where Termalin fits

Termalin is a cross-platform SSH client (Windows, macOS, Linux) that removes the day-one friction above: it stores keys with correct permissions so the chmod/icacls dance never happens, keeps each host’s user, key and port together so a server is a nickname and not a remembered command, and shows the machine’s disk, memory and load live in the status bar. And when you want an agent to help, it connects through the built-in MCP server — working over your SSH access without ever holding the key. The Free tier has no host limits — grab it and add your new server.

Termalin is a free, cross-platform SSH client — clean first connections, and safe access for the AI agents you let help. Download it, or read the security model.