Exposing SSH directly on your production servers creates a larger attack surface. Each server becomes a potential entry point for attackers attempting password spraying, brute force attacks, or exploiting SSH vulnerabilities. The more servers you have listening on port 22, the more targets you’re giving potential attackers.
An SSH jump server, also called a bastion host or proxy server, solves this problem by acting as a single gateway for all SSH connections to your infrastructure. Instead of allowing SSH access directly to your production servers, you configure your firewall to only allow SSH from the jump server’s IP address. All other SSH traffic is blocked. This reduces your attack surface from dozens of servers down to a single hardened system that you can monitor closely.
The jump server acts as an intermediary. You SSH into the jump server first, which then forwards your connection to the destination server. Modern SSH configurations make this transparent—you don’t have to manually log into two servers. Instead, SSH automatically routes your connection through the jump server behind the scenes.
This guide shows you how to create SSH keys, install them on both the jump server and destination servers, and configure your SSH client to automatically use the jump server.
Prerequisites
- A server to act as your jump server (Ubuntu 20.04 or later recommended)
- One or more destination servers you want to access through the jump server
- SSH access to all servers (initially using passwords or existing keys)
- OpenSSH installed on your local machine
Important: This guide assumes you’re using SSH key-based authentication exclusively. Password-based authentication should be disabled on all servers for security reasons. Password logins are vulnerable to brute force attacks and credential stuffing, which is exactly what the jump server architecture is designed to prevent.
Step 1: Create Your SSH Keys
You’ll need two separate key pairs. The first key authenticates you to the jump server. The second key authenticates from the jump server to your destination servers.
On your local machine, create the jump server key:
ssh-keygen
When prompted for the file location, give it a descriptive name:
Enter file in which to save the key: /home/username/.ssh/jumpserver
For production environments, use a passphrase for extra security:
Enter passphrase (empty for no passphrase): [enter a strong passphrase]
Enter same passphrase again: [repeat the passphrase]
For development or test environments, you can skip the passphrase by pressing Enter twice.
Your keys are now created:
Your identification has been saved in /home/username/.ssh/jumpserver
Your public key has been saved in /home/username/.ssh/jumpserver.pub
Next, create the destination server key using the same process, but with a different name:
ssh-keygen
Enter file in which to save the key: /home/username/.ssh/production
Enter passphrase (empty for no passphrase):
You now have four files in your .ssh directory:
- jumpserver and jumpserver.pub (for accessing the jump server)
- production and production.pub (for accessing production servers)
Step 2: Install the Jump Server Key
Copy your public key to the jump server using ssh-copy-id:
ssh-copy-id -i ~/.ssh/jumpserver [email protected]
Enter your password when prompted. This is the last time you’ll need to use a password for the jump server.
Test the key:
ssh -i ~/.ssh/jumpserver [email protected]
If successful, you’ll be logged in without entering a password.
Step 3: Install the Production Key on the Jump Server
The production key needs to be installed on the jump server so it can authenticate to your destination servers. From your local machine, copy the private key to the jump server:
scp ~/.ssh/production [email protected]:~/.ssh/
SSH into the jump server and set proper permissions:
ssh -i ~/.ssh/jumpserver [email protected]
chmod 600 ~/.ssh/production
Step 4: Install the Production Key on Destination Servers
Still logged into the jump server, copy the public key to each destination server:
ssh-copy-id -i ~/.ssh/production [email protected]
ssh-copy-id -i ~/.ssh/production [email protected]
Test that the jump server can now access the destination servers:
ssh -i ~/.ssh/production [email protected]
Once confirmed, log out of the jump server and return to your local machine.
Step 5: Configure SSH to Use ProxyJump
Now configure your local SSH client to automatically route connections through the jump server. Create or edit ~/.ssh/config on your local machine:
vim ~/.ssh/config
Add the following configuration:
# Jump Server
Host jump
Hostname jumpserver.example.com
User username
IdentityFile ~/.ssh/jumpserver
# Production Servers
Host prod1 prod2 prod3
User username
ProxyJump jump
IdentityFile ~/.ssh/production
# Specific server mappings
Host prod1
Hostname server1.example.com
Host prod2
Hostname server2.example.com
Host prod3
Hostname server3.example.com
Let’s break down what each setting does:
Host: A short alias you can use instead of typing the full hostname
Hostname: The actual FQDN or IP address of the server
User: Your username on the remote server
IdentityFile: The path to the private key for authentication
ProxyJump: Tells SSH to route the connection through the specified jump server
With this configuration in place, you can now connect to any production server with a simple command:
ssh prod1
SSH automatically connects to the jump server first, then forwards your connection to the destination server—all in one seamless step. From your perspective, it works just like a direct connection.
Step 6: Configure Firewall Rules
The final step is to lock down your production servers. Configure UFW on each destination server to only allow SSH connections from the jump server’s IP address:
ufw allow from 203.0.113.10 to any port 22
ufw deny 22
Replace 203.0.113.10 with your jump server’s actual IP address. With these rules in place, SSH attempts from any IP other than your jump server will be blocked at the firewall level.
Using Wildcards for Multiple Servers
If you have many servers with similar hostnames, you can use wildcards in your SSH config:
# All development servers
Host dev-*
User username
ProxyJump jump
IdentityFile ~/.ssh/development
# All production web servers
Host web-prod-*
User username
ProxyJump jump
IdentityFile ~/.ssh/production
Now any hostname matching the pattern automatically uses the jump server. For example, ssh dev-database or ssh web-prod-01 will route through your jump server without additional configuration.
Best Practices
Harden the jump server: Since the jump server is your single point of entry, secure it thoroughly. Disable password authentication, keep it patched, enable automatic security updates, and monitor logs for suspicious activity.
Use different keys for different environments: Create separate key pairs for development, staging, and production. This limits the blast radius if a key is compromised.
Rotate keys regularly: Treat SSH keys like passwords. Rotate them periodically, especially when someone with access leaves the team.
Monitor jump server logs: All SSH connections flow through the jump server, making it a central point for security monitoring and audit logging.
Consider a backup jump server: If your jump server goes down, you lose access to all your infrastructure. Configure a secondary jump server for redundancy.
Use with Ansible deployments: If you use Ansible for deployments, it can also route connections through your jump server. Add the ProxyJump configuration to your Ansible inventory or set the ansible_ssh_common_args variable to use the same jump server for automated deployments:
ansible_ssh_common_args: '-o [email protected]'
With an SSH jump server properly configured, you’ve added a critical security layer to your infrastructure. Your production servers are now protected behind a single hardened gateway, dramatically reducing your exposure to SSH-based attacks.