Secure and Keyless Access to Amazon EC2 Instances

Traditional SSH access to EC2 instances relies on long-lived private keys .pem files—that are often shared, misplaced, or stored with excessive permissions. As organizations scale their cloud footprint, key sprawl becomes a security and operational burden. In this deep dive, we’ll deconstruct the risks of PEM-based access, examine stopgap hardening measures, and then design a modern, keyless architecture using native AWS services that align with Zero Trust principles.

1. The Problem with PEM Files

A .pem file is essentially a private key that, when compromised, grants an attacker interactive shell access. Common failure modes include:

• Key sharing between developers, often over chat or email.
• Weak file permissions (world-readable on shared drives or laptops).
• No audit trail tying a specific user to a specific session.
• Static credentials that rarely rotate, surviving employee offboarding.
• Exposed SSH ports (22) on the internet that invite brute‑force attacks.

While you can mitigate these risks with bastion hosts, key encryption, and rigorous rotation, those measures add operational complexity without fundamentally removing the long-lived secret. The architectural goal should be to eliminate the static key entirely.

2. Stopgap Hardening (If You Must Keep PEM Files)

Before replacing the PEM model, many teams apply these tactical controls. They buy time but are not a long-term strategy:

# Correct file permissions
chmod 400 my-key.pem

# Encrypt at rest with a passphrase
openssl rsa -aes256 -in my-key.pem -out my-key-encrypted.pem

• Bastion hosts in a public subnet act as the single entry point; all other instances remain in private subnets. Security groups allow SSH only from the bastion’s IP.
• Rotation scripts remove old public keys from ~/.ssh/authorized_keys and replace key pairs periodically.

These practices address some symptoms but still require protecting the bastion’s PEM file and managing user provisioning on each instance.

3. The Modern AWS Approach: Keyless, IAM-Driven Access

AWS offers a suite of services that replace the PEM file with temporary, IAM‑authenticated sessions. The architecture converges on:

• No open inbound ports
• No long-lived keys
• Full session logging to S3 and CloudWatch
• Just-in-time access controlled by IAM policies and Identity Center

The primary building blocks are AWS Systems Manager Session Manager, EC2 Instance Connect, and optional enterprise SSH CAs integrated with your identity provider.

4. Deep Dive: AWS Systems Manager Session Manager

Session Manager is the recommended native service. It creates a secure, interactive shell or port forwarding channel from your local client (or the AWS Console) to the instance without requiring an inbound security group rule, a public IP, or an SSH key.

4.1 Architecture

[Engineer Workstation]            [AWS Cloud]
       |                              |
   AWS CLI (aws ssm start-session)   IAM (permissions check)
       |                              |
       v                              v
   [Session Manager API Endpoint] ---> [SSM Agent on EC2]
                                          |
                                          v
                                   [EC2 Instance (private subnet)]

• The SSM Agent (pre‑installed on Amazon Linux 2/2023, Ubuntu, Windows, etc.) initiates an outbound HTTPS connection to the Session Manager service.
• The engineer’s client also calls the Session Manager API; IAM evaluates whether the principal is authorized to start a session on the target instance.
• A two-way WebSocket tunnel is established over TLS, carrying the interactive session.

Because the connection is outbound from the instance, no security group ingress rules are required—the instance can sit in a private subnet with zero inbound ports open.

4.2 Prerequisites & IAM Configuration

Instance IAM Role must include the managed policy AmazonSSMManagedInstanceCore (or custom equivalent that allows ssmmessages:*, ec2messages:*, cloudwatch:* if logging is desired).

Example minimal role (CloudFormation snippet):

SessionManagerRole:
  Type: AWS::IAM::Role
  Properties:
    AssumeRolePolicyDocument:
      Statement:
        - Effect: Allow
          Principal:
            Service: ec2.amazonaws.com
          Action: sts:AssumeRole
    ManagedPolicyArns:
      - arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore

User/Principle IAM Policy for engineers:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ssm:StartSession",
        "ssm:TerminateSession",
        "ssm:DescribeSessions"
      ],
      "Resource": [
        "arn:aws:ec2:region:account-id:instance/i-*",
        "arn:aws:ssm:region:account-id:document/SSM-SessionManagerRunShell"
      ],
      "Condition": {
        "StringEquals": {
          "ssm:resourceTag/Environment": "dev"
        }
      }
    }
  ]
}

Use resource tags and conditions to scope which instances a user can access—this enables least‑privilege and multi‑team isolation without additional bastions.

4.3 VPC Endpoints for Private Instances

For instances in private subnets with no internet access, SSM Agent needs to reach the Session Manager API. Deploy VPC endpoints:

• com.amazonaws.region.ssmmessages
• com.amazonaws.region.ssm
• com.amazonaws.region.ec2messages

Attach these endpoints to the private subnets and ensure the attached security groups allow outbound HTTPS. This keeps all session traffic within the AWS backbone.

4.4 Session Logging & Auditing

Enable session logging in the Session Manager preferences:

• S3 bucket for full transcript logs (encrypted with KMS).
• CloudWatch Logs for near‑real‑time streaming and metric filters.

Configuration (console or via update-document for SSM-SessionManagerRunShell):

{
  "inputs": {
    "s3BucketName": "my-ssm-logs-bucket",
    "s3KeyPrefix": "sessions/",
    "s3EncryptionEnabled": true,
    "cloudWatchLogGroupName": "/ssm/sessions",
    "cloudWatchEncryptionEnabled": true
  }
}

All session commands and output are captured. Coupled with AWS CloudTrail (which records StartSession API calls with the user’s identity), you get non‑repudiable audit trails mapping every action to a specific IAM principal.

4.5 Using Session Manager

From the AWS CLI, after installing the Session Manager plugin:

aws ssm start-session --target i-0abcdef1234567890

Port forwarding for SSH‑dependent tools (e.g., scp replacement):

aws ssm start-session \
    --target i-0abcdef1234567890 \
    --document-name AWS-StartPortForwardingSession \
    --parameters '{"portNumber":["22"],"localPortNumber":["2222"]}'

Then ssh user@localhost -p 2222 works over the tunnel, still without opening port 22 to the world.

5. EC2 Instance Connect

EC2 Instance Connect provides a console‑based SSH alternative that temporarily pushes a one‑time SSH public key to the instance metadata. When you click Connect in the EC2 console, AWS generates a short‑lived key pair, uploads the public key, and opens the browser‑based SSH client.

• IAM controlled: You can deny ec2-instance-connect:SendSSHPublicKey unless specific conditions are met (e.g., MFA).
• No PEM management: The key expires in 60 seconds and is discarded.
• Best suited for ad‑hoc debugging by small teams who occasionally need traditional SSH.

Limitations: it still relies on SSH protocol, requires port 22 open from the EC2 Instance Connect service IP range, and doesn’t provide the deep auditing of Session Manager.

6. Enterprise SSH Certificate Authority

For organizations that must keep native SSH (e.g., non‑AWS servers, multi‑cloud) but want to eliminate long‑lived keys, an SSH Certificate Authority (CA) offers a middle ground. Products like Teleport, Okta Advanced Server Access, Smallstep, or BLESS (Netflix’s open‑source tool) can:

1. Authenticate users against an IdP (SAML/OIDC).
2. Issue a short‑lived SSH certificate (valid minutes).
3. The EC2 instance trusts the CA’s public key (configured in TrustedUserCAKeys).

This model moves the trust anchor from individual .pem files to a central, audited CA. You can combine it with AWS IAM Identity Center: users log in with corporate credentials, receive a temporary certificate, and then SSH into the instance. While more complex than Session Manager, it unifies access across AWS and on‑premises.

7. IAM Identity Center + Session Manager: The Gold Standard

Most cloud‑native teams converge on a pattern:

• IAM Identity Center (successor to AWS SSO) with your corporate directory (Azure AD, Okta, etc.) and MFA.
• Session Manager for all interactive access.
• Instances in private subnets with no public IPs.
• No inbound security group rules for SSH.

Users sign in via the AWS CLI (aws sso login) or the console, assume a permission set that grants ssm:StartSession on a specific set of instances, and start a session—all without ever touching a private key.

This design aligns with the AWS Well‑Architected Framework security pillar: protect data in transit (TLS 1.2+), centralize identity, enforce least privilege, and log all access.

8. Network Architecture: Removing SSH from the Equation

An illustrative architecture

• No bastion host to patch, harden, and audit.
• No SSH keys to rotate.
• All traffic flows over the AWS internal network or through a VPN if the user is remote.

If you must support a short transition period, maintain a bastion with SSH forwarding but begin tagging instances for Session Manager access and migrating users immediately.

9. Auditing and Compliance

Compliance frameworks (PCI DSS, HIPAA, SOC 2) demand individual accountability for server access. Session Manager delivers:

• CloudTrail: StartSession calls with userIdentity, sourceIP, sessionId.
• Session logs: full input/output in S3/CloudWatch.
• Recording capability (via shell profiles) to capture visual sessions.

Set up CloudWatch alerts for StartSession without MFA or from unusual locations. Use AWS Config rules to detect security groups with port 22 open to 0.0.0.0/0.

10. Migrating from PEM‑Based SSH

A practical migration roadmap:

1. Enable SSM Agent on all instances (via AMI bake or user data).
2. Attach the AmazonSSMManagedInstanceCore IAM role to the EC2 instances.
3. Deploy VPC endpoints if your instances lack internet access.
4. Create IAM policies that mirror existing team boundaries (tag‑based).
5. Train users on aws ssm start-session and console access.
6. Disable SSH security group rules incrementally, starting with non‑production.
7. Decommission bastion hosts once all access is migrated.

For file transfer, replace scp with:

• aws ssm start-session port forwarding + rsync,
• Or directly use aws s3 cp to an S3 bucket with VPC endpoints, removing the need for any interactive session.

11. Trade‑offs and When You Still Might Need SSH

Session Manager cannot fully replace every scenario today:

• Non‑AWS resources: On‑premises servers require a hybrid activation, but some shops prefer a unified SSH CA.
• Legacy applications that programmatically SSH into instances must be refactored to use SSM Run Command or port forwarding.
• High‑touch interactive tools that rely on raw PTY features may need tuning with Session Manager’s shell profiles.

However, for 90% of human‑operator access to EC2, keyless IAM‑based access is more secure, auditable, and scalable.

Conclusion

The evolution from .pem files to IAM‑powered, keyless access is not merely a trend it is the security baseline for modern cloud operations. By combining AWS Systems Manager Session Manager, IAM Identity Center, and private networking, you eliminate the risks of key sprawl, gain comprehensive auditing, and simplify user lifecycle management. As a Solutions Architect, I recommend this architecture as the starting point for any new workload and a priority migration path for existing environments. The result: a Zero Trust access model where every session is authenticated, authorized, and recorded without a single private key in sight.

Tags: #EC2, #Security

Share: LinkedIn, Twitter, Email