I have a problem that I think every engineer running automated tooling against AWS eventually hits: long-lived access keys are terrible, and everyone knows it, and most people use them anyway because the alternatives feel complicated. Static IAM user credentials are hard to rotate, easy to leak, and a disaster to manage across multiple accounts. If one key gets committed to a repo, you're rotating every key in every account. If a new account gets added, you create another user with another key that has to be managed forever. And there's nothing in the AWS console that stops you from generating a key and never touching it again for three years. I've configured IAM roles across dozens of client accounts. The static key problem is universal, and the enterprise answer is usually instance profiles or OIDC federation for CI pipelines. Those work great for workloads running inside AWS. For a machine sitting on a home network that needs to call AWS APIs, IAM Roles Anywhere is the solution most teams haven't reached yet. I run a personal homelab on AWS Landing Zone Accelerator with 8 accounts: Management, Network, SharedServices, Dev, AI, Backup, Audit, and LogArchive. Off-AWS automation—including CLI commands from the gateway host, infrastructure deployments, and backup scripts—needs programmatic credentials. Workloads already running in AWS use execution roles instead. The SCP I put in place blocks iam:CreateUser across all member accounts. No IAM users allowed. This forced me to solve the external-machine problem properly. The solution is IAM Roles Anywhere. A machine presents a certificate signed by a trusted CA, the service validates the certificate and issues temporary credentials scoped to an IAM role. Maximum 12-hour session duration, no static keys, no users. Here's how I set it up. One thing has tightened since I wrote this. The original multi-account LZA pattern below is still the conceptual foundation, but my active homelab automation now uses a SharedServices Terraform layer with a dedicated IAM Roles Anywhere trust anchor and CN-pinned profiles for specific workloads: the Atlantis Terraform runner, the Burrito datastore writer, and the database backup path. Those use one-hour sessions and workload-specific certificates instead of a single broad host identity. The same pattern survived; the trust policy just got more specific.

How IAM Roles Anywhere Works

The mental model is simpler than the name suggests. You register a certificate authority with AWS as a "trust anchor" in an account. Any certificate signed by that CA can authenticate to that account. You define profiles that control which roles the certificate can assume. When a machine needs credentials, it presents its certificate to the IAM Roles Anywhere endpoint and gets back temporary STS credentials. Three resources per account:

1. Trust Anchor: your CA certificate, registered with IAM Roles Anywhere in the account
2. Profile: configuration for which roles can be assumed, what session policies apply, how long sessions last
3. Role: the IAM role itself, with a trust policy allowing rolesanywhere.amazonaws.com to assume it

The actual credential exchange is handled by aws_signing_helper (available via Homebrew as rolesanywhere-credential-helper). It takes your certificate and private key, signs a request to IAM Roles Anywhere, and returns JSON-formatted credentials that the AWS CLI's credential_process mechanism can consume directly.

The Certificate Setup

I created a custom CA certificate specifically for this purpose. The CA is named Homelab-CA with a 10-year validity period (expires March 2036). From that CA, I issued a machine certificate for the gateway host (the machine running all the automated tooling), with CN=homelab-host, digitalSignature + clientAuth extensions. The certificate and key live in ~/.homelab/certs/. The CA cert is in host-aws.pem, which also contains the machine cert. The private key is host-aws-key.pem, chmod 600. Creating the CA and machine certificate with openssl: IAM Roles Anywhere needs to see the full chain in the certificate file when you have intermediates. For a self-signed CA, it's just the cert followed by the CA.

Deploying to All 8 Accounts via LZA

This is where things get interesting. I could have set up the Trust Anchor, Profile, and Role in each account manually. But with 8 accounts and the expectation of adding more over time, I wanted LZA to handle it. LZA supports deploying arbitrary CloudFormation stacks to specific accounts or OUs via customizations-config.yaml. The stack deploys in each target account during the pipeline run. The CloudFormation template (cfn-templates/roles-anywhere.yaml) creates all three resources: And in customizations-config.yaml, the deployment target covers every account: Run the LZA pipeline and all 8 accounts get the same Trust Anchor, Profile, and Role deployed consistently.

The PEM String Corruption Problem

Here's the part that isn't in any documentation. LZA's customizations support parameter substitution via parameters: in the stack config. The natural approach is to pass the CA certificate as a parameter, something like: This doesn't work. LZA's string replacement engine corrupts multiline PEM strings. The newlines in a PEM-encoded certificate are essential. When LZA processes the parameter substitution, it handles the multiline value incorrectly, and the resulting X509CertificateData in the template is malformed. IAM Roles Anywhere then rejects it with a cryptic validation error. The fix is straightforward: hardcode the CA certificate directly in the template. It's not elegant, but it works reliably. The CA certificate is not a secret (it's the public certificate, not the private key), so embedding it in source control is fine from a security standpoint. The private key never touches the template. There are two other approaches that don't work cleanly with LZA:

• Storing in Parameter Store and using a Custom:: Lambda resource to fetch it at deploy time adds complexity
• Using {{resolve:ssm:...}} only works for simple string values, and LZA processes the resolution before CloudFormation does, which is where the corruption happens

Hardcode the PEM. Move on. To be clear: the CA certificate is a public artifact, the same thing you'd install in any trust store. The private key that signs machine certificates never leaves the host and is never referenced in any template or repo. Embedding the public cert in source control is no different from distributing a root CA to clients that need to verify the chain.

Configuring the AWS CLI

Each account profile in ~/.aws/config uses credential_process to invoke aws_signing_helper: The pattern repeats for all 8 accounts. Each account has its own Trust Anchor ARN and Profile ARN (generated when the CloudFormation stack deploys), but the same certificate, private key, and role name. Running a command is exactly like using any other profile: Each AWS CLI invocation runs aws_signing_helper and performs the certificate exchange, which adds a brief pause. The AWS CLI documentation is explicit that it does not cache credentials returned by credential_process; if process-level caching is needed, the external helper or a wrapper must implement it. The 43,200-second profile duration controls the lifetime of each issued credential set, not an AWS CLI disk cache. You can also test the credential exchange directly: This returns the raw JSON with AccessKeyId, SecretAccessKey, SessionToken, and Expiration. The AWS CLI's credential_process mechanism reads that JSON and uses the credentials transparently.

What This Replaced

Before this setup, the honest answer is I was using long-lived IAM keys stored in ~/.aws/credentials. One key per account. Keys that were valid indefinitely. Keys that, if they ended up in a commit or a log file, would give anyone full administrative access. With IAM Roles Anywhere, the worst case is different. An attacker would need both the certificate AND the private key to authenticate. The private key is a file on the machine, not an environment variable or a config file that gets checked in. The credentials that get issued expire in 12 hours. There are no credentials to steal from ~/.aws/credentials because the file doesn't have any. The SCP that blocks iam:CreateUser across all member accounts makes sure this stays the only path. There's no fallback to "I'll just create a quick IAM user for this." Machine access goes through certificates.

What This Doesn't Solve

Console access still uses AWS Identity Center (SSO). That's the right tool for human interactive access. IAM Roles Anywhere is specifically for workload and machine authentication, where you need programmatic credentials without a human in the loop. You also still need to think about certificate lifecycle. A 10-year CA cert is convenient but means you're relying on the security of that CA key for a decade. If the private key is compromised, you need to rotate the CA, re-register the trust anchor in all 8 accounts, and re-issue machine certificates. That's manageable with 1 machine cert, less manageable at scale. For a homelab with one machine running automated tooling, the complexity tradeoff is worth it. No IAM users, no static keys, every API call is accountable to a specific machine identity.

Tech Stack

• IAM Roles Anywhere for certificate-based workload authentication
• aws_signing_helper (v1.8.x, Homebrew: rolesanywhere-credential-helper) for credential exchange
• Custom CA (OpenSSL, self-signed, CN=Homelab-CA, 10-year validity)
• Machine certificate (CN=homelab-host, digitalSignature + clientAuth, signed by Homelab-CA)
• AWS Landing Zone Accelerator for multi-account CloudFormation deployment
• AWS Identity Center (SSO) for human console access

The CloudFormation template above covers all three resources and deploys consistently across accounts via LZA. The newer Terraform-managed SharedServices profiles use the same IAM Roles Anywhere primitives with tighter certificate subject matching per workload.