Ansible for Infrastructure Automation: Dynamic Inventory, Vault, and CI/CD Integration

Beyond Basic Playbooks

Static inventory and simple playbooks take you far. For managing real infrastructure at scale, you need dynamic inventory (hosts from cloud APIs), encrypted secrets (Vault), and integration with CI/CD pipelines.

This guide covers the patterns you'll need when Ansible is managing production infrastructure across hundreds of hosts.

---

Dynamic Inventory

Static inventory files become a maintenance burden when infrastructure scales. Dynamic inventory plugins pull host lists from cloud APIs.

AWS EC2 Dynamic Inventory

# Install boto3 (AWS SDK)
pip install boto3

# Install the amazon.aws collection
ansible-galaxy collection install amazon.aws

Create inventory/aws_ec2.yml:

plugin: amazon.aws.aws_ec2
regions:
  - us-east-1
  - eu-west-1

# Filter to specific instances
filters:
  instance-state-name: running
  tag:Environment: production

# Group by tags
keyed_groups:
  - key: tags.Role
    prefix: role
  - key: tags.Environment
    prefix: env
  - key: placement.region
    prefix: region

# Hostnames
hostnames:
  - tag:Name
  - private-ip-address

compose:
  ansible_host: private_ip_address

Test it:

ansible-inventory -i inventory/aws_ec2.yml --list
ansible-inventory -i inventory/aws_ec2.yml --graph

# Use in playbook
ansible-playbook -i inventory/aws_ec2.yml site.yml --limit role_webserver

GCP Dynamic Inventory

pip install requests google-auth
ansible-galaxy collection install google.cloud

# inventory/gcp.yml
plugin: google.cloud.gcp_compute
projects:
  - my-gcp-project
regions:
  - us-central1
auth_kind: serviceaccount
service_account_file: /path/to/service-account.json

keyed_groups:
  - key: labels.role
    prefix: role
  - key: zone
    prefix: zone

---

Ansible Vault: Encrypting Secrets

Never store passwords or API keys in plaintext in your repository.

Encrypting Secrets

# Create a vault password file (don't commit this)
echo "my-vault-password" > ~/.vault_pass
chmod 600 ~/.vault_pass

# Encrypt a value inline (for use in vars)
ansible-vault encrypt_string 'my-db-password' --name 'db_password'
# Output:
# db_password: !vault |
#   $ANSIBLE_VAULT;1.1;AES256
#   3661333...

# Encrypt a whole file
ansible-vault encrypt inventory/group_vars/production/secrets.yml

# Decrypt for viewing
ansible-vault view inventory/group_vars/production/secrets.yml

# Edit in place
ansible-vault edit inventory/group_vars/production/secrets.yml

Vault File Structure

# inventory/group_vars/production/secrets.yml (encrypted)
---
db_password: "supersecret"
api_key: "sk-abc123..."
ssl_key: |
  -----BEGIN PRIVATE KEY-----
  ...

Using Vault Secrets in Playbooks

# inventory/group_vars/production/vars.yml (plaintext)
db_host: db01.example.com
db_user: appuser

# inventory/group_vars/production/secrets.yml (vault-encrypted)
db_password: "supersecret"

The db_password variable is available in tasks without any special handling — Ansible decrypts it at runtime:

- name: Configure database
  ansible.builtin.template:
    src: templates/db.conf.j2
    dest: /etc/myapp/db.conf
  # Template uses {{ db_host }}, {{ db_user }}, {{ db_password }}

Run with vault password:

# Pass vault password file
ansible-playbook -i inventory/ site.yml --vault-password-file ~/.vault_pass

# Or prompt
ansible-playbook -i inventory/ site.yml --ask-vault-pass

# Set in ansible.cfg
vault_password_file = ~/.vault_pass

---

ansible.cfg: Project Configuration

Create ansible.cfg at the project root:

[defaults]
inventory = inventory/
roles_path = roles/
collections_paths = collections/
vault_password_file = ~/.vault_pass
remote_user = deploy
private_key_file = ~/.ssh/id_ed25519
host_key_checking = False
retry_files_enabled = False
stdout_callback = yaml
callbacks_enabled = timer, profile_tasks

[ssh_connection]
ssh_args = -o ControlMaster=auto -o ControlPersist=60s -o ServerAliveInterval=30
pipelining = True
forks = 20    # default is 5; increase for large inventories

pipelining = True significantly speeds up Ansible by reducing SSH round-trips (requires requiretty disabled in sudoers).

---

Rolling Updates with serial

Deploy to hosts in batches to avoid taking down everything at once:

- name: Deploy application
  hosts: webservers
  serial: 2         # deploy to 2 hosts at a time (number or percentage)
  max_fail_percentage: 25   # abort if >25% of hosts fail

  pre_tasks:
    - name: Remove from load balancer
      community.general.haproxy:
        state: disabled
        host: "{{ inventory_hostname }}"
        socket: /run/haproxy/admin.sock
        wait: yes
        wait_retries: 10

  roles:
    - app_deploy

  post_tasks:
    - name: Wait for application to start
      ansible.builtin.uri:
        url: "http://{{ inventory_hostname }}/health"
        status_code: 200
      register: health
      until: health.status == 200
      retries: 10
      delay: 5

    - name: Add back to load balancer
      community.general.haproxy:
        state: enabled
        host: "{{ inventory_hostname }}"
        socket: /run/haproxy/admin.sock

---

CI/CD Integration

GitHub Actions Workflow

# .github/workflows/deploy.yml
name: Deploy Infrastructure

on:
  push:
    branches: [main]
  workflow_dispatch:

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: production   # requires manual approval

    steps:
      - uses: actions/checkout@v4

      - name: Install Ansible
        run: |
          pip install ansible boto3 botocore
          ansible-galaxy install -r requirements.yml

      - name: Write vault password
        run: echo "${{ secrets.ANSIBLE_VAULT_PASSWORD }}" > /tmp/vault_pass
        
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1

      - name: Write SSH key
        run: |
          echo "${{ secrets.DEPLOY_SSH_KEY }}" > /tmp/deploy_key
          chmod 600 /tmp/deploy_key

      - name: Run playbook
        run: |
          ansible-playbook \
            -i inventory/aws_ec2.yml \
            --vault-password-file /tmp/vault_pass \
            --private-key /tmp/deploy_key \
            site.yml

      - name: Cleanup secrets
        if: always()
        run: rm -f /tmp/vault_pass /tmp/deploy_key

---

Collections: Modern Role Packaging

Collections bundle roles, modules, and plugins for distribution:

# Install collections
ansible-galaxy collection install community.general
ansible-galaxy collection install ansible.posix

# requirements.yml
collections:
  - name: community.general
    version: ">=8.0.0"
  - name: amazon.aws
    version: ">=7.0.0"
  - name: ansible.posix
    version: ">=1.5.0"

ansible-galaxy collection install -r requirements.yml -p ./collections/

Use collection modules with FQCN (Fully Qualified Collection Name):

- name: Mount NFS share
  ansible.posix.mount:
    path: /mnt/data
    src: nfs-server:/exports/data
    fstype: nfs
    state: mounted

- name: Set file ACL
  ansible.posix.acl:
    path: /var/www/app
    entity: www-data
    etype: user
    permissions: rwx
    state: present

---

Performance: Tuning for Large Inventories

# ansible.cfg
[defaults]
forks = 50              # parallel SSH connections
gathering = smart       # cache facts; only gather if not cached
fact_caching = jsonfile
fact_caching_connection = /tmp/ansible_facts
fact_caching_timeout = 86400   # cache for 24 hours

[ssh_connection]
pipelining = True
ssh_args = -o ControlMaster=auto -o ControlPersist=600s

Skip fact gathering when not needed:

- name: Deploy config files (no facts needed)
  hosts: webservers
  gather_facts: no

  tasks:
    - ansible.builtin.template:
        src: templates/app.conf.j2
        dest: /etc/app/app.conf

---

Useful Patterns

Assert Conditions Before Proceeding

- name: Verify prerequisites
  ansible.builtin.assert:
    that:
      - ansible_distribution in ["Ubuntu", "Rocky", "RedHat"]
      - ansible_memtotal_mb >= 2048
      - db_password is defined
    fail_msg: "Prerequisites not met: check OS, RAM, and vault variables"

Conditional Task Execution

- name: Configure SELinux (RHEL only)
  ansible.posix.selinux:
    policy: targeted
    state: enforcing
  when: ansible_os_family == "RedHat"

- name: Configure AppArmor (Ubuntu only)
  community.general.ufw:
    state: enabled
  when: ansible_distribution == "Ubuntu"

Register and Use Task Output

- name: Check if service is deployed
  ansible.builtin.stat:
    path: /opt/myapp/current
  register: app_deploy_stat

- name: Run database migrations
  ansible.builtin.command:
    cmd: /opt/myapp/current/bin/migrate
  when: app_deploy_stat.stat.exists
  changed_when: true    # always mark as changed if run