Interview Prep: Senior & Architect Questions¶
These questions test architectural judgment across a whole organization, not a single playbook. There's rarely one right answer; interviewers listen for trade-offs, sequencing, and how you'd make the decision stick with other teams.
Each question includes the structure of a strong answer and the signals that separate senior from mid-level responses.
1. Project structure for a 30-engineer organization¶
"How would you design an Ansible project structure that a 30-engineer organization can safely contribute to, without one team's changes breaking another's environment?"
Strong answer structure:
- Separate content from configuration. Reusable roles, modules, and plugins live in versioned collections owned by platform or domain teams. Environment-specific data lives in inventory repositories.
- Isolate environments in inventory. One inventory per environment (
inventories/staging,inventories/production), each with its owngroup_vars/and vault ID, so a staging change can't alter production values. See Multi-Environment Inventory. - Pin everything. Playbook repositories consume collections through pinned
requirements.yml, ideally built into Execution Environments. A role change reaches a team only when that team bumps the version. - Ownership and review.
CODEOWNERSper role and per inventory; production inventory changes need approval from the owning team. - Automated gates. Lint, syntax check, Molecule for roles,
--check --diffagainst staging for playbooks, all in CI. See CI/CD and Linting. - Run from one place. Production runs happen from a controller or CI with audit logs, not laptops.
Senior signals: talks about versioning boundaries and blast radius, not just directory layout; mentions how breaking changes are communicated (changelogs, deprecation periods).
Mid-level trap: one giant monorepo with shared group_vars/all.yml and every role at HEAD.
Related: Project Layout, Production Role Design
2. Terraform + Ansible + Kubernetes¶
"When would you recommend Terraform + Ansible + Kubernetes together, versus one tool trying to do all three jobs?"
Strong answer structure:
| Layer | Best tool | Why |
|---|---|---|
| Cloud resources: networks, IAM, managed databases, clusters | Terraform | State tracking, plan/diff of infrastructure, dependency graph |
| Operating system and long-lived host configuration | Ansible | Agentless, procedural where needed, great for VMs and bare metal |
| Containerized application lifecycle | Kubernetes (with Helm/Kustomize, GitOps) | Continuous reconciliation, self-healing, scaling |
- Use all three when the estate has all three layers: cloud infrastructure, VMs or bare metal that need configuration (databases, legacy apps, Kubernetes nodes themselves), and containerized services.
- Use fewer when layers are missing. A fully managed Kubernetes platform with immutable node images may need no Ansible at all for hosts; a VM-only estate may not need Kubernetes.
- Avoid overlap: Ansible creating cloud resources without state leads to drift; Terraform provisioners running configuration scripts are hard to debug;
kubectl applyfrom Ansible competes with GitOps controllers. - Handoffs: Terraform outputs and tags feed Ansible dynamic inventory; Ansible builds golden images or configures nodes; Kubernetes owns everything inside the cluster.
Senior signals: defines clear ownership boundaries and handoff points; can explain where the answer changes (immutable infrastructure, small teams).
Related: What Is Ansible?, Terraform, Kubernetes
3. Introducing secrets management to a messy organization¶
"How would you introduce Ansible Vault or an external secret manager into an organization that currently has secrets scattered across plaintext files and Slack messages?"
Strong answer structure:
- Assume compromise. Secrets in Slack and plaintext files are already exposed; the plan must include rotation, not just encryption.
- Inventory first. Scan repositories and history (
gitleaks,trufflehog) and catalog secrets by owner and system. - Pick the target by scale. Small team: Ansible Vault with a vault ID per environment. Larger organization: an external manager (HashiCorp Vault, cloud secret managers) with per-identity access and audit logs, consumed through lookups.
- Migrate in waves, highest risk first (production database, cloud root credentials). For each: store in the new system, change consumers, rotate, verify, then delete the old copy.
- Prevent regression. Pre-commit and CI secret scanning,
no_logreview rules, and a clear, easy path for engineers to request and use secrets — people leak secrets when the correct way is too hard. - Measure. Count of secrets migrated and rotated, scanner findings trending to zero.
Senior signals: prioritizes rotation and developer experience; knows Vault ciphertext stays in Git history forever; plans for the vault password or manager credential itself.
Related: Secrets and Vault, Case Study: Vault Secrets
4. Governing an internal collection ten teams depend on¶
"A team wants to publish an internal collection that ten other teams will depend on — what does your review process for changes to it look like?"
Strong answer structure:
- Ownership: a named owning team with
CODEOWNERS; consumers can propose changes, owners approve. - Compatibility contract: semantic versioning; documented public interface (module arguments, role
argument_specs); breaking changes only in majors, preceded by deprecation warnings viameta/runtime.yml. - Automated checks on every pull request:
ansible-test sanityand units, integration tests or Molecule scenarios, lint, and a changelog fragment required. - Consumer protection: consumers pin major versions; a canary consumer (or a test repository exercising real consumer playbooks) runs against release candidates.
- Release process: tagged releases from CI to a private Automation Hub with an approval step; no manual uploads.
- Communication: release notes, a migration guide for majors, and a support window for the previous major.
Senior signals: treats the collection as a product with users; balances contribution speed with stability; has a plan for deprecating and removing things.
Related: Publishing Collections, Production Role Design
5. When to replace shell scripts with Ansible — and when not to¶
"How do you decide when a growing shell-script-based deployment process should be replaced with Ansible, versus when it shouldn't be?"
Signals that favor moving to Ansible:
- The scripts run against many hosts and hand-roll SSH loops, parallelism, and retries.
- They need to be re-runnable, but aren't idempotent — re-running causes damage.
- Several people modify them and there's no review or testing.
- Logic branches per OS or environment and variables are scattered across files.
- You need dry runs, diffs, or auditability.
Signals that favor keeping scripts:
- A single-host, one-shot task (a container entrypoint, a build step).
- Work already inside a system that manages state, such as a Dockerfile, a CI job, or a Kubernetes Job.
- The process is being retired or replaced by immutable images or GitOps soon.
- The scripts are small, stable, tested, and well understood.
How to migrate if you do: wrap first (run the existing script with ansible.builtin.script or command plus creates/changed_when) to gain inventory, parallelism, and logging immediately; then replace the riskiest steps with proper modules one at a time, verifying with --check --diff.
Senior signals: doesn't reflexively say "always Ansible"; considers team skills, lifetime of the system, and incremental migration with measurable wins.
Related: Shell Scripts, Command vs. Shell, Idempotency
6. Scaling Ansible to thousands of hosts¶
"Playbook runs across 5,000 hosts now take two hours. How would you approach it?"
Strong answer structure:
- Measure before changing. Enable the
ansible.posix.profile_taskscallback to find the slow tasks. - Cheap wins: pipelining and ControlPersist, raised
forkssized to the controller, fact caching withgathering = smartorgather_facts: falsewhere unused,gather_subset. - Task-level fixes: replace
commandloops with modules that take lists (packagewith a list, not a loop); avoid per-item lookups against slow APIs. - Execution strategy:
strategy: freewhere tasks don't need to wait for each other;serialonly where rollout safety requires it. - Architecture: split monolithic playbooks by concern and cadence; use Automation Controller with Mesh to run near the hosts in each region; run changes on demand instead of everything every time.
Senior signals: starts with profiling, separates speed from safety (not removing serial from critical rollouts), and understands controller resource limits.
Related: Forks, Serial, Strategy, and Throttle, Performance, Automation Controller and Mesh
Next¶
Return to Interview Preparation, or continue to Quick Reference.