DevOps Troubleshooting Interview Questions and Answers

Name what you have used. Show that you know the matching services in other clouds.

Sample answer: "I have the most hands-on experience with AWS ([your services]). I have used [GCP/Azure services] for [your project]. The ideas (networks, IAM, compute, storage) carry over. So I can learn the rest quickly from documentation and infrastructure as code." Be honest, and give a real project.

What they are checking: Whether you are honest about experience and understand the common concepts across clouds.

Work from the outside in: status, events, logs, then a shell inside the container.

kubectl get pods -n app                     # status: Pending, CrashLoopBackOff, ImagePullBackOff
kubectl describe pod web-7d9 -n app         # events: scheduling, probes, OOMKilled, image errors
kubectl logs web-7d9 -n app --previous      # logs of the last crashed container
kubectl exec -it web-7d9 -n app -- sh       # look around inside
kubectl get events -n app --sort-by=.lastTimestamp

• Pending: not enough resources, taints, or an unbound PVC.
• ImagePullBackOff: wrong image name or tag, or a missing registry secret.
• CrashLoopBackOff: the app exits, so read the previous logs; check env vars, config and probes.
• OOMKilled: raise memory limits or fix the leak.

If the image has no shell, use kubectl debug -it web-7d9 --image=busybox --target=web.

What they are checking: Whether you have a repeatable kubectl debugging routine rather than guessing.

Use built-in tools for a quick look. Use a metrics agent for history.

• top / htop: live CPU, memory and per-process usage. Sort by CPU or memory.
• free -h: memory and swap. Look at "available", not "free".
• vmstat 1 and iostat -x 1: run queue, swapping and disk wait.
• ps aux --sort=-%mem | head: the top memory users.
• uptime: load average. Compare it with the number of cores (nproc).
• df -h and du -sh: disk space.

For long-term monitoring, run Node Exporter with Prometheus and Grafana, or the CloudWatch agent. Alert on high CPU, low available memory and full disks.

What they are checking: Whether you know the basic Linux toolkit and the difference between live checks and ongoing monitoring.

Keep data outside the container's writable layer so it survives restarts.

• Locally with Docker: use a named volume.

docker run -d --name pg -e POSTGRES_PASSWORD=secret \
  -v pgdata:/var/lib/postgresql/data -p 5432:5432 postgres:16

• On Kubernetes: use a StatefulSet with volumeClaimTemplates. Each pod gets a stable name and its own PersistentVolumeClaim. A StorageClass backs it (for example the EBS CSI driver), so data stays after the pod moves. A headless Service gives stable DNS names. Store passwords in Secrets.

For production, a managed database such as RDS is usually safer than running your own. Backups, failover and patching are handled for you.

What they are checking: Whether you know volumes, PVCs and why StatefulSets exist.

ETL means extract, transform, load.

• Extract: read files from S3 (CSV, JSON, Parquet) or databases.
• Transform: clean the data. Drop or fill nulls, remove duplicates, fix types and formats, and join sources. In a graphical tool such as AWS Glue Studio, Talend or Azure Data Factory, you drag source, transform and target nodes and set the rules.
• Load: write into the warehouse. For Redshift use COPY from S3 (parallel and fast). For Snowflake use COPY INTO from a stage.

COPY sales FROM 's3://my-bucket/sales/'
IAM_ROLE 'arn:aws:iam::123456789012:role/redshift-load'
FORMAT AS PARQUET;

• Make queries fast: use columnar formats (Parquet), partitioned files, good distribution and sort keys, and compression. Avoid SELECT *. Use pre-aggregated tables.
• Schedule the job. Add data quality checks and alerts.

What they are checking: Whether you understand ETL stages and the basics of warehouse loading.

The path from code to a live site, in a few steps.

1. Build: develop the front end (HTML, CSS, JavaScript framework) and the backend API. Store data in a database. Keep code in Git.
2. Package: build the app, or a Docker image.
3. Infrastructure: a server or cloud service (VM, container service, or static hosting), a database, and a firewall that opens only ports 80 and 443.
4. Deploy: a CI/CD pipeline (GitHub Actions) tests, builds and ships the release. A web server such as Nginx serves the app and sends requests on to the backend.
5. Domain and HTTPS: point a DNS record to the server or load balancer. Add a TLS certificate (Let's Encrypt or ACM).
6. Operate: logs, monitoring, backups, and scaling or a CDN for speed.

This also touches operating systems (processes, files), networks (DNS, TCP, HTTP) and web technologies.

What they are checking: Whether you understand the whole lifecycle at a high level and the pieces involved.

Prometheus collects and stores metrics. Grafana shows them.

Check whether the app, the host and the network each allow the traffic.

Failures usually come from the network, the key, or the user.

Nagios runs checks on a schedule using plugins, and agents expose host data.

SQL lets you create tables and query rows.

The backend uses a driver or ORM with a connection string, host, port and credentials.

Snowflake separates storage and compute and needs little care. Redshift is tied closer to AWS and its cluster sizing.

Scrapy is a fast crawler for static pages. Selenium controls a real browser for pages that need JavaScript or interaction.

RAG (retrieval augmented generation) gives a language model relevant documents at question time, so answers are grounded in your data.

Virtualization lets one physical machine run several isolated virtual machines. Each has its own OS. They share the hardware through a hypervisor.

Error 525 means Cloudflare could not finish a TLS handshake with your origin server.

TLS 1.3 sets up an encrypted connection in one round trip.

OOP organises code into objects that combine data and behaviour.

In event-driven programming, events control the flow of the program. An event can be a click, a message or a file upload. Handlers react to them.

• An event is a signal that something happened.
• A handler/listener is code that runs in response.
• An event loop or broker waits for events and passes them on.

A common open source answer is Prometheus, Grafana, Loki and Alertmanager.

• Metrics: Node Exporter for hosts, kube-state-metrics and cAdvisor for Kubernetes, and client libraries for application metrics. Prometheus scrapes (pulls) them. On Kubernetes, use the kube-prometheus-stack Helm chart.
• Logs: Promtail or Grafana Alloy sends container and system logs to Loki. Query them with LogQL in Grafana.
• Dashboards: Grafana with Prometheus and Loki as data sources.
• Alerting: Prometheus alert rules fire to Alertmanager. It groups alerts, removes duplicates, and sends them to Slack, email or PagerDuty by severity and team.
• Good alerts: alert on symptoms (error rate, latency, pod restarts, pending pods, disk filling within 4 hours). Do not alert on every CPU spike. Each alert should link to a runbook.
• Managed option: CloudWatch with SNS, or Datadog, when you do not want to run the stack yourself.

- alert: HighErrorRate
  expr: sum(rate(http_requests_total{code=~"5.."}[5m])) / sum(rate(http_requests_total[5m])) > 0.05
  for: 10m
  labels: {severity: page}

What they are checking: Whether you have really built an observability stack and know how alerts reach people.

First make things stable, then find the cause, then recover. Do not reboot blindly, because you may lose evidence.

• Check status: look at the EC2 console status checks. A failed system check points to the AWS host. Stop and start the instance to move it to new hardware. A failed instance check points to the OS (kernel panic, full disk, bad fstab).
• Metrics and logs: look at CloudWatch CPU, credit balance, disk and network graphs. Also read the console output and screenshot (aws ec2 get-console-output --instance-id i-0abc).
• If SSH fails: try SSM Session Manager first. If that fails, stop the instance and detach the root volume. Attach it to a healthy rescue instance and mount it. Then read /var/log, fix fstab, free disk space or fix sshd_config. Finally reattach the volume and start the instance.
• If it is just too small: stop it, change the instance type, and start it. Or snapshot the volume and launch a larger instance from it.
• Avoid a repeat: set alarms on CPU, disk and status checks. Use an Auto Scaling group with health checks so it heals itself. Hold a short review after the incident.

aws ec2 describe-instance-status --instance-ids i-0abc --include-all-instances

What they are checking: Whether you have a calm, ordered recovery process and know the detach-volume rescue trick.

Start with the user impact, then narrow down layer by layer using data, not guesses.

1. Scope it: which endpoint, since when, which users? Check recent deploys or config changes.
2. Golden signals (latency, traffic, errors, saturation) on a Grafana or CloudWatch dashboard.
3. Resources: CPU throttling, memory pressure and swap, disk I/O, network, connection limits.
4. Distributed tracing (OpenTelemetry with Jaeger or Tempo, or X-Ray): the slowest span shows which service or call is guilty.
5. Dependencies: slow SQL (slow query log, pg_stat_statements), locks, cache misses, third-party APIs, DNS.
6. Logs: correlate by request ID, look for timeouts and retries.
7. Fix and verify: add an index, cache, scale out (HPA), tune pool sizes, or roll back. Then confirm the latency graph recovered.

The key is to change one thing at a time and measure.

What they are checking: Whether you follow a structured, data driven approach and know tracing.

Aurora is AWS's cloud native engine (compatible with MySQL and PostgreSQL). Standard RDS runs the regular engine on EBS volumes.

Writer crash in Aurora: the cluster detects the failure and promotes a reader to writer automatically. It picks by failover priority tier (then size). Readers share storage, so no data copy is needed. The cluster endpoint then points at the new writer, so applications that reconnect keep working. The failed instance is replaced and joins again as a reader. Clients must retry dropped connections. They must use the cluster (writer) endpoint and the reader endpoint, not instance endpoints.

For RDS Multi-AZ, the standby is promoted and DNS flips. So keep the DNS TTL low and retry connections.

What they are checking: Whether you know the shared storage design and how failover is exposed to applications.

Look at what the database is doing. Then fix the query or the connection handling.

Make costs visible per team. Set budgets. Then act on the biggest items.

Run active copies in each region. Send users to the nearest healthy one. Replicate data.

Default to a managed database outside the cluster, and self-host only with a good reason.

Choose by data shape, consistency needs and access pattern.

Treat AI tools like any other powerful third party that can reach your code and systems.

Pick the service by how much control you need.

They measure different layers.

Alert from metrics. Then investigate with kubectl.

Check each hop between the two, starting at the backend.

Use the exit information Kubernetes already records.

HPA adds or removes pod replicas. VPA changes the CPU and memory requests of each pod.

Find the blocker. Decide whether to kill it. Stop it from happening again.

Most problems come from what an EC2 app assumes about its machine.

Find out which layer is still serving old content.

Separate the resolver, the DNS records and the network path.

Restore service first. Find the root cause second.

Retention is set per log group, and search uses Logs Insights or filter patterns.

• Retention: by default logs never expire, which costs money. Set a retention period in the console, with Terraform (retention_in_days), or the CLI:

Ship logs from every host and pod to one searchable store.

• Collect: Fluent Bit, Promtail or Grafana Alloy as a DaemonSet or agent, tagging logs with labels like namespace, pod and app. Prefer structured JSON logs with a request ID.
• Store and search: Loki or Elasticsearch/OpenSearch.

Instrument with OpenTelemetry, collect with a collector, and view in a tracing backend.

Check from outside the system, as a user would. Alert on failures.

Aurora Global Database spans multiple regions. It has one primary region and read-only secondary regions. It copies data at the storage level.

Put RDS in private subnets. Never expose it to the internet.

Combine automated backups, copies in another region and account, and tested restores.

Choose by recovery targets: how much data you can lose (RPO) and how fast you must be back (RTO).

DynamoDB queries must use a key, so indexes give you extra access patterns.

Describe the path from model to a secure, scalable endpoint.

Automate it from a template instead of clicking 50 times.

• Template: build one VM, install the OS and agents, generalise it (cloud-init or sysprep), and convert it to a template.
• Terraform with the vSphere provider:

Make the environments hard to confuse and hard to touch by accident.

Scale turns small issues into daily problems.

First find which role is making the call, and which policy is missing or blocking it.

Give a short map of what you built. Then name the tooling.