AI and ML Engineer | 28 questions | 2026
Scenario based AI Engineer interview questions
Machine Learning
ExperiencedMost askedScenarioFree with sign-inYour model has 95% accuracy in testing but performs poorly in production. How do you investigate?
AnswerHide
I would look for a mismatch between offline and live conditions.
ExperiencedMost askedScenarioFree with sign-inYou are asked to build a churn prediction model with only 3% churners. Walk through your approach.
AnswerHide
- Define the problem: what counts as churn, how far ahead you predict, and what action the business takes. This sets the cost of false positives (a wasted offer) and false negatives (a lost customer).
- Split by time, not randomly. Build features only from data before the prediction date. This prevents...
ExperiencedMost askedScenarioFree with sign-inA stakeholder wants a fraud model that never misses fraud. How do you respond?
AnswerHide
I would explain the trade-off. Pushing recall toward 100% means flagging many more good transactions. This hurts customers and costs reviewer time. Instead, agree on a target based on cost.
ExperiencedMost askedScenarioFree with sign-inTraining loss keeps decreasing but validation loss starts increasing after epoch 10. What do you do?
AnswerHide
This is overfitting from epoch 10 onward. What I would do:
ExperiencedMost askedScenarioFree with sign-inHow do you detect and handle data drift after a model goes live?
AnswerHide
Watch the inputs, the predictions and the outcomes.
ExperiencedMost askedScenarioFree with sign-inHow would you choose between a simple interpretable model and a complex high-accuracy model for loan approvals?
AnswerHide
Loan decisions are regulated and affect people. So interpretability and fairness matter as much as accuracy.
ExperiencedMost askedScenarioFree with sign-inYour dataset has 500 rows and 2,000 features. How do you build a reliable model?
AnswerHide
With far more features than samples, overfitting is the main risk.
ExperiencedMost askedScenarioFree with sign-inYour model must predict for brand new users, but your data has many rows per user. How do you design validation?
AnswerHide
Suppose the same user (or patient, or device) appears in both train and test rows. The model can memorize patterns of that user. Then the score is too high for new users. Use group-aware splits (GroupKFold) when you must predict for new entities. Use time-based splits when you must predict future behavior. For cold-start problems, hold out whole users or items in validation. Make the split match how the model will be used in production. A random split answers a different question than the real one.
LLM and Generative AI
ExperiencedMost askedScenarioFree with sign-inYour support chatbot sometimes gives confident but wrong answers about refund policy. How do you fix it?
AnswerHide
- Reproduce and measure: collect the failed conversations into an evaluation set with the correct policy answers.
- Ground it: retrieve the current policy text. Tell the model to answer only from it and to quote the clause. It should say it does not know when the text does not cover the case.
- **Check...
ExperiencedMost askedScenarioFree with sign-inYour LLM feature costs far more than budgeted after launch. What do you do?
AnswerHide
First measure. Use token logs to break cost down by feature, prompt, user and model. Find the biggest spenders and unusual cases, such as very long inputs, retry loops and agent runs.
ExperiencedMost askedScenarioFree with sign-inUsers report that the assistant answers differently each time for the same question. How would you make it more consistent?
AnswerHide
First separate random variation from real ambiguity.
ExperiencedMost askedScenarioFree with sign-inA prompt that worked well starts failing after the provider updates the model. How do you handle this?
AnswerHide
Treat model changes like dependency upgrades.
ExperiencedMost askedScenarioFree with sign-inDesign a system that summarizes long legal documents of 500 pages using an LLM with a limited context window.
AnswerHide
Use hierarchical (map-reduce) summarization.
ExperiencedMost askedScenarioFree with sign-inHow would you build a text-to-SQL feature safely?
AnswerHide
Give the model the schema (tables, columns, types, short descriptions and a few example queries), not the data. For large schemas, retrieve only the relevant tables.
ExperiencedMost askedScenarioFree with sign-inUsers complain that your LLM feature takes 8 seconds before showing anything. How do you reduce it?
AnswerHide
First find where the time goes. Log retrieval time, prompt size, time to first token and total generation time.
ExperiencedMost askedScenarioFree with sign-inHow would you detect and handle toxic or unsafe output in a public-facing chatbot?
AnswerHide
Use layers.
- Add moderation classifiers for input and output (provider moderation APIs or your own classifier). Block or rewrite flagged content with a safe fallback message.
- Set clear policies in the system prompt, but enforce them in code.
- Log flagged cases for review and add them to the evaluation set.
- Test...
RAG and AI Agents
ExperiencedMost askedScenarioFree with sign-inDesign a RAG chatbot for a company's internal documents (HR policies, wikis, PDFs). Walk through the architecture.
AnswerHide
Ingestion: use connectors for the wiki, drive and PDFs. Parse with layout-aware extraction so tables stay correct. Clean the text. Chunk by headings with overlap. Attach metadata (source URL, owner, updated date, access groups). Embed the chunks and store them in a vector index, with a keyword index for hybrid search. Sync when documents change, including deletions.
ExperiencedMost askedScenarioFree with sign-inYour RAG chatbot answers 'I don't know' to questions that are clearly answered in the documents. How do you debug?
AnswerHide
Check each stage, using logs for the failing questions:
ExperiencedMost askedScenarioFree with sign-inDesign a customer support agent that can look up orders, issue refunds and escalate to humans.
AnswerHide
Use a limited agent with clear tools.
Tools: get_order(order_id), check_refund_eligibility(order_id) (the policy lives in code), issue_refund(order_id, amount), search_help_articles(query) (RAG), handoff_to_human(summary).
ExperiencedMost askedScenarioFree with sign-inRetrieval works for most queries but fails on acronyms, product codes and part numbers. What do you change?
AnswerHide
Pure embeddings often blur exact identifiers. Add hybrid search with BM25 so exact tokens match. Check that the analyzer or tokenizer keeps codes whole (no splitting on hyphens).
ExperiencedMost askedScenarioFree with sign-inYour agent calls the same tool repeatedly with the same arguments and never finishes. What do you do?
AnswerHide
Read the trace to see why. The tool may return an error or an empty result that the model misreads. The tool description may be unclear. Or the goal may be impossible.
ExperiencedMost askedScenarioFree with sign-inHow would you reduce the cost of a RAG system serving millions of queries?
AnswerHide
- Cache answers for frequent questions (exact match and semantic match). Cache embeddings too.
- Retrieve fewer, better chunks with reranking, to make prompts shorter.
- Send easy queries to a small model and hard ones to a larger model.
- Use prompt caching for the fixed instructions.
- Choose cheaper embedding...
ExperiencedMost askedScenarioFree with sign-inUsers ask follow-up questions like 'and what about the premium plan?'. Retrieval returns irrelevant chunks. How do you fix it?
AnswerHide
The raw follow-up does not have enough context to retrieve well. Add a question condensation step. Use the chat history to rewrite it as a standalone query ('What are the features of the premium plan?'). Then retrieve with the rewritten query.
ExperiencedMost askedScenarioFree with sign-inHow would you test and secure a RAG system against prompt injection in retrieved documents?
AnswerHide
Assume that any document may contain hostile instructions. Controls:
- Separate instructions from retrieved content with clear delimiters. Say that the retrieved text is data.
- Strip or flag text that looks like instructions.
- Limit what the model can do. In the RAG path, give it no tools, or only read-only ones.
- Prevent data theft. Do not render arbitrary external URLs or images.
- Apply output filters.
- Restrict who can add documents to the index.
FresherMost askedScenarioFree with sign-inLive task: write a prompt for a ticket classification task, then implement it in Python so it runs with any LLM provider.
AnswerHide
I would write the prompt first, with a clear role, the allowed labels, an output format and one example. Then I would wrap the model call in a small function, so the provider can be swapped.
ExperiencedMost askedScenarioFree with sign-inLive task: build a multi-step chain or agent, for example summarise a document, extract action items, then draft an email. How would you structure it?
AnswerHide
Make each step a small function with one prompt and one clear output. Then join the steps in order. Frameworks such as LangChain express this as a chain. In current LangChain, steps are joined with the pipe operator: prompt | model | parser. But the structure is just function composition, so I would first show it in plain Python.
ExperiencedMost askedScenarioFree with sign-inHow would you build a research agent that browses sources and writes a cited report, while avoiding fabricated citations?
AnswerHide
Plan, gather, then write.
ExperiencedMost askedScenarioFree with sign-inA document corpus contains many scanned PDFs, tables and images. How do you make RAG work on it?
AnswerHide
Invest in parsing before chunking.