Machine Learning Engineer interview questions
63 questions asked in Machine Learning Engineer interviews, on models in production, evaluation, LLMs and ML infrastructure. They are built from thousands of real job ads. Under each one: what the interviewer is listening for, and the follow-up they may ask. For the recruiter call, see the HR screen questions.
Behavioral questions
Questions about how you work in this role. More in behavioral questions for any tech role.
-
1
Tell me about a time a simple approach beat a more complex model you'd tried.
All levels
Listening for Honest comparison with numbers and no ego about the fancy model.
Follow-up Ask what they learned about where to start next time.
-
2
Tell me about a time someone expected more from a model or from AI than it could actually do. How did you handle it?
All levels
Listening for Setting honest expectations with examples instead of just saying no.
Follow-up Ask how the conversation ended.
-
3
Tell me about a time you tried a technique from a paper or a blog post at work. How did it go?
Junior
Listening for Curiosity with judgement about whether it fit the problem.
Follow-up Ask how they decided it was worth the time.
-
4
Tell me about a time your results looked too good to be true. What did you find?
Junior, Mid
Listening for Healthy suspicion and how they found the cause, like data leakage.
Follow-up Ask what check they now always run.
-
5
Tell me about a model you worked hard on that never made it to production. Why not?
Mid, Senior
Listening for Clear reasons, their own part in it, and what they would do differently.
Follow-up Ask when they first saw it might not ship.
-
6
Tell me about a time you worked with researchers or data scientists to get their model into production.
Mid, Senior
Listening for Bridging two ways of working and agreeing on what production needs.
Follow-up Ask what they changed in the handover after that.
-
7
Tell me about a time you raised a concern that a model could treat some users unfairly or cause harm.
Mid, Senior
Listening for Spotting the risk, backing it with data and pushing it to a decision.
Follow-up Ask what changed because of it.
Technical discussion questions
Talking through your experience and decisions, not a coding test. Grouped by topic.
Python
-
1
Walk me through a model you trained recently, from the data to the result.
Junior
Listening for Data prep, a baseline, the metric and what they did themselves.
Follow-up Ask what they tried that didn't work.
-
2
How do you keep research code from notebooks maintainable once it goes to production?
Mid, Senior
Listening for Moving logic into tested modules, configs, reproducible runs, code review.
Follow-up Ask about a time notebook code caused a production problem.
-
3
How do you raise the bar on ML code and analysis quality across the team?
Senior
Listening for Reviews, shared evaluation tools, templates and pairing that actually changed habits.
Follow-up Ask about one change that stuck.
LLMs
-
4
How would you evaluate an LLM feature where there's no single right answer?
Mid, Senior
Listening for Rubrics, a golden set, LLM-as-judge checked against people, regression runs.
Follow-up Ask how they would know the judge can be trusted.
-
5
How would you design an assistant that answers customer questions using our help docs and the customer's account data?
Senior
Listening for Retrieval, tool calls for account data, permissions, guardrails, hand-off to people and evaluation.
Follow-up Ask what happens when it is confidently wrong.
-
6
How would you decide between a hosted LLM API and fine-tuning and serving our own open model?
Senior
Listening for Quality, cost at our volume, latency, data privacy and team effort.
Follow-up Ask what would make them change the decision later.
Evaluation
-
7
Only one percent of our transactions are fraud. Which metric would you use to judge a fraud model?
Junior
Listening for Why accuracy misleads and a choice like precision, recall or PR-AUC tied to the cost of mistakes.
Follow-up Ask how they would pick the threshold.
-
8
How would you set up evaluation so the team can ship model changes every week without breaking things?
Senior
Listening for Offline regression suites, fast online checks, guardrail metrics, clear release rules.
Follow-up Ask who decides when a result is good enough.
-
9
Offline metrics keep improving, but online tests rarely show wins. What would you look into?
Senior
Listening for Metric mismatch, biased offline data, feedback loops, test power.
Follow-up Ask how they would change the offline evaluation.
Embeddings
-
10
How would you explain embeddings to a backend engineer on your team?
Junior
Listening for A simple correct picture of meaning as nearby vectors, with a use case.
Follow-up Ask how they would check two embeddings are really similar in meaning.
-
11
How would you choose an embedding model for search over our product catalogue?
Mid, Senior
Listening for A small labelled test set, comparing a few models on recall, cost and latency.
Follow-up Ask when they would fine-tune the embedding model.
MLOps
-
12
Tell me about a model you owned from training to production. What did you monitor after launch?
Mid
Listening for Data and prediction drift, latency, errors and business metrics with alerts.
Follow-up Ask about something monitoring caught.
-
13
Walk me through how your team releases a new model version safely.
Mid
Listening for Versioned artefacts, shadow or canary traffic, metrics to watch, quick rollback.
Follow-up Ask about a release that went wrong.
-
14
We have a dozen models, each deployed in its own way. How would you design a shared ML platform?
Senior
Listening for Common training, registry, serving and monitoring, built from real team needs.
Follow-up Ask what they would build first and what they would leave to teams.
Model serving
-
15
You've trained a model in a notebook. What has to happen before our app can call it?
Junior
Listening for Packaging, an API, versioning, monitoring and matching preprocessing.
Follow-up Ask what they would monitor first.
-
16
Our model takes 800 milliseconds to respond, and the product needs under 100. What options do you have?
Mid
Listening for Smaller or distilled models, quantisation, batching, caching, better hardware.
Follow-up Ask what quality loss they would accept.
-
17
Our LLM inference bill tripled in a quarter. How would you bring it down without hurting quality?
Senior
Listening for Measuring cost per feature, caching, routing to smaller models, shorter prompts, batching.
Follow-up Ask how they would prove quality held.
Pytorch
-
18
Your training run crashes with a GPU out-of-memory error. What would you try?
Junior
Listening for Smaller batch, gradient accumulation, mixed precision, checkpointing, more GPUs.
Follow-up Ask what each fix costs.
-
19
Tell me about a project where you used PyTorch or TensorFlow. What got in your way?
Junior, Mid
Listening for Real friction like debugging, data loading or deployment, and how they solved it.
Follow-up Ask which framework they would pick for a new project and why.
Distributed training
-
20
Training takes four days on one GPU. How would you move it to several GPUs?
Mid
Listening for Data parallel training, batch size and learning rate changes, data loading bottlenecks.
Follow-up Ask what stops it from scaling well.
-
21
We need to train a model that's too large for one GPU. What options would you weigh?
Senior
Listening for Sharded data parallel, tensor and pipeline parallelism, memory savings and communication cost.
Follow-up Ask how they would debug a run that gets slower with more GPUs.
Rag
-
22
Walk me through how you'd build a simple chatbot that answers questions from our internal docs.
Junior
Listening for Chunking, embedding, retrieval, then a prompt with the retrieved text.
Follow-up Ask what they would do when the answer isn't in the docs.
-
23
When you build search over long documents for an LLM, how do you decide how to split them into chunks?
Junior
Listening for Chunk size against context, overlap, keeping sections whole, testing with real questions.
Follow-up Ask how they would tell that chunking is the problem.
-
24
Your RAG assistant often answers from the wrong document. How would you debug it?
Mid
Listening for Checking retrieval separately from generation, chunking, embedding model, hybrid search or reranking.
Follow-up Ask how they would build an evaluation set.
Fine tuning
-
25
When would you fine-tune a model instead of just writing a better prompt?
Junior, Mid
Listening for Trade-offs of data, cost, latency, style and control.
Follow-up Ask how much data they would want before fine-tuning.
-
26
Walk me through a fine-tuning project you did, from the data to how you judged the result.
Mid, Senior
Listening for Data collection and cleaning, method like LoRA, a baseline and an honest evaluation.
Follow-up Ask what they would change with twice the budget.
Recommendation systems
-
27
Say you're building a 'you might also like' section for a small online shop. Where would you start?
Junior
Listening for A simple baseline like co-purchases from order data, then cold start.
Follow-up Ask how they would know it's working.
-
28
A recommendation model's click rate slowly drops over two months. How would you find out why?
Mid
Listening for Separating data drift, product changes, stale features and feedback loops.
Follow-up Ask how often they would retrain and why.
-
29
How would you design recommendations for a feed with millions of users and items?
Senior
Listening for Candidate generation with embeddings, ranking, re-ranking, freshness and cold start.
Follow-up Ask what changes when traffic grows ten times.
Deep learning
-
30
Your model does great on the training data but poorly on the validation set. What would you try?
Junior
Listening for Spotting overfitting and practical fixes like more data, regularisation, early stopping.
Follow-up Ask how they would make sure the split itself is fair.
Ranking
-
31
Users say our search results feel off. How would you measure how good the ranking actually is?
Junior
Listening for Ranking metrics like NDCG or MRR, labelled examples and click data.
Follow-up Ask how position bias affects click data.
-
32
You're improving the ranking of our home feed. How would you decide which new signals to try?
Mid
Listening for Hypotheses from user behaviour, cost to build, offline checks before tests.
Follow-up Ask how they would avoid making the feed too narrow.
Transformers
-
33
Tell me about a text task you've worked on. Which model did you use, and why that one?
Junior
Listening for Choosing a model for the task, not reciting theory.
Follow-up Ask what they compared it against and what would make them switch.
Experimentation
-
34
We're testing a new personalised homepage against the old one. What would you check before trusting the result?
Junior
Listening for Enough users and time, a clear main metric, a fair split, novelty effects.
Follow-up Ask what they would do if new and old users react differently.
-
35
Your new model beats the old one offline. Why do we still run an A/B test before launching it?
Junior, Mid
Listening for Offline metrics miss user behaviour, feedback loops and business metrics.
Follow-up Ask what they would do if the test shows no difference.
-
36
Your ranking model wins the A/B test on clicks, but time spent in the app goes down. What would you do?
Mid
Listening for Questioning the target metric, guardrails, looking at segments, a decision with the product team.
Follow-up Ask which metric they would optimise next.
Prompt engineering
-
37
How do you know a prompt change actually made things better?
Junior, Mid
Listening for A fixed test set, clear criteria, comparing before and after, not trusting a few examples.
Follow-up Ask how big their test set was.
-
38
Many teams are now building prompts and agents. What standards would you set for how they're built and reviewed?
Senior
Listening for Shared evaluation, prompt versioning, review, safety and cost limits.
Follow-up Ask how they would get teams to adopt them.
Reinforcement learning
-
39
Tell me about a problem where reinforcement learning fits better than ordinary supervised learning.
Junior
Listening for Sequential decisions with delayed reward, explained with a real example.
Follow-up Ask what makes RL hard to use in practice.
-
40
In simple terms, what does a reward model learn when we tune an LLM with human feedback?
Junior, Mid
Listening for Learning human preferences from compared answers and using that to guide the model.
Follow-up Ask what can go wrong if the reward model is weak.
-
41
How would you use user feedback, like thumbs up and down, to improve an LLM feature over time?
Mid, Senior
Listening for Collecting and cleaning noisy feedback, preference data, eval before shipping.
Follow-up Ask how they would handle biased or fake feedback.
-
42
When would you use a bandit or reinforcement learning for recommendations instead of a normal ranking model?
Senior
Listening for Exploration, delayed rewards and fast-changing items against complexity and safety.
Follow-up Ask how they would evaluate it offline.
Tensorflow
-
43
You inherit an old TensorFlow model, and the team now uses PyTorch. How would you approach moving it?
Junior, Mid
Listening for Whether to move at all, matching outputs on the same inputs, a careful switch.
Follow-up Ask how they would prove the new model behaves the same.
-
44
Half our production models run on an old TensorFlow serving setup that nobody wants to touch. What would you do about it?
Senior
Listening for Weighing risk and cost, adding tests and monitoring before changing anything, a gradual plan.
Follow-up Ask how they would get time for it on the roadmap.
-
45
We have models in both TensorFlow and PyTorch. How would you decide whether to standardise on one?
Senior
Listening for Cost of migration against the cost of two stacks, decided with data.
Follow-up Ask how they would move the models that stay behind.
Kubernetes
-
46
Our models run as containers on Kubernetes. What do you think about when you package a model that way?
Junior, Mid
Listening for Image size, model loading, CPU or GPU resources, health checks, scaling.
Follow-up Ask how they would roll back a bad version.
-
47
Several teams share a GPU cluster on Kubernetes and keep fighting over it. How would you organise it?
Senior
Listening for Quotas, priorities, queueing, preemption and visible usage per team.
Follow-up Ask how they would handle an urgent big training run.
AWS
-
48
Your training jobs run on cloud GPUs and cost a lot. How would you keep the cost down?
Junior, Mid
Listening for Spot instances with checkpoints, right-sized machines, shutting idle machines, smaller experiments first.
Follow-up Ask how they would track cost per experiment.
SQL
-
49
Walk me through how you usually pull and check training data before you train anything.
Junior
Listening for Querying, checking labels, missing values, duplicates and time leaks.
Follow-up Ask how they handle data too big for one machine.
-
50
How would you build user features, like activity over the last 30 days, without leaking future data into training?
Mid, Senior
Listening for Point-in-time correct joins, time-based splits, matching batch and online features.
Follow-up Ask how they would test for leakage.
Agents
-
51
Walk me through what happens when an LLM calls a tool, like looking up an order, and what your code has to do.
Junior
Listening for The model asks for a tool with arguments, the code checks and runs it, then sends the result back.
Follow-up Ask what the code should do with bad arguments.
-
52
When would you use an LLM agent that calls tools, instead of a single LLM call?
Junior, Mid
Listening for Multi-step tasks needing outside data versus the cost, latency and risk of agents.
Follow-up Ask how they would limit what the agent can do.
-
53
An agent that calls our internal tools sometimes loops or picks the wrong tool. How would you make it more reliable?
Mid
Listening for Clear tool descriptions, step limits, traces, an eval set of tasks, fallbacks.
Follow-up Ask how they would catch these failures in production.
Spark
-
54
Your feature data no longer fits in pandas on one machine. What would you do?
Junior
Listening for Pushing work into SQL, sampling, chunking or moving to Spark, with the trade-offs.
Follow-up Ask what new problems Spark brings.
-
55
The features at serving time don't match what the model saw in training. How would you find and fix that?
Mid
Listening for Logging served features, comparing distributions, sharing feature code or a feature store.
Follow-up Ask how they would stop it coming back.
-
56
How would you make training data reproducible, so anyone can rebuild last month's model exactly?
Senior
Listening for Versioned datasets and snapshots, recorded queries and code, a registry linking them.
Follow-up Ask what it costs to keep all that.