Fix Skewed Viewing Events Pipeline
A hard Spark mock interview question on DataDriven, with an AI interviewer who asks follow-up questions and a hire or no-hire decision at the end.
- Domain
- Spark
- Difficulty
- hard
- Seniority
- L5
Interview Prompt
A streaming company's nightly `viewing_engagement` Spark job, which combines a large `event_data` table with a small `users` dimension on `user_id` to produce daily engagement counts by event type and account status, has blown its 60-minute SLA and is still running after two hours. Read the Spark UI evidence to find the root cause and fix the job so it meets SLA.
Summary
Your nightly Spark job just paged you. One task has 40% of the data.
How This Interview Works
- Read the vague prompt (just like a real interview)
- Ask clarifying questions to the AI interviewer
- Write and run your Spark code against the tables
- Answer the interviewer's follow-up questions, then get a hire or no-hire decision