ECBS5256 – Managing Data Science Teams

Growth & Performance

Day 2 — Block D | 11:00–12:40

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

Learning Outcomes

By the end of this block you will be able to:

  • Deliver structured feedback using the SBI model
  • Conduct evidence-based performance calibration
  • Distinguish growth planning (PGP) from performance improvement (PIP)
  • Apply the PGP framework to articulate specific, actionable growth areas
CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

Welcome Back — Where We Are

Day 1 recap — you built:

  • A Manager OS with cadences, rituals, artifacts, and decision hygiene
  • A hiring plan with structured interviews and work samples
  • A roadmap tied to business outcomes
CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

Welcome Back — Where We Are

Today — Day 2:

  • Block D (now): Career development, feedback, performance — with practice
  • Block E (13:30): Infrastructure & Cross-Functional Interfaces
  • Block F (15:30): Leading Up & Executive Communication

The arc: Day 1 was about building the system. Day 2 is about running the system — people, feedback, and communication.

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

Why Growth & Performance Matter

The #1 reason data scientists leave: They don't feel like they're growing.

  • Bain/LinkedIn data: "Lack of career growth" is the top reason analytics professionals leave — ahead of compensation, work-life balance, and management quality
  • Turnover costs 1.5-2x salary — for a senior DS making $180K, that's $270K-$360K
  • The median tenure for a data scientist is 2.3 years (LinkedIn Workforce Report). Every departure resets your team's institutional knowledge
  • The best analytics people have options. Your job is to make staying the obvious choice

Growth isn't a perk — it's retention strategy.

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

Career Ladders for Analytics

Two tracks — different jobs, not a promotion hierarchy:

Level IC Track Management Track
Entry Analyst
Mid Senior Analyst Team Lead
Senior Staff Analyst Manager
Principal Principal Analyst Director

Key insight: Moving from IC to management is not a promotion. It is a career change.

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

What "Senior" Actually Means

It is not years of experience. Someone can be "senior" at year 3 or still not senior at year 10.

Senior = three dimensions:

Dimension Junior Senior
Scope Completes assigned tasks Identifies the right problems
Autonomy Needs direction Operates independently
Ambiguity Struggles without clear requirements Thrives in undefined spaces
CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

What "Senior" Actually Means

The litmus test: Can you give them an ambiguous problem and trust them to come back with a well-scoped analysis, clear findings, and actionable recommendations?

Great Mid (L3) Senior (L4+)
Given "why is churn increasing?" Thorough analysis — but needs you to scope it first Defines churn, selects cohort, identifies drivers, drafts recommendations — without being told how
When blocked Escalates and waits Finds an alternative path, tells you after
Impact radius Their own work is excellent The team is better because they're on it

If you promote people to senior just because they've been around for four years, you devalue the title for everyone.

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

The SBI Model — Feedback That Lands

Component What it means Example
S — Situation When and where "In yesterday's stakeholder meeting..."
B — Behavior What you observed "...you presented without confidence intervals..."
I — Impact The effect it had "...the VP questioned our methodology."

Why SBI works: It separates observation from judgment. You're not saying "you're sloppy." You're describing a specific behavior and its consequence.

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

Two Tools, Opposite Purposes

As a manager, you have two formal tools for developing people. They sound similar but serve opposite purposes:

PGP — Personal Growth Plan: A proactive, aspirational document you co-create with someone who is performing well. "You're doing great — here's how we accelerate your career." The individual owns it. You do this for everyone, ideally in the first month.

PIP — Performance Improvement Plan: A corrective, HR-involved process for someone who is genuinely underperforming after you've already given clear feedback multiple times. "This isn't working — here's what needs to change in 30-90 days." PIPs should be rare. If you're putting more than one person on a PIP per year, the problem might be your management, not their performance.

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

PGP vs. PIP — Know the Difference

PGP (Growth Plan) PIP (Improvement Plan)
Tone Aspirational Corrective
Owner The individual The manager / HR
Trigger Proactive — for everyone Reactive — after repeated feedback fails
Timeline 6-12 months 30-90 days
Outcome Promotion, new skills, career growth Improvement or exit
Frequency Every direct report, every cycle Rare — 1-2 per year at most

When does a DS get PIPed? Repeated errors reaching stakeholders, refusal to incorporate feedback, or missed commitments after clear expectations — and you've already given SBI feedback multiple times.

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

PGP Structure

PGP core sections:

  1. Context — current role and what you're working with
  2. Strengths — what you're known for (with evidence)
  3. Growth areas — what's holding you back (specific, not vague)
  4. Actions — concrete, time-bound development steps
  5. Vision — where you want to be in 6 months
  6. Check-in cadence — how and when you revisit this

The individual owns this document. You co-create it.

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

PGP in Practice — Growth Areas by Level

Level Example Growth Area Example Action
L2 (Junior) "Move from guided to independent analysis scoping" "Scope next 3 analyses independently; debrief with manager after each"
L3 (Mid) "Present technical findings to non-technical stakeholders in under 5 minutes" "Present at next 2 product reviews; manager coaches on deck structure beforehand"
L4 (Senior) "Develop the team's experimentation capability, not just your own" "Mentor one L2 through their first A/B test by Q3; create a reusable runbook"

The specificity test: If your growth area could apply to any analyst anywhere (e.g., "improve communication"), it is too vague.

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

Gold Standard — One Complete PGP Entry

This is what 15% of your grade looks like:

Growth area: Present technical findings to non-technical stakeholders in under 5 minutes
Action: Present at next 2 product reviews; manager coaches on deck structure beforehand
Measure: VP Product can summarize my key finding after the presentation without asking clarifying questions
Support needed: Manager pre-reviews deck; team lead shadows first presentation for feedback
Timeline: Demonstrate by end of Q2

If your PGP entries look like this, you're in excellent shape for the portfolio.

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

Activity: SBI Feedback Role-Play

25 Minutes | materials/sbi-scenario-cards.md

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

Activity: SBI Feedback Role-Play

Time: 25 minutes | Open materials/sbi-scenario-cards.md

Format:

  • Pair up. Each pair gets a scenario card.
  • Round 1 (10 min): Partner A gives SBI feedback. Partner B receives in character. Then 2 min pair debrief.
  • Round 2 (10 min): Swap roles. New scenario or continue.
CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

Activity: SBI Feedback Role-Play

Rules:

  • Stay in character during the practice
  • Giver: Use SBI structure. Be specific.
  • Receiver: React naturally — defensive, confused, emotional
  • Debrief honestly: "What worked? What felt off?"

Pick up your scenario cards. You have 25 minutes. Go.

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

Radical Candor + Anti-Patterns

Radical Candor (Kim Scott): The best feedback = caring personally + challenging directly.

Challenge Directly Don't Challenge
Care Radical Candor Ruinous Empathy
Don't Care Obnoxious Aggression Manipulative Insincerity

Anti-patterns: The Feedback Sandwich (people ignore the bread). Delayed feedback (half-life). "Be more strategic" (fortune cookie, not feedback).

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

The Manager Log

Keep a running document per direct report. Update it weekly. 3 bullets per person, 5 minutes.

Log this Don't log this
Key accomplishments with business impact Opinions or feelings about the person
Feedback you gave (with date) Hearsay from others without verification
Growth moments and stretch assignments Attendance or hours worked
Specific examples of collaboration or conflict Anything you wouldn't share with HR if asked
Risks you're tracking (skills gaps, engagement) Protected characteristics or personal details

The goal: no surprises at review time. When review season comes, you have 50 weeks of evidence instead of vague impressions.

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

Calibration Sessions

What: Managers present their direct reports to peer managers. The group norms on performance ratings together.

Why calibration matters:

  • Prevents grade inflation
  • Ensures fairness across teams
  • Forces managers to articulate impact, not effort

Your job in calibration: Advocate with evidence, not emotion.

  • Bad: "She worked really hard"
  • Good: "She shipped the churn model that saved $2.4M in annual revenue"

The manager log is what makes this possible.

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

The Ethical PIP

Before you ever start a PIP, check yourself:

  1. Clarity — Do they know the expectations?
  2. Support — Have you provided what they need?
  3. Feedback — Specific, timely, actionable? Multiple times?
  4. Documentation — Is it fair and specific?
  5. Timeline — Is 30-90 days reasonable?
  6. Dignity — Always. A PIP is not punishment.

Before a PIP, ask: "If this person's best friend were watching, would they say I've been fair?"

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

Activity: Mock Calibration

20 Minutes | materials/calibration-profiles.md

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

Activity: Mock Calibration

Time: 20 minutes | Open materials/calibration-profiles.md

Format:

  • Form groups of 4.
  • 5 min: Read all three profiles silently. Write your independent ratings on a card (Exceeds / Meets / Below).
  • 10 min: Reveal ratings simultaneously. Discuss disagreements. Reach consensus.
  • 2 min: Group reflection — Which profile was most contested? Why?

Rating Scale: Exceeds / Meets / Below — relative to the person's current level.

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

Debrief

From the SBI role-play:

  • What was harder — identifying the behavior, or articulating the impact?
  • Did anyone find positive feedback harder than constructive?

From mock calibration:

  • Which profile generated the most disagreement? Why?
  • Did anyone change their mind? What convinced you?
CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

Quick Draft: One SBI Development Area

5 minutes | Open templates/performance-summary.md

Pick one person you just calibrated. Write one development area using SBI:

Situation: [When/where did you observe this?]
Behavior: [What specifically did they do?]
Impact: [What was the consequence?]
Development recommendation: [What should they do differently?]

This is the exact format your Performance Summary requires. You're practicing it now so it's not blank when you sit down to write.

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

Key Takeaways

  • SBI separates observation from judgment. That's what makes feedback hearable.
  • Calibration is uncomfortable by design. The discomfort surfaces hidden assumptions.
  • PGP is not a PIP. Growth plans are proactive and aspirational.
  • Senior = scope + autonomy + ambiguity tolerance. Not years.
  • Document continuously. The manager log defeats recency bias.
  • Dignity is non-negotiable. In PIPs, feedback, calibration — always.

These aren't just frameworks. They're commitments you make to the people who report to you.

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

Up Next: Block E

Infrastructure & Cross-Functional Interfaces

13:30–15:10

CEU Vienna | Day 2 – Block D
ECBS5256 – Managing Data Science Teams

Transition to Block E

After lunch: Block E — Infrastructure & Cross-Functional Interfaces (13:30–15:10)

We'll cover the data stack, build vs. buy decisions, working with engineering and IT, and you'll sketch your case context's infrastructure.

Over lunch, think about: What does the data infrastructure look like in your case context?

See you at 13:30.

CEU Vienna | Day 2 – Block D

⏱ Expected: 11:00 (min 0/100)

Talk track: Welcome back. Today we shift from building the machine to growing the people inside it. This block is about the hardest part of management: feedback, performance, and growth. Some of it will be uncomfortable — that's by design. Let's get started.

⏱ Expected: 11:00 (min 0/100)

Talk track: Three outcomes for today's first block. Feedback, calibration, and growth planning — the people side of management.

⏱ Expected: 11:03 (min 3/100)

Talk track: Let me orient you. A week ago, you left here with a Team Charter, a Stakeholder Map, a hiring packet in progress, and a roadmap. Today we complete the arc.

⏱ Expected: 11:03 (min 3/100)

Talk track: This block is about growing the people you hired. Block E covers data infrastructure and cross-functional relationships. Block F is about executive communication — how to lead up, handle failure, and present to leadership.

⏱ Expected: 11:06 (min 6/100)

Talk track: Before we dive into frameworks, the business case. The number one reason data scientists leave their jobs is not compensation — it's that they don't feel like they're growing. LinkedIn's workforce data shows median data scientist tenure at about two and a half years. Every time someone leaves, you lose six months of institutional knowledge and spend three to six months ramping their replacement. That's nearly a year of reduced output. The best people have options. Your job is not to lock them in — it's to make staying the obvious choice by investing in their growth, giving honest feedback, and showing them a path forward.

⏱ Expected: 11:09 (min 9/100)

Talk track: Most analytics orgs have two tracks — IC and management. Moving from IC to management is NOT a promotion. It is a lateral career change into a fundamentally different job. The best analyst on your team might be a terrible manager. If your org doesn't have a Staff or Principal IC level, your best ICs will leave — or worse, become reluctant managers who make everyone miserable.

⏱ Expected: 11:13 (min 13/100)

Talk track: This slide is one of the most important in the entire course. Senior is about three things: scope, autonomy, and ambiguity tolerance. A senior person doesn't just do the work you assign them — they identify what work SHOULD be done, execute it independently, and thrive in undefined problem spaces.

⏱ Expected: 11:13 (min 13/100)

Talk track: Here's the trap: a great mid-level person can be incredibly productive. They deliver thorough, high-quality work — but they need you to scope it first. Tell them "analyze churn," they'll ask: how are we defining churn? Which cohort? What time window? Those are good questions — that's a strong mid. A senior person, given the same ambiguous question, comes back having made all those decisions themselves, identified three actionable drivers, and drafted recommendations. Both are valuable. But they're different levels. The difference isn't quality of output — it's who does the scoping.

⏱ Expected: 11:18 (min 18/100)

Talk track: Before we get into growth plans, let me give you the tool you'll use to deliver feedback on growth areas — and everything else. The SBI model. Situation anchors it in time. Behavior describes what you observed, not your interpretation. Impact explains the consequence. This separates observation from judgment. You're not saying "you're sloppy." You're describing a specific behavior and its specific consequence. SBI works even when you manage someone more technically skilled than you — you're observing behaviors and their impact, not grading their methodology. We'll practice this in a few minutes.

⏱ Expected: 11:21 (min 21/100)

Talk track: Let me introduce two tools you'll use as a manager. They have similar-sounding acronyms but they could not be more different in purpose. A PGP — Personal Growth Plan — is something you do proactively with people who are doing well. It's aspirational. "You're a strong L3, here's how we get you to L4." A PIP — Performance Improvement Plan — is corrective. It's what happens when someone is genuinely underperforming and you've already tried feedback, support, and clear expectations. PIPs involve HR. They have hard deadlines. They sometimes end in termination. These are fundamentally different tools, and confusing them is one of the most common mistakes new managers make.

⏱ Expected: 11:18 (min 18/100)

Talk track: If your team conflates PGPs and PIPs, nobody will engage honestly with a growth plan because they'll think they're in trouble. A PGP is a gift — it means you're investing in someone. A PIP is a last resort — it means the investment didn't work. When would you PIP a data scientist? Not for a single mistake. Not for being slow to ramp. You PIP when there's a sustained pattern — analyses with errors reaching stakeholders, refusal to incorporate feedback, commitments missed after expectations were explicitly set — and you've already given SBI feedback multiple times. If you haven't done those things first, you're not ready for a PIP. You're ready for better management.

⏱ Expected: 11:21 (min 21/100)

Talk track: The PGP template has six sections. Context grounds the plan in your current reality. Strengths and growth areas are the diagnosis. Actions are the treatment plan — concrete, time-bound steps. Vision gives a direction. And the check-in cadence makes sure this isn't a document you write once and forget. The individual owns this document — you co-create it. Push for specificity: "Improve communication" is not a growth area. "Learn to present technical findings to non-technical stakeholders in under 5 minutes" is. You'll draft your own PGP at home — we're not doing it in class today, but you have the template and a starter template with example growth areas to help you get going. It's a portfolio deliverable, so take it seriously.

⏱ Expected: 11:21 (min 21/100)

Talk track: What does a good PGP actually look like? It depends on level. For a junior, growth is about moving from guided to independent work. For a mid-level person, it's about expanding their impact beyond their own analyses. For a senior, it's about elevating the team, not just their own output.

⏱ Expected: 11:21 (min 21/100)

Talk track: Let me show you a gold standard entry — this is what the rubric is looking for when it says "actionability" and "specificity." Growth area: specific. Action: concrete. Measure: observable — you'd know if it happened. Support needed: what the manager commits to, not just what the person does. Timeline: a real date. This is a complete entry. If yours look like this, you're in excellent shape.

⏱ Expected: 11:27 (min 27/100) | Activity: SBI Feedback Role-Play (25 min)

Talk track: Time to practice. This is where the learning happens — not in hearing about SBI, but in doing it.

⏱ Expected: 11:27 (min 27/100) | Activity: SBI Feedback Role-Play (25 min)

Talk track: Time to practice. This is where the learning happens — not in hearing about SBI, but in doing it. Pair up. Each pair gets a scenario card. You'll do two rounds: first one partner gives feedback and the other receives, then you swap.

⏱ Expected: 11:27 (min 27/100) | Activity: SBI Feedback Role-Play (25 min)

Talk track: Stay in character. If you're the receiver and the feedback makes you defensive, be defensive. That's real. The goal is to feel what it's like to give structured feedback under pressure. Pick up your scenario cards. Go.

⏱ Expected: 11:52 (min 52/100)

Talk track: You just felt what it's like to give direct feedback. That discomfort has a name — you were choosing Radical Candor over Ruinous Empathy. Most new managers default to Ruinous Empathy — they care about the person but avoid the hard conversation. The anti-patterns: the feedback sandwich doesn't work because people pattern-match and discount the positive — they hear "nice thing, BUT..." and only remember the but. Delayed feedback loses its power because neither of you can reconstruct the moment. And "be more strategic" tells the person nothing about what to actually do differently. Always ask: is this specific enough that the person knows exactly what to change? One caveat — Radical Candor is a US-centric framework. In cross-cultural teams, and in this room, "challenging directly" may need to be calibrated to your team's norms. The principle holds — don't avoid hard conversations — but the style of directness varies.

⏱ Expected: 11:55 (min 55/100)

Talk track: The secret weapon is the manager log. Every week, three bullets per person, five minutes. What did they accomplish? What feedback did you give? What should you track for their development? Without documentation, you default to recency bias, availability bias, and often unconscious demographic bias. A quick note on what NOT to log: opinions, feelings, hearsay, hours worked. Log observable behaviors and their impact — this is SBI applied to documentation. One legal note: your manager log may be discoverable in an employment dispute. Write every entry as if HR and the employee will both read it. That's not paranoia — it's professionalism.

⏱ Expected: 11:59 (min 59/100)

Talk track: Calibration is where performance management gets real. A group of managers sit in a room and each presents their people. Without calibration, every manager becomes Lake Wobegon — everyone is above average. Your job is to advocate with evidence. "She worked really hard" is not a calibration argument. "She shipped the churn model that saved $2.4M in annual revenue" is. The manager log is what makes this possible — without it, you're going in with vague impressions. With it, you're going in with receipts.

⏱ Expected: 12:02 (min 62/100)

Talk track: PIPs are the hardest part of management. Before you ever get to one, check yourself. Did you set clear expectations? Did you provide support? Did you give timely feedback? If the answer to any of those is no, the first fix is on you. And sometimes what looks like underperformance is actually wrong role fit, unclear expectations, personal crisis, burnout, or your own failure as a manager. Diagnose before you prescribe. The best-friend test: if this person's best friend were watching everything you've done as their manager, would they say you've been fair? If yes, proceed. If no, fix your management first.

⏱ Expected: 12:02 (min 62/100) | Activity: Mock Calibration (20 min)

Talk track: Now we're going to do something uncomfortable — and that's exactly the point.

⏱ Expected: 12:02 (min 62/100) | Activity: Mock Calibration (20 min)

Talk track: Form groups of four. You'll get three fictional performance profiles. Read them all silently, then independently rate each person: Exceeds, Meets, or Below expectations — for their current level. Do NOT discuss yet. Once everyone has their ratings, reveal simultaneously. Where you disagree — that's where the real learning happens. You have 20 minutes. Go.

⏱ Expected: 12:22 (min 82/100) | Debrief

Talk track: Let's come back together. SBI first — what was the hardest part? Usually it's the impact statement. People can describe what happened but struggle to articulate why it mattered. That's the coaching muscle you're building. Calibration — which profile generated the most disagreement in your group? What made it hard? Notice how "what's expected at this level" changes the entire conversation.

⏱ Expected: 12:24 (min 84/100) | Micro-draft: bridge to portfolio (5 min)

Talk track: Before we wrap up, I want to bridge what you just did to what you'll submit. Open the performance summary template. Pick one of the three people you just calibrated — Dana, Kevin, or Mika. Write one development area using SBI format. Situation, behavior, impact, then your recommendation. Five minutes. This is what the rubric means by "evidence of SBI application." Go.

⏱ Expected: 12:29 (min 89/100) | Key Takeaways + Transition

Talk track: Key takeaways. SBI separates what you saw from what you think. Calibration is uncomfortable by design — if it feels easy, you're rubber-stamping. PGP is not a PIP. Senior means scope, autonomy, ambiguity tolerance — not tenure. Document continuously. And dignity is non-negotiable in every interaction. These aren't just frameworks. They're commitments you make to the people who report to you.

⏱ Expected: 12:29 (min 89/100) | Key Takeaways + Transition

Talk track: That wraps Block D. After lunch, Block E shifts gears — we'll talk about data infrastructure, working with engineering and IT, and the build versus buy decisions you'll face as a manager.

⏱ Expected: 12:29 (min 89/100) | Key Takeaways + Transition

Talk track: We're breaking for lunch. When you come back, Block E shifts to infrastructure — the data stack, build versus buy decisions, working with engineering and IT. You'll also do a hands-on activity sketching the infrastructure for your case context. Over lunch, start thinking about what the data stack looks like in your case. Enjoy lunch, and I'll see you at 1:30.