Designing a Research Plan

A research plan is a decision contract

Elena's worst meetings end with "let's get data" and no owner, timeline, or success metric. Priya's best meetings end with a one-page research plan signed by marketing, analytics, and finance: decision question, methods, sample, analysis plan, budget, and what happens if results are inconclusive.

A research plan is not bureaucracy. It is how BrightBrew prevents the onboarding email debate from restarting every week with new vanity metrics. With 142,000 subscribers and continuous A/B tests, undocumented studies also create technical debt: analysts cannot reproduce definitions, and experiment platforms fill with overlapping tests.

BrightBrew is a direct-to-consumer (DTC) specialty coffee subscription company and the anchor company for MKT 202. As of the latest reporting period, BrightBrew serves 142,000 active subscribers with 4.2% monthly churn, average revenue per user (ARPU, average monthly subscription revenue per active subscriber) of $28, and customer acquisition cost (CAC, marketing and sales spend to win one new paying subscriber) near $42. VP Marketing Elena Okonkwo, Head of Growth Sam Rivera, and Director of Customer Insights Priya Nair run active-subscriber and churned-subscriber survey panels refreshed quarterly, A/B tests on onboarding (welcome email sequence, first-shipment timing, grinder add-on offer), and cohort retention dashboards by signup month, acquisition channel, and plan type.

You met BrightBrew in MKT 201 (Marketing Management) positioning and STP work on BrightBrew's value proposition. This course adds the evidence layer: how to translate marketing decisions into research questions, collect valid qualitative and quantitative data, analyze cohorts and funnels, run experiments, and present recommendations leaders can act on.

Core sections of a research plan

Every plan should open with the business decision and decision date. Methods section lists mode (exploratory, descriptive, causal), instruments, and population. Analysis section pre-registers primary metrics and success thresholds. Operations covers timeline, owners, budget, and dependencies (engineering for experiments). Ethics and privacy references consent and storage.

The analysis plan should specify how missing data, outliers, and multiple comparisons will be handled before results arrive. Post-hoc storytelling erodes trust.

Aligning stakeholders before fieldwork

Sam cares about CAC and scale. Priya cares about validity. Finance cares about ROI proof. Legal cares about consent. A 30-minute alignment meeting prevents mid-field changes that invalidate surveys. Pre-register primary metric: for onboarding test, 30-day churn, not open rate.

Document disagreements. If Sam insists on open rate as primary, note it as secondary in the plan so Priya's objection is on record.

Budget and timeline realism

Translate plan into weeks and dollars. Interview recruiting for niche B2B takes longer than consumer coffee subscribers. Survey programming and panel pulls need review cycles. Experiments need engineering slots and minimum runtime for power.

BrightBrew rule of thumb: exploratory sprint 2 weeks, panel survey 4-6 weeks, A/B test 3-6 weeks runtime after 1-2 weeks setup. Parallel work only when samples do not overlap.

Risk register and inconclusive outcomes

List risks: low response rate, seasonality, instrumentation failure, underpowered test. Define inconclusive: "If CI on churn difference spans zero and width < 0.5pp, extend test 2 weeks; else default to status quo."

Kill criteria protect against sunk-cost bias: stop survey if <400 completes after 10 days and fix incentives.

Handoff to dashboard and memo

Plan should name deliverables: executive memo template, dashboard tiles, experiment readout deck. BrightBrew's cohort dashboard updates are not automatic; assign analyst to ship SQL and definitions with the readout.

Decision by Aug 15: double-sided $10 referral vs current single-sided $10.

Part A: Plan excerpt

Primary metric: 90-day referee retention. Guardrail: referrer fraud rate. Population: U.S. new signups. Runtime: 4 weeks or 8,000 assigns. Budget: $35k incentives + $8k analyst.

Part B: Analysis

Compare retention and blended CAC. Pre-register chi-square on retention; report CI. Inconclusive if n < 6,000 → extend 2 weeks.

Part C: Check

Timeline: setup Jul 1, launch Jul 8, readout Aug 12. Decision Aug 15 ✓

Part D: Managerial read

Finance signs off because primary metric is retention-based CAC, not referral count alone.

Without plans, BrightBrew would survey the same panel weekly with overlapping questions, burning response rates from 45% to 19% in one quarter. Coordinated calendar fixes fatigue.

Draft a one-page outline (section headers + 2 bullets each) for BrightBrew's onboarding A/B test research plan.

Solution

Include Decision (rollout by Sep 1), RQs (30-day churn), Design (RCT 50/50 new signups), Metrics (churn primary; first-brew secondary; support tickets guardrail), Analysis (two-proportion test; ITT), Ops ($12k; Priya owner; 5-week runtime), Ethics (support script), Inconclusive (extend if CI width > 0.6pp). Check: all sections present ✓

Key takeaways

• Research plans are decision contracts with dates, metrics, and owners.
• Pre-register primary metrics before fieldwork to prevent metric shopping.
• Budget time for recruiting, engineering, and panel fatigue.
• Define inconclusive outcomes and kill criteria up front.
• Name deliverables: memo, dashboards, and definition docs together.

1. Convert a past ad hoc analysis into a retroactive one-page plan. What was missing?
2. Build a quarter-level research calendar for BrightBrew with non-overlapping panel pulls.
3. Return to the unit page for the knowledge quiz, or continue to Unit 2.

Applying Designing a Research Plan at BrightBrew scale

When BrightBrew evaluates designing a research plan, the team starts from operational facts: 142,000 active subscribers, 4.2% monthly logo churn, $28 ARPU, and $42 blended CAC. VP Marketing Elena Okonkwo, Head of Growth Sam Rivera, and Director of Customer Insights Priya Nair align research strategy and decision translation with Monday dashboard reviews and pre-written research plans. A lesson concept that sounds abstract becomes concrete when tied to signup cohorts, panel waves, and experiment cells logged in the warehouse.

Consider how a 0.5 percentage point change in monthly churn affects BrightBrew. At current scale, that shift moves roughly 710 subscriber logos per month before accounting for mix effects across Classic Bag, Espresso Pod, and Starter Kit promos. Contribution margin near 16.24 dollars per month per subscriber turns small rate changes into five-figure monthly impact. That is why designing a research plan is not an academic exercise for Elena Okonkwo's marketing org; it is how the company avoids scaling a channel that fills the top of the funnel while leaking high-churn promo cohorts at month three.

The research strategy and decision translation workflow at BrightBrew deliberately separates exploratory, descriptive, and causal claims. Priya Nair's analysts label outputs before they reach Sam Rivera's growth standups. Exploratory interview themes become survey items only after codebook review. Descriptive cohort spikes trigger pre-registered experiments rather than same-day pricing changes. Causal A/B wins still require guardrail checks on support tickets, refunds, and grinder attach rates so a churn win does not hide margin erosion. You should copy that labeling habit even if you work outside subscription coffee: name the mode, name the population, name the comparison, and name the decision date before numbers hit a slide.

Document definitions alongside every metric tile. BrightBrew's churn formula specifies grace days after failed payment, pause versus cancel handling, and exclusion of fraud flags. Funnel steps define eligible denominators for visit, signup, shipment, first brew, and month-two active status. Survey estimates document weighting targets by plan mix. Experiment readouts specify intent-to-treat estimands and pre-registered minimum detectable effects on 30-day churn. When definitions live in a shared dictionary, the company builds institutional memory instead of re-debating the same SQL every quarter.

Extended BrightBrew scenario: cross-functional read

Imagine BrightBrew's Q3 review for designing a research plan. Finance asks whether improved onboarding justifies higher podcast CAC. Product asks whether compatibility tooling belongs in mobile web or email only. Operations asks whether shipment forecast accuracy supports green coffee buys. A weak research strategy and decision translation answer addresses only one function. A strong answer shows how evidence flows: qualitative language from churn interviews becomes survey prevalence estimates, descriptive cohort curves localize the leak to Starter Kit promos, and a randomized onboarding test estimates causal churn reduction with confidence intervals translated into saved logos and monthly contribution.

Work the arithmetic on a conservative example. Suppose an onboarding test shows 30-day churn falling from 5.0% to 4.3% intent-to-treat among ten thousand assigns per arm. Absolute reduction 0.7 percentage points yields about seventy fewer churners per ten thousand signups. If BrightBrew adds forty thousand new subscribers in a month after rollout, a sustained effect near the point estimate implies roughly two hundred eighty retained logos in the first month cohort, with compounding value as retained subscribers continue billing. Multiply by monthly contribution near sixteen dollars to communicate magnitude to executives who do not live in p-values. Pair the point estimate with a confidence interval and a pre-written rule: roll out if the interval excludes zero harm on guardrails and includes materially positive churn impact.

Stakeholder conflict is normal. Sam Rivera may push to scale spend while the test is immature. Priya Nair may push to extend runtime for power. Elena Okonkwo must decide under calendar pressure from holiday inventory commits. Designing a Research Plan gives you language to negotiate those tensions with evidence quality standards rather than charisma. If power is insufficient, the decision is extend or accept uncertainty, not pretend a noisy week-one lift is definitive. If qualitative sample is thin, the decision is fund ten more interviews, not quote three vivid anecdotes as national truth.

Translate lessons to your own context by replacing BrightBrew names while keeping structure. Pick one decision you face this quarter. Write the business question, three hypotheses, population rules, comparison group, primary metric, guardrails, and inconclusive outcome before collecting new data. If you cannot write those elements, you are not ready to field a survey or launch an experiment regardless of how easy the vendor dashboard makes clicks look.

Technical mechanics and checks (worked patterns)

For designing a research plan, BrightBrew analysts show work the way finance shows reconciliations. A cohort retention table prints signup month, eligible n, month-zero through month-three retention, and a check that weighted plan mix matches the dashboard within one percentage point. A funnel table multiplies step conversions and compares the product to observed month-two actives within rounding tolerance. A survey proportion reports weighted point estimate, ninety-five percent confidence interval, effective n, and exclusion counts from speeders or straight-liners. An experiment appendix lists assignment counts per arm, sample ratio mismatch p-value, intent-to-treat churn with interval, and support ticket guardrail delta.

Use plain-language hypothesis statements before formulas. Example for onboarding: null hypothesis states sequence B does not change 30-day churn versus sequence A; alternative states churn differs. Randomization creates comparable arms so differences after large n are plausibly treatment-related rather than channel mix artifacts. Still verify seasonality with year-over-year cohort comparisons and document concurrent campaigns that could violate independence assumptions.

For spreadsheet or SQL replication, write the grain first. Customer-month tables suit retention. Customer-level tables suit funnel conversion if timestamps exist for each stage. Survey tables suit respondent weights. Experiment tables suit assignment at signup with outcome flags thirty days later. BrightBrew forbids ambiguous one-word metrics like engagement without operational definition. Engagement might mean logged brew events, email opens, or app sessions; each definition implies different SQL joins and different managerial meaning.

Common executive questions (and disciplined answers)

Executives ask short questions that require long disciplined answers. "How sure are we?" maps to confidence intervals, power, and replication plans, not bravado. "What is the dollar impact?" maps to logos saved times contribution margin with explicit stationarity assumptions. "Can we ship faster?" maps to risk of rolling out biased assignment or underpowered tests that will reverse after holiday spend commits. "Why trust panel data?" maps to sampling frame, weighting, response quality rules, and consent governance. "Why not just ask sales?" maps to selection bias and absent counterfactuals in anecdote.

BrightBrew's credible answer format for designing a research plan is three bullets: decision recommendation, evidence strength label exploratory descriptive or causal, and next study if limitations matter. A fourth bullet lists what would falsify the recommendation within sixty days. That discipline prevents the analytics team from becoming either a bottleneck or a rubber stamp.

Practice the translation loop until it is habit. Business question to research questions to design to analysis plan to dashboard tile to memo ask. When the loop is complete, BrightBrew scales what survives skepticism. When the loop is broken, the company buys false confidence cheaply and pays for it in subscriber logos later.

Practice extension: self-check without peeking

Before reading any solution in this lesson again, open a blank document and complete four rows. Row one: write BrightBrew's business question that designing a research plan helps answer. Row two: list population inclusion and exclusion rules for that question. Row three: name primary metric, one secondary metric, and one guardrail metric. Row four: state the decision you would make if the metric moves favorably versus unfavorably. Compare your rows to the worked example and practice problem. Gaps indicate what to re-read.

If you are studying outside coffee subscriptions, substitute your company but keep numeric discipline. A B2B SaaS team might replace churn with logo retention and ARPU with average recurring revenue per account. A marketplace might replace funnel steps with search, booking, and repeat purchase. The structural habits from MKT 202 remain: define terms, show checks, label evidence mode, and tie results to decisions with explicit limitations.

Connection to MKT 201 and OMBA 102

MKT 201 positioned BrightBrew's value proposition, segments, and channel strategy. MKT 202 adds evidence standards for those strategic choices. OMBA 102 deepens inference, confidence intervals, and decision analysis that underpin experiment readouts and survey precision. Treat the three courses as a stack: strategy names where to play, analytics names how to validate, statistics names how much certainty the data earns.

When you present to executives, integrate the stack in one narrative arc rather than three jargon layers. Example: MKT 201 chose referral emphasis for Ritualist personas; MKT 202 shows referral cohort three-month retention ninety-one percent versus Starter promo seventy-two percent; OMBA 102 quantifies uncertainty on the difference with an interval and sample size plan for continued monitoring. That integrated story is what capstone memos in Unit six require.