Template: Reporting experiment or project results

Keeping track of the performance of an experiment or feature that you shipped, either in emails or an evolving document, is essential for maintaining stakeholder alignment and boosting your visibility as a product manager. This template is a guide for reporting on a completed project to colleagues that have varying degrees of familiarity with your work.

Subject / Title

[Project Name – Date]

A descriptive email subject or document title will help your teammates quickly find your documentation. Describe your project and include relevant dates.

Examples:

• Home Page Copy Experiment – 8/21/2020 update
• [Android] Onboarding Funnel Optimizations (8/1 – 9/7)

TL;DR

The goal of this [project/experiment] was to test [X]. We learned [Y]. Next, we plan to do [Z].

Include a 2 – 3 sentence summary of your goal (project objective or hypotheses tested in an experiment) along with the high-level results. This is the first thing a reader will see. It should give readers a quick insight into your project, and those who are interesting in learning more can read further. This section is essentially an executive summary.

Context

• Background info 1
• Background info 2
• etc.

Answer the following questions in a short paragraph or a few bullet points:

• What user or business problem were you trying to solve?
• Why were you trying to solve this problem? i.e. how do you know this problem is real?
• What was your solution? Describe the feature, project, or experiment as you would to a friend who has never heard about it before.

Include screenshots where helpful.

Experiment Setup

Experiment name

Start Date – End Date

• Variant 1 (allocation %): Description, hypothesis
• Variant 2 (allocation %): Description, hypothesis
• etc.

If you’re reporting on an A/B test, this section is useful for detailing the logistics of your experiment. Think about what your audience would need to know to understand the results you are reporting.

For example:

• Experiment duration, start and end dates
• Number of variations you tested vs the control group
• Allocation in each variant
• Differences in the treatments of each variant
• Hypotheses being tested

Results

Metric 1 moved because of X. Metric 2 moved because of Y.

	Variant 1	Variant 2
Metric 1	x% (stat sig status)	x% (stat sig status)
Metric 2	x% (stat sig status)	x% (stat sig status)

Report the results of important metrics, like your core product KPIs or predefined success indicators. With the data provided in this section, readers should understand the reasoning behind your decisions in the Next Steps section. Save any deeper analysis for the end of the email.

For experiments, it is important to indicate the percentage lift vs your control group and the statistical significance of each metric.

Example:

	Variant 1	Variant 2
ARPDAU	-1.2% (stat sig)	+0.7% (not stat sig)
Conversion	– 3.0% (stat sig)	-0.1% (not stat sig)

Next Steps

• Plan for this project going forward – roll out to more users, end, iterate, etc.
• Ideas for using the insights in other projects

What are you planning to do next and why? Use bullet points to talk about how you will use what you learned from this project to improve your product going forward.

Further Analysis

This optional section is where you can include interesting analysis or insights for readers that are looking for more details on your project, such as:

• Funnel analyses
• User interaction patterns – how are people using what you shipped?
• Trends over time
• Trends by different cohorts

Shoutouts

Huge thank you to X and Y for their great work on making this feature possible!

It’s always nice to recognize your colleagues for their contributions! If applicable, this is a great opportunity to recognize your awesome teammates for their hard work on the project you talked about in this email or document.