Senior Product Manager - Tech, Experimentation & Failure
Description
SENIOR PRODUCT MANAGER - TECH, EXPERIMENTATION & FAILURE
COMPANY: STEVEN.COM
REPORTING TO: CITO
LOCATION: LONDON
ABOUT STEVEN.COM
Steven.com is building the operating system for the creator economy, forecast to pass a trillion dollars by the early 2030's. This industry is currently held back by fragmented tools and a lack of professional infrastructure. Steven.com is the unlock. We are the end-to-end Operating System designed to scale what is irreplaceably human.
We have built proprietary technology and obsessed teams to identify and scale the highest-potential creators across our core pillars:
-
Creator Media: Amplifying reach, influence, and trust
-
Creator Community: Transforming audiences into connected tribes
-
Creator Products: Providing creators with the infrastructure to build and back aligned products and ventures
-
Powered by Creator Tech & Intelligence: A proprietary data and technology suite that fuels smarter decisions and drives innovation across the entire flywheel.
Our Experimentation & Failure team has one of the most strategically important and unusual mandates at Steven.com : increase the rate of failure. As Steven Bartlett puts it: "the path to the correct answer is out-failing your competition." This isn't a growth team or an optimisation function. It's the team that exists to make sure we learn faster than anyone else.
ROLE MISSION
Lead the Experimentation & Failure team, reporting to the CITO. You'll out-experiment and out-fail the competition - running high-velocity, rigorous experiments across every show, creator, piece of content, and commercial bet at Steven.com , while building a team and a culture that treats deliberate failure as the primary learning mechanism. This is deeply hands-on: you'll be setting hypotheses, isolating variables, checking statistical power, reading results, and moving to the next test - not directing from a distance.
KEY OUTCOMES
-
Own and drive the experimentation agenda across every show, creator, and IP property in the FlightStory portfolio - from podcast topic selection and episode structure through to thumbnail design and social tile copy. No detail too small to test.
-
Lead hypothesis formation for every experiment, with clear success, failure, and inconclusive criteria defined in advance - and enforce single-variable discipline across the board.
-
Ensure every experiment is adequately powered before launch: sample sizes calculated, measurement windows defined, results interpretable by design.
-
Systematically increase experimentation velocity and build the intake process that makes high-volume testing the default across FlightStory and Steven.com .
-
Build institutional memory - a searchable, structured record of every experiment run, what was learned, and what was decided - as a compounding organisational asset.
-
Partner with the VP of Engineering & Applied AI to apply AI tooling to experiment design, analysis, and reporting, and to ensure infrastructure supports testing at this velocity.
CORE COMPETENCIES
-
Deep, first-principles command of experimentation mathematics: statistical significance, power, sample size calculation, p-values, confidence intervals, Type I/II errors, and the difference between statistical and practical significance.
-
Genuine mastery of the scientific method applied to product and content - hypothesis formation, single-variable isolation, measurement design, and result interpretation.
-
A track record of building experimentation culture, not just running tests - creating an environment where the whole team experiments and failure is rewarded.
-
Comfortable querying data and working shoulder-to-shoulder with engineers and data scientists at implementation depth.
-
Strong written and verbal communication - able to write a hypothesis an engineer respects and explain a result a producer will act on.
-
Experience operating at pace, in high-volume testing environments where speed of learning is the competitive edge.
YOU'LL THRIVE HERE IF
-
You think in hypotheses, not features.
-
You isolate one variable at a time, and understand viscerally why changing five things at once makes a result meaningless.
-
You give a null or inconclusive result the same intellectual respect as a win - you know how to extract the signal either way.
-
You're obsessive about measurement: an experiment that can't be measured is just a change, not an experiment.
-
You're deeply sceptical of your own results, and design experiments to prove yourself wrong rather than confirm what you already believe.
-
You move fast, expect others to move fast, and don't wait for perfect conditions to run a test.
-
You believe failure is feedback, feedback is knowledge, and knowledge is power - and you build systems to generate that knowledge at the highest possible rate.
IDEAL BACKGROUND
-
Demonstrable experience leading (not just participating in) product or content experimentation programs at a technology, media, or creator-economy company - owning the methodology, volume, and culture.
-
Strong advantage: experience with algorithmic platforms and how to design experiments against platform-specific metrics; podcasting, video, social, or creator-economy background; familiarity with YouTube/Spotify/social analytics (CTR, retention, watch time, audience behaviour).
-
Nice to have: experience building an experimentation platform from scratch; familiarity with causal inference beyond standard A/B testing (holdouts, quasi-experiments, diff-in-diff); experience experimenting on AI/ML systems or prompt variations in production; a background in statistics, maths, CS, economics, or a natural science; exposure to early-stage environments where you had to build the experimentation infrastructure yourself.
##