Title 

Why Innovation Teams have to be Measured Differently and How to do it - with Tristan Kromer

About the Episode

Tristan Kromer is talking to us about why innovation teams need to be measured differently from other product teams and how to do it. In other words, we are talking about how you can take practical steps to implement innovation accounting on a team level. 

Please let us know what measures you would add and why in the comments!

Innovation Team Metrics Covered in this Episode

• Experimentation Velocity / % of teams that run experiments 
• Insight Velocity / % of experiments that generate insights
• % of falsified hypothesis (validated/invalidated experiments ratio) 
• Retrospectives / % of weeks teams have run retrospectives in the last month 
• Qualitative data - team sprint demo (weekly, biweekly) 

Topics and Insights

• (01:40) The problem - why do we need to measure innovation teams differently
• (05:00) All teams should be measured against something they can control 
• (06:00) For innovation teams, even product related lead indicators often can not be applied as the lead indicator might not yet be known
• (10:00) If Zombies were chasing Tristan, he might be able to run 5 blocks
• (11:00) Nobody knows what product teams will win, but we do know that teams which experiment and learn faster, have a much better chance to achieve Product Market Fit. Tom Eisenmann from Harvard University just released data to back up that claim
• (14:30) The solution - how to fairly and effectively measure innovation team 
• (15:00) Separating the measurement of the product from the measurement of the team
• (15:40) Experimentation Velocity and Insight Velocity 
• (19:00) Accounting for experiment and research cost 
• (22:20) Elijah is having problems formulating a proper sentence and Yoga with Adriene comes up 
• (23:30) The difference between an evaluative experiment and generative research
• (29:00) Should the evaluative experiment and generative research ratio be measured?
• (31:00) Product teams should never stop running interviews
• (31:30) Teams are likely to be more successful when they use a greater variety of experiments and call for more academic research into this topic!
• (33:15) We need better tools to record experiments and insights from innovation teams. 
• (36:50) Tools for recording experiments 
• (41:30) Going deeper into measuring Experimentation and Insight Velocity. 
• (46:20) The amount of falsified hypotheses are another team performance indicator
• (52:00) Measure but don’t count
• (52:50) Teams also need to be measured against lead product/project indicators but each team needs to find out their own indicators or levers for success 
• (55:00) Retrospectives, another important team measure
• (56:00) The importance of qualitative data like sprint demos 

Questions for our Listeners

• What team metrics do you recommend?
• How do you record experiments in your organisation? What can you recommend? 

Please comment below

About the Cohost 

Tristan Kromer helps product teams go fast.

As a Silicon Valley-based lean startup coach and founder of Kromatic, Tristan works with innovation teams to run at least one experiment/research per week to improve their product and business model.

For larger companies and governments, Tristan and his team work with corporate teams and leaders to build innovation ecosystems.

Tristan designed lean startup programs such as the Build or Die Bootcamp for TechBA (Mexico) and the Innovation Partnership Program (Vietnam-Finland) in addition to being part of Luxr, whose Core Curriculum has been used by 13 accelerators internationally, including Singularity University, 500 Startups, & The United States Innovation Fellows.

He has worked with companies ranging from early stage startups with zero revenue to established businesses with >$10M USD revenue (Kiva, Cancer Research U.K., TES) to enterprise companies with >$1B USD revenue (Unilever, Swisscom, Salesforce, Fujitsu, LinkedIn).

Tristan regularly speaks, appears on panels, and gives workshops internationally with organizations such as the Stanford Center for Entrepreneurial Studies & D-school, Dubai Chamber of Commerce, General Electric (GE), and more.

Originally from New York City, he has lived in Germany, Switzerland, Taiwan, and Vietnam, and currently resides in San Francisco, USA. With his remaining hours, Tristan volunteers his time with early stage startups.

Connect with Tristan 

Website / LinkedIn / Twitter