About

Shop talk on "intricately managed miracles" and early-stage subculture edited by four professionals in the throes of growing and funding early-to-mid stage tech companies.  For bios and other goodness click here.

Subscribe
Tweets
Search
Community
Powered by Squarespace
The Five & Dime
« Business Development - The Basics | Main | The Curbed Chronicles: Don’t be a Jerk & Don’t Quit Your Day Job »
Thursday
Nov182010

Measuring User Engagement with Cohort Reports

Getting a reporting infastructure in place is always a challenge because it requires dev resources.

The good news for your beta period is that you can save your dev hours for features - you don't need a big dashboard report right now. Just focus on measuring user engagement.

Why? Because beta is all about nailing product/market fit and your on-boarding process – and user engagement is the metric that most directly reflects both.

There are 2 critical reports to measuring engagement. The first is a blunt instrument; the second more parsed out – and during beta, lots more useful.

Here’s how they work.

1.       Rolling % Active

The idea here is that you want to get some sense of what % of your overall user base is active in any given week or month. The calendar week or month is an artificial and rigid way to measure engagement activity. So instead of saying ‘50% of our users were active in November, vs. 45% in October’ – you generate a weekly report showing the rolling % of users active in the past 7 and 30 days. It gives you a smoother curve.

While overall % active is clearly a key stat, it has some shortcomings especially during the beta phase.

The primary shortcoming is that you can’t tell *which* of your users are active or inactive.

For example, are your active users mostly just new sign-ups who go nuts with rookie enthusiasm for a few days and then go silent, or are they your old powerhouse users who’ve been with you since the start -- or both?

Relatedly, an overall % active number doesn’t allow you to easily map engagement fluctuations to product changes. For example, have users who signed up after you implemented some new on-boarding technique engaged better than the users who signed up before that technique was implemented?

Also, I think it’s common during the first 12-24 months of a new product to have some subset of people sign up, go dormant, and then come back as real, engaged users x months later (it took me at least 9 months to actually start using Twitter, and I’m still dragging my feet on Evernote). An overall % Active metric won’t give you a sense of whether or not the initial sign-ups who did not convert to active users are coming back to you over time - or just staying dormant.

2.       Weekly Sign-Up Cohorts

Cohort reports are the best way to get behind the overall % Active stat and understand which users are active in any given time period.

The best way to understand cohorts is to look at an example.

Let’s say you launched your product on October 4th and have therefore been live in beta for 6 weeks. The following report shows for every week you’ve been live – how many people signed up each week, and how many of those users (per weekly sign-up cohort) remained active in each subsequent week:

(*these numbers are goofy btw, just trying to show the concept)



So if 30 people signed up in your first week of being live, then per this report, 14 of them were still active 6 weeks later.

Notice that of those first 30 sign-ups, lots of them went dormant in weeks 2-4, but came roaring back in week 5. Why did they come back? Maybe you implemented your weekly digest in that 5th week and reminded users you existed. Or maybe social proof kicked in and they realized how awesome you are.

Whatever the causes – cohorts let you map product or marketing activities to user engagement outcomes.

Note that you can run these *monthly* as well – so for all users who signed up in any given month, what number of those users are still active in each subsequent month.

There are lots of other helpful ‘flavors’ of cohort reports that you can easily churn out once you’ve got your basic queries in place. The most obvious of these is % of users active. This is really the same report as above, just showing % of each cohort active instead of absolute #, like this:

The cool thing about the % view is that you can benchmark engagement very easily week to week. So in the above report, look at the ‘Week 2’ column. The November 1st cohort is doing lots better in its second week than the October cohorts. Maybe you tested a new tutorial video that week – looks like it worked!

Although I won’t get into queries, one fundamental input to mention is your definition of ‘active user.’

As a product person, I always want to define ‘active’ very narrowly or stringently - but as big Rick Eaton always says, you don’t get points for ‘difficulty level’ in the active user contest :) Maybe more on this in another post.

At any rate – cohorts are helpful. Hope it makes sense! (Also interested what other folks are doing to measure engagement out there.)

References (20)

References allow you to track sources for this article, as well as articles that were written in response to this article.
  • Response
    ブランド時計通販
  • Response
    Response: video sexe gratuit
    Measuring User Engagement with Cohort Reports - Home - EarlyStager
  • Response
    Response: landscape design
    Measuring User Engagement with Cohort Reports - Home - EarlyStager
  • Response
    Response: natural acai berry
    Measuring User Engagement with Cohort Reports - Home - EarlyStager
  • Response
    Response: natural acai berry
    Measuring User Engagement with Cohort Reports - Home - EarlyStager
  • Response
    男教师强奸女学生:男教师强奸女学生图片 教师强奸10女生并包 宁波信用卡套现 女生为二奶男教师强奸女学生:男教师强奸女学生图片 教师强奸10女生并包女生为二奶 男教师强奸女学生:重庆一 湛江信用卡取现 小学教师强奸10名女生 并包女学生为二奶 辛勤的园丁,人类灵魂的工程师,宁波信用卡套现这是人们心目中对教师的崇高评价湛江信用卡取现。然而,在重庆大足县,有一名小学教师,从去年至今,先后对自己所教的10名未成年女学生实施猥亵;10年前他教过的一名女生也成了他的 二奶 目前,禽兽教师已被刑拘。 老师查夜查出色魔 6月4日,记者 贵阳信用卡套现 来到大足县
  • Response
    前中宣部部长朱厚泽揭秘胡耀邦下台是被谁“解决”了?前中宣部部长朱厚泽揭 上海信用卡套现 秘胡耀邦下台是被谁 解决 了?2010年5月9日凌晨,朱厚泽在北京医院逝世,到现在已经两年了。在这两年间,朋友相聚,时常谈起朱厚泽的人品和学识。他是实 乐山私家侦探 践家,更是思想家,他是中国当代改革的思想先驱之一。朋友们深深感到,他的离世是中国改革的重大损失。 上海信用卡套现初识 我和朱厚泽相识是唐宝心先生介绍的。唐先生1936年毕业于清华大学,抗战时期在贵阳参与创办清华中学,任校长。1980年代,唐先生是天津师范大学荣休教授,是清华大学 重庆信用卡取现 天
  • Response
    盘点国内 北京套现 八大“土豪金”建筑:河豚塔、圆大厦上榜原标题:全国八 徐州信用卡取现 大 土豪金 建筑人民日报新大楼并不是唯一一栋金大楼,中国各地不乏 土豪金 建筑,看来,国人几千年来还是酷爱金色。比之过去皇帝只是屋顶上弄个琉璃瓦的土砖,当今盛世可是真有金箔、纯金的建筑。细数起来,共有八大建筑:江苏中华河豚塔江苏扬中园博园,一条宽44米,高62米,长90米的巨型 河豚 ,是对着河豚标本,按比例 北京信用卡代还 放大设计建造的。为扬中市新地标建筑。由同济大学建筑设计院设计,重约2100吨,外表由8920块不同形状的黄铜金属表皮单元空间定位拼
  • Response
    影响信用卡额度提升的6大因素影响信用卡额度提无锡办理信用卡升的6 无锡办理信用卡 大因素 影响信用卡额度提升的6大因素:经济能力、住房情况、信用记录、婚姻状况、技术职称和工作状况、学历情况。* 北京套现 ***理财网小编为您一一介绍下: 1、经济能力 。工作单位的个人北京套现收入证明是决定信用卡额度的重要因素,能表明持卡人的收入是否稳定和偿镇江信用卡代还还能力等北京信用卡还款情况。如果申请人是高收入行业,会在评级的时候加分,对提额非常有利。 2、住房情况。重要因素之一。如果信用卡申请人拥 镇江信用卡代还 有个人住房,那么就证明他有一定的经济基
  • Response
    招商银行新年积分活动——“小积分抽大奖!”传统的加分换礼成都信用卡北京信用卡套现公司套现已经无法满足人们日趋增长 成都信用卡 的欲望了。招行针对 北京代还信用卡 人们目前非常热衷的两极旅游,开展了这次的积分抽奖活动。小编特地去官网了解了详情,在此为大家介绍。 2012年11月1日至2013年2月28日期间,仅需99积分起,即可参与2012幸运抽奖活动!抽取以9,999积分兑换南、北极探访之旅资格(南极、北极各3名)! 注:受出行时间北京代还信用卡限制,在2012年11月1日至12月15日期间参与活动者中,将抽取南极探 达州侦探公司 访游3名。
  • Response
    Measuring User Engagement with Cohort Reports - Home - EarlyStager
  • Response
    Response: shipping service
    Measuring User Engagement with Cohort Reports - Home - EarlyStager
  • Response
    Response: shipping service
    Measuring User Engagement with Cohort Reports - Home - EarlyStager
  • Response
    Thanks goes out to Ray for this recent review at his . "Something switche Womens Nike Flyknit Lunar 1+ Running Shoes Fireberry/White/Dark Grey/Pure Platinum d in 2010 with my thinking and I went to the heavier shoes, and I have been paying for that ever since. I had ...
  • Response
    农行北京分行截至今年3月底发卡数量已达689.52万张 总量居全国第一时间:2007-04-24 17:11来源:未知作者:admin 芜湖信用卡取现 编辑:Cindy阅读: 据统计芜湖信用卡取现,截至今年3月底,中苏州信用卡办理国农业银行北京分行银行卡发卡数量已达689.52万张,成为全国第一大发卡行。 “农业 苏州信用卡办理 银行在全国的网点最多,而金穗贷记卡不用存钱,先透支,后还款,56日之内还款还没有利息,非常划算。”日前一位用户在接受记者采兰州信用卡取现访时说。 今年初以 据统计,截至今年3月底,中国农业银行北京分行 兰州信用卡取现 银
  • Response
    农行北京分行截至今年3月底发卡数量已达689.52万张 总量居全国第一时间:2007-04-24 17:11来源:未知作者:admin 芜湖信用卡取现 编辑:Cindy阅读: 据统计芜湖信用卡取现,截至今年3月底,中苏州信用卡办理国农业银行北京分行银行卡发卡数量已达689.52万张,成为全国第一大发卡行。 “农业 苏州信用卡办理 银行在全国的网点最多,而金穗贷记卡不用存钱,先透支,后还款,56日之内还款还没有利息,非常划算。”日前一位用户在接受记者采兰州信用卡取现访时说。 今年初以 据统计,截至今年3月底,中国农业银行北京分行 兰州信用卡取现 银
  • Response
    Response: Alleure Eye System
    Measuring User Engagement with Cohort Reports - Home - EarlyStager
  • Response
    Response: Pure Life Cleanse
    Measuring User Engagement with Cohort Reports - Home - EarlyStager
  • Response
    Response: helpful site
    Measuring User Engagement with Cohort Reports - Home - EarlyStager
  • Response
    Measuring User Engagement with Cohort Reports - Home - EarlyStager

Reader Comments

There are no comments for this journal entry. To create a new comment, use the form below.

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>