Wilson Mar bio photo

Wilson Mar


Calendar YouTube Github


7+ strategies to make betas more useful

US (English)   Norsk (Norwegian)   Español (Spanish)   Français (French)   Deutsch (German)   Italiano   Português   Estonian   اَلْعَرَبِيَّةُ (Egypt Arabic)   Napali   中文 (简体) Chinese (Simplified)   日本語 Japanese   한국어 Korean


Public beta releases are often the first time that partners and friends have substantial hands-on experience with the upcoming version.

I think too many developers waste the opportunity to build goodwill and obtain additional assistance that beta participants can provide.

Here is a summary of my advice explained further below:

  1. Define actors and use cases.
  2. Automate use case flows by each actor.
  3. Capture detailed factual proof points.
  4. Detail history of run speeds.
  5. Help those who participate.
  6. Delegate socialization of specifics.
  7. Make it easy to track progress and get reports.

More about each is described below.

Define actors and use cases

Too many organizations release only the software and expect everyone to somehow just figure it all out on their own.

This results in not enough coverage simply because people don’t know about all the features.

Clarification of who is being served by the product and what each type of user does would provide a richer basis for suggestions.

Automate use case flows by each actor

Automation of keystrokes are useful not just for testing.

Being able to run the demo sequence automatically would provide salespeople and others confidence that their live demo actually works rather than having to say sorry.

Ideally, planning for automation activities could start when a wireframe is available, especially when a tool such as Figma allows some user activity without programming.

Having automated use-case runs also helps to speed the development of product vidoes needed for not just launch but throughout the product lifecycle.

PROTIP: Involve functional automation testers, who may be able to code automation scripts instead of writing manual step-by-step procedures. This is so that when workflows change, demo videos can be recreated quickly rather than going through all the workflows to capture even small changes. Cypress.io is popular among functional testers because it’s easier to code than Selenium from a previous generation. See https://www.npmjs.com/package/cypress-movie

The more people who views a video, the more varied the extent of feedback.

Capture detailed factual proof points

General platitudes such as “overall it’s great” does not really provide excitement nor actionable improvement ideas.

Marketers need specific and relatable “sound bites” and quotes that can be published. An example:

“It now takes me 2 minutes to do what used to take 30 minutes. Love the automatic features of this new release.”

So we want to encourage beta reviewers to take measurements before, during, and after demo activities.

Valid measurements require planning and discipline, and that takes pre-planning and encouragement before and all the way through the process.

Detail history of run speeds

If timings of each action taken by the automated scripts are captured during each run, there would be data to diagnose complaints about a particular interaction being slow.

If those timings are captured at various points in various environments (including runs on developer laptops), troubleshooting can include comparison across different environments and under varying conditions over time.

This also enables anomalies to be identified earlier.

Help those who participated

Many consultants make hay from the fact that they had a hand in the creation of the product.

So have a webpage that acknowledges their participation so they can point to proof that they mattered.

Some beta participants had to fight hard to get approval for taking time to participate.

So a letter or email to their bosses from one senior manager to another would go a long way to ensure continued enthusiam in the future.

Such communication is an opportunity to “sell” the product, and contribute to continued willingness to pay maintenance fees.

As importantly, help beta participants avoid the need to run through a gauntlet of inexperience people when technical support is needed.

Delegate socialization of specifics

It takes a tremendous amount of time to stay in touch and clarify feedback.

So consider “deputizing” select individuals (“ambassadors”) from among the user community to take responsibility for communicating personally with his/her peers about the demo.

Ambassadors who live the product are in the best position to integrate comments from various others into single cohesive actionable statements.

Communications can be emails or, better yet, regular phone calls and video conference calls.

It helps if communication events are planned ahead, with several reminders ahead of each event, and follow-ups after.

Direct and personal communications can help to smooth over inevitable gliches so disappointments do not turn into long-term bitterness.

Make it easy to track progress and get reports

Provide a system for demo participants to track their activities and communications.

Make it easy by providing a special “bcc” address for ambassadors when they send out emails.

A system can then parse emails to collate the information, and present it in reports.

Design the tracking and reports based on specific actors and use case flows so decisions and actions can be specifically actionable.

Let us help you! We’ve done it.

More on Evanglism

This post is one of a (planned) series:

  1. Evangelist Job Description (dreaming on both sides)

  2. Evangelism dependencies
  3. Deliverables and Events Budgeting spreadsheet
  4. Evangelism Cost-Benefit analysis (Bang-for-the-buck comparion)

  5. Events and Conferences
  6. Calendar of Announcements
  7. Milestone events

  8. Social media
  9. Social media strategy
  10. Tweets
  11. Sentiment analysis (of favorable words or not)
  12. Responses to social media
  13. Tech Press mentions

  14. Competitive Comparisons (by analysts and others)
  15. Competitive Strategy
  16. Competitive Comparison Kit (for people to run for themselves)
  17. Objections, and how to handle them

  18. Target Customers
  19. Customer References
  20. Proof Points
  21. Email databases (User Acquisition)
  22. Remarketing of current or lapsed users

  23. Product websites
  24. Product wikis
  25. User Forums (Google, LinkedIn, Reddit, etc.)
  26. Surveys

  27. Product Roadmaps
  28. Release Schedule
  29. Tasks
  30. Issues

  31. environments for development, testing, demos, training, etc.
  32. Demo presentation scripts
  33. Beta activities
  34. Release Notes

  35. Tutorials
  36. Live event sign-up administration
  37. Webinars
  38. Video production capability
  39. Live Video Streaming

  40. Spiffs
  41. Hackathons
  42. Participating in Conferences
  43. Running Conferences
  44. Dynamic projections of tweets, etc.

  45. Recruitment follow-up