Yeah, that title’s kinda click bait-y, isn’t it? Still, it does the trick (you are reading this sentence right now, aren’t you?) and it is entirely accurate. There are seven things I walked away from feeling all the wiser now that ScreenPlay’s public beta test is complete.
From October 20, 2015 to January 29, 2016, it’s been sixteen weeks of feedback, correspondence and rewrites. Oh, the rewrites. But from the very beginning, the game was very well received and has scored an average of 4.6 out of 5 on the playtest surveys. The nice part about this result was that I wasn’t trying to fix the game’s first release, I was trying to improve it. Shore up a few weak areas, tighten up the text for greater clarity, even play around with a few add-ons for good measure. More importantly, I had a chance to not only garner the attention of total strangers willing to give this game a shot, but I could see my game played without me. As a game designer, that’s what it’s all about. That, and never having to hold down a real job because your games sell a crapload of copies. One step at a time, ok?
The process was also very educational because it helped me learn more about the demographic for this game, which will come into play during crowdfunding, marketing, and sales later this year. There’s lots more I could get into, but it’s not what I want to write about today. The 7 things I learned from playtesting, remember. And here they.
First, it might be a good idea to see what I did that was unique compared to a simple Facebook post looking for playtesters otherwise none of this will make as much sense. The short and sweet of it? I developed a points reward system and built a Kickstarter-style campaign offering incentives based on the number of playtest points earned during the playtesting. Points were awarded based on completing one of three surveys per update: a Reader’s Survey for those who’ve only read the material, a Writer’s Survey for those who played as Writers, and a Director’s Survey for those who hosted their own playtest.
1. Four Months Is Too Long
Allow me to clarify something here: it’s not that running a playtest for four month is too long, just running a playtest campaign for four months is too long. During the first two months, there was a really nice back and forth. By the middle of December, right around the start of the holidays, it practically stopped.
Truthfully, running a playtest campaign like this was very similar to a crowdfunding campaign in that you have only that first week to get people’s attention. Anyone who’s truly interested and excited to be a part of it will grab hold within that first week and never let go. There’s so much going on within the roleplaying community alone that it won’t take long before the next shiny object comes along and garners all the attention. Even the way in which sites like DriveThruRPG display product is based on both popularity and freshness. (It also doesn’t help that DTRPG doesn’t rank free products, though I can’t say with any certainty such a public ranking would have improved any long term exposure.) What I do know is within the first couple of weeks, there were nearly 400 downloads of ScreenPlay: The Rehearsal Edition. It trickled after that. My playtesters signed on within the first week with a couple more on the following week and even they started to drift after a while.
This is not bitterness you’re reading, simply fact. I can’t blame them. Playtesting is more than simply reading and potentially playing; it’s asking the buyer/downloader to commit to something more than your average download. It’s a purchase with homework. It’s not for everyone. Which brings me to #2…
2. If Only 1% Of The Downloaders Respond, It’s Successful
Of the 700+ downloads, only eight people completed at least one survey. There were five others who earned playtest points for posting about it and using the #ScreenPlayRPG hashtag (and I’ll get into that soon), but those eight are the focus of this lesson. That’s just over 1% of downloaders. At first, it was disappointing until talking it over with my publisher, Aaron, who shared the same percentage with his last public beta test for his Entropic Gaming System.
Does this mean there are 692+ copies of ScreenPlay out there in world at risk of polluting my sales numbers when the final product is released? Possibly, but those chances are slim. I look at it with the same viewpoint as people who pirate movies – you were never going to get them to invest. Some people are curious, others simply grab something because it’s free and it gets lost in the shuffle. I’ve heard from a few others who read it but never had the opportunity to participate in the playtest, so this 1% is not a measurement for final sales nor is it an indication of lost consumers. The majority of these downloads were never going to participate, maybe 5% were, and 1% did. For a small indie publisher reaching out to total strangers, that’s still damn good. As a safety net, I did not update the final draft of the playtest document with significant rule changes now cannon for the game. In other words, only those who were truly invested in ScreenPlay know how the final version works. Just in case.
3. Reward Levels Should Be Based On Position, Not Total Points
The drawback to these single digit playtesters is with the playtest points themselves. The highest “scoring” playtester had 13 points, second place had 11, and third had 3 points. That’s a big gap. If there were more playtesters, the range could have been tightened. Then again, if everyone started to disconnect after the second month, would the points have risen to double digits in the first place? Whatever the determination, it all comes down to a flaw in how rewards were handed out.
Rather than use playtest point totals for prize levels next time, rankings would work better. Doesn’t matter how many you get, it all comes down to what rank you earn in the big scheme of things. The top ranked playtester earns the big prize, the second/third/fourth get another, fifth to tenth get this, and so forth. This approach allows everyone to still feel like they have a shot based on how everyone else is performing rather than racking up points when you may not have the time to do so.
4. Write Out Your “Stretch Goals” In Advance
Towards the end of November, as the playtest points were rolling in, I was scrambling to keep up the unlocked treatments. There were a couple of long nights spent locked in the Lab pouring out words, flipping pages, and tearing through post-it notes getting new drafts ready for release. All this on top of assembling new and revised rules to include in the same release. As my plan was to reveal these unlocked treatments over the course of the playtest (it was simply a matter of when they would be released), I really should have prepped them in advance then modified them to suit the current playtest draft’s revisions. Lesson learned.
5. Graveyard Writing Sessions Can Only Last So Long
With a newborn baby in the house and midnight feedings a regular occurrence during my one month paternity leave, connecting those times awake in the middle of the night with writing ScreenPlay was a match made in heaven. It’s 3:00 am, I’m up anyways, give me a keyboard. And I got a LOT of words pumped out in those graveyard writing sessions. The entire first draft of ScreenPlay was fully developed in that month.
By the time the playtest started, I was convinced my best chance to maintain the needed word output was through a weekly graveyard shift. For the first couple of months, it worked out great. (Though there was that one time when I started writing about some American college in a bout of sleep writing. Weird, huh?) Once I took some time off in December for the holidays, that was all she wrote. The ability to stay up late was gone. Plus it was around the time my son started sleeping through the night, so perhaps the waning instinct played a factor too. Either way, it’s something I’ll have to keep in mind as this game heads into final production, new treatments and supplements, and a crowdfunding campaign.
6. There’s A Demand For More Than Just Action Stories
When ScreenPlay first started out, it was designed to be an action-packed storytelling system. I built it to simulate fight scenes, car chases, that kind of thing. And it worked beautifully. There were two factors I hadn’t counted on. First, the game could actually handle multiple genres with little translation. Horror, drama, romance, you name it. Second, what I was providing to the playtesters was incredibly biased. I was writing material for ScreenPlay as if I was the only person running it, stories I wanted to tell with my group. That did not mean the playtesters wanted nothing but. As one playtesters put it, “Your work is very dark.”
Busted. By the time I wanted to make adjustments and offer something a little more Steven Spielberg 1980s excitement-y, the playtesting started to lose steam. This is now a major refocus moving forward and it’s also why I’m looking at bringing on other writers/designers to provide new treatments in the future, including stretch goals for the upcoming crowdfunding.
7. Hashtags Do Not Guarantee Marketing Success
Working in Marketing has taught me one thing: just because all the cool kids do it doesn’t mean it’ll work for you. I added a promotional feature to the playtest in the hopes it would create a more organic awareness by awarding 1 playtest point per version to anyone who used the #ScreenPlayRPG hashtag. During the first couple of weeks, there were a couple… and that was about it. No such luck. Tracking the hashtag, it was 95% me. Trends aren’t forced. Encouraged, yes, but a true organic campaign happens on its own and it’s a beautiful thing when it does.
This shows one of the other strengths of this playtest format – it’s not just useful for the game. It gave me a better understanding of the audience who would play this game, redefine its branding, and taught me what works and what might work for the next phase of marketing. So while the hashtag didn’t work out, that’s still a playtest success. Anything that teaches you something and helps build a better campaign for when you’ve got all your gears spinning at full speed is an essential tool.
Would I run this style of playtesting again? You betcha!
It was incredibly successful on multiple, even in its failures. The time invested in this playtest was worth the effort and lack of sleep, though there are some definite adjustments for next time. If you’re looking to run a public playtest in your near future, I would seriously advise consider a format like this.
Now if you’ll excuse me, there’s a final draft of ScreenPlay calling my name.