Google Play's closed testing requirement is brutal for indie developers

I have been building a small Android app.

Nothing controversial. Nothing clever with permissions. Nothing involving financial services, medical data, crypto, gambling or anything else that might reasonably justify a substantial approval process.

It is a baby-name app.

And apparently getting a baby-name app onto Google Play now requires running what feels like a miniature corporate QA programme.

The 12 testers, 14 days rule

For personal Google Play developer accounts created after 13 November 2023, Google requires developers to run a closed test before they can release an app to production. The published requirement sounds straightforward enough:

• Publish a closed testing release.
• Get at least 12 people to opt in.
• Keep at least 12 testers opted in continuously for 14 days.
• Apply for production access.

Annoying for an independent developer, perhaps, but manageable.

So I did it.

I found the testers. I distributed the app. I waited the required period. I finally got the option to apply for production access.

Google reviewed the application and responded with:

Your app requires more testing to access Google Play production.

Not because the app crashed.

Not because Google identified a policy violation.

Not because there was a security problem.

Just: more testing.

And underneath that:

Run your closed test with at least 12 testers for 14 more days starting from the review date.

Brilliant.

Meeting the requirement does not mean you meet the requirement

This is the part I find particularly frustrating.

Google presents the closed-testing process as a measurable hurdle.

Twelve testers.

Fourteen days.

Those are objective requirements. You naturally assume that once you satisfy them, you have completed the testing requirement.

Except you have not.

What you have actually earned is the ability to ask Google whether your testing was good enough.

The production-access application asks how testers used the app, whether they tried all its features, what feedback they provided and what you changed as a result. Those are reasonable questions in isolation.

The problem is that they turn a supposedly objective publishing requirement into a subjective review. Google's own guidance says it may require continued testing because of "insufficient tester engagement", but it does not publish a measurable engagement threshold.

There is apparently a hidden second requirement:

Convince Google that your beta test looked sufficiently like a beta test.

And if you do not, you can be sent back for another two weeks.

This hits indie developers completely differently

For a company with a QA department, 12 testers is nothing.

For a single developer building something in their spare time, finding 12 people who are willing to do all of this is surprisingly difficult:

1. Join a Google testing programme.
2. Install an unfinished application.
3. Actually use it.
4. Keep it installed.
5. Stay opted in for weeks.
6. Give meaningful feedback.

Your friends and family quickly become unpaid QA engineers.

And because dropping below 12 continuous testers risks breaking the eligibility period, the sensible approach is not even to find 12 people. You probably want 15 or 20, just in case.

All of this before your application has a single real user.

The incentives are weird

I understand what Google is trying to achieve.

Google Play has historically suffered from huge amounts of low-quality, abandoned, cloned, spammy and outright malicious software. Making developers prove that a real group of humans has tested an application probably filters out some of that.

But it also creates a strange barrier.

Someone producing hundreds of questionable apps commercially can build a testing process around Google's requirements.

A hobbyist who spends evenings building one genuinely useful app is suddenly messaging friends saying:

Could you please open my baby-name app a few times this week so Google believes I am a real developer?

That is backwards.

What counts as enough testing?

That is the most uncomfortable part. Nobody really knows.

Should every tester use the app every day?

Should they use every feature?

How much feedback is enough?

Does Google expect an update during the closed test?

How many testers should actively engage rather than simply remain opted in?

The published minimum is 12 testers for 14 days. The practical minimum can clearly be higher.

So now I will spend another two weeks making absolutely certain there is a visible testing trail: real usage, real feedback, documented bugs and suggestions, and changes released in response.

Not necessarily because the app needs another two weeks of testing.

Because I need to demonstrate to Google's approval process that testing happened.

Those are subtly different things.

I will try again in two weeks

Google reviewed my production-access application on 11 September. My reward for successfully completing the first closed-test period is another 14-day closed-test period.

So I will apply again near the end of September.

This time I will have more testers than the minimum, more structured feedback and a clearer record of what changed during testing. Hopefully that satisfies the production-access review.

And then, eventually, my terrifying baby-name application might finally be considered safe enough to unleash upon the billions of Android users Google keeps telling me are waiting for it.

I will update this post when I find out.