How We Test and Score Short Drama Apps

Ranked by value, store reputation and what real users report.

Short version: some apps on this site were tested by a real person on a real phone, and some weren't (yet). We never blur the two. Every review tells you which kind it is right under the title, and the score is labeled the same way.

What are the two kinds of reviews?

Tested by our team. Someone on our team installed the app, watched it, hit the paywall and wrote down what happened: how many episodes were actually free, what the coins and VIP cost inside the app, how hard it was to cancel. Only these reviews say "we tested," "we paid" or "when we tried." You'll see the test date and the phones we used in a "What happened when we tested" box.

Based on store data and user reviews. The review is built from facts we can check: the App Store and Google Play listings, the prices listed in the App Store, store ratings, and a few hundred recent user reviews. Before it goes live, automatic checks make sure every price and number in it matches that data. Sometimes someone on our team also checks the prices against the app itself, and then the review says so at the bottom, with the date. These reviews never claim that we watched the shows or paid for coins. They say things like "according to the App Store listing" or "users complain that…", and the score is labeled data-based score.

When we test an app that used to be data-based, its review switches to the tested version and the change shows up in the "What we changed on this page" log at the bottom.

Where does the data come from?

• Store listings. App Store data comes from Apple's official lookup service. Google Play data comes from the public store page. We record the developer, the last update date, the content rating and the in-app purchase prices that the App Store lists.
• Ratings. App Store and Google Play ratings with the number of ratings, rechecked every week. Every ratings table says when it was checked.
• User reviews. Up to 150 App Store and up to 150 Google Play reviews per app, spread evenly over the past 12 months. Each review is sorted into topics (paywall, unexpected charges, ads, bugs, dubbing and so on), and the share of reviews for each topic is counted exactly, not estimated. We don't publish the reviews themselves, only the shares and short paraphrases.
• Our own notes. For tested apps, the tester's notes: free episodes, coin and VIP prices seen inside the app, what one series cost to finish, how canceling went.

We don't collect Trustpilot data automatically. Their terms don't allow it, so a Trustpilot rating only appears when we've added it by hand.

The ratings, prices, review shares and scores for every app in our ranking are also published as an open dataset you can download and reuse.

How is the score calculated?

Every app gets a score from 1 to 10, made of seven criteria with fixed weights. For data-based reviews, each criterion comes from a formula. "share(x)" is the share of analyzed user reviews that mention topic x, from 0 to 1.

Why we compare apps with each other

People mostly write store reviews when something annoys them, so every short drama app collects plenty of complaints. A raw "18% complain about the paywall" doesn't tell you much on its own. What matters is whether that's better or worse than the other apps.

So for the criteria built on user complaints (free tier, English audio, billing honesty, and the review-based parts of content and app quality), we compare each app with all the apps we've analyzed. An app with a typical amount of complaints gets about 6.5. Apps with clearly fewer complaints than the rest land around 8.5 to 9.5, and apps with clearly more land around 3 to 4. A difference of a couple of percent is noise, so it barely moves a score. Value and the store-rating part of app quality stay on the fixed scale from the table above.

Comparing only makes sense once there's enough to compare with. Until we've analyzed reviews for at least 8 apps, every criterion uses the fixed formulas from the table, and the review says so.

Store reputation: millions of ratings count

A few hundred written reviews are a small window. Store ratings are not: the biggest apps have millions of them. So store reputation is its own criterion. It combines the average App Store and Google Play rating (weighted by how many people rated each) with a bonus for how many ratings the app has, and it's compared with the other apps the same way.

Catalog size

Catalog size isn't part of the score yet. Most apps don't publish how many shows they have, and scoring only the few that do would punish them for being open while everyone else got the middle of the scale. For now, the content score comes only from what users say about the shows.

We collect title counts from official sources: the app's own store description, press releases and trade coverage. Numbers that count something else don't make the cut: "30,000 dramas across 18 languages" counts translations, and "10,000 episodes" counts episodes, not shows. Once we have a reliable number for at least 15 apps, catalog size will count for half of the content score, and this page will say so.

Only the last 12 months of reviews

We only analyze user reviews from the past 12 months. Older reviews describe an app that may have had different prices, a different paywall or fewer bugs.

Short review windows count for less

For very popular apps, the most recent few hundred reviews can cover just a few days, and one bad update can fill them with complaints. When the reviews we analyzed cover less than 30 days, the complaint-based criteria are pulled toward the middle of the scale in proportion. Reviews from 15 days count half as much as a full month.

A cap for billing complaints

Great shows don't make up for surprise charges. If an app's billing honesty score is below 4, its overall score can't go above 6.4, and its review opens with a "Watch your wallet" box: what users report and how to cancel.

For tested apps, the tester's 1–10 rating replaces the formula for every criterion they rated. The final score is the weighted sum, rounded to one decimal, and it's calculated by code. Nobody types a score in by hand, and nobody can buy one. Paid placements on this site are always labeled Sponsored and never change a score or a position in our rankings.

Who writes the reviews?

The Vertical Drama Reviews team builds every review from the data above, and automatic quality checks run on it before anything goes live. The checks confirm that every number in the text matches our data, look for copy-pasted text from other pages and catch banned filler phrases. A page that fails stays unpublished. Data-based reviews and best lists go live as soon as they pass. Reviews of apps we tested by hand are different: a person reads and approves each one before it goes live.

How often do you recheck?

Store ratings, listings and in-app prices are checked every week. User reviews are re-analyzed every month. When something on a page changes, the date under the title moves and the change is listed at the bottom of the page. If nothing changed, the date stays put.

Spotted something wrong? Tell us and we'll fix it. Who we are and how we make money is on our about page.