The Long Video Trap
I got an interesting graph out of YouTube analytics this week.
It's interesting to me because it helped me finally figure out what I've been seeing in my channel metrics for the last few videos.
As a creator, you're constantly aware of YouTube's recommendation algorithm. The majority of views will come from in-platform recommendations. No matter how good you make something, no matter how much you publicise it elsewhere, if the algorithm doesn't pick it up then it'll flop.
There are a whole bunch of steps to the algorithm where it attempts to match videos to viewers based on topics and interests, but most of our interaction with it as creators comes after this. YouTube has a potential audience member browsing the recommendations page. Does your video get shown?
For YouTube, the algorithm has a simple job to do. They want that viewer to click on something and watch it, and to keep doing that for long enough that they watch multiple adverts or feel they're getting value for money out of their Premium subscription. Scrolling through the feed without finding anything is bad, and clicking on videos only to stop watching immediately is bad, because people doing this are not watching ads and are at high risk of disengaging to do something else entirely.
As a result, YouTube heavily prioritises two metrics about your video:
- How many people clicked the thumbnail (the click through rate, or CTR)
- How long the ones who clicked watched for
Initially, the video will be shown to a small audience with very closely matched interests to see how it does. If I make a video about bus simulators, the initial audience will be people who are really into games and really into simulation and really into buses. If my thumbnail is attractive to those people, and my video keeps them entertained, the net then widens. Maybe people who are into simulation games, but aren't so fussed about buses. Maybe people who love buses and have watched the occasional video about games.
This is the "wider audience" stage, where YouTube keeps widening the net until the CTR and watch time drop to the point where your video is no longer a good thing to recommend.
Isolation
The problem for YouTube is the alogorithm does not exist in isolation. Creators are constantly trying to reverse-engineer it and game it to a greater or lesser degree to get their videos given favourable treatment.
This causes problems.
If we go back a year or two, YouTube appeared to give CTR by far the strongest weighting in its recommendations. This worked well in making us creators think not just about the quality of the video, but also how you got your potential audience to go, "that looks like it might be interesting". But then the gaming started, and YouTube hit a well-known problem: clickbait.
Clickbait is where someone decides that if CTR is given such a high weighting, you don't actually need to care about the video they see after you got that click - it can be a lazy compilation of unnarrated clips or even something completely unrelated to the thumbail and title and it doesn't matter, you got the click.
YouTube's first response, as far as I can tell, was to penalise videos where viewers didn't get through the "intro", which it classifies as the first 30 seconds. This forced a lot of us to think about those first 30 seconds, and it's why you've seen so many of those flashy prerendered channel intros be condensed or even removed in favour of jumping straight into the topic at hand, but was itself easily gameable: providing you keep people engaged for 31 seconds or more at the start, you don't need to care about the rest.
This is the root of the infamous, "without further ado, let's get into it", followed by what is noticeably more ado and a significant lack of getting into it. There was never any intention of getting into it; only of making you believe there was for long enough to get over that critical 30 second mark.
Then something changed, around early 2023. Something which at first had us creators very confused.
CTR
"I don't understand what's happening here. The CTR is still looking great, my intro retention is over 70%, but it's stopped dead."
Thus began a lot of head scratching about what had happened to the conventional wisdom we'd been working with for the last year or so. I theorised that it might be the rate of change of CTR as videos went to wider audience, as it seemed consistent with what I was seeing, but then other people found counter-examples and nothing anywhere seemed to make much sense.
That is, except for one thing. YouTube kept recommending longer and longer videos. We'd always had the infamous 8-minute mark, the point above which midroll adverts can be included, but in 2023 videos suddenly shot up in length. Whereas before 30 minutes or more had been the rare preserve of deluxe documentaries in the mould of Defunctland, even relatively ordinary channels about things which can absolutely be covered in about the time of a tea break were putting out 40 minute, 50 minute, even hour plus videos.
We thought there was a new metric in town: absolute watch time. The longer someone watched a video, the more it got recommended. 12 minutes of an hour long video is better than 8 minutes of a 10 minute video. YouTube openly admitted this, in their usual style. "Recommendations are based on a combination of relative and absolute watch time, with relative time weighted more heavily for short videos and absolute time given more weight for longer videos".
Which is what all those hour-long videos are gaming. Yes, it makes for infuriating watching as someone insists on saying, "Superman: The Ride at Six Flags New England" every time instead of "the coaster" and describing the colour of individual items on their stock footage, but the metrics say someone giving up a video as unwatchably slow after 15 minutes is better than them going away happy, informed and entertained after just ten.
Or do they?
Red Shirt
Through all of this I and millions of other YouTube viewers are being recommended nearly every Tom Scott video, in the svelte range of 3-6 minutes long. On the more specialist interest side, Big Clive is still popping up in feeds for 7 minutes of electronics and 3 minutes of ill-advised Soda Streaming, and Jago Hazzard's 10 minute timetable is so reliable you could run a railway on it.
Once more, I refer you to this graph.
The point up to the elbow is me, despite early misgivings about a video, doing everything right based on our understanding of the algorithm. I had a topic which was currently popular. I had a thumbnail and title with a high CTR. I had a suitably long video with above-average retention (and therefore watch time) for the channel.
You can see this by how far the blue line climbs above my expected range in the dark grey band. But then there's that sudden elbow. It doesn't completely kill the video growth, indeed that's my usual gradient afterward, but it did arrest that crazy upward momentum. As we'd seen and be confused by earlier in the year, the CTR was still great, indeed it was increasing slightly at that point. But one very noticeable thing happened at exactly the point of that elbow. This:
YouTube stopped pushing the video to wider and wider audiences at exactly the point average viewing time dropped below my typical range. Now this is just one data point, but it also ties in with something else: while channels I used to watch are releasing painfully long epics, my feed is returning more and more to the mix of 7, 15 and 25-minute videos I used to see.
I think I know what I'm seeing here. YouTube is not rewarding long videos. It's rewarding consistency.
If I put out a video, YouTube's expectation is that I engage the average viewer for between 8:36 and 9:57, whether that's by slightly under a third of people watching a 30-minute video all the way through, or nearly everyone watching a 10-minute video. It doesn't mind if I exceed that, but it will get unhappy if I fall below it.
What this is saying is that while I can alternate between more engaging, shorter videos and longer videos with higher dropoff, I can't release a 5-minute video and expect it to do well because that's not what YouTube expects of me. It's also subtly telling me that if I want wide reach, my video has to be equally engaging to both niche and wider audiences. That elbow happened because once you get a broad enough audience, the people in it don't really want to watch a British man make a sandwich and play with virtual trains for half an hour.
Length
The vital takeaway from this is that if I am right, making longer and longer videos in the hope of cheating the algorithm is a terrible idea. Not only are you going to annoy regular viewers with dragging, unentertaining scripts but you're forcing yourself into a trap you can't get away from. Every time you extend the lower bound of that average watch time upward, you're making the point at which YouTube turns off the tap easier and easier to reach.
Worse, while you may have the most enticing thumbnail and intriguing title, you're getting closer to the point at which you destroy your CTR because for each potential viewer there's a certain length above which they just won't click, especially if they're feeling suspicious of long videos the same way they'd be suspicious of obvious clickbait. CTR is still a critical metric for getting that early push. But now your video has to be long because YouTube expects 15-20 minutes of watch time from you as an absolute minimum, you can't get that without going above 90 minutes, nobody clicks a 90-minute video recommendation and... oops, YouTube have picked new winners for your category that they'd rather recommend instead.
Of course, I'm only guessing at things based on what I see, educated as those guesses may be, and even the basic structure of this article is based around the idea YouTube changes their algorithm regularly in response to exploits; for all I know, the reason I'm seeing more shorter videos again is they've put more weight on relative watch time across all lengths of video and that elbow I noticed in my stats is just an odd coincidence.
Still. It feels like it explains a lot, especially combined with the OMSI 2 vid which also had a great CTR but just didn't get the average watch time as it was a third shorter than usual and thus slightly undershot that typical range.
Which suggests in terms of actual advice, if I'm even close to correct it makes sense to do the following:
- Make videos of consistent length and engagement, in the range which works for you and your audience.
- Be cautious of long videos increasing the lower bound of average watch time beyond what you feel comfortable delivering regularly.
- Consider how videos remain engaging once they're shown to a less niche audience.
- It may even be worth releasing a few deliberately short videos with the intention of having them underperform, but also keep your expected average watch time lower and easier to achieve.
- Forget all of the above, just have fun and make stuff you enjoy making.
And most importantly, don't make videos longer than their "natural" length in the hope of gaming the algorithm, because it's likely you're gaming it against yourself. At least, until it next changes in mystifying and ineffable ways, which will probably happen about 5 seconds after I click Publish.
Addendum
Speaking with other creators as a result of this has turned up some interesting counterexamples along with some confirmation.
Firstly, videos above the expected time range by a large amount not being boosted heavily. I can fit this into my model if YouTube really is going so hard after consistency that it wants you to stay within a particular range. One thing I nearly put in and then couldn't figure out a nice place to put it is an idea that in the same way YouTube has "gaming" and "simulation" categories for viewers, it may have "likes watching for 9-12 minutes", "dislikes long videos" and similar categories behind the scenes, and is trying to match people who like particular lengths of content with creators who make particular lengths of content. I shied away from this because it feels weird that they'd show users videos based on average watch time rather than overall length, but maybe there's some weird engagement maths going on behind the scenes.
The second counterexample is more interesting, though. A video which started out with average watch time below the creator's typical range, stayed below it throughout... and yet received two huge pushes from YouTube.
One thing about the behaviour I observed is that the elbow occurred for a video which had been in my typical range, then fell below it. So there may be something different in play that I'm not seeing in my stats, or that there's a different criteria for suddenly pushing a video which has already been out a while, or again I've just found a coincidence and extrapolated too much from it. And it could be that we're underestimating the influence of the topic and interest matching which takes place before all the CTR/retention magic happens - YouTube may be more lenient for topics where it has relatively few videos to show the audience, and if it can't figure out a useful wider audience it won't give videos the boost even if everything else lines up.
And of course there's the question of whether multiple variations are being A/B tested concurrently, and I just happened to see one particular version of the recommendation algorithm on one particular day. We are, after all, standing outside the machine with divining rods trying to figure out how it works without being able to look inside it.