Most of my YouTube playlists say something like "15 unavailable videos are hidden", or about ~10% for a 5 years old playlist. The same ratio holds true for most bookmarks of YouTube videos I have.
There's videos from ~15 years ago that I consider gone for good - lost to the great copyright apocalypse - having been unable to find them anywhere on the today's internet.
I've had luck going to my liked videos, hitting the 3 dots under "Liked videos", selecting "Show unavailable videos", and copying the video watch?= ID into google. If the video is popular enough its text should pop up in a forum somewhere with some discussion at least where you'll find the right clues. Example: 9_1Qn370IIY
Same experience. I have years of notes and many link to specific youtube video sources (with time stamp). Unfortunately those URLs don't even give a title of a video, so tracking down removed videos can be hard. However, I've had some luck using LLMs! (obviously only when I can remember the context of a quote or some details about the video, even if vague). Once you know the video title it can often be found on unofficial youtube mirror sites.
Downloading your favourite stuff is absolutely what you should do, but it doesn't solve the bigger problem.
Many from my generation, who grew up during an era of information scarcity, have huge stashes of hoarded data. But what can we do with it? There is no way to legally share it, so we take it with us in our graves.
EDIT: Many good comments, but they all suffer from
one blind spot, so I answer here instead of individually.
A lot of content is only legal because it is on YouTube (or any other platform that has deals with the content industry). Sure, we can share illegally in a way that is obscure enough that no one gets sued. But what help is that, eventually it will get either popular and sued out of existence or forgotten and fade.
Then there is old content that is not legally shareable at all, like old game ROMs. Plenty of millennial stuff there but the Gen-X world is already half forgotten.
Cooyright is just too
long and too unreliable. What we need at least is exemptions from Copyright for archivists, otherwise a lot of stuff will
simply be lost in the coming decades.
I suppose you can distribute YouTube videos the same you distribute Linux ISOs. Something like IPFS (RIP(?)) would work even better for random videos than it ever did for Linux ISOs.
I went down an IA hole a month or so ago, and discovered someone uploading random Jaz + Zip drives images, that appeared to be backups from random businesses and people in the late 1990s There were diary entries, letters to a lawyer about divorce, business accounting data, invoices, and period-era porn, that if i were to guess, was organized to be offered on BBSes. tl;dr: encrypt that data at rest, you never know where it will end up.
I have mixed feelings about that stuff (which I've seen there too). There's historic value in it and I think it should be preserved. It also feels a bit icky and wrong for it to be publicly available so soon after its creation.
honestly, for stuff that old i kinda like the idea of seeding it anyway... i know that is a naïve PoV in the modern era, but i kinda miss the days of the internet being innocent enough that you got a real slice of someone's life sometimes.
I agree with the sentiment (and have several TB of archived Youtube videos) but I can't help but think "Keep this on the down-low!". I feel the same way every time someone mentions Youtube RSS feeds (which drive my archiving rig). Keep quiet or they'll take them away!
TBF when I wrote the post I wasn't planning for it to reach any kind of sizeable audience. That said, I feel like yt-dlp is mainstream enough that talking about ways to make it suit your needs probably won't ruin everything?
Tough balance, and not one where I have any real confidence in my own answer.
I have a nightly job that looks at all my youtube playlists, and downloads them to disk. It also sends me an email whenever a video that I haven't watched becomes unavailable so I can watch the local version. It took about 2 days to setup, but probably 30 minutes with AI today.
would be interested to have a look if you happened to have a repo setup or a blog post explaining! prob won't necessarily setup something myself but always been curious about this kind of thing
It's really as easy as asking Opus 5.5 and/or DeepSeek 4.1 (if Opus has moral issues) to set this up. I have the same. I've got a Download playlist which synchronises everything automatically. It had some issues where YouTube detected ytdl and throttled the download or limited video quality, so I set up a container with a full browser and fake browser fingerprint attributes, then downloaded at 1.5x (that was the limit before it throttled again). Just spar with Opus a bit if you encounter these issues.
One of the things I wish I could solve better, was a way to search for and show decent search results that you could then pipe into `yt-dlp`. Every method of doing this I've seen is basically a "hack" and often breaks or doesn't work at all. Anyone have a better solution to this that lets you basically build a frontend/proxy to Youtube or lets you Search, Show results and Download what you want to see lateR?
It's rate-limited but other hacks to scrape Youtube without the API will also be rate-limited as well unless you try to trick Google via unapproved hostile methods such as commanding an army of residential ip zombies and aggregate it.
Fundamentally, the index of Youtube videos lives in Google's servers so you're beholden to Google to get results from it. If the issue is that the Google's official results from search queries are not surfacing the videos you think are actually there, it means you'd have to build a duplicate (massive) "shadow" index to return those missing videos. To build such a beast, you're back to hitting the brick wall of rate-limits again.
I was messing around with the new coding agent in Xcode and built an app that would let me put in a bunch of channels I subscribe to, then basically made a YouTube RSS app around that. It would fetch new videos and show the thumbnails, times, descriptions, etc. If I wanted to watch something it will download it with yt-dlp on the backend. Watched videos would get cleaned up periodically, but videos marked as favorites would stick around.
It worked decently well. However, I started running into API limits when it would refresh to fetch updates. Things could likely be tweaked to avoid this, but I haven’t revisited it yet.
I didn’t make a generic search, but did allow pulling up one-off videos to download and favorite them.
What hacks have you found? I guess my first thought is some sort of snippet to run in the browser console, or an extension, that you can just run when using youtube itself that copies the yt-dlp command to downloads all the page's search results to your clipboard. Definitely a hack I'm just curious what else you've seen.
copy(
'yt-dlp ' +
[...document.querySelectorAll('ytd-video-renderer a#video-title')]
.map(a => {
const u = new URL(a.href);
return `"https://www.youtube.com/watch?v=${u.searchParams.get('v')}"`;
})
.join(' ')
)
Due to the nature of youtube updating so often all methods utilizing youtube's search would break often, but in the age of coding agents I don't think it matters as much anymore. Tell that to claude code and it will do it in 15 minutes. Being annoyed with youtube's search interface regressions I've basically built something similar by having an alternative frontend to better sort by date ranges, view count etc. It has already broken twice in last 6 months but fixing it is trivial by pointing a coding agent at it. The way it works is by piping the search query to automated playwright browser which returns the parsed search results and thumbnails. Also dabbled with Youtube's API in the past which sometimes shows results that don't show up on its regular search.
Since youtube api won't help with this someone needs to build a video index and a good search algorithm on top. Would be nice to be able to search through indexed subtitles too.
Freetube but if you want HD you have to separate out the video and audio downloads then combine them after. A little annoying but it does have download right there in the client
i use freetube to search. i can select videos that i want to watch with a single click into a playlist and then export that playlist and feed it to yt-dlp. since my internet is often slow, that is the main way i watch videos. i don't even open them directly in freetube, but wait until yt-dlp is done downloading.
Podsync will download videos from a playlist and then publish an XML feed of videos it's downloaded. You can set it up so that you interact with it by just adding videos to a (public or unlisted) playlist.
It's intended to be used as a youtube-to-podcast-player tool, but seems like it fits your use, too.
Read the comments and combined with my personal experience, made me wanna have a full on offline library, and in a better case, "decentralized" libraries of such things, although at the very least videos prolly won't work all that well.
I regret not archiving all the videos from the youtuber Myles Power (pseudoscience-debunker). He was one of my more favorite YouTubers but a few years ago he nuked his entire channel and all the videos on it (I suspect because his employer didn't like him arguing with people on the internet).
I wanted to rewatch some of his videos recently and as far as I can tell most of his videos are now lost media. I should have downloaded them when I had the chance.
I've always downloaded entire YT channels I enjoyed. I now have free/forever access to tons of content that now sits behind paywalls on topics I care about.
Not the poster you're replying to, but my archiving rig uses my personal fork of "tt-rss" to pull in the RSS feeds for the channels I care about and a script to query the database. That script uses yt-dlp to download the videos and subs, then marks them downloaded in the database. It's a bunch of glue-and-tape scripting code but was easy to hack together in an afternoon. Tweaking yt-dlp to run at a velocity that didn't get my IP address put into "you must login" hell was the most difficult part. The only long-term care-and-feeding has been keeping yt-dlp updated in the face of the arms race.
Yeah what the other person said - the Tdarr instance was for my other media and I figured, might as well use it across the board. It was less work to just add another library to Tdarr and let the server go brrrr than to work out how to get ytdl-sub to do that, especially when the library was already in the hundreds of GB.
`--recode-video` is probably the better solution if starting from scratch.
I did so with regards from Tales from the crypt; the
early seasons were great, so I have them locally.
Or, at the least the better ones; some are not so good.
For this purpose I also kept list, in .yml files, for
the most recent URL. Not just on youtube but also dailymotion
etc...
Interestingly there are some surges of URLs suddenly being
eliminated. Naturally we know the copyright mafia is trying
to hinder us, the people here, but it is interesting how that
works. Because ... it sometimes works. And then it does not
work. Which is quite strange to me.
For instance, you can often still find new videos on youtube
here, even full movies. Sometimes you can download them; then
suddenly they are gone, but they were up before that for months
or even years. So it is really really strange to see some videos
suddenly vanishing, others being around for decades literally.
Even when you'd assume these have ... "copyright issues".
>The corporate internet is hostile to art and the things you love
Agreed. I mourn the deletion of legendary Hair cake video from FilthyFrank every now and then. But at least we have AI slop videos or in better case videos with "The inevitable rise and fall of XXX" title" with human narrator reading AI written script. But hey at least everyone censor word fck and sht.
Every time you watch a youtube video you are downloading the youtube video to disk. It is just to prevent the lawyers from exploding that we all pretend that deleting it afterword means it never was downloaded.
But yes, everyone should be not deleting the videos they've downloaded.
Technically watching a video only streams video chunks into memory. But of course there are tools pretending to be a youtube client to stream it to a file.
to the layman download means the whole video is stored in its entirety on my disk. and possibly to lawyers too. that is not happening with streaming videos. youtube videos in particular most likely never hit the disk. not even the cache. when i am watching youtube then i am not deleting the video after it is done. the "downloaded" segments disappear automatically as i watch. so there is no delete step that i could skip in order to keep the file. rather, actually downloading a video requires an entirely different operation.
It used to be but not now. Your browser is served little slices of the video and audio, and the player adapts in realtime, giving you lower or higher bitrate, depending on your network congestion.
yt-dlp, as far as I know, always asks for the same consistent quality/bitrate of your choosing (and saves them to disk and joins the segments and muxes the video and audio into one stream)
> ytdl-sub
> This is the only yt-dlp wrapper that allowed me to properly maintain the Playlist structure to any extent
I have the same requirement and it's possible to do it with the default yt-dlp:
yt-dlp --embed-metadata --parse-metadata "playlist_index:%(track_number)s" "PLAYLIST_URL"
Most of my YouTube playlists say something like "15 unavailable videos are hidden", or about ~10% for a 5 years old playlist. The same ratio holds true for most bookmarks of YouTube videos I have.
There's videos from ~15 years ago that I consider gone for good - lost to the great copyright apocalypse - having been unable to find them anywhere on the today's internet.
Scrape, my friend.
I saw a similar thing some years back: playlists I had created began to be missing videos.
Been scraping ever since…
I've had luck going to my liked videos, hitting the 3 dots under "Liked videos", selecting "Show unavailable videos", and copying the video watch?= ID into google. If the video is popular enough its text should pop up in a forum somewhere with some discussion at least where you'll find the right clues. Example: 9_1Qn370IIY
internet archive also has a decent portion of old popular videos
Same experience. I have years of notes and many link to specific youtube video sources (with time stamp). Unfortunately those URLs don't even give a title of a video, so tracking down removed videos can be hard. However, I've had some luck using LLMs! (obviously only when I can remember the context of a quote or some details about the video, even if vague). Once you know the video title it can often be found on unofficial youtube mirror sites.
Any luck with the wayback machine?
Downloading your favourite stuff is absolutely what you should do, but it doesn't solve the bigger problem.
Many from my generation, who grew up during an era of information scarcity, have huge stashes of hoarded data. But what can we do with it? There is no way to legally share it, so we take it with us in our graves.
EDIT: Many good comments, but they all suffer from one blind spot, so I answer here instead of individually.
A lot of content is only legal because it is on YouTube (or any other platform that has deals with the content industry). Sure, we can share illegally in a way that is obscure enough that no one gets sued. But what help is that, eventually it will get either popular and sued out of existence or forgotten and fade.
Then there is old content that is not legally shareable at all, like old game ROMs. Plenty of millennial stuff there but the Gen-X world is already half forgotten.
Cooyright is just too long and too unreliable. What we need at least is exemptions from Copyright for archivists, otherwise a lot of stuff will simply be lost in the coming decades.
Seed it. That is how you share it.
I suppose you can distribute YouTube videos the same you distribute Linux ISOs. Something like IPFS (RIP(?)) would work even better for random videos than it ever did for Linux ISOs.
Put it on the Internet Archive and let the future discover it.
IA accepts and follows takedown requests from almost anybody about almost anything.
I went down an IA hole a month or so ago, and discovered someone uploading random Jaz + Zip drives images, that appeared to be backups from random businesses and people in the late 1990s There were diary entries, letters to a lawyer about divorce, business accounting data, invoices, and period-era porn, that if i were to guess, was organized to be offered on BBSes. tl;dr: encrypt that data at rest, you never know where it will end up.
I have mixed feelings about that stuff (which I've seen there too). There's historic value in it and I think it should be preserved. It also feels a bit icky and wrong for it to be publicly available so soon after its creation.
honestly, for stuff that old i kinda like the idea of seeding it anyway... i know that is a naïve PoV in the modern era, but i kinda miss the days of the internet being innocent enough that you got a real slice of someone's life sometimes.
I agree with the sentiment (and have several TB of archived Youtube videos) but I can't help but think "Keep this on the down-low!". I feel the same way every time someone mentions Youtube RSS feeds (which drive my archiving rig). Keep quiet or they'll take them away!
TBF when I wrote the post I wasn't planning for it to reach any kind of sizeable audience. That said, I feel like yt-dlp is mainstream enough that talking about ways to make it suit your needs probably won't ruin everything?
Tough balance, and not one where I have any real confidence in my own answer.
>Keep quiet or they'll take them away!
But you just mentioned the RSS thing ... Doctor, heal thyself!
> the RSS thing
Ssshh, you just did it again!
I'm using the External Application Button browser extension along with yt-dlp so that I can download YT videos with a single click. https://webextension.org/listing/external-application-button...
My arguments line for the extension looks something like this:
I have a nightly job that looks at all my youtube playlists, and downloads them to disk. It also sends me an email whenever a video that I haven't watched becomes unavailable so I can watch the local version. It took about 2 days to setup, but probably 30 minutes with AI today.
How often do you get those emails?
Are you running this locally? This is something I've considered but never felt like putting the effort to complete
Yes, on my local box that's on 24x7.
would be interested to have a look if you happened to have a repo setup or a blog post explaining! prob won't necessarily setup something myself but always been curious about this kind of thing
It's really as easy as asking Opus 5.5 and/or DeepSeek 4.1 (if Opus has moral issues) to set this up. I have the same. I've got a Download playlist which synchronises everything automatically. It had some issues where YouTube detected ytdl and throttled the download or limited video quality, so I set up a container with a full browser and fake browser fingerprint attributes, then downloaded at 1.5x (that was the limit before it throttled again). Just spar with Opus a bit if you encounter these issues.
One of the things I wish I could solve better, was a way to search for and show decent search results that you could then pipe into `yt-dlp`. Every method of doing this I've seen is basically a "hack" and often breaks or doesn't work at all. Anyone have a better solution to this that lets you basically build a frontend/proxy to Youtube or lets you Search, Show results and Download what you want to see lateR?
>, was a way to search for and show decent search results
Not sure what you're exactly asking for. Google/Youtube does provide an "official" API for youtube searches:
https://developers.google.com/youtube/data/reference/rest/v3...
It's rate-limited but other hacks to scrape Youtube without the API will also be rate-limited as well unless you try to trick Google via unapproved hostile methods such as commanding an army of residential ip zombies and aggregate it.
Fundamentally, the index of Youtube videos lives in Google's servers so you're beholden to Google to get results from it. If the issue is that the Google's official results from search queries are not surfacing the videos you think are actually there, it means you'd have to build a duplicate (massive) "shadow" index to return those missing videos. To build such a beast, you're back to hitting the brick wall of rate-limits again.
Do you know if this requires a GCP/Google account or can be it be used freely, but rate-limited to a reasonable rate for personal use?
Answering my own question. No.
> API key not valid. Please pass a valid API key.
https://github.com/offline-ant/await-clipboard/blob/main/awa...
A script that streams clipboard changes.
Next i run a bunch of user services that do:
`await-clipboard.sh | grep --line-buffered -z -e "youtu" | ...`
I just use youtube like normal, but instead of clicking a video i right-click copy to clipboard and a mpv+sponsorblock window pops up
I was messing around with the new coding agent in Xcode and built an app that would let me put in a bunch of channels I subscribe to, then basically made a YouTube RSS app around that. It would fetch new videos and show the thumbnails, times, descriptions, etc. If I wanted to watch something it will download it with yt-dlp on the backend. Watched videos would get cleaned up periodically, but videos marked as favorites would stick around.
It worked decently well. However, I started running into API limits when it would refresh to fetch updates. Things could likely be tweaked to avoid this, but I haven’t revisited it yet.
I didn’t make a generic search, but did allow pulling up one-off videos to download and favorite them.
What hacks have you found? I guess my first thought is some sort of snippet to run in the browser console, or an extension, that you can just run when using youtube itself that copies the yt-dlp command to downloads all the page's search results to your clipboard. Definitely a hack I'm just curious what else you've seen.
Due to the nature of youtube updating so often all methods utilizing youtube's search would break often, but in the age of coding agents I don't think it matters as much anymore. Tell that to claude code and it will do it in 15 minutes. Being annoyed with youtube's search interface regressions I've basically built something similar by having an alternative frontend to better sort by date ranges, view count etc. It has already broken twice in last 6 months but fixing it is trivial by pointing a coding agent at it. The way it works is by piping the search query to automated playwright browser which returns the parsed search results and thumbnails. Also dabbled with Youtube's API in the past which sometimes shows results that don't show up on its regular search.
I used a self-hosted Invidious instance in the past. Im not sure of the current state, but it’s a privacy front end similar to redlib.
> Anyone have a better solution
Since youtube api won't help with this someone needs to build a video index and a good search algorithm on top. Would be nice to be able to search through indexed subtitles too.
I don't even want an "algorihtm" per se. Just search, show results and a download or paly. That's it.
By algorithm I mean a scoring function to tell which video is relevant to the search.
Freetube but if you want HD you have to separate out the video and audio downloads then combine them after. A little annoying but it does have download right there in the client
freetube removed the download feature a while ago. see my other comment above on how i use freetube to download videos.
Dang really? Guess I just haven’t needed it in a while
i use freetube to search. i can select videos that i want to watch with a single click into a playlist and then export that playlist and feed it to yt-dlp. since my internet is often slow, that is the main way i watch videos. i don't even open them directly in freetube, but wait until yt-dlp is done downloading.
Podsync will download videos from a playlist and then publish an XML feed of videos it's downloaded. You can set it up so that you interact with it by just adding videos to a (public or unlisted) playlist.
It's intended to be used as a youtube-to-podcast-player tool, but seems like it fits your use, too.
You can use google site search operator to get a better youtube search
> and often breaks
That's kinda the nature of YouTube downloaders/scrapers including yt-dlp. It's not exactly a public API that YouTube's offering to support.
Read the comments and combined with my personal experience, made me wanna have a full on offline library, and in a better case, "decentralized" libraries of such things, although at the very least videos prolly won't work all that well.
I regret not archiving all the videos from the youtuber Myles Power (pseudoscience-debunker). He was one of my more favorite YouTubers but a few years ago he nuked his entire channel and all the videos on it (I suspect because his employer didn't like him arguing with people on the internet).
I wanted to rewatch some of his videos recently and as far as I can tell most of his videos are now lost media. I should have downloaded them when I had the chance.
I've been (manually) download my favourite youtube videos to my cloud since I've noticed some videos gone forever for any reasons.
I've always downloaded entire YT channels I enjoyed. I now have free/forever access to tons of content that now sits behind paywalls on topics I care about.
All automated for years
What are you using?
Not the poster you're replying to, but my archiving rig uses my personal fork of "tt-rss" to pull in the RSS feeds for the channels I care about and a script to query the database. That script uses yt-dlp to download the videos and subs, then marks them downloaded in the database. It's a bunch of glue-and-tape scripting code but was easy to hack together in an afternoon. Tweaking yt-dlp to run at a velocity that didn't get my IP address put into "you must login" hell was the most difficult part. The only long-term care-and-feeding has been keeping yt-dlp updated in the face of the arms race.
Rather than using Tdarr, wouldn't it be simpler to use yt-dlp's `--recode-video` option?
Yeah what the other person said - the Tdarr instance was for my other media and I figured, might as well use it across the board. It was less work to just add another library to Tdarr and let the server go brrrr than to work out how to get ytdl-sub to do that, especially when the library was already in the hundreds of GB.
`--recode-video` is probably the better solution if starting from scratch.
If the jellyfin receives content from anywhere else, a separate job for transcoding would cover those too.
I did so with regards from Tales from the crypt; the early seasons were great, so I have them locally. Or, at the least the better ones; some are not so good.
For this purpose I also kept list, in .yml files, for the most recent URL. Not just on youtube but also dailymotion etc...
Interestingly there are some surges of URLs suddenly being eliminated. Naturally we know the copyright mafia is trying to hinder us, the people here, but it is interesting how that works. Because ... it sometimes works. And then it does not work. Which is quite strange to me.
For instance, you can often still find new videos on youtube here, even full movies. Sometimes you can download them; then suddenly they are gone, but they were up before that for months or even years. So it is really really strange to see some videos suddenly vanishing, others being around for decades literally. Even when you'd assume these have ... "copyright issues".
>The corporate internet is hostile to art and the things you love
Agreed. I mourn the deletion of legendary Hair cake video from FilthyFrank every now and then. But at least we have AI slop videos or in better case videos with "The inevitable rise and fall of XXX" title" with human narrator reading AI written script. But hey at least everyone censor word fck and sht.
Is this another yt-dlp wrapper?
No, it's a blog post. Next time you can click the link instead of being dismissive.
No, not really. It's a dude showing how he used Jellyfin to organize his youtube archive.
yes
Every time you watch a youtube video you are downloading the youtube video to disk. It is just to prevent the lawyers from exploding that we all pretend that deleting it afterword means it never was downloaded.
But yes, everyone should be not deleting the videos they've downloaded.
Technically watching a video only streams video chunks into memory. But of course there are tools pretending to be a youtube client to stream it to a file.
to the layman download means the whole video is stored in its entirety on my disk. and possibly to lawyers too. that is not happening with streaming videos. youtube videos in particular most likely never hit the disk. not even the cache. when i am watching youtube then i am not deleting the video after it is done. the "downloaded" segments disappear automatically as i watch. so there is no delete step that i could skip in order to keep the file. rather, actually downloading a video requires an entirely different operation.
No its the same operation just you save instead of not save.
It used to be but not now. Your browser is served little slices of the video and audio, and the player adapts in realtime, giving you lower or higher bitrate, depending on your network congestion.
yt-dlp, as far as I know, always asks for the same consistent quality/bitrate of your choosing (and saves them to disk and joins the segments and muxes the video and audio into one stream)
"to the layman download means the whole video is stored in its entirety on my disk."
Yes, that's exactly what it means.