Cultural rot caused by YouTube taking down old videos is a serious problem.
There was a YouTube channel I was following for about 15 years, of an older woman who had pet foxes. Her health started declining and she always had on her channel something like, "I'll leave these videos up forever." She passed away and YouTube deleted all her content.
The problem is 100x worse for any political content. My YouTube bookmarks from just the past few years is a graveyard of information that's not convenient to have around. It's even true for politics-related jokes, songs, and memes. They seem to be selected for deletion more often than not, within a few years.
I have a playlist with 285 videos, and youtube just says "Unavailable videos are hidden".
It's the worst, I have no idea what is missing. I don't even know how much is missing and I am sure the number will keep increasing over time, not even sure they will bother to tell me that there are missing videos at this rate.
If you view your playlist on a desktop browser you can make it show you the unavailable videos in the list. Then you can grab those IDs and try to find them via the Wayback Machine, etc. Hope this helps.
Also Filmot has a userscript for this that searches their archive for metadata and shows thumbnails from the Wayback Machine: https://filmot.com/moresearch
Content uploaded to the Internet Archive is stored as <service>-<identifier>. Please don’t abuse, use judiciously. When in doubt, contact Patron Services (info at archive dot org) prior to archival ops. They can also assist with establishing a collection for multiple items if warranted. It costs the Internet Archive $2/GB to store bits in perpetuity; consider donating if it is within your means.
(content uploaded with this will be excluded by Archive search due to spammers and other bad actors, you will have to reference the identifier directly to access or use tools like this post; anything flagged by a copyright holder using the DMCA will be made unavailable, so keeping a local copy might be prudent)
I hate to do the HN thing, but nextcloud has a feature to use yt-dlp (or whatever), feed it a url, it will put it in your folder. You can share the folder or the file, even with the same URI (the hash yt gives videos, I mean).
I've done this by hand a lot - I have probably a half TB of 480p rips from YouTube and Twitter, as well as a bunch from reddit - though reddit has always been a pain in the ass.
I say use something like nextcloud because there's turnkey Linux images available, so you only need a host that supports that feature to get started. Furthermore, once you have the video "local", in the future if some decentralized video sharing service launches you'll be 90% of the way there by merely having canonical links to content available publicly.
A while back there was a hacker news story about someone who used AI to clean up the long lost Steely Dan song "the second arrangement". I was looking for it again on YouTube and it's now private, I presume thanks to a complaint from the record company. (There's still a wav file out there thankfully.) I feel like if the company was smart they'd put the track on all the streaming services, maybe even print a 7" 45 of it (vinyl boom anyone?) or include it on a Gaucho remaster... Then it could make some revenue.
I know there are probably a ton of much more academic use cases for this kind of tool, but for me personally, I'm mostly interested to use it to figure out what exactly got removed from a playlist (or liked videos or etc.) because it always bothers me when I can't remember what's no longer there.
The tricky bit is that you have to get your youtube cookie in order for yt-dlp to get your own liked videos. I expect it to be a lot more LoC to authenticate and get the cookie than to do everything else (which is a one-liner in yt-dlp), so I just do it manually once in a while.
I recently found ytdl-sub which is basically a metadata wrapper around yt-dlp to help you save videos in a format library software like Plex etc supports. Rather than having some horribly named videos with metadata files just taking up space.
There's also tube archivist, which is a whole archival front end and back end. Bit more complicated to set up but once you do, you also get a nice web UI of your videos that is similar to YouTube. It even shows comments.
How timely! Just the other day I was looking for an old conference talk I was involved in, that for some reason had been taken down from YouTube. It was a decent amount of work even finding a link to convince myself it had ever existed.
Fortunately, the Wayback Machine had a copy at decent resolution.
YouTube recently deleted a video of mine for violating their ToS (my video just showcased a YouTube DL UI I built). I'm super sad because somehow I did not have a backup as I expected it to last forever in YouTube. I miss the video and it seems it's not archived anywhere
IMO the top motivation and method of using this tool both is when you notice videos in a playlist going dark.
On desktop, you'd view the full playlist and notice "Unavailable videos are hidden" at the top. Click the kebab menu and "Show unavailable videos" and the list will populate grey-icon videos which do still link to the broken video so you can get the ID.
But videos will also be broken in a way that this method does not reveal (it will only show private and deleted videos). Copystruck or georestricted videos you'll only find out once you attempt to view. Use a YouTube playlist downloader to quickly see which those are.
And then there will be videos you know existed but aren't in a playlist--I have relied on googling around for old forum and blog posts that include the link in order to get the original ID, if I didn't click on a broken link in the first place.
Few years ago, I was following a person who did a work van conversion on a nissan nv200, very nice detail and work. Noticed it was missing and unavailable. Looks like he passed away, and they removed his channel. Notice many van/rv rebuild videos are missing. I'd hate to see how many videos get removed on future history, culture, politics and tech.
Tried it twice in the Archive Finder tool: first time had no hits (and the Wayback Machine timed out), but the second time the Wayback Machine didn't time out and returned a download link for me that included an intact video file.
For this particular video I did find a reupload on Niconico when I was originally searching (https://www.nicovideo.jp/watch/sm11640795, if you're curious) but the archive finder tool is a game-changer.
I always wondered how a company could offer free video hosting in exchange for mining them and creating AI. Once they've been mined and they know who watches them, the content becomes superfluous and I suppose it can get deleted. So that is where we are now.
Cultural rot caused by YouTube taking down old videos is a serious problem.
There was a YouTube channel I was following for about 15 years, of an older woman who had pet foxes. Her health started declining and she always had on her channel something like, "I'll leave these videos up forever." She passed away and YouTube deleted all her content.
The problem is 100x worse for any political content. My YouTube bookmarks from just the past few years is a graveyard of information that's not convenient to have around. It's even true for politics-related jokes, songs, and memes. They seem to be selected for deletion more often than not, within a few years.
I have a playlist with 285 videos, and youtube just says "Unavailable videos are hidden".
It's the worst, I have no idea what is missing. I don't even know how much is missing and I am sure the number will keep increasing over time, not even sure they will bother to tell me that there are missing videos at this rate.
If you view your playlist on a desktop browser you can make it show you the unavailable videos in the list. Then you can grab those IDs and try to find them via the Wayback Machine, etc. Hope this helps.
hum I don't see how to do that anymore
It's in the ... menu at the left.
Also Filmot has a userscript for this that searches their archive for metadata and shows thumbnails from the Wayback Machine: https://filmot.com/moresearch
https://github.com/bibanon/tubeup
Docker: https://github.com/etnguyen03/docker-tubeup
Content uploaded to the Internet Archive is stored as <service>-<identifier>. Please don’t abuse, use judiciously. When in doubt, contact Patron Services (info at archive dot org) prior to archival ops. They can also assist with establishing a collection for multiple items if warranted. It costs the Internet Archive $2/GB to store bits in perpetuity; consider donating if it is within your means.
(content uploaded with this will be excluded by Archive search due to spammers and other bad actors, you will have to reference the identifier directly to access or use tools like this post; anything flagged by a copyright holder using the DMCA will be made unavailable, so keeping a local copy might be prudent)
Now you've learned the hard way that bookmarking is not enough, you need to save a local copy as well.
I have, but it's not easy to share those with others, and of course no one is discovering them now
I hate to do the HN thing, but nextcloud has a feature to use yt-dlp (or whatever), feed it a url, it will put it in your folder. You can share the folder or the file, even with the same URI (the hash yt gives videos, I mean).
I've done this by hand a lot - I have probably a half TB of 480p rips from YouTube and Twitter, as well as a bunch from reddit - though reddit has always been a pain in the ass.
I say use something like nextcloud because there's turnkey Linux images available, so you only need a host that supports that feature to get started. Furthermore, once you have the video "local", in the future if some decentralized video sharing service launches you'll be 90% of the way there by merely having canonical links to content available publicly.
Peertube can also do some fanciness: https://docs.joinpeertube.org/use/channel-sync
What happened to the foxes.
Now we'll never know what they say
A while back there was a hacker news story about someone who used AI to clean up the long lost Steely Dan song "the second arrangement". I was looking for it again on YouTube and it's now private, I presume thanks to a complaint from the record company. (There's still a wav file out there thankfully.) I feel like if the company was smart they'd put the track on all the streaming services, maybe even print a 7" 45 of it (vinyl boom anyone?) or include it on a Gaucho remaster... Then it could make some revenue.
I know there are probably a ton of much more academic use cases for this kind of tool, but for me personally, I'm mostly interested to use it to figure out what exactly got removed from a playlist (or liked videos or etc.) because it always bothers me when I can't remember what's no longer there.
Yes. I always feel violated when a Liked video or a Watch Later video is suddenly gone without even the video title available.
I almost feel like I should have a backup of my BB liked or saved videos
A dash of yt-dlp and a cron job. If you've got a media server, it's the way to go for subscribing to content you're interested in.
The tricky bit is that you have to get your youtube cookie in order for yt-dlp to get your own liked videos. I expect it to be a lot more LoC to authenticate and get the cookie than to do everything else (which is a one-liner in yt-dlp), so I just do it manually once in a while.
I solved this by making my own "To watch..." playlist which was available to anyone who had the link.
But I'm one of those "solve the unstable server by rebooting it once a day" guys, so...
If you're logged in on a browser on the same machine you can use the --cookies-from-browser option instead of a cookies.txt
For all the responses to my post, I’m glad I did
Lots to dive into.
I recently found ytdl-sub which is basically a metadata wrapper around yt-dlp to help you save videos in a format library software like Plex etc supports. Rather than having some horribly named videos with metadata files just taking up space.
https://github.com/jmbannon/ytdl-sub
This looks handy thanks!
Naming is definitely the hardest problem in coding and file naming.
Part of me feels like chaining a few of these tools into a docker project.
There's also tube archivist, which is a whole archival front end and back end. Bit more complicated to set up but once you do, you also get a nice web UI of your videos that is similar to YouTube. It even shows comments.
Filmot has a userscript for this that searches their archive for metadata and shows thumbnails from the Wayback Machine: https://filmot.com/moresearch
How timely! Just the other day I was looking for an old conference talk I was involved in, that for some reason had been taken down from YouTube. It was a decent amount of work even finding a link to convince myself it had ever existed.
Fortunately, the Wayback Machine had a copy at decent resolution.
Thanks for sharing. <3
YouTube recently deleted a video of mine for violating their ToS (my video just showcased a YouTube DL UI I built). I'm super sad because somehow I did not have a backup as I expected it to last forever in YouTube. I miss the video and it seems it's not archived anywhere
It's almost definitely in Google takeout if you select your YouTube videos.
But also losing something small like that is a small price to pay for ensuring what you truly care about (photos/documents) are truly backed up.
But didn't you get involved in youtube-dl tooling knowing that nothing can be assumed to last forever in youtube?
Interestingly, my YouTube playlists now say "unavailable videos are hidden" so I can't get the link.
There's a "Show Unavailable Videos" button in the ... menu at the left.
I’ve been looking for something like this for years.
> If you don't have the link, this site won't help you.
Nvm.
https://archive.org/details/movies?tab=collection
You can search the internet archive directly though
I looked through the source code and it doesn’t do anything I haven’t already done. Such is life lol
IMO the top motivation and method of using this tool both is when you notice videos in a playlist going dark.
On desktop, you'd view the full playlist and notice "Unavailable videos are hidden" at the top. Click the kebab menu and "Show unavailable videos" and the list will populate grey-icon videos which do still link to the broken video so you can get the ID.
But videos will also be broken in a way that this method does not reveal (it will only show private and deleted videos). Copystruck or georestricted videos you'll only find out once you attempt to view. Use a YouTube playlist downloader to quickly see which those are.
And then there will be videos you know existed but aren't in a playlist--I have relied on googling around for old forum and blog posts that include the link in order to get the original ID, if I didn't click on a broken link in the first place.
Few years ago, I was following a person who did a work van conversion on a nissan nv200, very nice detail and work. Noticed it was missing and unavailable. Looks like he passed away, and they removed his channel. Notice many van/rv rebuild videos are missing. I'd hate to see how many videos get removed on future history, culture, politics and tech.
Does anyone have a good example video that's no longer around to test with?
I do! Just the other day I was trying to find this: https://www.youtube.com/watch?v=__-NKgcRM2c
Tried it twice in the Archive Finder tool: first time had no hits (and the Wayback Machine timed out), but the second time the Wayback Machine didn't time out and returned a download link for me that included an intact video file.
For this particular video I did find a reupload on Niconico when I was originally searching (https://www.nicovideo.jp/watch/sm11640795, if you're curious) but the archive finder tool is a game-changer.
Not archived across the board for a video I tried.
I always wondered how a company could offer free video hosting in exchange for mining them and creating AI. Once they've been mined and they know who watches them, the content becomes superfluous and I suppose it can get deleted. So that is where we are now.