How I archive YouTube videos
The system I currently have down is this: Go through my subs and set it to notify me via email of new uploads. This makes a target channel easier to maintain it's item on archive.org.
First we gotta set up YouTube DL. We're using Linux/Mac for this, and sourcing the youtube-dl binary from the python repository Pip.
Setting up Youtube-dl
Do not install youtube-dl from Ubuntu/Apple repository. It's probably stale, and may lack key features we need, or buggy and not updated by the maintainer.
We'll use the python Pip repository instead. We'll also be assuming you're using Ubuntu.
- sudo apt-get install ffmpeg h.264 python-pip atomicparsley
- sudo -H pip install --upgrade pip youtube-dl
Using a config
Youtube-dl allows you to use a config to set default behavior. This is useful instead of aliasing in .bashrc because pipes or screen instances or trickle do not recognize aliases when running a command prepend with screen or trickle.
- echo "--download-archive ~/.ytdlarchive --retries 100 --no-overwrites --call-home --continue --write-info-json --write-description --write-thumbnail --write-annotations --all-subs --sub-format srt --convert-subs srt --write-sub --add-metadata -f bestvideo+bestaudio/best --merge-output-format 'mkv' --prefer-ffmpeg --embed-thumbnail" > ~/.config/youtube-dl/config
To temporarily ignore these changes use --ignore-config. To revert back to normal, rm ~/.config/youtube-dl/config.
- sudo -H pip install --upgrade youtube_dl
The upgrade schedule for the script is bi-weekly, so try to check for updates once every week or so, or daily if you're really thirsting for a fix to a problem you've reported. I've seen examples of multiple updates being pushed in a week.
Use the -r flag to limit bandwidth consumption. Example:
- youtube-dl -r 100K [URL]
To rip a video or channel at 100kbps.
Ripping private Google Drive videos
Okay so you got a video shared with you and the owner doesn't alllow downloading. while this method doesn't get a 1:1 copy of the file it's still quite good and much better than nothing or screen capturing the file.
This involves just just a few steps. You'll need:
- The latest youtube-dl.
This downloads the actual video.
- The Chrome extension Cookies.txt
We need to get your Google cookie. We need this because you need to be logged in to see the video, or more accurately youtube-dl needs to be logged in to see the page so it can extract the video.
Go to your email and find the share notification email. This is the easiest method of getting the link we need to feed into youtube-dl.
It should look something like this:
Put that somewhere you can get back to because you'll need it later.
You need to be signed in so Youtube-dl can see the video to extract it. In order for Youtube-dl to sign in we need to use your cookie. With Cookies.txt installed, open a new tab and head to drive.google.com. A new button appeared when you installed cookies.txt. Click this button and it will give you the contents of your cookie. Copy and paste this into a file on your desktop.
I'm calling mine drive-cookie.txt.
Ripping the video
Put youtube-dl on your desktop along with the coolie text file. open cmd or your command line program, navigate to your desktop. Remember that video link sent in the email when the item was shared with you? We need that too.
youtube-dl.exe --continue --cookies drive-cookie.txt -f best https://drive.google.com/file/d/0B_vK3fbXlLEFM19MQl9nT3dBVEE/view
This will use the cookie to sign in to your google account, and download that video.
MAD reuploaders who don't create content
These guys require you only rip their video descriptions and filter them for NicoNico links:
youtube-dl --ignore-config --write-description --skip-download https://www.youtube.com/user/HelloSibho/videos ; cat *.description | grep niconico > ~/sibho.txt
This will get you most of the way to cleaning up the links. There will be some text that has to be manually cut out. However once that's done you can:
cat ~/sibho.txt | xargs | > sibho-processed.txt
And it will be put into a nice string of URLs that can be pasted into the end of tubeup.
Future targets for archival
Kraut And Tea