UC Berkeley Course Captures

From Archiveteam
Revision as of 20:26, 26 August 2018 by Znak (talk | contribs) (→‎coursecapture.berkeley.edu: Fix anchor link.)
Jump to navigation Jump to search
UC Berkeley Course Captures
Status Offline on March 16, 2017
Archiving status Saved!
Archiving type Unknown
Project source https://www.bamsoftware.com/git/archive-ucberkeley-webcast.git
IRC channel #berklost (on hackint)

On 2017-03-15, the University of California, Berkeley began to delete or put behind authentication nearly a decade's worth of "course captures" (recordings of lectures, audio and video) that had previously been public. There were tens of thousands of files on YouTube and iTunes U, dating from 2006 to spring 2015. The removal affected these public channels that we know of:

Archive Team and others managed to save nearly all of the to-be-removed content from both YouTube and iTunes U. It is all now preserved at the Internet Archive:

The YouTube videos are actually still present at youtube.com, but are now private and unlisted, requiring a CalNet login to view. The playlist metadata that links together the multiple videos of a course is gone. The iTunes U course content is completely gone, with or without authentication.

Since fall 2015, new course captures have been private by default. See #How to preserve recent course captures for information on how you can help preserve them, if you have a CalNet account.


Archive organization

The files archived from YouTube are organized with one item per video, and one collection per playlist. Collection identifiers look like ucberkeley-webcast-playlistid and item identifiers look like ucberkeley_webcast_videoid. Example collection: Sociology 1, 001 - Fall 2012.

The files archived from iTunes U are organized with one item per playlist (iTunes or RSS feed), and multiple audio/video files within the item. Item identifiers look like ucberkeley_webcast_itunesu_itunesid. Example item: Practice of Art 8 - Fall 2014. The filenames are constructed as track. YYYY-mm-dd - title.ext, and characters like / that would cause problems in a filename are hex-escaped, URL style. For example: 04. 2006-01-25 - Lecture%2fDialogue: Intelligence Comes in Many Flavors.mp3. The naming convention is an attempt to make sorting by filename preserve the lecture order, but it doesn't always work, because of occasional missing/duplicate track numbers or dates that are out of order relative to the track numbers (the RSS date doesn't always match the lecture date).

In many cases, there are both YouTube and iTunes U versions of the same course. iTunes U often has both an audio and a video version of the same course.

The results of this archiving effort ended up in the ucberkeley-webcast collection. There are a few other places where related videos have ended up:

How to preserve recent course captures

A screenshot of the CalNet login page at auth.berkeley.edu.
If this login screen looks familiar to you, then you have what it takes to preserve course captures since fall 2015.

We managed to preserve all the content that was public at the time of the shutdown, which was everything up through spring 2015. However, since fall 2015, course captures have been private by default, and are similarly at risk of being lost. Also, a minority of videos before fall 2015 were not made public. If you have a CalNet account (an @berkeley.edu email address), then you have the ability to download and preserve videos.

Requirements:

  • A CalNet account (email address ending in @berkeley.edu).
  • youtube-dl
  • Knowledge of how to run programs from the command line.

See #coursecapture.berkeley.edu for the status of downloading recent course captures.

Set up YouTube OAuth access

In order to download private videos from the command line, you need to enable OAuth authentication for the YouTube API. You only have to do this once. The outcome of this step is a client_secrets.json file.

  1. Sign out of any Google account that may be signed in. (Or use a private browsing window for the following steps.)
  2. Go to https://console.cloud.google.com/projectcreate and sign in with your CalNet credentials. Create a new project. The name doesn't matter. I used archive-ucberkeley-webcast.
  3. Select your new project. (The name of the currently selected project is at the top of the page.)
  4. Go to https://console.cloud.google.com/apis/library/youtube.googleapis.com/ and click Enable to enable the YouTube Data API v3 API.
  5. Go to https://console.cloud.google.com/apis/credentials/consent and set the Project name shown to users. The name doesn't matter; this is only for your own personal use.
  6. Now go to https://console.cloud.google.com/apis/credentials and click Create credentials, OAuth client ID, with type Other. Again, the name doesn't matter.
  7. Back on the Credentials screen, under OAuth 2.0 client IDs, click on the download button. Save the file to client_secrets.json.

Make a list of video URLs

  1. Go to https://coursecapture.berkeley.edu/ and find the course you want.
  2. Copy and paste all the URLs for the course and put them into a plain text file urls.txt, with one URL per line, like this:
    https://www.youtube.com/watch?v=7g7P49utu58
    https://www.youtube.com/watch?v=Xed5Zg7MEFM
    https://www.youtube.com/watch?v=1q7RbmGXyMs

If you are a programmer, you may prefer to work with machine-readable data rather than copy and paste. See coursecapture.berkeley.edu-20180706.json under #Indexes of files for a pre-parsed scrape of https://coursecapture.berkeley.edu/. Here is an example of using jq to extract the video URLs for one course:

jq -r '.courses[]|select(.name == "STAT 20" and .term=="Spring" and .year=="2017").videos[].url' coursecapture.berkeley.edu-20180706.json

Download the videos

In this step you will use the client_secrets.json file to generate an OAuth token, which you will then provide to youtube-dl.

  1. Download youtube-oauth2.py. (This program is also included in the #Archiving scripts. It requires the oauth2client library (pip install oauth2client or apt install python-oauth2client).
  2. Run youtube-oauth2.py once to generate credentials.json. This is an interactive process, but once you have done it, you don't have to do it again until your token expires. You can change the path ~ to wherever you wish to store the client_secrets.json and credentials.json files—just don't put them where you might accidentally upload them.
    python2 youtube-oauth2.py ~/client_secrets.json ~/credentials.json

    You will see a prompt like this:

    Open this URL in a browser, sign in, and paste the code.
    https://accounts.google.com/o/oauth2/auth?scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fyoutube&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&response_type=code&client_id=..client_id..&access_type=offline
    code: 
    Open the URL shown by the program in a web browser, sign in to your CalNet account, approve the access to YouTube, and finally it will show you a code.
  3. Paste the code into the code: prompt. If everything worked, the program will output a line that starts with Authorization: Bearer . This is your temporary OAuth access token. You don't have to save it; now that credentials.json exists, running youtube-dl again will produce an equivalent token, without having to paste a code again.

Finally, run youtube-dl, providing it with your list of URLs. (See the YouTube page for more information about youtube-dl options.) In the command line, we call youtube-oauth2.py again, using the cached credentials.

youtube-dl -o "%(title)s-%(id)s.%(ext)s" --continue --retries 4 --write-info-json --write-description --write-thumbnail --write-annotations --all-subs --ignore-errors -f best -a urls.txt --add-header "$(python2 youtube-oauth2.py ~/client_secrets.json ~/credentials.json)"

When the download is finished, check to make sure there are no .ytdl, .part, or .part-Frag* files among the downloads. Their presence indicates an incomplete download. Delete them and run the download command again. youtube-dl will not re-download any videos that are already complete.

Bulk parallel downloading

download-coursecaptures-parallel.sh is a script that automates the extraction of video URLs from coursecapture.berkeley.edu-20180706.json and runs multiple parallel copies of youtube-dl to download them. (You can get all the necessary files from the repo under #Archiving scripts.) You'll need to set up OAuth as described under #Download the videos.

Status

All the files from both YouTube and iTunes have been preserved and uploaded to the Internet Archive. The "x"s in the tables below are, for the most part, the videos that we downloaded in all available YouTube formats. Others have downloaded more videos (though perhaps in fewer formats); see #Archiving efforts. YouTube comments were not preserved, unless someone else at #Archiving efforts got them.

If you would like to continue the work, see #How to preserve recent course captures. Another thing you can do to help is to find and link course home pages and other materials. Lots of courses have online slides, homeworks, practice exams, etc. At least for EE/CS courses, there's a good archive of past course home pages at https://inst.eecs.berkeley.edu/classes-eecs.html. When you find a course home page, leave a review on the Internet Archive page with a link to it.

coursecapture.berkeley.edu

There is an index of YouTube videos at https://coursecapture.berkeley.edu/ that goes back to Fall 2008. It substantially overlaps with #YouTube playlists, #YouTube videos in semester playlists but not course playlists, and #YouTube videos without playlists, but the coverage is not identical. The YouTube-sourced lists go back farther in time, to 2006; but the coursecapture.berkeley.edu list has videos since fall 2015 and also older videos that were not made public on the YouTube channel.

YouTube playlists

playlist downloaded uploaded
Environ Sci, Policy, and Management C11, 001 - Spring 2015 (22 videos) x archive.org 2
Biology 1A, 001 - Spring 2015 (38 videos) archive.org
Economics 270D, 001 - Spring 2015 (14 videos) x archive.org 2
Computer Science 61B, 002 - Spring 2015 (37 videos) archive.org
Computer Science 61B, 001 - Spring 2015 (0 videos) x archive.org
Computer Science 170, 001 - Spring 2015 (26 videos) archive.org
Computer Science 162, 001 - Spring 2015 (25 videos) x archive.org
Physics 8B, 001 - Spring 2015 (43 videos) archive.org
Statistics 131A, 001 - Spring 2015 (40 videos) x archive.org 2
Computer Science 188, 001 - Spring 2015 (26 videos) archive.org
Biology 1AL, 001 - Spring 2015 (12 videos) x archive.org 2
Computer Science 10, 001 - Spring 2015 (26 videos) archive.org
Political Science 179, 001 - Spring 2015 (10 videos) archive.org
Electrical Engineering C247B, 001 - Spring 2015 (22 videos) archive.org
Computer Science 70, 001 - Spring 2015 (28 videos) archive.org
Psychology 1, 001 - Spring 2015 (23 videos) archive.org
Computer Science 98, 052 - Spring 2015 (12 videos) archive.org
Geography 130, 001 - Spring 2015 (25 videos) x archive.org 2
Computer Science 186, 001 - Spring 2015 (28 videos) archive.org
Public Health 142, 001 - Spring 2015 (38 videos) archive.org
Psychology 131, 001 - Spring 2015 (24 videos) archive.org
Letters and Science C70V, 001 - Spring 2015 (26 videos) archive.org
Computer Science 61C, 001 - Spring 2015 (26 videos) archive.org
Bioengineering 200, 001 - Spring 2015 (5 videos) archive.org
Economics 113, 001 - Spring 2015 (28 videos) archive.org
Media Studies 104A, 001 - Spring 2015 (27 videos) archive.org
Biology 1B, 001 - Spring 2015 (43 videos) x archive.org 2
Computer Science 169, 001 - Spring 2015 (25 videos) archive.org
Environ Sci, Policy, and Management 114, 001 - Spring 2015 (23 videos) archive.org
Psychology 140, 001 - Spring 2015 (23 videos) archive.org
Public Health 241, 001 - Spring 2015 (42 videos) archive.org
Electrical Engineering 123, 001 - Spring 2015 (36 videos) x archive.org 2
International and Area Studies 106, 001 - Spring 2015 (25 videos) archive.org
Computer Science 195, 001 - Spring 2015 (13 videos) x archive.org 2
Computer Science 61A, 001 - Spring 2015 (38 videos) archive.org
Japanese 7B, 001 - Spring 2015 (26 videos) archive.org
Cars (0 videos) x archive.org
Public Health 181, 001 - Fall 2014 (7 videos) archive.org
Electrical Engineering 143, 001 - Fall 2014 (22 videos) archive.org
Political Science 179, 001 - Fall 2014 (9 videos) archive.org
Nuclear Engineering 101, 001 - Fall 2014 (29 videos) archive.org
History 5, 001 - Fall 2014 (25 videos) archive.org
Industrial Engin and Oper Research 191, 001 - Fall 2014 (12 videos) archive.org
Environmental Economics and Policy 145, 001 - Fall 2014 (39 videos) archive.org
Nuclear Engineering 290D, 001 - Fall 2014 (0 videos) x archive.org
Biology 1B, 001 - Fall 2014 (43 videos) archive.org
Computer Science 61C, 001 - Fall 2014 (38 videos) archive.org
Electrical Engineering 227BT, 001 - Fall 2014 (15 videos) archive.org
Nutritional Sciences and Toxicology 10, 001 - Fall 2014 (24 videos) archive.org
Psychology 160, 001 - Fall 2014 (24 videos) archive.org
Environmental Economics and Policy C115, 001 - Fall 2014 (26 videos) archive.org
Computer Science 61B, 003 - Fall 2014 (39 videos) archive.org
Public Health 245, 001 - Fall 2014 (39 videos) archive.org
Astronomy C13, 001 - Fall 2014 (24 videos) archive.org
Computer Science 70, 001 - Fall 2014 (27 videos) archive.org
Philosophy 3, 001 - Fall 2014 (38 videos) x archive.org 2
Astronomy C12, 001 - Fall 2014 (25 videos) archive.org
Agricultural and Resource Economics 213, 001 - Fall 2014 (26 videos) archive.org
Economics 100B, 001 - Fall 2014 (25 videos) archive.org
Computer Science 61A, 001 - Fall 2014 (38 videos) archive.org
Computer Science 170, 001 - Fall 2014 (39 videos) archive.org
Computer Science C149, 001 - Fall 2014 (25 videos) archive.org
Computer Science 61B, 001 - Fall 2014 (41 videos) archive.org
Statistics 21, 001 - Fall 2014 (41 videos) x archive.org
Earth and Planetary Science C20, 001 - Fall 2014 (23 videos) archive.org
Economics 1, 001 - Fall 2014 (26 videos) archive.org
Chemistry 3B, 002 - Fall 2014 (26 videos) archive.org
Legal Studies 103, 001 - Fall 2014 (41 videos) archive.org
American Studies C132B, 001 - Fall 2014 (24 videos) archive.org
English 127, 001 - Fall 2014 (25 videos) archive.org
Computer Science 10, 001 - Fall 2014 (25 videos) archive.org
Practice of Art 8, 001 - Fall 2014 (12 videos) archive.org
History 162A, 001 - Fall 2014 (24 videos) archive.org
Biology 1AL, 001 - Fall 2014 (12 videos) archive.org
Energy and Resources Group C200, 001 - Fall 2014 (26 videos) archive.org
Economics C3, 001 - Fall 2014 (21 videos) archive.org
Bioengineering 200, 002 - Fall 2014 (9 videos) archive.org
Chemical & Biomolecular Engineering 179, 001 - Fall 2014 (35 videos) archive.org
Physics 7A, 001 - Fall 2014 (40 videos) archive.org
Physics 7B, 001 - Fall 2014 (42 videos) archive.org
Sociology 1, 001 - Fall 2014 (29 videos) archive.org
Economics 119, 001 - Fall 2014 (24 videos) archive.org
Cognitive Science C102, 001 - Fall 2014 (27 videos) archive.org
Computer Science 162, 001 - Fall 2014 (36 videos) archive.org
East Asian Languages and Cultures 109, 001 - Fall 2014 (28 videos) archive.org
Japanese 7A, 001 - Fall 2014 (23 videos) archive.org
Chemistry 1A, 009 - Summer 2014 (0 videos) x archive.org
Peace and Conflict Studies 99, 006 - Summer 2014 (1 video) archive.org
East Asian Languages and Cultures 105, 001 - Spring 2014 (0 videos) x archive.org
Architecture 180AC, 001, 001 - Spring 2014 (6 videos) archive.org
Architecture 140, 001 - Spring 2014 (1 video) archive.org
Cognitive Science 98/198 - Fall 2013 (12 videos) archive.org
Electrical Engineering 127, 001 - Spring 2014 (25 videos) archive.org
Edible Education 101 - Spring 2014 (13 videos) archive.org
Biology 1AL, 001 - Spring 2014 (13 videos) archive.org
Electrical Engineering 140, 001 - Spring 2014 (35 videos) archive.org
Computer Science 162, 001 - Spring 2014 (23 videos) archive.org
Economics 172, 001 - Spring 2014 (27 videos) archive.org
Chemistry 1AL, 001 - Spring 2014 (14 videos) archive.org
Biology 1A, 001 - Spring 2014 (38 videos) archive.org
Political Science 179, 001 - Spring 2014 (11 videos) archive.org
American Studies C132B, 001 - Spring 2014 (24 videos) archive.org
Psychology 131, 001 - Spring 2014 (24 videos) archive.org
Computer Science 61B, 001 - Spring 2014 (25 videos) archive.org
Biology 1B, 001 - Spring 2014 (40 videos) archive.org
Computer Science 61C, 001 - Spring 2014 (41 videos) archive.org
Computer Science 61A, 001 - Spring 2014 (38 videos) archive.org
Environ Sci, Policy, and Management 114, 001 - Spring 2014 (24 videos) archive.org
Public Health 241, 001 - Spring 2014 (35 videos) archive.org
Bioengineering 200, 001 - Spring 2014 (9 videos) archive.org
Engineering 45, 001 - Spring 2014 (36 videos) archive.org
Philosophy 135, 001 - Spring 2014 (38 videos) archive.org
Psychology 140, 001 - Spring 2014 (22 videos) archive.org
Statistics 131A, 001 - Spring 2014 (32 videos) archive.org
Computer Science 10, 001 - Spring 2014 (23 videos) archive.org
Psychology 164, 001 - Spring 2014 (25 videos) archive.org
Electrical Engineering 130, 001 - Spring 2014 (26 videos) archive.org
Mathematics 10B, 001 - Spring 2014 (25 videos) archive.org
International and Area Studies 106, 001 - Spring 2014 (27 videos) archive.org
Physics C10, 001 - Spring 2014 (27 videos) archive.org
Cognitive Science C103, 001 - Spring 2014 (26 videos) archive.org
Nutritional Sciences and Toxicology 11, 001 - Spring 2014 (17 videos) archive.org
Mathematics 128A, 001 - Spring 2014 (27 videos) archive.org
Economics C3, 001 - Spring 2014 (28 videos) archive.org
Mechanical Engineering 233 - Spring 2014: Advanced Control Systems II (21 videos) archive.org
Bioengineering 200, 002 - Fall 2013 (5 videos) archive.org
Environ Sci, Policy, and Management 117 Lab - Fall 2013 (6 videos) x archive.org 2
Computer Science 169, 001 - Fall 2013 (26 videos) archive.org
Biology 1AL, 001 - Fall 2013 (9 videos) x archive.org 2
Economics 119, 001 - Fall 2013 (24 videos) archive.org
Practice of Art 8, 001 - Fall 2013 (11 videos) x archive.org 2
Public Health 250A, 001 - Fall 2013 (37 videos) archive.org
Computer Science 162, 001 - Fall 2013 (24 videos) archive.org
Nutritional Sciences and Toxicology 10, 001 - Fall 2013 (23 videos) archive.org
Computer Science 70, 001 - Fall 2013 (26 videos) archive.org
Letters and Science C70V, 001 - Fall 2013 (25 videos) x archive.org 2
Computer Science 170, 001 - Fall 2013 (36 videos) archive.org
Environ Sci, Policy, and Management 117, 001 - Fall 2013 (8 videos) archive.org
Psychology 1, 001 - Fall 2013 (23 videos) archive.org
Biology 1A, 001 - Fall 2013 (39 videos) archive.org
Environmental Economics and Policy C1, 001 - Fall 2013 (28 videos) archive.org
Nuclear Engineering 101, 001 - Fall 2013 (40 videos) archive.org
Psychology 130, 001 - Fall 2013 (25 videos) archive.org
Environmental Economics and Policy C115, 001 - Fall 2013 (26 videos) archive.org
Letters and Science 120C, 001 - Fall 2013 (26 videos) archive.org
Statistics 133, 002 - Fall 2013 (34 videos) archive.org
Computer Science 294, 073 - Fall 2013 (25 videos) archive.org
Physics 8B, 001 - Fall 2013 (42 videos) archive.org
Computer Science 61A, 001 - Fall 2013 (36 videos) archive.org
Computer Science 61B, 001 - Fall 2013 (38 videos) archive.org
Mathematics 16A, 002 - Fall 2013 (27 videos) archive.org
Electrical Engineering 130, 001 - Fall 2013 (28 videos) archive.org
Philosophy 3, 001 - Fall 2013 (27 videos) archive.org
Public Health 250B, 001 - Fall 2013 (25 videos) archive.org
Biology 1B, 001 - Fall 2013 (40 videos) archive.org
Earth and Planetary Science C20, 001 - Fall 2013 (21 videos) archive.org
Physics 151, 001 - Fall 2013 (24 videos) archive.org
Electrical Engineering 140, 001 - Fall 2013 (27 videos) archive.org
Chem 1a, 001 - Spring 2013 (1 video) x archive.org 2
African American Studies 107, 001 - Spring 2013 (5 videos) archive.org
Public Health C242C, 001, Statistics C247C, 00 - Spring 2013 (22 videos) archive.org
Biology 1AL, 001 - Spring 2013 (12 videos) archive.org
Political Science 149F, 001 - Spring 2013 (17 videos) archive.org
Practice of Art 8, 001 - Spring 2013 (11 videos) archive.org
Near Eastern Studies 15, 001 - Spring 2013 (32 videos) archive.org
Mechanical Engineering 106, 001 - Spring 2013 (12 videos) archive.org
Chem 1A, 002 - Spring 2013 (42 videos) archive.org
Energy and Resources Group 280, 001, Energy an - Spring 2013 (27 videos) archive.org
Education 182AC, 001 - Spring 2013 (23 videos) archive.org
History 151C, 001 - Spring 2013 (21 videos) archive.org
Cognitive Science C103, 001, History C192, 001 - Spring 2013 (25 videos) archive.org
Computer Science 61B, 001 - Spring 2013 (38 videos) archive.org
Computer Science 162, 001 - Spring 2013 (24 videos) archive.org
Computer Science 194, 024 - Spring 2013 (25 videos) archive.org
Environ Sci, Policy, and Management 114, 001 - Spring 2013 (24 videos) archive.org
Public Health 241, 001 - Spring 2013 (41 videos) archive.org
Computer Science 61A, 001 - Spring 2013 (39 videos) archive.org
Bioengineering 200, 001 - Spring 2013 (9 videos) archive.org
Electrical Engineering 105, 001 - Spring 2013 (32 videos) archive.org
Psychology 140, 001 - Spring 2013 (22 videos) archive.org
Physics 8A, 002 - Spring 2013 (28 videos) archive.org
Cognitive Science C102, 001, Psychology C129, - Spring 2013 (25 videos) archive.org
Biology 1A, 001 - Spring 2013 (38 videos) archive.org
Biology 1B, 001 - Spring 2013 (40 videos) archive.org
Letters and Science C70V, 001, Physics C10, 00 - Spring 2013 (25 videos) archive.org
Economics 113, 001 - Spring 2013 (25 videos) archive.org
Economics 174, 001, Economics 274, 001 - Spring 2013 (25 videos) archive.org
History 5, 001 - Spring 2013 (26 videos) archive.org
Media Studies 104A, 001 - Spring 2013 (26 videos) x archive.org 2
Public Health 200C2, 101 - Fall 2012 (6 videos) archive.org
Public Health 200C2, 001 - Fall 2012 (11 videos) archive.org
Economics C181, 001, Environmental Economics and - Fall 2012 (1 video) archive.org
Legal Studies 145, 001 - Fall 2012 (5 videos) archive.org
Computer Science 184 - Computer Graphics Lecture Series (39 videos) x archive.org
Bioengineering 200, 002 - Fall 2012 (7 videos) archive.org
Computer Science 169, 001 - Fall 2012 (24 videos) archive.org
History 124A, 001 - Fall 2012 (14 videos) archive.org
Philosophy 135, 001 - Fall 2012 (34 videos) archive.org
Education 140AC, 001 - Fall 2012 (1 video) archive.org
Electrical Engineering 130, 001, Electrical Engi - Fall 2012 (22 videos) archive.org
Economics 140, 001 - Fall 2012 (2 videos) archive.org
Environmental Economics and Policy 145, 001 - Fall 2012 (27 videos) archive.org
Edible Education 103 with Michael Pollan (15 videos) archive.org
Mechanical Engineering 220, 001 - Fall 2012 (25 videos) archive.org
Legal Studies 163, 001 - Fall 2012 (1 video) archive.org
Physics H7C, 001 - Fall 2012 (16 videos) archive.org
Biology 1AL, 001 - Fall 2012 (11 videos) archive.org
Computer Science 162, 001 - Fall 2012 (23 videos) archive.org
Public Health 250B, 001 - Fall 2012 (26 videos) archive.org
Sociology 1, 001 - Fall 2012 (26 videos) archive.org
Computer Science 10, 001 - Fall 2012 (22 videos) archive.org
Electrical Engineering 100, 001, Electrical Engi - Fall 2012 (34 videos) archive.org
Letters and Science C70V, 001, Physics C10, 001 - Fall 2012 (26 videos) archive.org
Computer Science 61A, 001 - Fall 2012 (37 videos) archive.org
Computer Science 61B, 001 - Fall 2012 (37 videos) archive.org
Philosophy 3, 001 - Fall 2012 (36 videos) archive.org
Public Health 290, 001 - Fall 2012 (10 videos) archive.org
Biology 1A, 001 - Fall 2012 (37 videos) archive.org
Computer Science 70, 001 - Fall 2012 (25 videos) archive.org
Chemistry 3B, 002 - Fall 2012 (26 videos) archive.org
Chemistry 1A, 001 - Fall 2012 (39 videos) archive.org
Biology 1B, 001 - Fall 2012 (40 videos) archive.org
Nuclear Engineering 101, 001 - Fall 2012 (29 videos) archive.org
Public Health 250A, 001 - Fall 2012 (39 videos) archive.org
International and Area Studies 106, 001, Interna - Fall 2012 (26 videos) archive.org
Electrical Engineering C245, 001, Mechanical Eng - Fall 2012 (24 videos) archive.org
Electrical Engineering 105, 001 - Fall 2012 (21 videos) archive.org
Earth and Planetary Science C20, 001, Letters an - Fall 2012 (24 videos) archive.org
Economics C3, 001, Environmental Economics and P - Fall 2012 (23 videos) archive.org
Mathematics 128A, 001 - Fall 2012 (26 videos) archive.org
eCHEM 1A: Online General Chemistry (Part 4 of 4) (79 videos) archive.org
eCHEM 1A: Online General Chemistry (Part 3 of 4) (107 videos) archive.org
eCHEM 1A: Online General Chemistry (Part 2 of 4) (121 videos) archive.org
eCHEM 1A: Online General Chemistry (Part 1 of 4) (111 videos) archive.org
Physics 111 (37 videos) archive.org
Journalism 102AC, 001 - Spring 2012 (4 videos) archive.org
Public Sociology, Live! (11 videos) archive.org
Economics 104, 001 - Spring 2012 (16 videos) archive.org
Environ Sci, Policy, and Management 114, 001 - Spring 2012 (23 videos) archive.org
Legal Studies 145, 001 - Spring 2012 (5 videos) archive.org
Biology 1AL, 001 - Spring 2012 (12 videos) archive.org
Astronomy 250, 001 - Spring 2012 (12 videos) archive.org
Computer Science 170, 001 - Spring 2012 (24 videos) archive.org
American Studies 10AC, 001 - Spring 2012 (26 videos) archive.org
Computer Science 162, 001 - Spring 2012 (24 videos) archive.org
Computer Science 61A, 001 - Spring 2012 (40 videos) archive.org
Computer Science 10, 001 - Spring 2012 (25 videos) archive.org
Chemistry 1A, 002 - Spring 2012 (43 videos) archive.org
Public Health 241, 001 - Spring 2012 (41 videos) archive.org
Plant and Microbial Biology 160, 001 - Spring 2012 (37 videos) archive.org
Physics 112, 001 - Spring 2012 (35 videos) archive.org
Civil and Environmental Engineering 120, 001 - Spring 2012 (1 video) archive.org
Biology 1A, 001 - Spring 2012 (37 videos) archive.org
Biology 1B, 001 - Spring 2012 (40 videos) archive.org
Computer Science 188, 001 - Spring 2012 (24 videos) archive.org
Computer Science 164, 001 - Spring 2012 (21 videos) archive.org
Legal Studies 160, 001 - Spring 2012 (26 videos) archive.org
Economics C3, 001, Environmental Economics and - Spring 2012 (25 videos) archive.org
Electrical Engineering 140, 001 - Spring 2012 (20 videos) archive.org
Computer Science 169, 001 - Spring 2012 (26 videos) archive.org
Letters and Science C70V, 001, Physics C10, 00 - Spring 2012 (26 videos) archive.org
Cognitive Science C103, 001, History C192, 001 - Spring 2012 (27 videos) archive.org
Media Studies 104A, 001 - Spring 2012 (26 videos) archive.org
Computer Science C149, 001, Electrical Enginee - Spring 2012 (23 videos) archive.org
Edible Education: The Rise and Future of the Food Movement (12 videos) archive.org
Economics 119, 001 - Fall 2011 (23 videos) archive.org
Mathematics 16B, 001 - Fall 2011 (24 videos) archive.org
Public Health 250A, 001 - Fall 2011 (35 videos) archive.org
Electrical Engineering 141, 001 - Fall 2011 (18 videos) archive.org
Practice of Art 23AC, 001 - Fall 2011 (22 videos) archive.org
Electrical Engineering 105, 001 - Fall 2011 (21 videos) archive.org
History 151C, 001 - Fall 2011 (24 videos) archive.org
Statistics 20, 002 - Fall 2011 (22 videos) archive.org
Mathematics 16A, 002 - Fall 2011 (24 videos) archive.org
Biology 1AL, 001 - Fall 2011 (11 videos) archive.org
Computer Science 162, 001 - Fall 2011 (18 videos) archive.org
Computer Science 10, 001 - Fall 2011 (25 videos) archive.org
Public Health 250B, 001 - Fall 2011 (24 videos) archive.org
Economics 1, 001 - Fall 2011 (24 videos) archive.org
Earth and Planetary Science C20, 001 - Fall 2011 (20 videos) archive.org
Public Health 200C2, 001 - Fall 2011 (10 videos) archive.org
Mathematics 16A, 001 - Fall 2011 (36 videos) archive.org
Electrical Engineering 100, 001 - Fall 2011 (24 videos) archive.org
Computer Science 61B, 001 - Fall 2011 (34 videos) archive.org
Computer Science 61C, 001 - Fall 2011 (35 videos) archive.org
Philosophy 3, 001 - Fall 2011 (36 videos) archive.org
Biology 1B, 001 - Fall 2011 (40 videos) archive.org
Psychology 101, 001 - Fall 2011 (33 videos) archive.org
Computer Science 188, 001 - Fall 2011 (23 videos) archive.org
Chemistry 1A, 001 - Fall 2011 (39 videos) archive.org
Biology 1A, 001 - Fall 2011 (40 videos) archive.org
Chemistry 3B, 002 - Fall 2011 (26 videos) archive.org
Economics C3, 001 - Fall 2011 (23 videos) archive.org
Electrical Engineering C245, 001 - Fall 2011 (25 videos) archive.org
Geography C110, 001 - Fall 2011 (24 videos) archive.org
Astronomy 10, 002 - Fall 2011 (25 videos) archive.org
History C187, 001 - Fall 2011 (20 videos) archive.org
Environmental Economics and Policy 100, 001 - Fall 2011 (25 videos) archive.org
Physics C10, 001 - Fall 2011 (26 videos) archive.org
Statistics 21 - Fall 2009 (25 videos) archive.org
Public Health 253E, 001 - Spring 2010 (14 videos) archive.org
Political Science 179, 001 - Spring 2011 (1 video) archive.org
Sociology 150A - Spring 2009 (26 videos) archive.org
Molecular and Cell Biology C148, 001 - Spring 2011 (18 videos) archive.org
Public Health 253B, 001 - Spring 2011 (10 videos) archive.org
Biology 1AL, 001 - Spring 2011 (12 videos) archive.org
Practice of Art 8, 001 - Spring 2011 (10 videos) archive.org
Electrical Engineering 141, 001 - Spring 2011 (34 videos) archive.org
Economics C3, 001 - Spring 2011 (24 videos) archive.org
Cognitive Science C103, 001 - Spring 2011 (23 videos) archive.org
Public Health 253E, 001 - Spring 2011 (13 videos) archive.org
International and Area Studies 107, 001 - Spring 2011 (21 videos) archive.org
Sociology 185, 001 - Spring 2011 (12 videos) archive.org
Computer Science C149, 001 - Spring 2011 (23 videos) archive.org
Chemistry C234, 001 - Spring 2011 (22 videos) archive.org
Computer Science 61A, 001 - Spring 2011 (40 videos) archive.org
Computer Science 10, 001 - Spring 2011 (22 videos) archive.org
Environ Sci, Policy, and Management 114, 001 - Spring 2011 (21 videos) archive.org
Chemistry 1A, 001 - Spring 2011 (43 videos) archive.org
Public Health 241, 001 - Spring 2011 (38 videos) archive.org
Bioengineering 200, 001 - Spring 2011 (5 videos) archive.org
Practice of Art 23AC, 001 - Spring 2011 (23 videos) archive.org
Computer Science 164, 001 - Spring 2011 (35 videos) archive.org
Cognitive Science C102, 001 - Spring 2011 (24 videos) archive.org
Biology 1B, 001 - Spring 2011 (40 videos) archive.org
Biology 1A, 001 - Spring 2011 (38 videos) archive.org
Electrical Engineering 290C, 001 - Spring 2011 (22 videos) archive.org
Electrical Engineering 140, 001 - Spring 2011 (23 videos) archive.org
Astronomy 7B, 001 - Spring 2011 (20 videos) archive.org
Mechanical Engineering 110, 001 - Spring 2011 (25 videos) archive.org
Electrical Engineering 240, 001 - Spring 2011 (26 videos) archive.org
Public Health 257B - Fall 2009 (13 videos) archive.org
Development Studies C10, 001 - Fall 2010 (11 videos) archive.org
Mathematics 16B, 001 - Fall 2010 (19 videos) archive.org
Computer Science 169, 001 - Fall 2010 (21 videos) archive.org
Computer Science 10, 001 - Fall 2010 (25 videos) archive.org
Public Health 251D, 001 - Fall 2010 (12 videos) archive.org
Public Health 257B, 001 - Fall 2010 (13 videos) archive.org
Biology 1AL, 001 - Fall 2010 (11 videos) archive.org
Computer Science 162, 001 - Fall 2010 (27 videos) archive.org
Biology 1B, 001 - Fall 2010 (40 videos) archive.org
Computer Science 61A, 001 - Fall 2010 (42 videos) archive.org
Electrical Engineering 40, 001 - Fall 2010 (36 videos) archive.org
Statistics 20, 003 - Fall 2010 (34 videos) archive.org
Biology 1A, 001 - Fall 2010 (39 videos) archive.org
Physics C10, 001 - Fall 2010 (26 videos) archive.org
Electrical Engineering C245, 001 - Fall 2010 (26 videos) archive.org
Chemistry 3B, 002 - Fall 2010 (25 videos) archive.org
Economics C181, 001 - Fall 2010 (25 videos) archive.org
History 5, 001 - Fall 2010 (25 videos) archive.org
Economics C3, 001 - Fall 2010 (23 videos) archive.org
Electrical Engineering 141, 001 - Fall 2010 (28 videos) archive.org
Chemistry 1A, 001 - Fall 2010 (43 videos) archive.org
Law 271, Environmental Law and Policy - Fall 2009 (27 videos) archive.org
Using the miniTweezers Optical Trap (7 videos) archive.org
Statistics 21, 001 - Spring 2010 (25 videos) archive.org
Physics C10, 001 - Spring 2010 (26 videos) archive.org
Computer Science 61A, 001 - Spring 2010 (40 videos) archive.org
Biology 1AL, 001 - Spring 2010 (12 videos) archive.org
Chemistry 1A, 002 - Spring 2010 (43 videos) archive.org
Biology 1B, 001 - Spring 2010 (41 videos) archive.org
Biology 1A, 001 - Spring 2010 (39 videos) archive.org
Electrical Engineering 240, 001 - Spring 2010 (29 videos) archive.org
Electrical Engineering 141, 001 - Spring 2010 (26 videos) archive.org
Practice of Art 23AC, 001 - Fall 2009 (0 videos) x archive.org
Mathematics 53, 001 - Fall 2009 (25 videos) archive.org
Law 272.3 - Climate Change and the Law - Fall 2008 (12 videos) archive.org
Law 271.71 - International Environmental Law - Spring 2009 (13 videos) archive.org
Law 270.6 - Energy Regulations and the Env- Spring 2008 (15 videos) archive.org
Law 270.7 - Renewable Energy & Alternative Fuels - Fall 2008 (14 videos) archive.org
Computer Science 61A - Spring 2008 (41 videos) archive.org
Computer Science 61B - Fall 2006 (39 videos) archive.org
Law 271 - Environmental Law and Policy - Spring 2008 (28 videos) archive.org
Psychology 1 - General Psychology - Fall 2007 (25 videos) archive.org
My Top Videos (9 videos) archive.org
Political Science 179 - Spring 2008 (12 videos) archive.org
Electrical Engineering 290f - Spring 2007 (30 videos) archive.org
Peace and Conflict Studies 164A - Fall 2006 (28 videos) archive.org
Peace and Conflict Studies 164B - Spring 2007 (27 videos) archive.org
AST210/EE213 - Soft X-Rays and Extreme Ultraviolet Radiation (28 videos) archive.org
SIMS 141 - Search Engines - Fall 2005 (16 videos) archive.org
Physics 10, 001 - Spring 2006 (26 videos) archive.org
Integrative Biology 131 - General Human Anatomy - Fall 2005 (39 videos) archive.org
Chemistry 3B - Spring 2006 (23 videos) archive.org
ME C117 - Structural Aspects of Biomaterials - Spring 2007 (25 videos) archive.org


These three courses were erroneously split across multiple playlists. As of 2018-02-14, all of the videos had a review at the Internet Archive cross-referencing the other playlists.

playlist downloaded uploaded
Public Health 150E, 001 - Spring 2015 (14 videos) archive.org
Public Health 150E, 001 - Spring 2015 (1 video) archive.org
playlist downloaded uploaded
Computer Science 198, 032 - Spring 2015 (6 videos) archive.org
Computer Science 198, 032 - Spring 2015 (1 video) archive.org
Computer Science 198, 032 - Spring 2015 (1 video) archive.org
Computer Science 198, 032 - Spring 2015 (1 video) archive.org
playlist downloaded uploaded
Cognitive Science C103, 001 - Spring 2015 (13 videos) archive.org
Cognitive Science C103, 001 - Spring 2015 (1 video) x archive.org 2
Cognitive Science C103, 001 - Spring 2015 (1 video) archive.org
Cognitive Science C103, 001 - Spring 2015 (1 video) archive.org
Cognitive Science C103, 001 - Spring 2015 (2 videos) archive.org
Cognitive Science C103, 001 - Spring 2015 (1 video) archive.org
Cognitive Science C103, 001 - Spring 2015 (1 video) archive.org
Cognitive Science C103, 001 - Spring 2015 (3 videos) archive.org
Cognitive Science C103, 001 - Spring 2015 (2 videos) archive.org

There are 28 semester-long playlists whose constituent videos are almost a subset of the individual course videos found above. There are just 61 videos, most of them private, found in the semester-long playlists that are not accounted for above. Except for the exceptions listed in the table below, we don't have to download the videos of these playlists (we already have their metadata).


YouTube videos in semester playlists but not course playlists

There are 61 videos that are found in the semester-long playlists that don't have their own individual course playlist. As of 2018-02-14, all of the videos (except the ones that were private) have a review at the Internet Archive that cross-references the other videos in the course.

video downloaded uploaded
American Studies C132B - Lecture 10: Introduction to pragmat x archive.org 2
American Studies C132B - Lecture 11: : Pragmatism continued, x archive.org 2
American Studies C132B - Lecture 12: Pragmatism continued, W x archive.org 2
American Studies C132B - Lecture 13: Pragmatism review; prag x archive.org 2
American Studies C132B - Lecture 14: Pragmatism review; prag x archive.org 2
American Studies C132B - Lecture 15: : John Dewey as the thi x archive.org 2
American Studies C132B - Lecture 18 : Modernist social scien x archive.org 2
American Studies C132B - Lecture 19: Modernist social scienc x archive.org 2
American Studies C132B - Lecture 20: The heyday of manageria x archive.org 2
American Studies C132B - Lecture 21: Intellectual roots of t x archive.org 2
American Studies C132B - Lecture 22: Intellectual roots of t x archive.org 2
American Studies C132B - Lecture 23: 1968, when what was alr x archive.org 2
American Studies C132B - Lecture 2: Republicanism, liberalis x archive.org 2
American Studies C132B - Lecture 4: American political econo x archive.org 2
American Studies C132B - Lecture 5: American political econo x archive.org 2
American Studies C132B - Lecture 6: The new science of the m x archive.org 2
American Studies C132B - Lecture 7: The new science of the m x archive.org 2
American Studies C132B - Lecture 8: Economic growth and soci x archive.org 2
American Studies C132B - Lecture 9: Economic growth and soci x archive.org 2
Astronomy 7B - Lecture 28 x archive.org 2
Nutritional Sciences and Toxicology 120 - Lecture 4 x archive.org 2
Nutritional Sciences and Toxicology 120 - Lecture 5 x archive.org 2
Nutritional Sciences and Toxicology 120 - Lecture 6 x archive.org 2
Nutritional Sciences and Toxicology 120 - Lecture 7 x archive.org 2
Nutritional Sciences and Toxicology 120 - Lecture 8; Audio p x archive.org 2
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a
Private video n/a

YouTube videos without playlists

playlists-20170307.json and uploads-20170307.json cover almost the same set of videos, but not quite. playlists has 9,881 videos and uploads has 9,897. Their union has 9,953 videos and their intersection has 9,825. There are 56 videos that are in playlists but not uploads, and 72 videos that are in uploads but not playlists.

Of the 72 orphan videos that were in uploads but not playlists:

  • 31 are duplicates or near-duplicates of other videos that were in some existing playlist (and already archived under that playlist). In some cases, title changes like "Full version will be available soon"→"Low audio" suggest that the orphan videos are preliminary versions that were later replaced by better versions.
  • 9 have an existing playlist where they would fit well; it looks like they were just omitted due to an oversight.
  • 32 had no matching existing playlist. Of the 32:
video notes downloaded uploaded
American Studies C132B - 2014-04-16: Critiques of managerial liberalism—th Duplicate of American Studies C132B - 2014-04-16: Critiques of managerial liberalism—th in playlist American Studies C132B, 001 - Spring 2014. x archive.org
American Studies C132B - 2014-11-18: The linguistic turn Duplicate of American Studies C132B - 2014-11-18: The linguistic turn in playlist American Studies C132B, 001 - Fall 2014. x archive.org
Berkeley is fighting back [UC Berkeley] x archive.org
Bioengineering 200 - 2014-05-07 Duplicate of Bioengineering 200 - 2014-05-07 in playlist Bioengineering 200, 001 - Spring 2014. x archive.org
Biology 1AL - 2014-04-21 Duplicate of Biology 1AL - 2014-04-21 in playlist Biology 1AL, 001 - Spring 2014. x archive.org
Biology 1B - 2014-03-05 Duplicate of Biology 1B - 2014-03-05: audio improved in playlist Biology 1B, 001 - Spring 2014. x archive.org
Biology 1B - 2014-04-09 Duplicate of Biology 1B - 2014-04-09 in playlist Biology 1B, 001 - Spring 2014. x archive.org
Biology 1B - 2014-04-23 Duplicate of Biology 1B - 2014-04-23 in playlist Biology 1B, 001 - Spring 2014. x archive.org
CalCentral - Raymond has a block x archive.org
Chemistry 1AL - 2014-04-04: Biofuels 1: Ecotoxicity assay and synthesis Duplicate of Chemistry 1AL - 2014-04-04: Biofuels 1: Ecotoxicity assay and synthesis in playlist Chemistry 1AL, 001 - Spring 2014. x archive.org
Computer Science 162 - 2014-10-31 Duplicate of Computer Science 162 - 2014-10-31 in playlist Computer Science 162, 001 - Fall 2014. x archive.org
Computer Science 61B - 2014-11-26 Duplicate of Computer Science 61B - 2014-11-26 in playlist Computer Science 61B, 001 - Fall 2014. x archive.org
Computer Science 61C - 2014-04-14 Should have been in playlist Computer Science 61C, 001 - Spring 2014. x archive.org
Computer Science 70 - Lecture 1 Duplicate (more like a complete reshoot?) of Computer Science 70 - Lecture 1 in playlist Computer Science 70, 001 - Fall 2013. x archive.org
Computer Science 70 - Lecture 23: Full version will be available soon Duplicate of Computer Science 70 - Lecture 23: Low audio in playlist Computer Science 70, 001 - Fall 2013. x archive.org
Computer Science 70 - Lecture 28: Complete version will be available soon Duplicate of Computer Science 70 - Lecture 28: Low audio in playlist Computer Science 70, 001 - Fall 2013. x archive.org
Computer Science 70 - Lecture 6 Duplicate of Computer Science 70 - Lecture 6 in playlist Computer Science 70, 001 - Fall 2013. x archive.org
CS 61A Lecture 15: Generic Operators I Duplicate of CS 61A Lecture 15: Generic Operators I in playlist Computer Science 61A - Spring 2008. x archive.org
Earth and Planetary Science C20 - 2014-10-30: CS: Deadliest volcanoes Duplicate of Earth and Planetary Science C20 - 2014-10-30: CS: Deadliest volcanoes in playlist Earth and Planetary Science C20, 001 - Fall 2014. x archive.org
Economics 115 - 2014-03-04: Low audio x archive.org
Economics 115 - 2014-04-17: videos removed x archive.org
Economics 115 - 2014-04-22 x archive.org
Economics 115 - 2014-05-01 x archive.org
Economics 1 - 2014-11-10: Multipliers Duplicate of Economics 1 - 2014-11-10: Multipliers in playlist Economics 1, 001 - Fall 2014. x archive.org
Economics C3 - 2014-04-10 Duplicate of Economics C3 - 2014-04-10 in playlist Economics C3, 001 - Spring 2014. x archive.org
Electrical Engineering 130 - Lecture 14 Duplicate of Electrical Engineering 130 - Lecture 14 in playlist Electrical Engineering 130, 001, Electrical Engi - Fall 2012. x archive.org
Electrical Engineering 140 - 2014-04-07: Lecture 28: Last minutes of lectu Duplicate of Electrical Engineering 140 - 2014-04-07: Lecture 28: Last minutes of lectu in playlist Electrical Engineering 140, 001 - Spring 2014. x archive.org
Electrical Engineering 140 - 2014-04-16: Lecture 32: Last minutes of lectu Should have been in playlist Electrical Engineering 140, 001 - Spring 2014. x archive.org
Electrical Engineering 143 - 2014-10-16: Ion Implantation I -- Range & Str Duplicate of Electrical Engineering 143 - 2014-10-16: Ion Implantation I -- Range & Str in playlist Electrical Engineering 143, 001 - Fall 2014. x archive.org
Environmental Economics and Policy C1 - Lecture 3 Duplicate of Environmental Economics and Policy C1 - Lecture 3 in playlist Environmental Economics and Policy C1, 001 - Fall 2013. x archive.org
Environ Sci, Policy, and Management 114 - 2015-02-02 Should have been in playlist Environ Sci, Policy, and Management 114, 001 - Spring 2015. x archive.org
Integrative Biology 131 - Lecture 15: Cardiology Should have been in playlist Integrative Biology 131 - General Human Anatomy - Fall 2005. Still on YT as of 2018-02-25. x archive.org
International and Area Studies 106 - 2014-04-10 Duplicate of International and Area Studies 106 - 2014-04-10 in playlist International and Area Studies 106, 001 - Spring 2014. x archive.org
Japanese 7A - 2014-09-18 Duplicate of Japanese 7A - 2014-09-18 in playlist Japanese 7A, 001 - Fall 2014. x archive.org
Lecture 10: LC, RLC circuits, intro to phasors and impedance EE40, Summer 2010. x archive.org
Lecture 11: Phasors, Impedence, Transfer functions, Filters, Bode plots EE40, Summer 2010. x archive.org
Lecture 12: Transfer Functions, Filters, Bode Plots EE40, Summer 2010. x archive.org
Lecture 13: Bode Plots, Intro to Integrated Circuits and Digital Logic 1 EE40, Summer 2010. x archive.org
Lecture 14: Power, Intro to Integrated Circuits and Digital Logic 2 EE40, Summer 2010. x archive.org
Lecture 16: Power in Digital Circuits, CMOS, Intro to Diodes EE40, Summer 2010. x archive.org
Lecture 17: Tradeoffs in Digital Circuits (Power, Speed, Cost), Diodes EE40, Summer 2010. x archive.org
Lecture 18: Intro to Semiconductor Physics, PN Junctions, Chip Manufacturing EE40, Summer 2010. x archive.org
Lecture 1: Introduction, Ohm's Law, Sign Conventions EE40, Summer 2010. x archive.org
Lecture 2: Kirchoff's Laws, Voltage and Current Divider, Node Voltage Method EE40, Summer 2010. x archive.org
Lecture 3: Node Voltage contd., Superposition, Transformation Theorems, Simulation, Strain Gauge EE40, Summer 2010. x archive.org
Lecture 4: More Node Voltage and Thevenin Equivalents, Intro to Op-Amps EE40, Summer 2010. x archive.org
Lecture 5: Introduction to Op-Amps EE40, Summer 2010. x archive.org
Lecture 6: Op-Amps continued EE40, Summer 2010. x archive.org
Lecture 7: Op-amps in saturation and with positive feedback, MyDaq EE40, Summer 2010. x archive.org
Lecture 8: Capacitors and Inductors , Node Voltage with capacitors and inductors, ODEs EE40, Summer 2010. x archive.org
Lecture 9: DE review, solving RC, RL circuits EE40, Summer 2010. x archive.org
LIVE: Local, Clean and Green Conference 2011 x archive.org
Magnesium Dry Ice Probably should have been in one of the playlists eCHEM 1A: Online General Chemistry (Part 1 of 4), eCHEM 1A: Online General Chemistry (Part 2 of 4), eCHEM 1A: Online General Chemistry (Part 3 of 4), or eCHEM 1A: Online General Chemistry (Part 4 of 4). x archive.org
Master of Advanced Study in Integrated Circuits (MAS-IC) Curriculum x archive.org
Master of Advanced Study in Integrated Circuits (MAS-IC) Program x archive.org
Nuclear Engineering 101 - Lecture 3 Duplicate of Nuclear Engineering 101 - Lecture 3 in playlist Nuclear Engineering 101, 001 - Fall 2013. x archive.org
Nutritional Sciences and Toxicology 10 - 2014-11-04 Duplicate of Nutritional Sciences and Toxicology 10 - 2014-11-04 in playlist Nutritional Sciences and Toxicology 10, 001 - Fall 2014. x archive.org
On The Same Page Freedom's Orator Spot x archive.org
PACS 164B - Lecture 28 Should have been in playlist Peace and Conflict Studies 164B - Spring 2007. x archive.org
Philosophy 3 - 2014-11-19 Duplicate of Philosophy 3 - 2014-11-19 in playlist Philosophy 3, 001 - Fall 2014. x archive.org
Physics C10 - 2014-04-03 Duplicate of Physics C10 - 2014-04-03 in playlist Physics C10, 001 - Spring 2014. x archive.org
Psychology 131 - 2014-04-07 Duplicate of Psychology 131 - 2014-04-07 in playlist Psychology 131, 001 - Spring 2014. x archive.org
Psychology 140 - 2014-04-14 Duplicate of Psychology 140 - 2014-04-14 in playlist Psychology 140, 001 - Spring 2014. x archive.org
Psychology 160 - Lecture 7 x archive.org
Public Health 250A - Lecture 12 Duplicate of Public Health 250A - Lecture 12 in playlist Public Health 250A, 001 - Fall 2013. x archive.org
Public Health 253B - Lecture 3 Should have been in playlist Public Health 253B, 001 - Spring 2011. x archive.org
Sociology 185 - Lecture 13: Cesar Rodriguez-Garavito on Social Minefields in Latin America Should have been in playlist Sociology 185, 001 - Spring 2011. x archive.org
Sociology 185 - Lecture 14: The Class Discusses Global Sociology - What have we Learned? Should have been in playlist Sociology 185, 001 - Spring 2011. x archive.org
The Greek and EU Crisis x archive.org
UC Berkeley ETS - Classroom Audio/Visual Tutorial x archive.org
(Un)Making the Mexican Diabetic: Race, Science, and the Promise of Community Knowledge x archive.org
Welcome to UC Berkeley, Class of 2011! x archive.org

iTunes U

We have downloaded all of the 1,174 iTunes U items, except for two whose files are 404.

Because of its length, the table of iTunes U status is in a subpage: UC Berkeley Course Captures/iTunes Status.

Archiving scripts

Some source code to assist in archiving. The repo also includes #Indexes of files.

git clone https://www.bamsoftware.com/git/archive-ucberkeley-webcast.git

See the README for some usage hints. Although there are some YouTube-related scripts, they weren't actually used to download most of the YouTube videos; that was done by someone else.

Indexes of files

These files contain links to media downloads as well as valuable metadata.

playlists-20170307.json
JSON list of UCBerkeley channel playlists from the YouTube API. Each line is a playlists resource, with the addition of an array of playlistItems resources, which are the individual videos in the playlist.
uploads-20170307.json
JSON list of all uploads of the UCBerkeley channel from the YouTube API. The format is the same as playlists-20170307 (there is only one line because "uploads" is treated as its own playlist). playlists-20170307.json and uploads-20170307.json almost completely overlap in the videos they contain, but there are about 125 videos that are only in one or the other.
https://gist.github.com/Wundark/5a56ee2c9e49d441646ad2a6e7a2c0c0
List of YouTube videos, from a Reddit thread.
id354813951.tar.xz(missing a few videos) id354813951_2.tar.xz
Index of iTunes files. To download the video/audio files for a lecture, first fetch the URLs containing downloadTrack from course.json. This returns some XML containing a second URL (and some metadata) which points to the actual download location. All these requests need to use the iTunes user agent string ("iTunes/12.5" works).
itunes-minus-youtube-20170304.txt
List of 729 iTunes downloads that don't seem to be among the YouTube playlists (by comparison of course titles). It was produced like this:
jq -j '.id,"\t",.snippet.title,"\n"' indexes/playlists-20170307.json | sort | uniq > youtube.txt
tar -O -xf indexes/id354813951_2.tar.xz --wildcards -- '*/course.json' | jq -j '.storePlatformData."product-dv-product".results[]|(.id,"\t",.name,"\n")' | sort | uniq > itunes.txt
./dedup-youtube-itunes.py youtube.txt itunes.txt > indexes/itunes-minus-youtube.txt
./dedup-youtube-itunes.py youtube.txt itunes.txt
id354813951_2-rss.txt
RSS feed URLs for each iTunes item. These RSS feeds download from wbe-itunes.berkeley.edu rather than from iTunes directly, which is reportedly faster.
webcast.berkeley.edu-series-20170301.html.gz
HTML of http://webcast.berkeley.edu/series on 2017-03-01. The page is dynamically generated using JavaScript, so the HTML is taken from the inspector in a browser after the page has loaded. The page contains links to YouTube and iTunes.
coursecapture.berkeley.edu-20180706.json
A parsed crawl of https://coursecapture.berkeley.edu/ on 2018-07-06. This is an index of YouTube links for courses, including post-2015 ones. The format is:
{
  "date": "2018-02-25 22:10:40",
  "courses": [
    {
      "name": "BIOLOGY 1A",
      "term": "Spring",
      "year": "2010",
      "section": "001",
      "instructors": "Gary L. Firestone, Jasper D. Rine, Jennifer A Doudna",
      "videos": [
        {
          "title": "Biology 1A - Lecture 1: Course introduction. Introduction to macromolecules.",
          "url": "https://www.youtube.com/watch?v=DRiokVd-ID8"
        },
        ...
      ]
    }
  ]
}

Sample commands for working with JSON indexes (using jq):

jq -j '.id,"\t",.snippet.title,"\n"' indexes/playlists-20170307.json
Extract all playlist IDs and titles. Convert an ID into a URL as: https://www.youtube.com/playlist?list=id.
jq -r '.items[].snippet.resourceId.videoId' indexes/playlists-20170307.json
Get a list of all video IDs in playlists-20170307.json. Convert an ID into a URL as: https://www.youtube.com/watch?v=id.
jq -r '.items[].snippet.resourceId.videoId' indexes/uploads-20170307.json
Get a list of all video IDs in uploads-20170307.json.

Unlisted/private captures

Since after Spring 2015, course captures have been by default unlisted or private behind authentication. Course captures from past semesters (going back to 2008) are indexed (but require a CalNet login) at

Course captures from current courses (may only be available to students actually in those courses) are at

Videos that don't seem to be listed in any of the other lists:

Auxiliary information for courses for which we otherwise have videos:

  • The "Webcast Departmental" YouTube channel hosts some unlisted videos—though they are, of course, unlisted, even when authenticated, so you can't get a list of them.
  • The "Webcast-legacy Departmental" YouTube channel hosts some old videos, not sure what their relation to the rest is.

Archiving efforts

Shutdown notice

2017-03-01

http://news.berkeley.edu/2017/03/01/course-capture/[IAWcite.todayMemWeb]

Cathy Koshland, UC Berkeley vice chancellor for undergraduate education, sent this message to the campus community today:

Dear Campus Community,

I wanted to share with you the decision to restrict access to our legacy Course Capture (classroom lecture) videos and podcasts, currently searchable at webcast.berkeley.edu and found on YouTube and UC Berkeley iTunesU, to members of the campus community.

As part of the campus’s ongoing effort to improve the accessibility of online content, we have determined that instead of focusing on legacy content that is 3-10 years old, much of which sees very limited use, we will work to create new public content that includes accessible features. Our public legacy libraries on YouTube and iTunesU include over 20,000 publications. This move will also partially address recent findings by the Department of Justice which suggests that the YouTube and iTunesU content meet higher accessibility standards as a condition of remaining publicly available. Finally, moving our content behind authentication allows us to better protect instructor intellectual property from “pirates” who have reused content for personal profit without consent.

Since fall 2015 we have piloted publishing all of our Course Capture content behind CAS/CalNet authentication. This strategy has enhanced our ability to accommodate students and UC Berkeley community members who have demonstrated an accessibility need, and we have concluded that authentication is an intervention that is appropriately responsive to the Berkeley community.

We will continue to evaluate the role of online Course Capture and distribution in tandem with advances in technology befitting the No. 1 public institution in the country. Berkeley will maintain its commitment to sharing content to the public through our partnership with EdX (edx.org). This free and accessible content includes a wide range of educational opportunities and topics from across higher ed.

Beginning March 15, 2017, access to iTunesU course content will be suspended. On the same day we will begin the process of moving the publicly offered YouTube content made from the current legacy channel [youtube.com/ucberkeley] to a new authentication login required channel. The entire process is expected to take three to five months. During this time the ETS team will migrate the videos into the new channel behind CalNet/CAS authentication. Berkeley users seeking to view this older content will be able to access it by logging into YouTube with their bConnected/Google-supported identity.

To help manage the instructional impact, instructors with legacy content have been contacted. Instructors utilizing the ETS Course Capture service since fall 2015 will experience no changes in viewing or accessing content.

Enrolled Berkeley students requiring accommodations will continue to receive support through the Disabled Students Program.

Finally, as we continue to strive for inclusion and effective teaching and learning for all members of the campus community, we encourage you to reference a new campus website designed to help instructors identify best practices and techniques in creating accessible course content for all users: accesscontent.berkeley.edu.

For additional information, please review this FAQ document.

2017-02-24 http://news.berkeley.edu/2017/02/24/faq-on-legacy-public-course-capture-content/[IAWcite.todayMemWeb]

Here is additional information to assist the campus community and the public with upcoming changes to UC Berkeley’s library of legacy public Course Capture (classroom lecture) content from webcast.berkeley.edu, located on YouTube and UC Berkeley iTunesU.

Who uses this content? How much of the content is used/watched?
Course recordings are a study-tool for current students. Results from a recent review of our legacy (2006-2015) public course recordings on YouTube show that the average video is watched for less than eight minutes.
Who are the “pirates” mentioned in the CalMessage?
Pirates is a term used to describe websites that embed YouTube content without the permission of the original copyright holder for profit. UC Berkeley legacy Course Capture content has been discovered on for-profit websites, which use either a subscription fee or on-page advertising.
Why now? Is this related to the DOJ letter?
UC Berkeley stopped posting course lecture videos publicly through webcast.berkeley.edu in 2015 as a way to reduce costs and increase adoption. However, we left legacy content from 2006-2015 in place. The Department of Justice letter indicates that they believe our legacy Course Capture content from webcast.berkeley.edu and located on YouTube and iTunesU is in violation of the Americans with Disabilities Act. We are removing the legacy webcast.berkeley.edu content from public access to focus on making future public content more accessible. Instructors are encouraged to reference accesscontent.berkeley.edu for best practices and resources for making course content accessible.
If we don’t add captions and descriptions, what happens?
Failure to meet the expectations of the Department of Justice could mean potential legal and financial ramifications.
What about current students who need captioning?
ETS and the Disabled Students Program (DSP) have been partnering over the last several years to identify courses requiring captioning based on student need. The partnership and support of students working with DSP will continue.
What will happen to the recordings?
Beginning March 15, 2017, iTunesU Course Capture content will be removed. You may continue to use/download course capture content until that date. Other content in this location such as events, KALX and Public Affairs content will remain available after March 15. On the same day ETS will begin moving the publicly offered YouTube course capture content from the current legacy channel [youtube.com/ucberkeley] to a new authentication login-required channel. The entire process is expected to take three to five months. Berkeley users seeking to view this older content will be able to access it by logging into YouTube with their bConnected/Google supported identity. Instructors with course recordings on YouTube recorded fall 2015 or later will experience no change. Individual video URLs (links) will remain unchanged. Instructors currently using impacted recordings are encouraged to contact the Course Capture team to identify ways to mitigate any effect on their courses: coursecapture@berkeley.edu
How long will videos be interrupted?
The entire process to migrate the public YouTube videos from their current location to a new YouTube channel that will be accessible with campus member’s bConnected/Google supported identity will take 8-10 weeks and begin on March 15, 2017. Each video will be unavailable on bCourses for 2-3 business days. If you are a current instructor using impacted legacy recordings please contact the Course Capture team to review your needs: coursecapture@berkeley.edu
If I have other videos that I want to get captioned or audio described, how would I do that?
While speech-to-text tools continue to improve, effective captioning remains a very manual process. The UC System has recently introduced contracts with several vendors to provide captioning services.The vendor transcribes a recording and adds the text to the appropriate YouTube video, or a transcriber may be hired to caption an event live. At UC Berkeley, content created/captured by Berkeley Video and Berkeley AV is now being captioned. Information on audio description best practices are available at: https://webaccess.berkeley.edu/resources/tips/audio-description and https://webaccess.berkeley.edu/ask-pecan/descriptive-audio
I’m using the impacted recordings (iTunesU or spring 2015 or earlier YouTube content) in my course now. What should I do?
ETS is working hard to mitigate impacts to current instruction. If you already have a list of your video links, you have no additional steps to take. Video URLs will remain unchanged. If you need assistance or have additional concerns, please contact the Course Capture team to review your needs: coursecapture@berkeley.edu
I am an instructor who is using impacted recordings (iTunesU or spring 2015 or earlier YouTube content) for something outside of UC Berkeley. What should I do?
If you are an instructor using legacy recordings currently available to the public as an extension of your research or teaching, please contact the Course Capture team: coursecapture@berkeley.edu
Why was the public not notified before webcast.berkeley.edu content disappeared so that we had a chance to download iTunes legacy content?
We added notifications to our sites and provided a warning before content began to be removed. The legacy content on webcast.berkeley.edu located on YouTube and UC Berkeley’s iTunes U is three to ten years old.
I am a Berkeley instructor who wants to use old content in my class, where can I find the URL to share with my students?
Before videos are migrated: Instructors can copy/paste their YouTube links for future reference. Link URLs will remain unchanged. Educational Technology Services (ETS) is working to modify webcast.berkeley.edu so that videos are accessible to UC Berkeley CalNet users starting in April Instructors with immediate questions can contact the Course Capture team: coursecapture@berkeley.edu
Can I get a copy of my old lectures from YouTube to use personally?
Currently, ETS doesn’t have a service that provides copies of recordings to individuals.
I am a Berkeley CalNet user, so why can’t I search for videos and playlists that I used to be able to see on webcast.berkeley.edu?
The process that allows us to place the videos behind authentication removes playlists and content search options. ETS is working to provide campus users a new website that will function as a directory of recordings that should launch sometime in April on the existing webcast.berkeley.edu site.
Can I still find previous events and other non-Course Capture recordings on YouTube?
The public UC Berkeley Events Channel (youtube.com/ucberkeleyevents) will continue to be available. Many recordings at this location are already captioned and plans are in place to caption future content.