Tutorial: Experimenting with youtube-dl.exe

dosdan

Active Member
I've found the YouTube download programs to be problematic. What's on offer:
  1. Many programs appear to be riddled with spyware or adware.
  2. Limited in functionality e.g the "free" version will not download Full HD, only 720p.
  3. Not that flexible e.g. a 720p60 file is available in YT, but you end up downloading a 720p30 version.
However there is a free, open-source alternative: youtube-dl. Being open source, the program is open to public scrutiny, so it's a safer prospect than running scumware. I'm going to show you how to use it.

1. Download the windows executable (the Window exe link) on the Downloads page (there's no installation program). I placed this youtube-dl.exe file in its own directory, D:\YouTube_dl. I also appended this directory to my PATH list so I could run it from anywhere. (Control Panel | System | Advanced System Settings | Environment Variables...). Type PATH at the cmd prompt (Search | type cmd ) to see your current path list. Changes in the PATH will not appear until you reboot. (Type EXIT to close a cmd prompt window.)

2. If youtube.dl won't run, it's probably because you haven't got the Microsoft Visual C++ 2010 Redistributable Package (x86) installed. (I already had it.)

3. youtube.exe will make use of the versatile and powerful ffmpeg, if available, to increase its functionality. You should download it and place ffmpeg.exe in a directory in your PATH.

On YT, I mainly work with playlists as I upload soccer matches I've videoed as 4 "quarters". In this tutorial, I'll use a playlist containing 6 small clips (6 soccer saves by a goalie) as it's conveniently small and has 2 different formats which will illustrate some YT downloading problems and work-arounds. You will be able to download the whole playlist, or just 1 or more parts of it. ---https://www.youtube.com/playlist?list=PLvrzxFQc-Vzyi1u35aS4rDAG0HkEG-IcS (I'm adding an incorrect "---" in front of this URL whenever I mention it in the text, as otherwise this website will automatically preload it.)

I'll be using this batch file (adjust lines 2&3 to suit your situation): dl_playlist.bat which is also in the same directory as youtube-dl.exe:

Code:
@echo off
d:
cd\YouTube_dl
If [%3]==[] goto JUST_PLAYLIST

d:\YouTube_dl\youtube-dl.exe -o "%%(playlist)s/%%(playlist)s - %%(title)s.%%(ext)s" --playlist-items "%3" %1=%2
goto END

:JUST_PLAYLIST
d:\YouTube_dl\youtube-dl.exe  -o "%%(playlist)s/%%(playlist)s - %%(title)s.%%(ext)s" "%*"

:END
echo.

Unfortunately, while you can drag-and-drop files onto a batch-file icon on your desktop, you can't paste a string, i.e. a YT URL, onto a batch-file icon. So the pasting of the YT URL has to be done at the cmd prompt.

1. Get to a cmd prompt.
2.Type DL_PLAYLIST. The batch file will complain because you've given it no URL to work with, but it will have changed to D:\YouTube_dl, which will make further work easier. (BTW, using the Tab key will autocomplete the typing of file names. So if you type DL and then press Tab, and DL_PLAYLIST.BAT is in the current directory, its filename will be supplied. If more than 1 filename starting with DL exists in this directory, you may need to press Tab more than once to get the correct match.)
3. Copy the ---https://www.youtube.com/playlist?list=PLvrzxFQc-Vzyi1u35aS4rDAG0HkEG-IcS string to the Windows clipboard.
4. Switch back to the cmd prompt window, use the Up arrow to repeat your last command (DL_PLAYLIST), add a space after it and then paste the URL by either clicking on the icon at the top-left of the cmd window | Edit | Paste, or just by using the Ctrl-V Paste hotkey). The prompt should look like:

Code:
D:\YouTube_dl> DL_PLAYLIST https://www.youtube.com/playlist?list=PLvrzxFQc-Vzyi1u35aS4rDAG0HkEG-IcS

After entering this command, 6 small MP4s should be downloaded. We will be using this download directory a bit (D:\YouTube_dl\Kaiea's Saves), so I suggest opening it in Window Explorer in a 2nd window. Here's what you'll see in the cmd window:

Code:
D:\YouTube_dl>
D:\YouTube_dl>dl_playlist https://www.youtube.com/playlist?list=PLvrzxFQc-Vzyi1u35aS4rDAG0HkEG-IcS
[youtube:playlist] PLvrzxFQc-Vzyi1u35aS4rDAG0HkEG-IcS: Downloading webpage
[download] Downloading playlist: Kaiea's Saves
[youtube:playlist] playlist Kaiea's Saves: Downloading 6 videos
[download] Downloading video 1 of 6
[youtube] 9HzQvow8zF8: Downloading webpage
[youtube] 9HzQvow8zF8: Downloading video info webpage
[youtube] 9HzQvow8zF8: Extracting video information
[youtube] 9HzQvow8zF8: Downloading MPD manifest
[youtube] 9HzQvow8zF8: Downloading MPD manifest
[dashsegments] Total fragments: 7
[download] Destination: Kaiea's Saves\Kaiea's Saves - Save1.f299.mp4
[download] 100% of 20.70MiB in 00:22
[dashsegments] Total fragments: 8
[download] Destination: Kaiea's Saves\Kaiea's Saves - Save1.f140.m4a
[download] 100% of 468.06KiB in 00:01
[ffmpeg] Merging formats into "Kaiea's Saves\Kaiea's Saves - Save1.mp4"
Deleting original file Kaiea's Saves\Kaiea's Saves - Save1.f299.mp4 (pass -k to keep)
Deleting original file Kaiea's Saves\Kaiea's Saves - Save1.f140.m4a (pass -k to keep)
[download] Downloading video 2 of 6
[youtube] BnEn7X3Pr7o: Downloading webpage
[youtube] BnEn7X3Pr7o: Downloading video info webpage
[youtube] BnEn7X3Pr7o: Extracting video information
[youtube] BnEn7X3Pr7o: Downloading MPD manifest
[youtube] BnEn7X3Pr7o: Downloading MPD manifest
[dashsegments] Total fragments: 6
[download] Destination: Kaiea's Saves\Kaiea's Saves - Save2.f299.mp4
[download] 100% of 17.32MiB in 00:18
[dashsegments] Total fragments: 7
[download] Destination: Kaiea's Saves\Kaiea's Saves - Save2.f140.m4a
[download] 100% of 393.46KiB in 00:01
[ffmpeg] Merging formats into "Kaiea's Saves\Kaiea's Saves - Save2.mp4"
Deleting original file Kaiea's Saves\Kaiea's Saves - Save2.f299.mp4 (pass -k to keep)
Deleting original file Kaiea's Saves\Kaiea's Saves - Save2.f140.m4a (pass -k to keep)
[download] Downloading video 3 of 6
[youtube] V5VdLG-q9Is: Downloading webpage
[youtube] V5VdLG-q9Is: Downloading video info webpage
[youtube] V5VdLG-q9Is: Extracting video information
[youtube] V5VdLG-q9Is: Downloading MPD manifest
[youtube] V5VdLG-q9Is: Downloading MPD manifest
[dashsegments] Total fragments: 6
[download] Destination: Kaiea's Saves\Kaiea's Saves - Save3.f298.mp4
[download] 100% of 9.27MiB in 00:10
[dashsegments] Total fragments: 6
[download] Destination: Kaiea's Saves\Kaiea's Saves - Save3.f140.m4a
[download] 100% of 349.09KiB in 00:01
[ffmpeg] Merging formats into "Kaiea's Saves\Kaiea's Saves - Save3.mp4"
Deleting original file Kaiea's Saves\Kaiea's Saves - Save3.f298.mp4 (pass -k to keep)
Deleting original file Kaiea's Saves\Kaiea's Saves - Save3.f140.m4a (pass -k to keep)
[download] Downloading video 4 of 6
[youtube] jj9xuO2t-p0: Downloading webpage
[youtube] jj9xuO2t-p0: Downloading video info webpage
[youtube] jj9xuO2t-p0: Extracting video information
[youtube] jj9xuO2t-p0: Downloading MPD manifest
[youtube] jj9xuO2t-p0: Downloading MPD manifest
[dashsegments] Total fragments: 6
[download] Destination: Kaiea's Saves\Kaiea's Saves - Save4.f298.mp4
[download] 100% of 8.84MiB in 00:10
[dashsegments] Total fragments: 6
[download] Destination: Kaiea's Saves\Kaiea's Saves - Save4.f140.m4a
[download] 100% of 363.13KiB in 00:01
[ffmpeg] Merging formats into "Kaiea's Saves\Kaiea's Saves - Save4.mp4"
Deleting original file Kaiea's Saves\Kaiea's Saves - Save4.f298.mp4 (pass -k to keep)
Deleting original file Kaiea's Saves\Kaiea's Saves - Save4.f140.m4a (pass -k to keep)
[download] Downloading video 5 of 6
[youtube] aBSNt8H9CrE: Downloading webpage
[youtube] aBSNt8H9CrE: Downloading video info webpage
[youtube] aBSNt8H9CrE: Extracting video information
[youtube] aBSNt8H9CrE: Downloading MPD manifest
[youtube] aBSNt8H9CrE: Downloading MPD manifest
[dashsegments] Total fragments: 13
[download] Destination: Kaiea's Saves\Kaiea's Saves - Save5.f298.mp4
[download] 100% of 25.45MiB in 00:27
[dashsegments] Total fragments: 14
[download] Destination: Kaiea's Saves\Kaiea's Saves - Save5.f140.m4a
[download] 100% of 953.43KiB in 00:02
[ffmpeg] Merging formats into "Kaiea's Saves\Kaiea's Saves - Save5.mp4"
Deleting original file Kaiea's Saves\Kaiea's Saves - Save5.f298.mp4 (pass -k to keep)
Deleting original file Kaiea's Saves\Kaiea's Saves - Save5.f140.m4a (pass -k to keep)
[download] Downloading video 6 of 6
[youtube] DjrG3yTQG20: Downloading webpage
[youtube] DjrG3yTQG20: Downloading video info webpage
[youtube] DjrG3yTQG20: Extracting video information
[youtube] DjrG3yTQG20: Downloading MPD manifest
[youtube] DjrG3yTQG20: Downloading MPD manifest
[dashsegments] Total fragments: 7
[download] Destination: Kaiea's Saves\Kaiea's Saves - Save6.f298.mp4
[download] 100% of 9.81MiB in 00:10
[dashsegments] Total fragments: 7
[download] Destination: Kaiea's Saves\Kaiea's Saves - Save6.f140.m4a
[download] 100% of 413.61KiB in 00:01
[ffmpeg] Merging formats into "Kaiea's Saves\Kaiea's Saves - Save6.mp4"
Deleting original file Kaiea's Saves\Kaiea's Saves - Save6.f298.mp4 (pass -k to keep)
Deleting original file Kaiea's Saves\Kaiea's Saves - Save6.f140.m4a (pass -k to keep)
[download] Finished downloading playlist: Kaiea's Saves

youtube-dl.exe will use the output template (-o) specified and create the requested sub-directory, based on the playlist title, if it doesn't already exist. Here's what D:\YouTube_dl\Kaiea's Saves looks like in Windows Explorer:

dl_1.jpg


Here, "Total bitrate" is video+audio, while "Bit rate" is audio only. Explorer truncates the displayed frame-rate, rather than rounding 59.94fps to 60fps.

You can see we have 2 x 1080p60 & 4 x 720p60 clips. In the cmd window, f299 & f298 are the two video stream formats, while f140 is the audio stream format. More on this later.

The [ffmpeg] lines are when FFMPEG is invoked to merge these separate video and audio streams into a final MP4. You can add the -v (verbose) switch to youtube-dl to see more of what's going on. i.e. d:\YouTube_dl\youtube-dl.exe -v -o ...

This will then show the FFMPEG cmd line being used. (youtube-dl won't overwrite an existing "final" file so you'll need either to delete the downloaded files when you make changes, and want to retry, or you'll need to rename the existing files first.) For example, the merge command used for Save1.mp4 is:

Code:
[debug] ffmpeg command line: ffmpeg -y -i 'file:Kaiea'"'"'s Saves\Kaiea'"'"'s Saves - Save1.f299.mp4' -i 'file:Kaiea'"'"'s Saves\Kaiea'"'"'s Saves - Save1.f140.m4a' -c copy -map 0:v:0 -map 1:a:0 'file:Kaiea'"'"'s Saves\Kaiea'"'"'s Saves - Save1.temp.mp4'

The line looks more complex than usual due to the need to handle the apostrophe in Kaiea's.

The batch file is designed to also work with individual files. For example, to download only the 2nd clip in the playlist, you'd use:

Code:
D:\YouTube_dl> DL_PLAYLIST https://www.youtube.com/playlist?list=PLvrzxFQc-Vzyi1u35aS4rDAG0HkEG-IcS 2

However, if you want to download more than one file in the playlist e.g. 2&4 or 1,2&4 you need to surround this part with double inverted commas:
Code:
D:\YouTube_dl> DL_PLAYLIST https://www.youtube.com/playlist?list=PLvrzxFQc-Vzyi1u35aS4rDAG0HkEG-IcS "2,4"

The reason is that DOS treats both "=" and "," as parameter delimiters. To investigate this use BATCH_PARAMETER_PARSING.BAT:
Code:
@echo off
echo %%1 is %1
echo %%2 is %2
echo %%3 is %3
echo %%4 is %4
echo %%* is %*

Try BATCH_PARAMETER_PARSING list=test 1,2
and BATCH_PARAMETER_PARSING list=test "1,2"

The batch file is designed to handle the "=" in the playlist URL without the need for surrounding the URL with double inverted commas.

youtube-dl.exe will automatically continue an aborted download, if you run the same command a 2nd time (assuming it did not finish). To demonstrate this, download the first clip, and while download, press Ctrl-C to pause & abort the download. Do this with the download directory window visible, so you'll see the dynamic changes occurring. For example, in the D:\YouTube_dl\20160911 - Grand Final - Wolves (1) Vs Bayside (0) download directory where were dealing with a very large clip:

Code:
28/01/2017  06:02 PM        10,456,249 20160911 - Grand Final - Wolves (1) Vs Bayside (0) - Qtr1.f298.mp4.part
14/09/2016  02:58 PM               713 20160911 - Grand Final - Wolves (1) Vs Bayside (0) - Qtr1.f298.mp4.part-Frag0
14/09/2016  02:58 PM         2,199,324 20160911 - Grand Final - Wolves (1) Vs Bayside (0) - Qtr1.f298.mp4.part-Frag1
14/09/2016  02:58 PM         2,196,175 20160911 - Grand Final - Wolves (1) Vs Bayside (0) - Qtr1.f298.mp4.part-Frag2
14/09/2016  02:58 PM         2,194,758 20160911 - Grand Final - Wolves (1) Vs Bayside (0) - Qtr1.f298.mp4.part-Frag3
14/09/2016  02:58 PM         2,129,026 20160911 - Grand Final - Wolves (1) Vs Bayside (0) - Qtr1.f298.mp4.part-Frag4
14/09/2016  02:58 PM         1,736,253 20160911 - Grand Final - Wolves (1) Vs Bayside (0) - Qtr1.f298.mp4.part-Frag5
28/01/2017  06:02 PM           523,264 20160911 - Grand Final - Wolves (1) Vs Bayside (0) - Qtr1.f298.mp4.part-Frag6.part

The Qtr1 clip has a video stream downloading in 196 fragments, most of which are about 2MB each, and an audio stream of approx. 180 fragments, each 78kB. Running the same dl_pathlist.bat ---https://www.youtube.com/playlist?list=PLvrzxFQc-Vzyi1u35aS4rDAG0HkEG-IcS1 cmd line again, resumes the download. The individual stream fragments disappear once the download completes.

To see what formats and frame-sizes are available for downloading a particular clip, use the "-F" (case-sensitive) switch. Here are the formats the Save1 clip is offered in:

Code:
format code  extension  resolution note
140          m4a        audio only DASH audio  128k , m4a_dash container, [email protected] (44100Hz)
160          mp4        256x144    DASH video  108k , avc1.4d400b, 30fps, video only
133          mp4        426x240    DASH video  242k , avc1.4d400c, 30fps, video only
134          mp4        640x360    DASH video  635k , avc1.4d401e, 30fps, video only
135          mp4        854x480    DASH video 1155k , avc1.4d4014, 30fps, video only
298          mp4        1280x720   DASH video 3482k , avc1.4d4020, 60fps, video only
299          mp4        1920x1080  DASH video 5791k , avc1.64002a, 60fps, video only
17           3gp        176x144    small , mp4v.20.3,  [email protected] 24k
36           3gp        320x180    small , mp4v.20.3,  mp4a.40.2
43           webm       640x360    medium , vp8.0,  [email protected]
18           mp4        640x360    medium , avc1.42001E,  [email protected] 96k
22           mp4        1280x720   hd720 , avc1.64001F,  [email protected] (best)

This is a 1080p60 clip. The interesting thing is that an individual 128kbps audio version is available, but there is a 192kbps audio stream inside "22", a version that has both sound and video streams, although the video is only 720p. You can force the downloading of a particular format by using the -f switch e.g. -f 22. So to get both 1080p60 video and 192kbps audio, and have FFMPEG automatically merge them, use -f 299+22.

The question then arises of why did youtube-dl not pick this as the best possible quality combo?
According to the youtube-dl documentation:

Since the end of April 2015 and version 2015.04.26, youtube-dl uses -f bestvideo+bestaudio/best as the default format selection.

In plain English, that means:
  • Use the highest-bitrate video-only format.
  • Use the highest-bitrate audio-only format.
  • If no audio-only format is available, instead use the audio stream from the "best" combined format.

In this example, youtube-dl's format default of -f bestvideo+bestaudio/best produces an inferior result to using -f bestvideo+best/bestaudio.

If you want 720p, "-f 22" is problematic too, because, although not shown in the list, it is only 720p30. To get 720p60 video with 192kbps audio, use "-f 298+22".

Note: "-f 298+22" is inefficient in its use of web bandwidth as it downloads the complete f22-format file, but only uses the audio stream part (about 2% of f22). This is the reason why the default for the audio looks for a audio-only format first before considering using a combined format.

The following directory shot summarises these format combinations:

dl_2.jpg


Dan.
 
Last edited:

The latest video from AVForums

Movies Podcast: Star Trek in 4K. Is the new boxset worth it?
Subscribe to our YouTube channel
Support AVForums with Patreon

Top Bottom