kumi/yt-dlp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
18919 commits 1 branch 0 tags 49 MiB
Commit graph

290 commits

Author SHA1 Message Date
pukkandan
0d1bb027aa
Move option warnings to YoutubeDL 
Previously, these warnings did not obey `--no-warnings` and did not output colors
 2021-05-03 22:49:04 +05:30
pukkandan
4cd0a709aa
Fix preload_download_archive writing verbose message to stdout 
* And move it after all deprecated warnings
 2021-05-03 22:49:03 +05:30
pukkandan
1815d1028b
[zee5] Fix py2 compatibility 2021-05-03 22:49:03 +05:30
The Hatsune Daishi
0fa9a1e236
[whowatch] Add extractor #292 
closes #223

Authored by: nao20010128nao 
Modified from: 9e4a0e061a/youtube_dl/extractor/whowatch.py
 2021-05-02 19:43:37 +05:30
pukkandan
eb55bad5a0
[aria2c] Fix whitespace being stripped off 
Closes #276
 2021-05-02 14:03:13 +05:30
pukkandan
cc0ec3e161
Do not strip out whitespaces in -o and -P 
Related: https://github.com/yt-dlp/yt-dlp/issues/276#issuecomment-827361652
 2021-05-02 14:03:12 +05:30
pukkandan
80185155a1
[ukcolumn] Add Extractor 
Closes #287
 2021-05-02 13:57:50 +05:30
pukkandan
c755f1901f
[CBS] Improve _VALID_URL to support movies 
Closes #290
Tested by: BeeMuffins
 2021-05-01 21:32:14 +05:30
pukkandan
68b91dc905
[youtube] Add oembed to reserved names 2021-05-01 21:24:31 +05:30
pukkandan
88f06afc0c
[rmcdecouverte] Improve _VALID_URL 
Closes #291
 2021-05-01 21:24:31 +05:30
CXwudi
40078a55e2
[niconico] Fix bug in thumbnail extraction #289 
Bug from: 6b1d8c1e30
Authored by: CXwudi
 2021-05-01 19:35:47 +05:30
pukkandan
d2558234cf
[utils] Escape URL while sanitizing 
Closes #263

While this fixes the issue in question, it does not try to address the root-cause of the problem
Refer: 915f911e36, f5fa042c82
 2021-04-29 05:20:50 +05:30
pukkandan
f5fa042c82
Revert "[utils] Encode URLs in YoutubeDLCookieProcessor" 
This reverts commit 915f911e36.

When the request is copied, `unredirected_hdrs` are not copied, which causes issues elsewhere
Reopens #263
 2021-04-29 05:20:18 +05:30
pukkandan
07e4a40a9a
[crackle] Improve extraction (See desc) 
Closes #282

* Refactor authorization as an extension to `_download_json`
* Better error messages and warnings
* Respect `--ignore-no-formats-error`
* Extract subtitles from manifests
* Try with crackle's geo-location service if all hard-coded countries fail
 2021-04-29 05:20:16 +05:30
pukkandan
e28f1c0ae8
[cleanup] Fix linter and some typos 
* Also remove inconsistent use of `"` in setup.py
 2021-04-28 19:59:40 +05:30
pukkandan
ef39f8600a
[curiositystream] Fix collections 
Closes #277

* A bug with authentication was reported in <https://github.com/yt-dlp/yt-dlp/issues/277#issuecomment-828254721> but cannot be tested without an account
 2021-04-28 19:29:33 +05:30
pukkandan
2291dbce2a
[niconico] Fix HLS formats 
Closes #171

* The structure of the API JSON was changed
* Smile Video seems to be no longer available. So remove the warning
* Move ping to downloader
* Change heartbeat interval to 40sec
* Remove unnecessary API headers

Authored-by: CXwudi, tsukumijima, nao20010128nao, pukkandan
Tested by: tsukumijima
 2021-04-28 19:18:29 +05:30
pukkandan
58f197b76c
Revert "[core] be able to hand over id and title using url_result" 
This reverts commit 0704d2224b.

This is a commit from `youtube-dlc`. It is not clear what the original purpose of this was. It seems to be a way for extractors to pass `title` and `id` through when the entry is processed by another extractor

* But `title` can already be passed through using `url_transparent`
* `id` is never supposed to be passed through since it could cause issues with archiving
 2021-04-28 19:18:06 +05:30
pukkandan
895b0931e5
[youtube:tab] Detect playlists inside community posts 2021-04-28 19:18:06 +05:30
pukkandan
1ad047d0f7
[nebula] Move to nebula.app 
Closes #272
Tested by: Lamieur
 2021-04-28 19:18:06 +05:30
pukkandan
be6202f12b
Subtitle extraction from streaming media manifests #247 
Authored by fstirlitz
Modified from: https://github.com/ytdl-org/youtube-dl/pull/6144

Closes: #73
Fixes:
https://github.com/ytdl-org/youtube-dl/issues/6106
https://github.com/ytdl-org/youtube-dl/issues/14977
https://github.com/ytdl-org/youtube-dl/issues/21438
https://github.com/ytdl-org/youtube-dl/issues/23609
https://github.com/ytdl-org/youtube-dl/issues/28132

Might also fix (untested):
https://github.com/ytdl-org/youtube-dl/issues/15424
https://github.com/ytdl-org/youtube-dl/issues/18267
https://github.com/ytdl-org/youtube-dl/issues/23899
https://github.com/ytdl-org/youtube-dl/issues/24375
https://github.com/ytdl-org/youtube-dl/issues/24595
https://github.com/ytdl-org/youtube-dl/issues/27899

Related:
https://github.com/ytdl-org/youtube-dl/issues/22379
https://github.com/ytdl-org/youtube-dl/pull/24517
https://github.com/ytdl-org/youtube-dl/pull/24886
https://github.com/ytdl-org/youtube-dl/pull/27215

Notes:
* The functions `extractor.common._extract_..._formats` are still kept for compatibility
* Only some extractors have currently been moved to using `_extract_..._formats_and_subtitles`
* Direct subtitle manifests (without a master) are not supported and are wrongly identified as containing video formats
* AES support is untested
* The fragmented TTML subtitles extracted from DASH/ISM are valid, but are unsupported by `ffmpeg` and most video players
    * Their XML fragments can be dumped using `ffmpeg -i in.mp4 -f data -map 0 -c copy out.ttml`.
        Once the unnecessary headers are stripped out of this, it becomes a valid self-contained ttml file
    * The ttml subs downloaded from DASH manifests can also be directly opened with <https://github.com/SubtitleEdit>
* Fragmented WebVTT files extracted from DASH/ISM are also unsupported by most tools
    * Unlike the ttml files, the XML fragments of these cannot be dumped using `ffmpeg`
    * The webtt subs extracted from DASH can be parsed by <https://github.com/gpac/gpac>
    * But validity of the those extracted from ISM are untested
 2021-04-28 19:02:43 +05:30
Felix S
e8f834cd8d [threeqsdn] Extract subtitles from streaming manifests 2021-04-28 17:24:50 +05:30
Felix S
e0e624ca7f [canvas] Extract subtitles from streaming manifests 2021-04-28 17:24:19 +05:30
Felix S
ec4f374c05 [wat] Extract subtitles from streaming manifests 2021-04-28 17:24:08 +05:30
Felix S
c811e8d8bd [atresplayer] Extract subtitles from streaming manifests 2021-04-28 17:23:56 +05:30
Felix S
b2cd5da460 [francetv] Extract subtitles from the HLS manifest 2021-04-28 17:23:47 +05:30
Felix S
2de3b21e05 [uplynk] Extract subtitles from HLS manifests 2021-04-28 17:23:37 +05:30
Felix S
4bed436371 [twitter] Extract subtitles from HLS manifests 2021-04-28 17:23:27 +05:30
Felix S
efe9dba595 [srgssr] Extract subtitles from HLS manifests 2021-04-28 17:23:16 +05:30
Felix S
47f4203dd3 [nytimes] Extract subtitles from HLS manifests 2021-04-28 17:23:05 +05:30
Felix S
015c10aeec [roosterteeth] Use common code for subtitle extraction 2021-04-28 17:22:56 +05:30
Felix S
a00d781b73 [elonet] Use common code for subtitle extraction 2021-04-28 17:22:45 +05:30
Felix S
0c541b563f [tv4] Extract subtitles from streaming manifests 2021-04-28 17:22:36 +05:30
Felix S
64a5cf7929 [byutv] Extract subtitles from streaming manifests 2021-04-28 17:22:27 +05:30
Felix S
7a450a3b1c [generic] Extract subtitles from direct SSTR manifest links 2021-04-28 17:22:18 +05:30
Felix S
7de27caf16 [generic] Extract subtitles from direct DASH manifest links 2021-04-28 17:22:07 +05:30
Felix S
c26326c1be [generic] Extract subtitles from direct HLS manifest links 2021-04-28 17:21:55 +05:30
Felix S
66a1b8643a [downloader/ism] Support muxing TTML subtitles 2021-04-28 17:21:45 +05:30
Felix S
15828bcf25 [downloader/hls] Handle MPEG-2 PES timestamp overflow 2021-04-28 17:21:35 +05:30
Felix S
333217f43e [downloader/hls] Remove duplicate cues using a sliding window of candidates 2021-04-28 17:21:26 +05:30
Felix S
4a2f19abbd [downloader/hls] Assemble single-file WebVTT subtitles from HLS segments 2021-04-28 17:21:14 +05:30
Felix S
f6a1d69a87 [extractor/common] Extend _extract_akamai_formats to also extract subtitle tracks 2021-04-28 17:20:29 +05:30
Felix S
fd76a14259 [extractor/common, downloader/ism] Extract SSTR subtitle tracks 
_parse_ism_formats was extended into _parse_ism_formats_and_subtitles;
all direct users were updated, though _extract_ism_formats was left
as a compatibility wrapper.

The SSTR downloader was also modified in order to prepare for muxing
subtitle streams, although no support for any subtitle codecs was
added in this commit.
 2021-04-28 17:20:20 +05:30
Felix S
171e59edd4 [extractor/common] Extract DASH subtitle tracks 
_extract_mpd_formats and _parse_mpd_formats were extended into
_…_formats_and_subtitles; wrappers with old names are provided
for compatibility.
 2021-04-28 17:20:11 +05:30
Felix S
a0c3b2d5cf [extractor/common] Extract HLS subtitle tracks 
_extract_m3u8_formats is renamed to _extract_m3u8_formats_and_subtitles
and extended to handle subtitle tracks instead of skipping them;
a wrapper with the old name is provided for compatibility.

_parse_m3u8_formats is likewise renamed and extended, but without adding
the compatibility wrapper; the test suite is adjusted to test the enhanced
method instead.
 2021-04-28 17:19:57 +05:30
Felix S
19bb39202d [extractor/common] Generalise _merge_subtitles 
This allows modifying a subtitles dictionary in-place.
 2021-04-28 17:19:46 +05:30
Felix S
d4553567d2 [downloader/ism] Prevent writing the header again when resuming an interrupted download 2021-04-28 17:19:37 +05:30
Felix S
4d49884c58 [downloader/fragment] Allow persisting extra state when a download is interrupted 2021-04-28 17:19:31 +05:30
Felix S
5873d4ccdd [utils] Improve bug_report_message 
Add an optional argument specifying the text that should go before
the message.
 2021-04-28 17:19:23 +05:30
Hadi0609
db9a564b6a
[zee5] Fix extraction for some URLs (#279) 
Closes: #278
 2021-04-28 14:51:54 +05:30
Felix S
c72967d5de
[mediasite] Generalize URL pattern (#275) 
Authored by: fstirlitz
 2021-04-26 17:23:20 +05:30
pukkandan
598d185db1
Fix case sensitivity of format selector 
Bug introduced in f8d4ad9ab0
 2021-04-26 10:56:56 +05:30
pukkandan
b982cbdd0e
[limelight] Obey allow_unplayable_formats 2021-04-26 10:56:55 +05:30
pukkandan
6a04a74e8b
[FormatSort] Fix for when some formats have quality and others don't 2021-04-26 10:56:54 +05:30
pukkandan
88728713c8
Py2 compatibility for FileNotFoundError 2021-04-26 10:56:53 +05:30
CXwudi
6b1d8c1e30
[niconico] Fix title and thumbnail extraction (#273) 
Authored by: CXwudi
 2021-04-26 08:23:57 +05:30
Ashish
87c3d06271
[Mxplayer] Add MxplayerShowIE (#270) 
Authored by: Ashish0804
 2021-04-26 08:12:51 +05:30
pukkandan
915f911e36
[utils] Encode URLs in YoutubeDLCookieProcessor 
Closes #263
 2021-04-24 19:20:07 +05:30
pukkandan
cf9d6cfb0c
[tubi] Raise "no video formats" error when video url is empty 
Related: #266
 2021-04-24 17:52:33 +05:30
pukkandan
bbed5763f1
[francetvinfo] Improve video id extraction 
Closes #261
 2021-04-23 00:01:09 +05:30
pukkandan
ca0b91b39e
[version] update :ci skip all 2021-04-22 17:30:36 +05:30
pukkandan
e58c22a0f6
[documentation] Fix typos 2021-04-22 16:54:44 +05:30
pukkandan
1bdae7d312
Update to ytdl-commit-7e8b3f9 
[youtube] Remove unused code
7e8b3f9439
 2021-04-22 16:54:07 +05:30
Felix S
a471f21da6
[mildom] Remove proxy (#260) 
Closes #251
Makes 2cff495997, ab406a1c0e, #252 obsolete

Authored by: fstirlitz
 2021-04-22 16:52:22 +05:30
pukkandan
6efb071135
[BilibiliChannel] Fix pagination 
Closes #222

ccca21d7f5
Coauthored by: nao20010128nao, pukkandan
 2021-04-22 04:19:33 +05:30
pukkandan
a439a3a45c
Improve output template (see desc) 
* Objects can be traversed like `%(field.key1.key2)s`
* A number can be added to the field as `%(field+n)s`
* Deprecates `--autonumber-start`
 2021-04-22 04:19:33 +05:30
pukkandan
26e2805c3f
Add option --skip-playlist-after-errors 
Allows to skip the rest of a playlist after a given number of errors are encountered
 2021-04-22 02:16:31 +05:30
pukkandan
3b4775e021
[go] Fix _VALID_URL 
Closes #255
 2021-04-21 15:43:53 +05:30
pukkandan
ab406a1c0e
[mildom] Warn user of proxy 2021-04-21 15:43:22 +05:30
pukkandan
a3faeb7de4
[MetadataFromField] Improve regex and add tests 2021-04-21 11:12:04 +05:30
pukkandan
8c54a3051d
[youtube] Bugfix in _extract_ytcfg 2021-04-21 10:37:24 +05:30
pukkandan
c32b0aab8a
Improve --sub-langs (see desc) 
* Treat `--sub-langs` entries as regex
* `all` can be used to refer to all the subtitles
* the language code can be prefixed with `-` to exclude it
* Deprecates `--all-subs`
Closes #253
 2021-04-20 02:58:03 +05:30
pukkandan
3097d9e512
[mildom:user:vod] Download only necessary amount of pages 2021-04-19 11:41:49 +05:30
pukkandan
c1df120eda
[mildom:vod] Remove proxy 
* Proxy is needed only for live videos
 2021-04-19 11:41:40 +05:30
pukkandan
2cff495997
[mildom] Change proxy 
Related: #251
Closes #252
 2021-04-19 11:41:33 +05:30
pukkandan
d0491a1ebe
[twitcasting] Fix extractor 
* `Origin: https://twitcasting.tv` must be sent when requesting the webpage. Otherwise the extracted `m3u8` will always give a `502`
* Fix regex for when `data-movie-playlist` is a dict containing the needed list
* media initialization is fully supported; so change downloader to native

Closes #220
 2021-04-18 17:57:46 +05:30
nixxo
b9d68c199b
[rai] Add support for http formats (#208) 
Authored by: nixxo
 2021-04-17 22:42:28 +05:30
pukkandan
201c145953
Update to ytdl-commit-9f6c03 
[cbsnews] Fix extraction for python <3.6
9f6c03a006
 2021-04-17 08:40:31 +05:30
pukkandan
5d34200268
[youtube:tab] Reload with unavailable videos for all playlists 
If the unavailable video is in a later page, the warning and button are not shown in the initial webpage
So we force all playlists' initial page to reload with the correct params
 2021-04-17 08:40:30 +05:30
pukkandan
b7da73eb19
Add option --ignore-no-formats-error 
* Ignores the "no video format" and similar errors
* Experimental - Some extractors may still throw these errors
 2021-04-17 08:40:30 +05:30
pukkandan
6a39ee13f7
Fix inconsistent use of report_warning 2021-04-17 04:16:41 +05:30
pukkandan
33245766ab
[downloader] Fix ffmpeg selection for m3u8_native 2021-04-17 04:15:56 +05:30
coletdjnz
358de58c4d
[youtube:tab] Show unavailable videos in playlists (#242) 
Closes #231

Authored by: colethedj
 2021-04-17 04:09:08 +05:30
pukkandan
a7191c6f57
Fix some linter and typos 2021-04-16 05:31:47 +05:30
lkho
baa5873942
[viu:ott] Fix extractor (see desc) 
* add language_flag_id query param
* add support for premium account (untested since I dont have a premium account)
* support entire series

Code from:
https://github.com/blackjack4494/youtube-dlc/pull/211
https://github.com/ytdl-org/youtube-dl/pull/15182
https://github.com/ytdl-org/youtube-dl/pull/26775

Fixes:
https://github.com/yt-dlp/yt-dlp/issues/219
https://github.com/ytdl-org/youtube-dl/issues/27946
https://github.com/ytdl-org/youtube-dl/issues/27863
https://github.com/ytdl-org/youtube-dl/issues/27812
https://github.com/ytdl-org/youtube-dl/issues/27464
https://github.com/ytdl-org/youtube-dl/issues/26788
https://github.com/blackjack4494/yt-dlc/issues/136

Possibly also fixes (untested):
https://github.com/ytdl-org/youtube-dl/issues/16992
https://github.com/ytdl-org/youtube-dl/issues/26701

Co-authored by: lkho, pukkandan
 2021-04-16 05:19:46 +05:30
pukkandan
c6ce815461
[Exec] Ensure backward compatibility when the command contains % 2021-04-16 05:19:44 +05:30
coletdjnz
79360d99d3
[youtube] Standardize API calls for tabs, mixes and search (#245) 
Authored by: colethedj
 2021-04-15 16:52:59 +05:30
pukkandan
46fff7105e
[youtube] Ignore invalid stretch ratio 
Closes #244
 2021-04-14 15:22:17 +05:30
pukkandan
72e1fe969f
[downloader] Fix downloader selection for m3u8 
Bug introduced by: 52a8a1e1b9 and a31953b0e6
 2021-04-14 12:25:42 +05:30
Ashish
b5be6dd504
[TubiTv] Add TubiTvShowIE (#243) 
Authored by: Ashish0804
 2021-04-14 12:22:28 +05:30
coletdjnz
8ea3f7b909
[youtube] Improve channel syncid extraction to support ytcfg (#241) 
Authored by: colethedj
 2021-04-14 10:37:03 +05:30
pukkandan
921b76cab8
Ensure mergeall selects best format when multistreams are disabled 2021-04-13 10:53:25 +05:30
pukkandan
a31953b0e6
[downloader] Fix external downloader selection for m3u8 
Closes #239
 2021-04-12 22:34:11 +05:30
pukkandan
54670cf084
[version] update 
:ci skip all
 2021-04-12 03:30:55 +05:30
pukkandan
b31fdeedfd
[documentation] Improvements 2021-04-12 02:36:06 +05:30
pukkandan
8fa43c73d8
Add option --convert-thumbnails 
Closes: https://github.com/yt-dlp/yt-dlp/issues/99 https://github.com/yt-dlp/yt-dlp/issues/102
 2021-04-12 02:32:29 +05:30
pukkandan
56d868dbb7
Allow running some postprocessors before actual download 2021-04-12 02:12:46 +05:30
coletdjnz
f4f751af40
[youtube] Parse API parameters from initial webpage (#230) 
* Obtain innertube_context, api_key and x-goog-visitor-id from webpage
* Generalize the header & Innertube_context extraction across YouTube extractors

Related: 1b0a13f33c

Authored by: colethedj
 2021-04-12 01:53:04 +05:30
pukkandan
1988fab7e3
[youtube] Fix thumbnail URL 
Closes: https://github.com/yt-dlp/yt-dlp/issues/233 https://github.com/ytdl-org/youtube-dl/issues/28023
 2021-04-11 06:06:13 +05:30
pukkandan
9de3ea3126
Pass any field to --exec using similar syntax to output template 
Related: https://github.com/ytdl-org/youtube-dl/issues/28642
 2021-04-11 06:06:12 +05:30