3 commits

Author	SHA1	Message	Date
Yousong Zhou	04b9f85873	scripts/dl_github_archive.py: rename from download.py ... - Make the code more GitHub-specific - Requires mirror hash to work with .gitattributes - Use different API depending on whether PKG_SOURCE_VERSION is a complete commit id or other ref types like tags - Fix removing symbolic link - pre-clean dir_untar for possible leftovers from previous run Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>	2018-07-05 01:30:57 +08:00
Yousong Zhou	6e3c2d757a	scripts/download.py: use a more terse api for fetching git commit date ... The previous api [1] includes in its response patch data among other things, as such the response size can vary and be big. Use another api[2] to improve it a bit [1] Get a single commit, Repositories, https://developer.github.com/v3/repos/commits/#get-a-single-commit [2] Git Commits, Git Data, https://developer.github.com/v3/git/commits/#get-a-commit Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>	2018-06-27 15:32:01 +08:00
Yousong Zhou	75ab064d2b	build: download code from github using archive API ... A new python script scripts/download.py is added to fetch tarballs using GitHub archive API [1], then repack in a reproducible way same as the current DownloadMethod/git GitHub imposes a 60 reqs/hour rate limit on unauthenticated API access[2]. This affects fetching commit date for feeding tar --mtime= argument. However, observation indicates that archive download is NOT subject to this limit at the moment. In the rare cases where download fails because of this, we will falback to using DownloadMethod/git The missing piece in the GitHub API is that it cannot provide in the tarball dependent submodules's source code. In that case, the implementation will also fallback to using DownloadMethod/git [1] Get archive link, https://developer.github.com/v3/repos/contents/#get-archive-link [2] Rate limiting, https://developer.github.com/v3/#rate-limiting v2 <- v1: - allow passing multiple urls with --urls argument - add commit ts cache. can be helpful on retry Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>	2018-06-27 10:51:27 +08:00