Commit Graph

1023 Commits

Author SHA1 Message Date
Tianling Shen
a5e4185f46
Revert "scripts/feeds: use partial clone for git submodule update"
Unfortunately this feature requires very recent git version.

This reverts commit b6cc644e1f.

Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 98bb1a6344)
2023-04-29 12:30:36 +08:00
Tianling Shen
d113f64e8e
scripts/feeds: use partial clone for git submodule update
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit b6cc644e1f)
2023-04-29 11:41:24 +08:00
Glen Huang
f6932e9bd5
scripts/feeds: use partial clone for src-git-full
Partial clone is much faster without clipping history

Signed-off-by: Glen Huang <me@glenhuang.com>
[also apply to include/download.mk]
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 3151477727)
2023-04-29 11:22:35 +08:00
Tianling Shen
9aeea00793
download.pl: update cdn domain
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 3ddf2d6612)
2023-03-15 17:44:47 +08:00
Tianling Shen
d92f634312
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2023-01-14 13:49:55 +08:00
Christian Marangi
8b46a263ab
scripts/dl_github_archieve.py: fix generating unreproducible tar
Allign dl_github_archieve.py to 8252511dc0
change. On supported system the sigid bit is applied to files and tar
archieve that on tar creation. This cause unreproducible tar for these
system and these bit should be dropped to produce reproducible tar.

Add the missing option following the command options used in other
scripts.

Fixes: 75ab064d2b ("build: download code from github using archive API")
Suggested-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
Tested-by: Robert Marko <robimarko@gmail.com>
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
(cherry picked from commit 5f1758ef14)
2023-01-12 15:00:55 +01:00
Tianling Shen
243cb791ed
download.pl: fix typo error
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 9b3db09fe3)
2022-12-11 22:38:34 +08:00
Tianling Shen
cd25afbc7e
scripts/download.pl: update mirrors
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 3d37d7b1c0)
2022-12-11 13:46:52 +08:00
Tianling Shen
fb16e01fc9
Revert "download.pl: remove broken mirror"
mirror2 now points to mirrors.gigenet.com.

This reverts commit 9fbf1751ec.

Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 4fb0bd80d5)
2022-12-11 12:52:52 +08:00
Tianling Shen
36de5d6ab5
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-12-06 03:04:32 +08:00
Christian Marangi
fd90eed40c
scripts: ext-toolchain: add support for musl
Openwrt now supports only glibc and musl. Add support for musl and
rework the libc check to handle the new config flags and correctly
compile package basend on that.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
(cherry picked from commit 7be01fe13b)
2022-12-04 16:07:35 +01:00
Christian Marangi
462c5653e1
scripts: ext-toolchain: add support for info.mk in probe_cc
Openwrt generate info.mk that contains the libc type. For probe_cc check
if the file exist and parse directly it for LIBC type.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
(cherry picked from commit 75311977f5)
2022-12-04 16:07:35 +01:00
Christian Marangi
1f5b8a32e4
scripts: ext-toolchain: actually probe libc type on config generation
Currently we never call probe_cc before config generation, this cause
the script to never actually detect the correct libc type.
Call probe_cc before config generation to correctl set the .config file.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
(cherry picked from commit ddeabc75eb)
2022-12-04 16:07:35 +01:00
Christian Marangi
ed785589ea
scripts: ext-toolchain: add option to overwrite config
It can be useful to overwrite an already generated config.
Option are simply added at the end of the config and make defconfig
will overwrite the relevant option with the new one.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
(cherry picked from commit f4dd18ca39)
2022-12-04 16:07:34 +01:00
Christian Marangi
1e764ea7df
scripts: ext-toolchain: fix wrong prefix in print_config generation
The parsed prefix in print_config is wrong and this produce broken
generated .config that won't work with any external toolchain.

Currently the prefix from a CC of

'arm-openwrt-linux-muslgnueabi-gcc-12.1.0'

produce a prefix

'arm-openwrt-linux-muslgnueabi-gcc-'

This is wrong as the real prefix should be

'arm-openwrt-linux-muslgnueabi-'

This is probably caused by a change in how the toolchain is now handled
that now append also the gcc version. Probably in ancient days the
version wasn't part of the name and the prefix generation stripped the
'-gcc' instead of the gcc version.

Fix this and correctly strip the gcc version and the gcc suffix to
correctly call toolchain bins.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
(cherry picked from commit 53c293262f)
2022-12-04 16:07:34 +01:00
Josh Soref
997ab54373
scripts: fix various typos
This only affects typos in comments or user-facing output.

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
[only picks changes to scripts, drop "commandline" replacement,
 fix case for "arbitrary", improve commit message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
(cherry picked from commit 08622de7d6)
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-12-04 16:07:34 +01:00
Christian Marangi
fd7e8e4e75
scripts/ext-tools: introduce new script to install prebuilt tools
Add a simple script to make it easier to install a prebuilt tools tar.
Currently it will be used by our tools container and kernel workflow on
github.

Simple script that take a tar that contains prebuilt host tools, extract
them and refresh the timestamps to skip recompilation of such host
tools.

By default it refresh timestamps of build_dir/host and
staging_dir/host/stamp.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
(cherry picked from commit 5428bdc2df)
2022-12-04 16:07:34 +01:00
Tianling Shen
cec31d8c48
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-09-27 17:07:22 +08:00
Petr Štetiar
c07c565ea6 scripts/download.pl: fix downloads with wget
Several users of wget for downloads (curl is not available in the
system) have reported broken download functionality:

 wget --tries=5 --timeout=20 --output-document=-  https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.10.142.tar.xz
 http://: Invalid host name.

Thats all happening due to '' was passed as an argument, which got later
expanded to http://.

In the context of a list constructor '' is not nothing, it is an empty
string element.  So fix it by using () as it will yield "nothing" and
thus not introduce an empty string element.

Fixes: #10692
Fixes: 90c6e3aedf ("scripts: always check certificates")
Signed-off-by: Jo-Philipp Wich <jo@mein.io> [shellwords() -> ()]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
(cherry picked from commit 50a48faa1b)
2022-09-21 11:52:40 +02:00
Petr Štetiar
f14d7cef7c scripts/download.pl: silence can't exec curl warning
When running build in verbose mode `make V=s` we can see a lot of
following warnings when curl is not available in the system:

 Can't exec "curl": No such file or directory at scripts/download.pl line 77.

So lets fix it by redirecting of the stderr to null hole.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
(cherry picked from commit c836ca84e8)
2022-09-21 11:52:40 +02:00
Josh Roys
f0bca34f16 scripts: always check certificates
Remove flags from wget and curl instructing them to ignore bad server
certificates. Although other mechanisms can protect against malicious
modifications of downloads, other vectors of attack may be available
to an adversary.

TLS certificate verification can be disabled by turning oof the
"Enable TLS certificate verification during package download" option
enabled by default in the "Global build settings" in "make menuconfig"

Signed-off-by: Josh Roys <roysjosh@gmail.com>
[ add additional info on how to disable this option ]
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz> [backport]
(cherry picked from commit 90c6e3aedf)
2022-09-21 11:52:40 +02:00
Petr Štetiar
45a486bf93 scripts: xxdi.pl: add xxd -i compat mode
So it can serve as a standalone drop in replacement for xxd utility used
currently mostly in U-Boot packages with `xxd -i` mode which outputs C
include file style, with aim for byte to byte identical output, so the
eventual difference in the generated output is easily spottable.

Fixes: #10555
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Signed-off-by: Jo-Philipp Wich <jo@mein.io> [perl-fu]
(cherry picked from commit 06e01e817e)
2022-09-21 11:52:40 +02:00
Jo-Philipp Wich
70124b8579 scripts: xxdi.pl: remove File::Slurp dependency
In order to make it more portable.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
(cherry picked from commit 8b278a76d9)
2022-09-21 11:52:40 +02:00
Petr Štetiar
4e221757c4 scripts: add xxdi.pl
xxdi.pl is a Perl script that implements vim's 'xxd -i' mode so that
packages do not have to use all of vim just to get this functionality.

References: #10555
Source: 97a6bd5cee/xxdi.pl
Signed-off-by: Petr Štetiar <ynezz@true.cz>
(cherry picked from commit 2117d04a3a)
2022-09-21 11:52:40 +02:00
Tianling Shen
a08efaa103
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-08-30 10:39:29 +08:00
Paul Spooren
d445df84c5 feeds: use git-src-full to allow Git versioning
Both $(AUTORELEASE) and $(PKG_SRC_VERSION) (from luci.git) use the Git
log to determine releases and package timestamps.

Feeds are shallow cloned by default, resulting in an incomplete Git log
and therefore different local package versions than offered upstream.

This commits sets the default feeds to use `src-git-full` to solve that.

Add fixes from "2b1d92f: scripts/feeds: silence git warning by selecting
pull style" to `src-git-full`

Signed-off-by: Paul Spooren <mail@aparcar.org>
(cherry picked from commit 7fae1e5677)
2022-08-28 08:16:24 +02:00
Tianling Shen
f150238b1d
download.pl: add ImmortalWrt fallback
Fixes: 759

Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit a190804949)
2022-08-17 20:42:31 +08:00
Tianling Shen
117cd892e7
download.pl: remove broken mirror
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 9fbf1751ec)
2022-08-06 18:50:11 +08:00
Tianling Shen
009fbb76d2
scripts/download.pl: drop broken mirror
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 02edeb1469)
2022-05-06 22:11:22 +08:00
Tianling Shen
e80786fa7e
scripts/download.pl: drop cqu mirror
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 7807fb0119)
2022-03-17 20:00:08 +08:00
Tianling Shen
2c16ee6a6d
include/version: use ImmortalWrt's release repo
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2022-03-14 21:45:05 +08:00
Tianling Shen
af4829a642
scripts/download.pl: add new mirror
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 13c56d272d)
2021-11-30 20:43:52 +08:00
Tianling Shen
af64623418
scripts/download.pl: tweak default mirrors
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-11-29 13:24:39 +08:00
Tianling Shen
b3d3ea94b1
scripts/download.pl: reorder self-hosted mirrors
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit d26b4ff4de)
2021-11-29 00:44:47 +08:00
Tianling Shen
1480f5c9c4
scripts/download.pl: add immortalwrt's mirrors
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 30da20e2c5)
2021-11-23 02:08:23 +08:00
SuLingGG
67e8f2075e
scripts/download.pl: update openwrt.cc mirror url
[updated url]
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit 009c172171)
2021-08-30 23:07:08 +08:00
Tianling Shen
5ef4b8e1de
download.pl: add some mirrors disabled by default
To avoid abuse, these mirrors will not be used until you enable them
manually.

Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit a3325e5051)
2021-07-15 22:38:10 +08:00
Tianling Shen
e8492b9f83
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-06-22 15:25:54 +08:00
Paul Spooren
2e157714a8 build,json: fix generation with empty profiles
If the image generation doesn't add any profiles to the output the
*profile merge* will fail. To avoid that set an empty profile as
fallback.

Signed-off-by: Paul Spooren <mail@aparcar.org>
(cherry picked from commit fd0d9909bf)
2021-06-21 09:43:21 -10:00
Moritz Warning
8add3e139c build: preserve profiles.json between builds
Keep other profiles.json content if the data belongs to the current
build version.

Also useful for the ImageBuilder, which builds for a single model each
time. Without this commit the profiles.json would only contain the
latest build profile information.

Signed-off-by: Moritz Warning <moritzwarning@web.de>
[improve commit message]
Signed-off-by: Paul Spooren <mail@aparcar.org>
(cherry picked from commit a463b96241)
2021-06-21 09:43:21 -10:00
Tianling Shen
37f3ec2632
scripts/download: update mirror url
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-05-26 22:18:23 +08:00
Tianling Shen
929818e613
scripts/download.pl: only call our own mirror when failed to download
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-04-19 01:39:17 +08:00
Tianling Shen
244109d01b
scripts/download.pl: add aliyun(cn) mirror
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-04-19 01:38:54 +08:00
Tianling Shen
a058fd9880
scripts/download.pl: remove unavailable mirror
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-04-17 00:13:23 +08:00
Tianling Shen
b0f006510e
scripts/download.pl: update mirrors
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-04-13 21:35:09 +08:00
Tianling Shen
9312dbaafe
scripts/download.pl: re-order mirrors
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-03-26 22:17:32 +08:00
Tianling Shen
42f947d355
scripts/download.pl: correct branch name of self-hosted mirror
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-03-26 21:32:06 +08:00
Tianling Shen
ae2a20dbf1
scripts/download.pl: add self-hosted mirror
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-03-26 21:31:30 +08:00
Tianling Shen
56729f89c5
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-03-26 18:39:12 +08:00
Paul Spooren
22149da1f4 build,json: backport default_packages fixes
Remove duplicate packages by running in `target/linux/` rather than
`target/linux/<target>/<subtarget>` and sort packages alphabetically.

Squash commit of:
7880a64848 build,json: 3rd fixup of default_packages
b36068d35d build,json: fixup fixup of arch_packages
1bf2b3fe90 build,json: fixup missing arch_packages

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-25 23:15:42 -10:00