Commit Graph

1022 Commits

Author SHA1 Message Date
SuLingGG
009c172171
scripts/download.pl: update openwrt.cc mirror url
[updated url]
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-08-30 23:06:44 +08:00
Tianling Shen
71813d23c5
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-08-18 19:44:35 +08:00
Daniel Golle
98bccdafd7
base-files: rename 'sdcard' to 'legacy-sdcard'
While an image layout based on MBR and 'bootfs' partition may be easy
to understand for users who are very used to the IBM PC and always have
the option to access the SD card outside of the device (and hence don't
really depend on other recovery methods or dual-boot), in my opinion
it's a dead end for many desirable features on embedded systems,
especially when managed remotely (and hence without an easy option to
access the SD card using another device in case things go wrong, for
example).

Let me explain:

* using a MSDOS/VFAT filesystem to store kernel(s) is problematic, as a
  single corruption of the bootfs can render the system into a state
  that it no longer boots at all. This makes dual-boot useless, or at
  least very tedious to setup with then 2 independent boot partitions
  to avoid the single point of failure on a "hot" block (the FAT index
  of the boot partition, written every time a file is changed in
  bootfs). And well: most targets even store the bootloader environment
  in a file in that very same FAT filesystem, hence it cannot be used
  to script a reliable dual-boot method (as loading the environment
  itself will already fail if the filesystem is corrupted).

* loading the kernel uImage from bootfs and using rootfs inside an
  additional partition means the bootloader can only validate the
  kernel -- if rootfs is broken or corrupted, this can lead to a reboot
  loop, which is often a quite costly thing to happen in terms of
  hardware lifetime.

* imitating MBR-boot behavior with a FAT-formatted bootfs partition
  (like IBM PC in the 80s and 90s) is just one of many choices on
  embedded targets. There are much better options with modern U-Boot
  (which is what we use and build from source for all targets booting
  off SD cards), see examples in mediatek/mt7622 and mediatek/mt7623.

Hence rename the 'sdcard' feature to 'legacy-sdcard', and prefix
functions with 'legacy_sdcard_' instead of 'sdcard_'.

Tested-by: Stijn Tintel <stijn@linux-ipv6.be>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-08-16 12:22:17 +01:00
AmadeusGhost
33e772ad39 Merge Official Source 2021-08-11 12:08:20 +08:00
Stijn Tintel
8347d8b4fb base-files: add generic sdcard upgrade method
Add a generic sdcard upgrade method instead of duplicating code in yet
another target, and add a feature flag to only install this upgrade
method in targets that set this flag. Copied from mvebu.

Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
2021-08-07 01:34:40 +03:00
Stijn Tintel
84c9570aaa scripts/target-metadata.pl: order features alphabetically
We generaly do this in many other places, so let's do it here also.

Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
2021-08-07 01:30:37 +03:00
AmadeusGhost
1bb251a7e9 Merge Official Source 2021-07-20 12:18:13 +08:00
Yonghyu Ban
7270d768da scripts/mkits.sh: Fix the hash algorithm paramter
The mkits.sh script help message states hash algorithm can be
specified using the -H command-line option, but it does not work
currently due to a bug in the script.

This patch fixes this problem by changing the option from -S to
-H and specify getopts parameter after it

Signed-off-by: Yonghyu Ban <yonghyu@empo.im>
2021-07-18 19:37:30 +01:00
Tianling Shen
7679fe4f88
download.pl: add some mirrors disabled by default
To avoid abuse, these mirrors will not be used until you enable them
manually.

Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
(cherry picked from commit a3325e5051)
2021-07-15 22:38:30 +08:00
Tianling Shen
3035291b81
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-07-09 00:54:32 +08:00
Ansuel Smith
64ce35bfaf scripts: check if dl directory exist in dl_cleanup script
Check if the provided dl directory exist and return on error.

Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
2021-07-05 14:15:58 -10:00
Tianling Shen
d97b735f52
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-06-21 20:11:10 +08:00
Paul Spooren
fd0d9909bf build,json: fix generation with empty profiles
If the image generation doesn't add any profiles to the output the
*profile merge* will fail. To avoid that set an empty profile as
fallback.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-06-20 22:42:29 -10:00
Tianling Shen
4c46b256d1
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-06-17 20:51:30 +08:00
Moritz Warning
a463b96241 build: preserve profiles.json between builds
Keep other profiles.json content if the data belongs to the current
build version.

Also useful for the ImageBuilder, which builds for a single model each
time. Without this commit the profiles.json would only contain the
latest build profile information.

Signed-off-by: Moritz Warning <moritzwarning@web.de>
[improve commit message]
Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-06-16 21:28:53 -10:00
AmadeusGhost
a19da363be Merge Official Source 2021-06-10 23:09:38 +08:00
Karel Kočí
1aa80ce393 scripts/feeds: generate index after all feeds are updated
This separates index update from feed update. The result is that all
requested feeds are first updated and only then indexed.

The reason for this change is to prevent errors being reported and
potentially invalid index being generated thanks to cross feeds
dependency.
The feeds script pulls in default all feeds as they come and on install
prefers packages from first feeds (unless special feed is requested).
Thus order of feeds in some way specifies preferences. This is handy for
downstream distributions as they can simply override any package from
upstream feeds by placing their feed before them. This removes need to
patch or fork upstream feeds.
The problem is that such feed most likely depends in some way also on
subsequent feeds. The most likely feeds are 'packages' or 'luci'. The
example would be Python package that needs 'python.mk' from 'packages'
feed. Ordering custom feed after dependent feeds is sometimes just not
possible because of preference requirement described before.
The solution is to just first pull all feeds and generate indexes only
after that. In the end this ensures that index is generated correctly at
first try without any error.

In terms of code this removes 'perform_update' argument from
'update_feed' as with index update removal the update is the only action
performed in that subroutine. Thus this moves condition to 'update'
subroutine.

Signed-off-by: Karel Kočí <karel.koci@nic.cz>
2021-06-07 12:44:50 -10:00
Tianling Shen
f01d1d27b3
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-05-30 22:28:06 +08:00
Paul Spooren
ec4852f731 scripts: config.guess: update to 2021-05-24
This script hasn't seen an update in multiple years, update it to the
latest version provided upstream. Both `config.guess` and `config.sub`
are copied from upstream[1] and not modified.

The full changelog is available within the upstream repository[1].

[1]: https://git.savannah.gnu.org/git/config.git

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-05-29 13:03:07 +02:00
Tianling Shen
e821c29ca7
scripts/download: update mirror url
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-05-26 22:18:37 +08:00
Tianling Shen
bb56c18f7c
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-05-13 22:39:57 +08:00
Leonardo Mörlein
b993b68b6c build: introduce $(MKHASH)
Before this commit, it was assumed that mkhash is in the PATH. While
this was fine for the normal build workflow, this led to some issues if

    make TOPDIR="$(pwd)" -C "$pkgdir" compile

was called manually. In most of the cases, I just saw warnings like this:

    make: Entering directory '/home/.../package/gluon-status-page'
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    [...]

While these were only warnings and the package still compiled sucessfully,
I also observed that some package even fail to build because of this.

After applying this commit, the variable $(MKHASH) is introduced. This
variable points to $(STAGING_DIR_HOST)/bin/mkhash, which is always the
correct path.

Signed-off-by: Leonardo Mörlein <me@irrelefant.net>
2021-05-13 15:13:15 +02:00
Tianling Shen
dd5110d976
scripts/download.pl: only call our own mirror when failed to download
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-04-19 01:40:29 +08:00
Tianling Shen
cfab12e7cc
scripts/download.pl: add aliyun(cn) mirror
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-04-19 01:40:02 +08:00
Tianling Shen
1e4c245042
scripts/download.pl: remove unavailable mirror
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-04-17 00:13:51 +08:00
Tianling Shen
c0e262bb86
scripts/download.pl: update mirrors
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-04-13 21:35:34 +08:00
Tianling Shen
84874a2967
scripts/download.pl: re-order mirrors
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-03-26 22:18:34 +08:00
Tianling Shen
8f3bc0a046
scripts/download.pl: add self-hosted mirror
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-03-26 21:32:40 +08:00
Tianling Shen
0ee01d71cd
Merge Official Source
Closes: #332

Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-03-26 18:37:54 +08:00
Paul Spooren
7880a64848 build,json: 3rd fixup of default_packages
This became a bit of a tragedy, caused by a corner cases which wasn't
put into account during testing. DEFAULT_PACKAGES are defined in
target/linux/<target>/Makefile but also in
target/linux/<target>/<subtarget>/target.mk.

The latter was no longer imported when using DUMP=1, however not using
DUMP=1 while running the Makefile in target/linux/<target>/ caused duplicate
packages in the list.

As a solution, which should have been used from day 0, `make` runs in
target/linux/ without DUMP=1, resulting in no duplicate packages and all
inclusions from include/target.mk, linux/target/<target>/{Makefile,
<subtarget>/target.mk}

While at it, sort the list of default packages.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-25 10:25:54 -10:00
Paul Spooren
b36068d35d build,json: fixup fixup of arch_packages
The commit "1bf2b3fe90 build,json: fixup missing arch_packages" fixes
the missing package architecture locally but runs $(TOPDIR)/Makefile
rather than a target specific one. While this works on local builds just
fine, it causes the buildbots to add garbage to the `arch_packages`
variable:

    cd \"/builder/shared-workdir/build\"; git log --format=%h -1
    toolchain > /builder/shared-workdir/build/tmp/.ver_check\ncmp -s
    /builder/shared-workdir/build/tmp/.ver_check
    /builder/shared-workdir/build/staging_dir/toolchain-x86_64_gcc-8.4.0_musl/stamp/.ver_check
    || { \\\n\trm -rf
    /builder/shared-workdir/build/build_dir/target-x86_64_musl
    /builder/shared-workdir/build/staging_dir/target-x86_64_musl
    /builder/shared-workdir/build/staging_dir/toolchain-x86_64_gcc-8.4.0_musl
    /builder/shared-workdir/build/build_dir/toolchain-x86_64_gcc-8.4.0_musl;
    \\\n\tmkdir -p
    /builder/shared-workdir/build/staging_dir/toolchain-x86_64_gcc-8.4.0_musl/stamp;
    \\\n\tmv /builder/shared-workdir/build/tmp/.ver_check
    /builder/shared-workdir/build/staging_dir/toolchain-x86_64_gcc-8.4.0_musl/stamp/.ver_check;
    \\\n}\nx86_64

Only the last line contains the desired string.

Future investigation should check why the build system prints this to
stdout rather than stderr.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-24 19:03:24 -10:00
Paul Spooren
1bf2b3fe90 build,json: fixup missing arch_packages
Fix 7f4c2b1 "build,json: fix duplicates in default_packages" which
removed duplicate default packages but also removed the package
architecture from the profiles.json.

If DUMP=1 is set, the `ARCH_PACKAGES` is no longer exported and
therefore empty. Fix this by running make twice, once with DUMP=1 and
once without.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-24 13:41:34 -10:00
Tianling Shen
3c31b3fd88
Revert "treewide: add mirrors for GitHub Codeload"
This reverts commit 71ad441934.

Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-03-23 17:00:43 +08:00
Tianling Shen
a4e38ad14c
Revert "treewide: add mirrors for GitHub Archives"
This reverts commit 237595c1eb.
2021-03-23 17:00:18 +08:00
Tianling Shen
82388ff3ee
scripts/download.pl: add download mirrors hosted by SuLingGG
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-03-22 23:22:30 +08:00
Tianling Shen
46ce35d81c
download.pl: fix path to mirror 182.140.223.146
Closes: #327

Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-03-22 17:11:06 +08:00
Tianling Shen
18e21aff8f
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-03-22 16:35:07 +08:00
Paul Spooren
7f4c2b1a4f build,json: fix duplicates in default_packages
Calling without the DUMP=1 argument causes the target specific Makefile
to be "included" again which adds the target specific packages twice,
once on the actual run and once included from `include/target.mk`.

This led to duplicate package entries, causing confusion in downstream
projects using the generated JSON files.

While at it, apply `black` style to Python script.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-20 23:58:52 -10:00
AmadeusGhost
5726bab29c Merge Official Source 2021-03-18 23:36:35 +08:00
Daniel Golle
6890f6fe13 include/image*: add support for device-tree overlays
Add new target feature 'dt-overlay' which makes DTC keep the symbol
names in the generated dtb.
Make sure additional DT overlay sources specified by the new device
variable DEVICE_DTS_OVERLAY get compiled together with the main DTS
(currently overlays got to be in the same folder). Let Build/fit pass
the generated DT overlay blobs to mkits.sh.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-03-17 17:45:34 +00:00
Daniel Golle
4e6de4f093 scripts/mkits.sh: add support for adding DT overlay blobs to image
Allow adding multiple device tree overlay blobs to an image and
generate configurations for each of them.
This is useful on boards with modern U-Boot which allow e.g. user-
configurable peripherals ("shields") in that way.
Note that currently, each generated configuration adds exactly one
overlay on top of the base image, ie. adding multiple overlays at the
same time is not yet supported.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2021-03-17 17:41:24 +00:00
Tianling Shen
fad8da597e
Merge Official Source
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-03-16 23:59:03 +08:00
Tianling Shen
237595c1eb
treewide: add mirrors for GitHub Archives
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-03-15 15:04:16 +08:00
Tianling Shen
71ad441934
treewide: add mirrors for GitHub Codeload
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-03-15 14:29:36 +08:00
Paul Spooren
bb95be9265 scripts,ipkg-build: use realpath for pkg_dir
This allows manual execution of the ipkg-build script even with
releative path.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-14 19:32:30 -10:00
Paul Spooren
72f481b157 scripts,ipkg-build: apply shellcheck
This commit cleans the `ipkg-build` script via changes suggested by
shellcheck. These are mostly word splitting issues.

Remove the definition of GZIP, this adds three "lookups" of the `gzip`
binary but the rest of the build system doesn't seem to use such
improvements neither.

Signed-off-by: Paul Spooren <mail@aparcar.org>
2021-03-14 19:31:43 -10:00
Tianling Shen
0094964b3e
scripts/download.pl: add global mirror for mainland China users
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-03-15 11:22:56 +08:00
Tianling Shen
12c4fd437d
treewide: adjust tencent mirror
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
2021-03-11 14:59:09 +08:00
AmadeusGhost
8e1125c340 Merge Official Source 2021-03-06 12:05:05 +08:00
Robert Marko
5ec60cbe9d scripts: mkits.sh: replace @ with - in nodes
U-boot will reject the nodes with @ for the address since
commit:
79af75f777

This in turn will cause the failure to boot with OpenWrt
generated images.

So, to rectify that simply replace @ with -.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
(cover also newly added rootfs@1 and initrd@1 nodes)
Signed-off-by: Robert Marko <robert.marko@sartura.hr>
2021-03-05 15:45:33 +00:00