Remove \n that mangles output, and fix inconsistent version name check.
Example before:
Build dependency: Please install the GNU C++ Compiler (g++) 6 or later
Build dependency: \nPlease reinstall the GNU C++ Compiler (4.8 or later) - it appears to be broken
Build dependency: Please install ncurses. (Missing libncurses.so or ncurses.h)
Signed-off-by: Karl Palsson <karlp@tweak.net.au>
The final Python 3.5 release was 3.5.10 in September 2020 [0].
This release series is now End-of-Life (EOL).
The only LTS distribution that still only ships Python 3.5 is Ubuntu 16.04,
which will be EOL in April 2021 [1].
The meson build system bumped their python requirement to 3.6 for the 0.57.0
release. This patch ensures that OpenWrt can update meson while still
relying on the host python.
[0] https://www.python.org/dev/peps/pep-0478/#id4
[1] https://ubuntu.com/about/release-cycle
[2] https://mesonbuild.com/Release-notes-for-0-57-0.html#minimum-required-python-version-updated-to-36
Signed-off-by: Andre Heider <a.heider@gmail.com>
[python3.6+ in README.md]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
Before this commit, it was assumed that mkhash is in the PATH. While
this was fine for the normal build workflow, this led to some issues if
make TOPDIR="$(pwd)" -C "$pkgdir" compile
was called manually. In most of the cases, I just saw warnings like this:
make: Entering directory '/home/.../package/gluon-status-page'
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
bash: line 1: mkhash: command not found
[...]
While these were only warnings and the package still compiled sucessfully,
I also observed that some package even fail to build because of this.
After applying this commit, the variable $(MKHASH) is introduced. This
variable points to $(STAGING_DIR_HOST)/bin/mkhash, which is always the
correct path.
Signed-off-by: Leonardo Mörlein <me@irrelefant.net>
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
When building for MikroTik devices the kernel2minor tool will sometimes
fail with:
Can't get lstat from kernel file!: No such file or directory.
This is because kernel2minor expects paths no longer than 250 chars.
To work around this the include/image-commands.mk has been modified
to copy the kernel to a temporary file (/tmp/tmp.XXXXXXXXXX) before
calling kernel2minor.
Signed-off-by: François Chavant <francois@chavant.info>
OpenWRT requires a number of Perl modules to be installed. It wasn't checking on all of them.
This patch adds checks for Perl FindBin, File::Copy, File::Compare and Thread::Queue modules.
Failing to install these, will have the build break at some point. By adding these to the
prereq-build.mk script, they are checked on forehand.
Tested on a Fedora 33 and 34 (beta) that was freshly installed. Fedora appears to
break up Perl modules into small packages that need to be installed for the build to succeed.
Signed-off-by: Bas Mevissen <abuse@basmevissen.nl>
The recent removal of usbutils from core and replacement by hwdata in
packages has exposed hwdata's requirement for certain GNU options on
'install' (-T) Other packages (sqm-scripts) have openwrt specific
makefile sections to avoid GNU options but I suspect this is going to
get harder in the future.
Add GNU install as a prerequisite and link into
$STAGING_DIR/host/etc/bin as per similar GNU utils
This resolves an issue building under MacOS which would otherwise use a
non-GNU options aware version of 'install'
Signed-off-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
FS#3574
Adding cgroup support enables adding rules on processes
to limit resources in terms of iptable policies
Signed-off-by: Supriya Mane <sm.supriya@globaledgesoft.com>
Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>
With the existence of ABI versions there is no clean way to determine
the package name without an attached ABI version. The Packages index is
stored on device to know what packages are installed.
The ABIVersion was recently removed in c921650382 "build: drop ABI
version from metadata", while ABI versions still exists. This becomes a
problem if a user tries to export installed packages via `ubus call
rpcd-sys packagelist` which would return package names including the ABI
version. Trying to find these packages in a later release with changes
ABI version is impossible.
This commits adds the `ABIVersion` field again. Knowing both the
combined (SourceName + ABIVersion) and the `ABIVersion` it is possible
to calculate the package `SourceName` without storing it in the
on-device package list.
Signed-off-by: Paul Spooren <mail@aparcar.org>
`which` utility is not shipped by default for example on recent Arch
Linux and then any steps relying on its presence fails, like for example
following Python3 prereq build check:
$ python3 --version
Python 3.9.1
$ make
/bin/sh: line 1: which: command not found
...
Checking 'python3'... failed.
So make `which` utility host build requirement.
References: PR#3820 FS#3525
Signed-off-by: Petr Štetiar <ynezz@true.cz>
This reverts commit c7aec47e5e.
The original commit replaces 'which' with 'command'. Sadly most of
them are not equivalent and for 'which -a', there is no easy
replacements that would not reimplement PATH parsing logic. Hence
revert. Keeping a dependency on which is absolutely fine.
Signed-off-by: Clemens Fruhwirth <clemens@endorphin.org>
Currently minimal GNU supported GCC version is 7 (from May 2, 2017),
buildbots are using default GCC version 6 on Debian 9 (old stable),
current Debian stable has GCC version 8.3.0.
Signed-off-by: Petr Štetiar <ynezz@true.cz>
autotools.mk does not have any protection currently that would prevent
it from being sourced multiple times. Note that both package.mk and
host-build.mk source autotools.mk. So any package Makefile that includes
both will cause hooks to be added twice (at least twice).
This is fixed by declaring a new variable, __autotools_inc, and only
continuing if this variable doesn't equal 1. The same is done by
rules.mk already.
Also, this commit does away with an ifneq that checks PKG_FIXUP (instead
of HOST_FIXUP) for patch-libtool before adding to the host pre-configure
hook. This does not make sense.
The second ifneq is amended. The current one manually does what the
define patch_libtool_host is already doing. It can just use the define.
Signed-off-by: Sebastian Kemper <sebastian_ml@gmx.net>
Previously, build would fail for targets containing devices with not
initramfs image (such as mpc85xx-p1010). Only generate the JSON image
info for the initramfs image when we have one to avoid breaking the
builds.
Fixes commit d3140d0529 ("build/json: generate json file for initramfs")
Signed-off-by: David Bauer <mail@david-bauer.net>
The update to gettext 0.21 broke packages that use autotools and
gettext because the sed line was failing with the new version. Fix with
a better sed expression.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
The initramfs images are missing from the profiles.json files.
Signed-off-by: Moritz Warning <moritzwarning@web.de>
[fix code by exporting device variables]
Signed-off-by: Paul Spooren <mail@aparcar.org>
package/openwrt-packages is left over
from what is now a legacy repository
and has no other reference in the build tree
Signed-off-by: Michael Pratt <mcpratt@pm.me>
GetABISuffix does not work for intra-package ABI version of provided symbols,
since ABIV_$(provided) is not set.
Fix ABI version by using $(ABIV_$(1)) directly
Signed-off-by: Felix Fietkau <nbd@nbd.name>
Signed-off-by: CN_SZTL <cnsztl@project-openwrt.eu.org>
The sender domain has a DMARC Reject/Quarantine policy which disallows
sending mailing list messages using the original "From" header.
To mitigate this problem, the original message has been wrapped
automatically by the mailing list software.
Fixes: FS#2086
The logic for IS_TTY was broken, because it was testing stdin
instead of stdout.
MAKE_TERMOUT was introduced in GNU Make version 4.1 (05 Oct 2014),
so it should be available everywhere nowadays.
Signed-off-by: R. Diez <rdiezmail-openwrt@yahoo.com>
FS#2086 "IS_TTY in the makefile is broken" reports flawed detection of
stdout piping to a file. The issue describes how e.g. terminal color
codes and up in log files if running make like `make > log.txt`.
The proposed solution uses the make variable "MAKE_TERMOUT", which was
introduced in make 4.1. All major distributions seem to updated to 4.1
or later, so this ideally dosen't break anything.
Signed-off-by: Paul Spooren <mail@aparcar.org>
This was accidentally dropped in 27a4a71c24
("metadata: handle ABI version rebuild tracking for transient dependencies")
Signed-off-by: Felix Fietkau <nbd@nbd.name>
This reverts commit b12288fa69.
The patchelf approach is too fragile, and the only users of this have been
converted to make patching unnecessary
Leave the abi_version_str variable in place in rules.mk
Signed-off-by: Felix Fietkau <nbd@nbd.name>
Preparation for supporting dynamic ABI versions that depend on the runtime
configuration. Read the suffix from the staging dir pkginfo version files.
Signed-off-by: Felix Fietkau <nbd@nbd.name>
This makes it possible to declare a package ABI_VERSION independent from the
upstream soname by setting PKG_ABI_VERSION in the package makefile.
The library filename is fixed up for files installed to packages and to the
staging dir. References to the original from executables within the same
package are also fixed up
Signed-off-by: Felix Fietkau <nbd@nbd.name>
A stray comma was being appended to the last package version dependency,
causing it to be missed for ABI version checks
Signed-off-by: Felix Fietkau <nbd@nbd.name>
Multiple sources are hosted on OpenWrts source server only. The source
URLs to point to the server vary based on different epochs in OpenWrts
history.
Replace all by @OPENWRT which is an "empty" mirror, therefore using the
fallback servers sources.cdn.openwrt.org and sources.openwrt.org.
Signed-off-by: Paul Spooren <mail@aparcar.org>
The license folder is a core part of OpenWrt and all GPL-2.0 licensed.
Use SPDX license tags to allow machines to check licenses.
Signed-off-by: Paul Spooren <mail@aparcar.org>
[rebase, keep some Copyright lines, sharpen commit message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
The majority of our targets provide a default value for the variable
SUPPORTED_DEVICES, which is used in images to check against the
compatible on a running device:
SUPPORTED_DEVICES := $(subst _,$(comma),$(1))
At the moment, this is implemented in the Device/Default block of
the individual targets or even subtargets. However, since we
standardized device names and compatible in the recent past, almost
all targets are following the same scheme now:
device/image name: vendor_model
compatible: vendor,model
The equal redundant definitions are a symptom of this process.
Consequently, this patch moves the definition to image.mk making it
a global default. For the few targets not using the scheme above,
SUPPORTED_DEVICES will be defined to a different value in
Device/Default anyway, overwriting the default. In other words:
This change is supposed to be cosmetic.
This can be used as a global measure to get the current compatible
with: $(firstword $(SUPPORTED_DEVICES))
(Though this is not precisely an achievement of this commit.)
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Signed-off-by: CN_SZTL <cnsztl@project-openwrt.eu.org>
/lib/functions.sh was executable for no obvious reason and its
execute property was even checked in package-ipkg.mk just to
source it afterwards.
Remove the execute bit and shebang as this is clearly a library.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Reviewed-by: Philip Prindeville <philipp@redfish-solutions.com>
Signed-off-by: CN_SZTL <cnsztl@project-openwrt.eu.org>
This keeps the configuration, like the size of the cache, and the
statistics intact. Move the removal of the cache directory to the
distclean target, but only delete the .ccache directory inside of our
build tree, as we should not mess with a user-configured external ccache
directory this destructively.
Signed-off-by: Sven Wegener <sven.wegener@stealer.net>
clang's gcc emulation does the right thing with -print-file-name now,
drop the wrapper
Signed-off-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
`which` utility is not shipped by default for example on recent Arch
Linux and then any steps relying on its presence fails, like for example
following Python3 prereq build check:
$ python3 --version
Python 3.9.1
$ make
/bin/sh: line 1: which: command not found
/bin/sh: line 1: which: command not found
/bin/sh: line 1: which: command not found
...
Checking 'python3'... failed.
...
Fix this by switching to Bash builtin `command` which should provide
same functionality.
Fixes: FS#3525
Signed-off-by: Petr Štetiar <ynezz@true.cz>
With commit 2ca084cc ("build: improve ccache support") these variables
are being set globally and we don't need them for specific targets.
Signed-off-by: Sven Wegener <sven.wegener@stealer.net>
Set CCACHE_DIR to $(TOPDIR)/.ccache and CCACHE_BASEDIR to $(TOPDIR).
This allows to do clean and dirclean. Cache hit rate for test build
after dirclean is ~65%.
If CCACHE is enabled stats are printed out at the end of building process.
CCACHE_DIR config variable allows to override default, which could be useful
when sharing cache with many builds.
cacheclean make target allows to clean the cache.
Changes from v1:
- remove ccache directory using CCACHE_DIR variable
- remove ccache leftovers from sdk and toolchain make files
- introduce CONFIG_CCACHE_DIR variable
- introduce cacheclean make target
Signed-off-by: Roman Yeryomin <roman@advem.lv>
While parsing the nm output, we need to account for the fact that 64-bit kernels
have 64-bit wide addresses. While at it, replace the grep | sed combo with a
single awk invocation and a stronger regex.
Fixes: 2ef0acc5fc "kernel-build: fix
STRIP_KERNEL_EXPORTS for recent kernels"
Signed-off-by: Rui Salvaterra <rsalvaterra@gmail.com>
Multiple prereq checks are only required within the build system but not
for the ImageBuilder. These checks are excluded by using ifndef IB.
This commit merges the three ifndef IB blocks together.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Some images are created using different filesystems, most popular
squashfs and ext4. To allow downstream projects to distinguesh between
those, add the `filesystem` information to created json files.
Signed-off-by: Paul Spooren <mail@aparcar.org>
This is a neat project, but offers no benefit to OpenWrt. The initial
reason for it was to be a replacement for libstdcpp as it is smaller
and lacks compatibility for C++98. Unfortunately, compiling several
packages with it results in larger ipk sizes.
While not a member of the packages feed, this will be moved to
packages-abandoned to keep it somewhere.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
It was observed that the MD5 would not change after source files had been
modified, looking deeper into the build process it was discovered that
find_md5 build function makes a list of the files being built and then
passes the list to a summing utility on stdin. The resultant MD5 is of
the file list, not the contents of the files.
The MD5 would change if the ordering of the list changed, or items were
removed or deleted.
The proposed fix is to add the modification time after the filename and
then sort the list to prevent find returning files in a different order
falsely re-triggering a rebuild. The MD5 will now change when a file is
modified or files are added/removed from the list.
Using 'T@' to show time in epoch for timezone independent behaviour.
Signed-off-by: John Beckett <john.beckett@net2edge.com>
Commit f98878e4c1 ("cmake.mk: set C/CXX compiler for host builds as
well") has introduced regression as it didn't taken usage of ccache into
the account so fix it by handling ccache use cases as well.
In order to get this working we need to export HOSTCXX_NOCACHE in
rules.mk as well.
Fixes: f98878e4c1 ("cmake.mk: set C/CXX compiler for host builds as well")
Reported-by: Ansuel Smith <ansuelsmth@gmail.com>
Tested-by: Ansuel Smith <ansuelsmth@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Without this, cmake will use whatever CC/CXX is set to, which could be
clang. In that case, at least libjson-c/host will fail to compile.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
It is impossible to locate package that failed the build just from log
once more build is run in parallel (that is more than one make job). The
only way is to scout log files for failed package going back trough log.
This change makes it so error is printed for package that failed every
time.
Signed-off-by: Karel Kočí <karel.koci@nic.cz>
Without this, cmake will use whatever CC/CXX is set to, which could be
clang. In that case, at least libjson-c/host will fail to compile.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Currently it's assumed, that already downloaded tarballs are always
fine, so no checksum checking is performed and the tarball is used even
if it might be corrupted.
From now on, we're going to always check the downloaded tarballs before
considering them valid.
Steps to reproduce:
1. Remove cached tarball
rm dl/libubox-2020-08-06-9e52171d.tar.xz
2. Download valid tarball again
make package/libubox/download
3. Invalidate the tarball
sed -i 's/PKG_MIRROR_HASH:=../PKG_MIRROR_HASH:=ff/' package/libs/libubox/Makefile
4. Now compile with corrupt tarball source
make package/libubox/{clean,compile}
Signed-off-by: Petr Štetiar <ynezz@true.cz>
When setting the option IPK_FILES_CHECKSUMS the build system stores
checksums of all package file as metadata. In combination with pkg_check
this allows to see if a package is broken, e.g. caused by bad flash.
To create those checksums the tool `sha256sum` were used while the rest
of OpenWrt uses `mkhash`, a small & fast implementation of sha256. As
the build system does not check the existence of `sha256sum` and the
stderr output is moved to /dev/null, a situation where the option is
enabled but no actual checksum are created may occur.
Instead of adding `sha256sum` as a requirement, this replaces it with
`mkhash sha256` and adapts the `sed` pipe command to fit spacing.
CC: Xu Wang <xwang1498@gmx.com>
CC: Michal Hrusecky <Michal@Hrusecky.net>
Signed-off-by: Paul Spooren <mail@aparcar.org>
Apple ships a broken make version with the Xcode command line tools.
Homebrew installs make as gmake by default in order to not collide with
Apple's version.
Exit with an error if the broken one is used accidentally
Signed-off-by: Felix Fietkau <nbd@nbd.name>
The source date epoch is the only reproducible date close to the actual
build date. It can be used for tooling like the firmware wizard to show
the image age.
Signed-off-by: Paul Spooren <mail@aparcar.org>
With this commit the `profiles.json` contain both the target specific
`default_packages` as well as the device specific `device_packages` as a
array of strings.
This information is required for downstream projects like the various
web-based interactive firmware generators.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Signed-off-by: Paul Spooren <mail@aparcar.org>
JSON info files contain machine readable information of built profiles
and resulting images. These files were added in commit 881ed09ee6
("build: create JSON files containing image info").
They are useful for firmware wizards and script checking for
reproducibility.
Currently all JSON files are stored next to the built images, resulting
in up to 168 individual files for the ath79/generic target.
This patch refactors the JSON creation to store individual per image
(not per profile) files in $(BUILD_DIR)/json_info_files and create an
single overview file called `profiles.json` in the target directory.
Storing per image files and not per profile solves the problem of
parallel file writes. If a profiles sysupgrade and factory image are
finished at the same time both processes would write to the same JSON
file, resulting in randomly broken outputs.
Some target like x86/64 do not use the image code yet, resulting in
missing JSON files. If no JSON info files were created, no
`profiles.json` files is created as it would be empty anyway.
As before, this creation is enabled by default only if `BUILDBOT` is set.
Tested via buildroot & ImageBuilder on ath79/generic, imx6 and x86/64.
Signed-off-by: Paul Spooren <mail@aparcar.org>
[json_info_files dir handling in Make, if case refactoring]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Define wildcard patterns for filtering in target/linux/generic/config-filter
Preparation for supporting newer kernels
Signed-off-by: Felix Fietkau <nbd@nbd.name>
Nightly snapshot builds of OpenWrt change their kernels versions
frequently and lose thereby compatibility to kmods from the upstream
target specific packages feed.
To allow opkg to install packages over multiple days a kmod archive is
offered at $target/$subtarget/kmods/$kernelversion and added as a feed
to created snapshot images via a buildbot step[1].
Instead of using a buildstep add the kmod feed directly via
FeedSourcesAppend to be included in the ImageBuilder repositories.conf
as well. This is conditionally only done for SNAPSHOT builds and when
running as BUILDBOT. Releases are unaffected as they don't include
kernel version changes and local builds may use different kernel
versions or magics than available upstream.
This commit allows in a future step to ship ImageBuilders without a
locally stored kmod archive.
Signed-off-by: Paul Spooren <mail@aparcar.org>
