Commit graph

55 commits

Author SHA1 Message Date
Jo-Philipp Wich
b65dc04712 opkg: switch to own fork to improve memory usage
Switch to our own fork of opkg to significantly reduce the required amount
of memory when updating lists or installing packages.

Preliminary tests showed a usage drop of about 90% during these operations,
from ~3.7MB with unmodified opkg to ~360KB with our custom fork.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2017-02-19 19:08:46 +01:00
Hannu Nyman
7ba2062744 opkg: clarify messages and errors related to downloads
Clarify opkg's messages related to downloads:

* more visible error message for package list download failure
* separate error message for signature file download error
* if wget returns 4, signal the network error more clearly
* remove '.' from end of filenames and URLs

* try signature check only if the package list was downloaded ok.

Signed-off-by: Hannu Nyman <hannu.nyman@iki.fi>
2017-01-26 11:38:20 +01:00
Matthias Schiffer
621fd9fd62
opkg: use default PKG_BUILD_DIR
opkg doesn't have BUILD_VARIANTs anymore, so the previously defined
PKG_BUILD_DIR would lead to a weird 'opkg-' path component.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2017-01-16 09:02:42 +01:00
Felix Fietkau
9cdf852ae0 opkg: drop S/MIME support
It has never been used by default (due to being too bloated), and it is
properly replaced by usign (which has been the default for a long time
now).

Remove this feature to simplify the build system

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-10 13:44:25 +01:00
Jo-Philipp Wich
e1d1c31890 opkg: vfork external gzip command to uncompress data
Opkg's builtin decompression code is unsuitable to process nested archives as
it uses a single shared state and relies on undefined seek behaviour for pipes.

Rework the extraction logic to use the external gzip command as I/O filter for
decompressing data and remove the builtin inflate code entirely.

This shrinks the final opkg binary by about 4KB and results in less runtime
memory consumption due to efficient use of vfork() and less copy-on-write
operations in the forked child.

Rework by Felix: create a thread that relays data to the gzip process
instead of using a fragile poll loop

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-27 09:50:00 +01:00
Stijn Tintel
dc5f496a0d Revert "opkg: vfork external gzip command to uncompress data"
This reverts commit 0090adcd5c.
It breaks reading package list in /tmp/opkg-lists, making it impossible
to install packages from feeds in snapshots.
2016-12-27 04:41:30 +01:00
Felix Fietkau
d3489d899a opkg: add missing dependency on libpthread
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-26 15:52:00 +01:00
Jo-Philipp Wich
0090adcd5c opkg: vfork external gzip command to uncompress data
Opkg's builtin decompression code is unsuitable to process nested archives as
it uses a single shared state and relies on undefined seek behaviour for pipes.

Rework the extraction logic to use the external gzip command as I/O filter for
decompressing data and remove the builtin inflate code entirely.

This shrinks the final opkg binary by about 4KB and results in less runtime
memory consumption due to efficient use of vfork() and less copy-on-write
operations in the forked child.

Rework by Felix: create a thread that relays data to the gzip process
instead of using a fragile poll loop

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-26 11:17:32 +01:00
Felix Fietkau
c7c1cf5618 treewide: clean up and unify PKG_VERSION for git based downloads
Also use default defintions for PKG_SOURCE_SUBDIR, PKG_SOURCE

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-22 16:42:21 +01:00
Felix Fietkau
4e138663c6 Revert mirror hash fixes
They are based on obsolete download tarballs

Revert "usign: fix mirror hash" (4f874423fd)
Revert "ubox: fix mirror hash" (9ee74b5753)
Revert "opkg: fix mirror hash" (bf6f4bfad1)

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-20 16:32:08 +01:00
John Crispin
bf6f4bfad1 opkg: fix mirror hash
Signed-off-by: John Crispin <john@phrozen.org>
2016-12-20 13:43:29 +01:00
Felix Fietkau
720b99215d treewide: clean up download hashes
Replace *MD5SUM with *HASH, replace MD5 hashes with SHA256

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-16 22:39:22 +01:00
Jonas Gorski
88a14bfd1d opkg: run prerm scripts for the old version also on upgrade
To make sure we properly restart services on upgrade we need to
call the prerm script of the old package, in case the init script
changes (or vanishes).

Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
Acked-by: Jo-Philipp Wich <jo@mein.io>
2016-11-21 18:40:15 +01:00
Jonas Gorski
a58f176ef2 opkg: set PKG_UPGRADE also when running scripts for the old package
Allow scripts from the package to be upgraded to be aware of being
upgraded.

Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
Acked-by: Jo-Philipp Wich <jo@mein.io>
2016-11-21 18:40:15 +01:00
Daniel Engberg
9edfe7dd13 source: Switch to xz for packages and tools where possible
* Change git packages to xz
* Update mirror checksums in packages where they are used
* Change a few source tarballs to xz if available upstream
* Remove unused lines in packages we're touching, requested by jow- and blogic
* We're relying more on xz-utils so add official mirror as primary source, master site as secondary.
* Add SHA256 checksums to multiple git tarball packages

Signed-off-by: Daniel Engberg <daniel.engberg.lists@pyret.net>
2016-10-06 12:16:56 +02:00
Jo-Philipp Wich
bba8a1a9ba Revert "opkg: use vfork on gz_open by default (FS#120)"
This reverts commit 763f5d7873.

Currently the vfork() code path in opkg is broken and relies on unsupported
ftello() / fseeko() operations on pipes - we need to restructure the code
before we can reconsider this approach.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-08-28 23:03:13 +02:00
Jo-Philipp Wich
d0b88b6067 Revert "opkg: disable the use of vfork for the host build"
This reverts commit 02e3c718e9.

Currently the vfork() code path in opkg is broken and relies on unsupported
ftello() / fseeko() operations on pipes - we need to restructure the code
before we can reconsider this approach.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-08-28 23:03:13 +02:00
Felix Fietkau
02e3c718e9 opkg: disable the use of vfork for the host build
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-08-28 14:34:40 +02:00
Felix Fietkau
763f5d7873 opkg: use vfork on gz_open by default (FS#120)
Reduces memory consumption and binary size

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-08-28 14:18:54 +02:00
Jo-Philipp Wich
24a7ccb056 treewide: replace jow@openwrt.org with jo@mein.io
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-06-07 11:42:52 +02:00
Jo-Philipp Wich
4c60a6f803 opkg: fix use-after-free with duplicate packages on the command line
When the same package file is specified multiple times on the opkg install
command line, the name pointer on the argv array becomes stale after the
package structures have been merged, leading to invalid memory accesses
upon install.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-04-23 14:03:50 +02:00
Jo-Philipp Wich
9e04019024 package: flag essential components as nonshared
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2016-04-06 22:38:47 +02:00
John Crispin
1268b2e899 opkg: Fix handling of sha256sums for conffiles
MD5s are shorter than SHA256, so reading buffer has to be bigger to read the
whole hash.

Signed-off-by: Michal Hrusecky <michal.hrusecky@nic.cz>

SVN-Revision: 48917
2016-03-04 08:33:41 +00:00
John Crispin
61c839ddd7 opkg: Make opkg understand old md5
Opkg now uses sha256 by default and expects them. Making it optionally
understand md5s also and detect md5 sum so we can migrate from configuration
that used md5.

Signed-off-by: Michal Hrusecky <Michal.Hrusecky@nic.cz>

SVN-Revision: 48867
2016-03-01 09:17:12 +00:00
Felix Fietkau
e771a31b39 opkg: depend on uclient-fetch, disable busybox wget
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 48386
2016-01-19 23:35:04 +00:00
Felix Fietkau
1fb3259541 opkg: remove obsolete gcc version check
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 47444
2015-11-10 21:10:57 +00:00
John Crispin
5bf8dd5e5e opkg: Extend 'opkg list' command to optionally display package size
'opkg list' command only displays the available packages' name, version and
description. It would be useful to also see the approximate size of the
available package.

This patch extends "opkg list" command with "--size" to optionally show also
the *.ipk size.
* Default behaviour is to list the available packages as earlier:
  "name - version - description"
* with "--size" the output of is "name - version - size - description".

Signed-off-by: Hannu Nyman <hannu.nyman@iki.fi>

SVN-Revision: 46980
2015-09-16 09:37:39 +00:00
Jo-Philipp Wich
6497406220 opkg: expose lists_dir option as command line argument
Expose the "lists_dir" configuration file option as command line
switch so that we can override it for the Image Builder environment.

Also add a more standard PKG_MAINTAINER variable while touching
the Makefile.

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 46911
2015-09-14 16:56:40 +00:00
Felix Fietkau
3d6a411f14 opkg: work around unconditional libopenssl build dependency
As the OpenWrt build system only resolves build dependencies per directory,
all opkg variants were causing libopenssl to be downloaded and built,
not only opkg-smime. Fix this by applying the same workaround as in
ustream-ssl.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>

SVN-Revision: 46850
2015-09-11 16:31:01 +00:00
Jonas Gorski
ec428bccad opkg: ensure image provided feeds are available after sysupgrade
Split the opkg.conf into three files, to make it easier to support custom
feeds and configs:

* /etc/opkg.conf -> base opkg configuration
* /etc/opkg/distfeeds.conf -> default Openwrt package feeds
* /etc/opkg/customfeeds.conf -> custom package feeds

Of these three, only the base opkg.conf and the customfeeds.conf is marked
as to be kept, so that the distfeeds.conf from the image is always used.

To ease migration, a script is added that moves any feeds from /etc/opkg.conf
to /etc/opkg/customfeeds.conf on first boot.

Also ensure that any keys used for verification are also kept in upgrade.

Signed-off-by: Jonas Gorski <jogo@openwrt.org>

SVN-Revision: 46491
2015-07-25 12:12:28 +00:00
Felix Fietkau
278add2ab1 opkg: fix duplicate check_signature line for smime variant (#19863)
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 45999
2015-06-16 10:51:16 +00:00
John Crispin
72eb909a0d opkg: Fix --force-checksum when using SHA256
Allow installing packages with mismatching SHA256 checksums via

$ opkg install --force-checksum $PACKAGE

Signed-off-by: Florian Fieber <florian@florianfieber.de>

SVN-Revision: 45904
2015-06-05 14:12:45 +00:00
Jo-Philipp Wich
3c5357d6ef feeds: use common macro "FeedSourcesAppend" to populate opkg configurations
This introduces a common macro to assemble the correct url templates to
avoid code duplication and have the feed config handling in a central place.

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 45799
2015-05-28 09:47:37 +00:00
Felix Fietkau
4d67af61be opkg: fix md5 related #ifdef (thx, swalker)
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 45447
2015-04-14 22:10:40 +00:00
John Crispin
cc9a392e49 opkg: drop md5 in favour of sha256
Signed-off-by: John Crispin <blogic@openwrt.org>

SVN-Revision: 45437
2015-04-14 19:00:14 +00:00
Felix Fietkau
beca028bd6 build: add integration for managing opkg package feed keys
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 45286
2015-04-06 19:39:51 +00:00
Felix Fietkau
dde8214d16 opkg: add patch to circumvent signature checks on the command line
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 45285
2015-04-06 19:39:22 +00:00
Felix Fietkau
3efd1303d8 opkg: add patch for supporting signature checking through usign
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 45284
2015-04-06 19:38:37 +00:00
John Crispin
ac09fd9a13 opkg: make PKG_UPGRADE available during preinst hook
https://dev.openwrt.org/ticket/18479

Signed-off-by: John Crispin <blogic@openwrt.org>

SVN-Revision: 43581
2014-12-08 17:40:53 +00:00
John Crispin
5698e62ecb opkg: the PKG_UPGRADE env was not set properly
Signed-off-by: John Crispin <blogic@openwrt.org>

SVN-Revision: 43493
2014-12-02 15:21:50 +00:00
John Crispin
74a3a77bcd license info - revert r43155
turns out that r43155 adds duplicate info.

Signed-off-by: John Crispin <blogic@openwrt.org>

SVN-Revision: 43167
2014-11-03 09:56:44 +00:00
John Crispin
c10d97484a Add more license tags with SPDX identifiers
Note, that licensing stuff is a nightmare: many packages does not clearly
state their licenses, and often multiple source files are simply copied
together - each with different licensing information in the file headers.

I tried hard to ensure, that the license information extracted into the OpenWRT's
makefiles fit the "spirit" of the packages, e.g. such small packages which
come without a dedicated source archive "inherites" the OpenWRT's own license
in my opinion.

However, I can not garantee that I always picked the correct information
and/or did not miss license information.

Signed-off-by: Michael Heimpold <mhei@heimpold.de>

SVN-Revision: 43155
2014-11-03 08:01:08 +00:00
Steven Barth
bec9d38fa4 Add a few SPDX tags
Signed-off-by: Steven Barth <steven@midlink.org>

SVN-Revision: 43151
2014-11-02 12:20:54 +00:00
Jo-Philipp Wich
4f73613c26 build: improve feed handling for opkg.conf
- Consider not installed feeds as well
 - Add option to decide whether to comment disabled feeds

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 42931
2014-10-16 10:30:16 +00:00
John Crispin
b05c2aad3c opkg: indicate upgrade vs install
when running the postinst trigger we set PKG_UPGRADE=1 if this is an upgrade
and not an install.

Signed-off-by: John Crispin <blogic@openwrt.org>

SVN-Revision: 42468
2014-09-11 12:27:09 +00:00
Jo-Philipp Wich
960b7643c9 build: add new option to add disabled feeds commented out to opkg.conf
Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 42004
2014-08-06 09:12:37 +00:00
Jo-Philipp Wich
a720dd209f build: introduce per feed repository support
This changeset implements a new menuconfig option to generate separate
repositories for each enabled package feed instead of one monolithic one.

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 42002
2014-08-05 11:24:24 +00:00
Jo-Philipp Wich
1f3a36b9c3 opkg: introduce a --force-checksum cmdline flag to be ale to ignore mismatching md5sums
Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 40766
2014-05-14 11:19:07 +00:00
Jo-Philipp Wich
df57b8dcb6 opkg: switch to git repository (#14655)
Upstream development now takes place in a git repository which was clone
from the old svn repository. Switch the source location to the git tree
using the commit that corresponds to the old svn r618 revision.

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>

SVN-Revision: 39153
2013-12-21 13:47:46 +00:00
Felix Fietkau
594442a37e opkg: do not add blank "Provides:" fields to package metadata
Signed-off-by: Felix Fietkau <nbd@openwrt.org>

SVN-Revision: 39039
2013-12-13 10:40:26 +00:00