cache: move server names and local cache setup into one place

Signed-off-by: Tobias Ulmer <tu@emlix.com>

collect_project: convert to new object oriented structure

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2-fetch-sources: use new result objects

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2-playground: make use of result and e2build

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2-build: make use of result and e2build

Signed-off-by: Tobias Ulmer <tu@emlix.com>

cache: setup_cache: return false instead of nil

Signed-off-by: Tobias Ulmer <tu@emlix.com>

cache: new_cache() is an implementation detail

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2lib: verify_global_config(): always check for existence of servers table

Signed-off-by: Tobias Ulmer <tu@emlix.com>

cache: move e2lib.setup_cache into cache module

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2tool: restore fileid() behaviour and only verify when check-remote is true

Signed-off-by: Tobias Ulmer <tu@emlix.com>

policy: remove all traces of dep_set()

dep_set() used to point to the location of a dependent result in the out
directory when said result was built in working-copy mode. Since the
introduction of stable "scratch-hash" style BuildID's this has been
unnecessary.

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2tool: unify verify_hash() and fileid() to share the same interface and code basis

Signed-off-by: Tobias Ulmer <tu@emlix.com>

cache: rework the caching strategy

Introduce new functions to fetch and push files in a consistent way.

To query server flags we now have:

- cache.cache_enabled()
- cache.islocal_enabled()
- cache.writeback_enabled()

All take the optional flags argument, allowing for overrides when
necessary.

The flags table is now checked for unknown flags, throwing a runtime
error. Unused flags have been removed.

Removed the function to cache files explicitly, return the file path if
in cache, and removed some dead code.
If the cache is enabled, calls to any function requesting path or a
copy of the requested file will cache automatically. This behaviour can
now be suppressed with the flags argument.

Introduced a function to check for the existence of a file on the remote
side without downloading:
transport.file_exists()

Added new function
- cache.fetch_file_path()
that returns the requested files path to either the cache, local
file system or a temporary location.

The code base was adjusted to work without any caching if desired,
although this is not a recommended mode of operation due to speed and
temporary storage requirements.

Signed-off-by: Tobias Ulmer <tu@emlix.com>

cache: remove unused function file_local()

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2lib: make unlink_recursive() work when pathname is not a directory

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2lib: document unlink_recursive() does not follow symlinks

Signed-off-by: Tobias Ulmer <tu@emlix.com>

cache: file_path() remove unused flags and clean up

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2lib: add debug logging to callcmd()

Signed-off-by: Tobias Ulmer <tu@emlix.com>

cache: remove now unused check_only flag, add error checking

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2build: remove only use of check_only

In fact flags is entirely ignored by cache.file_path()

Signed-off-by: Tobias Ulmer <tu@emlix.com>

cache: remove unused flags.refresh

Signed-off-by: Tobias Ulmer <tu@emlix.com>

cache: add more error checking for flags

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2tool: tranport is an implementation detail of cache, stop using it

Signed-off-by: Tobias Ulmer <tu@emlix.com>

cache: remove unused flags.chmod feature

Signed-off-by: Tobias Ulmer <tu@emlix.com>

project: move chroot_call_prefix() to better location

Signed-off-by: Tobias Ulmer <tu@emlix.com>

result: modify build_mode() and drop the getter/setter style

Signed-off-by: Tobias Ulmer <tu@emlix.com>

result: build_process() returns a new build_process_class object

Signed-off-by: Tobias Ulmer <tu@emlix.com>

result: move my_chroot_list() implementation up into the descendent class

Signed-off-by: Tobias Ulmer <tu@emlix.com>

result: make the buildconfig table part of the result

Signed-off-by: Tobias Ulmer <tu@emlix.com>

transport: fix indentation

Signed-off-by: Tobias Ulmer <tu@emlix.com>

trace: reduce overhead in enable(), ignore more modules

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2lib: reduce debug log noise

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2tool: sprinkle asserts and fix indentation

Signed-off-by: Tobias Ulmer <tu@emlix.com>

cache: fetch_file() destname and flags are optional now

Signed-off-by: Tobias Ulmer <tu@emlix.com>

strict: add readonly() function

Enforces readonly access on a table. Also errors on accessing nil values

Signed-off-by: Tobias Ulmer <tu@emlix.com>

strict: __index() doesn't take a value

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2tool: error on configured/installed version mismatch

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2tool: remove info argument from umask funcs

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2tool: adjust to build process and build settings

Signed-off-by: Tobias Ulmer <tu@emlix.com>

result: make use of build_process_class

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2build: introduce the build_process class

Every result class has an associated build_process that knows how to
build results of this type.

If a new result class needs to extend the build process, it can inherhit
the class and overwrite methods or insert additional methods registering
them in the constructor.

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2build: build_result() never sets return_flags, delete

Can be revived when there's a need

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2build: replace environment_script()

environment.tofile() does the same thing

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2build: don't spam build output into debug.log

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2build: cosmetic cleanup

Signed-off-by: Tobias Ulmer <tu@emlix.com>

inline e2tool.bid_display()

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2tool: make set_info() local

Signed-off-by: Tobias Ulmer <tu@emlix.com>

result: use new asserts

Signed-off-by: Tobias Ulmer <tu@emlix.com>

remove package.loaded workaround

Signed-off-by: Tobias Ulmer <tu@emlix.com>

projenv: new module managing project environment

Signed-off-by: Tobias Ulmer <tu@emlix.com>

make.vars: remove unused variables

Signed-off-by: Tobias Ulmer <tu@emlix.com>

environment: add some assertions

Signed-off-by: Tobias Ulmer <tu@emlix.com>

cache: remove tooldebug noise

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2lib.callcmd(): allow multiple descriptors to use a pipe

Stdout and stderr used to get out of sync due to using a pipe each.

Signed-off-by: Tobias Ulmer <tu@emlix.com>

remove noisy debug output

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2lib: fix typo in error message

Signed-off-by: Tobias Ulmer <tu@emlix.com>

dlist: use result module, standard naming, drop info

Signed-off-by: Tobias Ulmer <tu@emlix.com>

project: move host system architecture check here

Signed-off-by: Tobias Ulmer <tu@emlix.com>

source: remove sources_sorted table

On closer investigation nothing was using this interface.

Signed-off-by: Tobias Ulmer <tu@emlix.com>

sl: add an unpack() method

Signed-off-by: Tobias Ulmer <tu@emlix.com>

sl: remove methods dealing with insertion order

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2build: switch to result module

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2build: r -> resultname

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2tool.dlist_recursive: rework cyclic error handling

Show only results involved in the cycle. End the cycle with the same
result as it started, making the circular nature more obvious.

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2tool: use new sources and results

Signed-off-by: Tobias Ulmer <tu@emlix.com>

Introduce result classes

basic_result class implements all common methods, but can't be used on
its own.

result_class represents the standard result type.

Special results can inherit from one of the two classes, overwriting
methods and such.

Composition is of course also possible, and may even be the better
option for plugins.

Signed-off-by: Tobias Ulmer <tu@emlix.com>

environment: don't format the hash input

Signed-off-by: Tobias Ulmer <tu@emlix.com>

git: use hash_append

Signed-off-by: Tobias Ulmer <tu@emlix.com>

svn: use hash_append

Signed-off-by: Tobias Ulmer <tu@emlix.com>

files: use hash_append

Signed-off-by: Tobias Ulmer <tu@emlix.com>

add ass(e)rt module, provide global assert functions

Assert functions are modeled after luaunit, and share some of its code.

Signed-off-by: Tobias Ulmer <tu@emlix.com>

project: use hash_append

Signed-off-by: Tobias Ulmer <tu@emlix.com>

licence: use hash_append

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2lib: sort required modules

Signed-off-by: Tobias Ulmer <tu@emlix.com>

chroot: use hash_append

Signed-off-by: Tobias Ulmer <tu@emlix.com>

cvs: use hash_append

Signed-off-by: Tobias Ulmer <tu@emlix.com>

chroot: unbreak module loading loop

Signed-off-by: Tobias Ulmer <tu@emlix.com>

policy: use eio

Signed-off-by: Tobias Ulmer <tu@emlix.com>

err: remove unused getcount() argument

Signed-off-by: Tobias Ulmer <tu@emlix.com>

project: move projid() to sensible location

Signed-off-by: Tobias Ulmer <tu@emlix.com>

Makefiles: stop build process at the error

set -e is no longer necessary since we pass it to SHELL directly.

Signed-off-by: Tobias Ulmer <tu@emlix.com>

err: improve assert error messages

Signed-off-by: Tobias Ulmer <tu@emlix.com>

hash: correct error message

Signed-off-by: Tobias Ulmer <tu@emlix.com>

files: remove unused environment module

Signed-off-by: Tobias Ulmer <tu@emlix.com>

Revert "class: add obj_(un)lock_rw methods for debugging"

This reverts commit 881b2326718b872f02e0bc53ec3676f5e90d20a2.

Improve error handling in hash module

Signed-off-by: Tobias Ulmer <tu@emlix.com>

Improve stringlist insertion and sorting

Signed-off-by: Tobias Ulmer <tu@emlix.com>

class: add obj_(un)lock_rw methods for debugging

Signed-off-by: Tobias Ulmer <tu@emlix.com>

err: add asserts

Signed-off-by: Tobias Ulmer <tu@emlix.com>

remove disabled lock test code

Signed-off-by: Tobias Ulmer <tu@emlix.com>

remove unused stringlist selftest

Signed-off-by: Tobias Ulmer <tu@emlix.com>

remove broken sha1 tests

Signed-off-by: Tobias Ulmer <tu@emlix.com>

environment: remove disabled unit tests

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2lib.join: fix indentation

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2lib.join: detect nil components and throw an error.

Signed-off-by: Tobias Ulmer <tu@emlix.com>

e2lib.log(): make newline optional

Signed-off-by: Tobias Ulmer <tu@emlix.com>

Handle fetching symlinks correctly, instead of creating 0 size file

This makes symlinks in proj/init work, and should also fix the use of
symlinks in other place. Also, the code should now detect other
non-regular files being fetched and return an error message in this case.

Fixes bz#141

Signed-off-by: Tobias Ulmer <tu@emlix.com>

show per-file licences

Signed-off-by: Tobias Ulmer <tu@emlix.com>

Improve accuracy of fetch-sources manual page

Signed-off-by: Tobias Ulmer <tu@emlix.com>

git source: explicitly fetch tags in addition to objects

Signed-off-by: Tobias Ulmer <tu@emlix.com>