[PATCH 1 of 3] scripts: new maintainer script validate-minimum-dependency-versions

[Mads Kiilerich]

On 10/21/19 10:19 PM, Thomas De Schampheleire wrote:
> # HG changeset patch
> # User Thomas De Schampheleire <thomas.de_schampheleire at nokia.com>
> # Date 1571684857 -7200
> #      Mon Oct 21 21:07:37 2019 +0200
> # Node ID d13fbe9d0a9e6b59e11cf8c546da66672b087c96
> # Parent  f00117816704857e610250e3ffca987a745a2284
> scripts: new maintainer script validate-minimum-dependency-versions
>
> Automate what we can.
>
> This script could be added later to an encompassing script
> 'prepare-for-release' (which would also do other release-related steps).
>
> The inverse script to test maximum versions is more difficult, because we
> currently express the upper bound as '< xxx' which we cannot change to '==
> xxx' because the original range does not include 'xxx'.
> It would be easier if we'd express the upper bound as '<= yyy' instead.


But we can't do that, as we want to allow for minor version bumps with 
bugfixes.

Also, the main use case for validation of upper bounds must be to see if 
pip has later versions that probably should be supported.
The best way to do that might be to first install as usual, dump `pip 
freeze`, then install again without upper bounds at all (with 
exceptions, that perhaps should have comments in setup.py), and compare 
pip freeze (for the targets that we install explicitly).


>
> diff --git a/scripts/validate-minimum-dependency-versions b/scripts/validate-minimum-dependency-versions
> new file mode 100755
> --- /dev/null
> +++ b/scripts/validate-minimum-dependency-versions
> @@ -0,0 +1,58 @@
> +#!/usr/bin/env bash
> +# Test that installation of all dependencies works fine if versions are set to
> +# the minimum ones.
> +
> +set -e
> +
> +if [ -n "$VIRTUAL_ENV" ]; then
> +    echo "This script will create its own virtualenv - please don't run it inside an existing one." >&2
> +    exit 1
> +fi
> +
> +if ! hg update --check -q .; then
> +    echo "Working dir is not clean, please commit/revert changes first." >&2
> +    exit 1
> +fi
> +
> +cd "$(hg root)"
> +
> +venv=build/minimum-dependency-versions-venv
> +log=build/minimum-dependency-versions-log


Perhaps make that .log


> +echo "virtualenv: $venv"
> +echo "log: $log"
> +
> +# clean up previous runs
> +rm -rf "$venv" "$log"
> +mkdir -p "$venv"
> +
> +# Set minimum versions
> +sed -i '/"/s/>=/==/' setup.py dev_requirements.txt


As mentioned before, perhaps first pip install on default setup.py (to 
make sure we get dependencies not covered by the following), then create 
a requirements.txt with explicit minimum versions.


> +
> +virtualenv -p "$(command -v python2)" "$venv"
> +source "$venv/bin/activate"
> +pip install --upgrade pip setuptools
> +# tee stderr to console as well as log file
> +# https://stackoverflow.com/a/692407/2941347
> +pip install -e . -r dev_requirements.txt python-ldap python-pam 2> >(tee -a "$log" >&2)


Why append when the log file just has been deleted?


> +
> +# Strip out the known Python 2.7 deprecation message.
> +sed -i '/DEPRECATION: Python 2\.7 will reach the end of its life/d' "$log"
> +
> +# Treat any message on stderr as a problem, for the caller to interpret.
> +if [ -s "$log" ]; then
> +    echo
> +    echo "Error: pip detected following problems:"
> +    cat "$log"
> +    echo
> +    exit 1
> +else
> +    rm -f "$log"
> +fi
> +
> +echo "Installation successful. Now running test suite..."
> +
> +pytest
> +
> +echo "Test suite successful."
> +echo "You can now do additional validation using virtual env '$venv'."
> +echo "When you are done, revert the changes in the repo."


LGTM, perhaps addressing the comments.

/Mads