stack 1.6.1 fails to pull in dependencies which stack 1.5.1 did

[phlummox]

General summary/comments (optional)

• Given the same resolver version (lts-6.35, though any lts-6 version seems to be the same), stack 1.6.1 no longer builds the http2 package correctly, nor packages that depend on it (like warp), where stack 1.5.1 did. It seems to be failing to resolve dependencies correctly - version 1.5.1 correctly downloads and builds them all.
  I haven't checked whether similar problems are occurring for other lts resolvers; I noticed this one because re-launching a Travis build of an old project resulted in build failures, despite nothing changing in the code and using the same resolver, OS, etc.

  update: this also seems to affect the OSX version of stack 1.6.1, from https://www.stackage.org/stack/osx-x86_64.

Steps to reproduce

On a clean Ubuntu instance (works with Docker images ubuntu:trusty and ubuntu:xenial), try the following commands:

(test-stack.sh)

## install stack

VER=1.6.1
# or VER=1.5.1

case $VER in
1.5.1)
  export STACK_URL="https://github.com/commercialhaskell/stack/releases/download/v1.5.1/stack-1.5.1-linux-x86_64.tar.gz"
  ;;
1.6.1)
  export STACK_URL="https://github.com/commercialhaskell/stack/releases/download/v1.6.1/stack-1.6.1-linux-x86_64.tar.gz"
  ;;
esac

sudo apt-get install -y libgmp-dev
mkdir -p ~/.local/bin
export PATH=~/.local/bin:$PATH
curl -L "${STACK_URL}" | tar xz --wildcards --strip-components=1 -C ~/.local/bin '*/stack'
stack --resolver=lts-6.35 setup

## install http2

stack --resolver=lts-6.35 install http2

Expected

... 
http2-1.6.2: configure
http2-1.6.2: build
http2-1.6.2: copy/register
Completed 14 action(s).

Actual

--  While building custom Setup.hs for package http2-1.6.2 using:
      /home/phlummox/.stack/setup-exe-cache/x86_64-linux/Cabal-simple_mPHDZzAJ_1.22.5.0_ghc-7.10.3 --builddir=.stack-work/dist/x86_64-linux/Cabal-1.22.5.0 configure --with-ghc=/home/phlummox/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc --with-ghc-pkg=/home/phlummox/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --user --package-db=clear --package-db=global --package-db=/home/phlummox/.stack/snapshots/x86_64-linux/lts-6.35/7.10.3/pkgdb --libdir=/home/phlummox/.stack/snapshots/x86_64-linux/lts-6.35/7.10.3/lib --bindir=/home/phlummox/.stack/snapshots/x86_64-linux/lts-6.35/7.10.3/bin --datadir=/home/phlummox/.stack/snapshots/x86_64-linux/lts-6.35/7.10.3/share --libexecdir=/home/phlummox/.stack/snapshots/x86_64-linux/lts-6.35/7.10.3/libexec --sysconfdir=/home/phlummox/.stack/snapshots/x86_64-linux/lts-6.35/7.10.3/etc --docdir=/home/phlummox/.stack/snapshots/x86_64-linux/lts-6.35/7.10.3/doc/http2-1.6.2 --htmldir=/home/phlummox/.stack/snapshots/x86_64-linux/lts-6.35/7.10.3/doc/http2-1.6.2 --haddockdir=/home/phlummox/.stack/snapshots/x86_64-linux/lts-6.35/7.10.3/doc/http2-1.6.2 --dependency=array=array-0.5.1.0-960bf9ae8875cc30355e086f8853a049 --dependency=base=base-4.8.2.0-0d6d1084fbc041e1cded9228e80e264d --dependency=bytestring=bytestring-0.10.6.0-9a873bcf33d6ce2fd2698ce69e2c1c66 --dependency=bytestring-builder=bytestring-builder-0.10.6.0.0-8aa4073730c676dbe210ea8bffd8d092 --dependency=case-insensitive=case-insensitive-1.2.0.10-2710882b4f658c001bb72a067083a3a8 --dependency=containers=containers-0.5.6.2-59326c33e30ec8f6afd574cbac625bbb --dependency=psqueues=psqueues-0.2.2.3-742bdf37b7dba2a5239a54cba5b43cc7 --dependency=stm=stm-2.4.4.1-7050c728ed5b2315e2c6b56d8bf9837f
    Process exited with code: ExitFailure 1
    Logs have been written to: /home/phlummox/.stack/global-project/.stack-work/logs/http2-1.6.2.log

    Configuring http2-1.6.2...
    Cabal-simple_mPHDZzAJ_1.22.5.0_ghc-7.10.3: At least the following dependencies
    are missing:
    aeson -any, aeson-pretty -any, hex -any, vector -any, word8 -any

Output of stack is in a gist.
Command run was:

$ stack --verbose --resolver=lts-6.35 install http2

I've also created a minimal project (which does nothing but have http2 as a dependency), and built it using Travis CI (link here): it builds using stack 1.5.1, and fails using 1.6.1.

Stack version

$ stack --version
Version 1.6.1, Git revision f25811329bbc40b0c21053a8160c56f923e1201b (5435 commits) x86_64 hpack-0.20.0

Method of installation

• Official binary, downloaded from stackage.org or fpcomplete's package repository

[decentral1se]

Hmmm, I had a copy of Stack 1.7.0 (from stack upgrade --git) and saw:

No packages found in snapshot which provide a "happy" executable, which is a build-tool dependency of "haskell-src-exts"
No packages found in snapshot which provide a "happy" executable, which is a build-tool dependency of "haskell-src-exts"
No packages found in snapshot which provide a "happy" executable, which is a build-tool dependency of "pretty-show"
No packages found in snapshot which provide a "happy" executable, which is a build-tool dependency of "pretty-show"

Error: While constructing the build plan, the following exceptions were encountered:

Error: While constructing the build plan, the following exceptions were encountered:

Dependency cycle detected in packages:
    [Glob,criterion,hashable,semigroups,Glob,http2]

Dependency cycle detected in packages:
    [QuickCheck,text,xml,test-framework,QuickCheck,Glob,http2]

Dependency cycle detected in packages:
    [criterion,case-insensitive,attoparsec,aeson,criterion,hashable,semigroups,Glob,http2]

Dependency cycle detected in packages:
    [hashable,case-insensitive,attoparsec,aeson,criterion,hashable,semigroups,Glob,http2]

Dependency cycle detected in packages:
    [hspec,nanospec,silently,hspec-core,hspec,base-compat,doctest,http-types,attoparsec,aeson,criterion,hashable,semigroups,Glob,http2]

Dependency cycle detected in packages:
    [mwc-random,monad-par,statistics,mwc-random,criterion,hashable,semigroups,Glob,http2]

Dependency cycle detected in packages:
    [semigroups,case-insensitive,attoparsec,aeson,criterion,hashable,semigroups,Glob,http2]


Dependency cycle detected in packages:
    [Glob,criterion,hashable,semigroups,Glob,http2]

Dependency cycle detected in packages:
    [QuickCheck,text,xml,test-framework,QuickCheck,Glob,http2]

Dependency cycle detected in packages:
    [criterion,case-insensitive,attoparsec,aeson,criterion,hashable,semigroups,Glob,http2]

Dependency cycle detected in packages:
    [hashable,case-insensitive,attoparsec,aeson,criterion,hashable,semigroups,Glob,http2]

Dependency cycle detected in packages:
    [hspec,nanospec,silently,hspec-core,hspec,base-compat,doctest,http-types,attoparsec,aeson,criterion,hashable,semigroups,Glob,http2]

Dependency cycle detected in packages:
    [mwc-random,monad-par,statistics,mwc-random,criterion,hashable,semigroups,Glob,http2]

Dependency cycle detected in packages:
    [semigroups,case-insensitive,attoparsec,aeson,criterion,hashable,semigroups,Glob,http2]

Dependency cycle detected in packages:
    [silently,nanospec,silently,hspec-core,hspec,base-compat,doctest,http-types,attoparsec,aeson,criterion,hashable,semigroups,Glob,http2]
Dependency cycle detected in packages:
    [silently,nanospec,silently,hspec-core,hspec,base-compat,doctest,http-types,attoparsec,aeson,criterion,hashable,semigroups,Glob,http2]

Dependency cycle detected in packages:

Dependency cycle detected in packages:
    [tasty,clock,tasty,scientific,quickcheck-instances,http-types,attoparsec,aeson,criterion,hashable,semigroups,Glob,http2]

    [tasty,clock,tasty,scientific,quickcheck-instances,http-types,attoparsec,aeson,criterion,hashable,semigroups,Glob,http2]

Dependency cycle detected in packages:
    [test-framework,text,xml,test-framework,QuickCheck,Glob,http2]
Dependency cycle detected in packages:
    [test-framework,text,xml,test-framework,QuickCheck,Glob,http2]

In the dependencies for unordered-containers-0.2.8.0:

In the dependencies for unordered-containers-0.2.8.0:
    hashmap must match -any, but the stack configuration has no specified version (latest matching version is 1.3.2)
    hashmap must match -any, but the stack configuration has no specified version (latest matching version is 1.3.2)
needed due to http2-1.6.2 -> unordered-containers-0.2.8.0

needed due to http2-1.6.2 -> unordered-containers-0.2.8.0

Some potential ways to resolve this:

Some potential ways to resolve this:

  * Recommended action: try adding the following to your extra-deps in /home/lwm/.stack/global-project/stack.yaml:

  * Recommended action: try adding the following to your extra-deps in /home/lwm/.stack/global-project/stack.yaml:

- hashmap-1.3.2

  * Set 'allow-newer: true' to ignore all version constraints and build anyway.

  * You may also want to try using the 'stack solver' command.

- hashmap-1.3.2

  * Set 'allow-newer: true' to ignore all version constraints and build anyway.

  * You may also want to try using the 'stack solver' command.

Plan construction failed.
Plan construction failed.

[mgsloan]

@lwm Whoah, something's definitely very wrong there, looks like it's outputting everything twice, possibly running the same thing concurrently.

I can reproduce the same thing @phlummox sees, but not the doubled build plan failure output, very curious.

[mgsloan]

I'd say that if fixed soon, this is a good candidate for a 1.6.3 release.

[snoyberg]

Agreed on including in a patch release. Due to travel, I can't look into this now, but I believe the issue @lwm is seeing about the cycles is likely caused by 2a52ac2.

[snoyberg]

OK, I've opened up a PR for this, #3654. It seems to fix the problem in my testing, but if you could test (stack upgrade --git --git-branch 3631-build-http2 should do the trick), that would be appreciated.

[snoyberg]

This should now be resolved on both master and the stable branch, and included in the next release. Thanks for the report, please comment if the issue remains.