Stack broken by latest integer-gmp

[TravisWhitaker]

A few hours ago integer-gmp-1.0.1.0 was uploaded to Hackage. The new package's cabal file's build-depends field uses the new ^>= syntax for version specification. The stable release (v1.5.1) doesn't use Cabal 2, so if you're working with a fresh index stack (any operation but stack update, it seems) will fail with Unable to parse cabal file for integer-gmp-1.0.1.0: NoParse "build-depends" 58. This seems to occur in all of my stack projects, no matter which snapshot is selected.

To reproduce (in some stack project's directory):

some/stack/project $ cp -r ~/.stack ~/.stack_known_good
some/stack/project $ rm -r ~/.stack
some/stack/project $ stack build
Downloaded nightly-2017-11-24 build plan.
Selected mirror https://s3.amazonaws.com/hackage.fpcomplete.com/
Downloading root
Selected mirror https://s3.amazonaws.com/hackage.fpcomplete.com/
Downloading timestamp
Downloading snapshot
Downloading mirrors
Cannot update index (no local copy)
Downloading index
Updated package list downloaded
Populated index cache.
Unable to parse cabal file for integer-gmp-1.0.1.0: NoParse "build-depends" 58

I'll note that all of the stack projects I've tried this with have non-empty extra-deps fields; perhaps stack functions correctly without checking the Hackage index if there are no extra-deps specified? Seems this happens even without any extra-deps.

If I've got something wrong here or if someone is aware of a workaround that isn't copying another machine's index, please do share.

[TravisWhitaker]

#3464 mentions that Cabal 2 syntax is not allowed in Stackage packages. However, integer-gmp-1.0.1.0 appears in the 12/01 nightly snapshot. Either integer-gmp-1.0.1.0 was re-uploaded, or some manual adjustment or exception was made when this version was added to Stackage.

[snoyberg]

The problem is that integer-gmp is included with GHC itself. This has a long history, quick summary:

• Stack through version 1.5.1 was too eager to parse Cabal files
• The work for extensible snapshots accidentally fixed this, and then we added a test case in Treat cabal file parsing failures from index as non-fatal #3396 to ensure it stayed that way
• With GHC-8.2.1, the only problem was ghc.cabal, so we made a patch release Stack 1.5.1 to work around that limitation with PR Support for Stackage Nightly based on GHC 8.2.1 #3304

In other words, this will be fixed permanently with the upcoming release.

[TravisWhitaker]

Is there a work-around for 1.5.1 given the current state of Hackage? Perhaps there are older Hackage indices somewhere that can be pointed to? If not I suppose using stack built from master is the way to go? This is quite frustrating brittleness, especially if it turns out that stack 1.5.1 and previous are permanently bricked by this change. It was surprising to find that such a change in the Hackage index not only broke reproducibility, but broke the tool itself without easy recourse short of copying a colleague’s local index. I’m sure my non-Haskeller colleagues whose code has Haskell dependencies are even more surprised to be blocked on this.

…

On Mon, Dec 4, 2017 at 8:43 PM Michael Snoyman ***@***.***> wrote: The problem is that integer-gmp is included with GHC itself. This has a long history, quick summary: - Stack through version 1.5.1 was too eager to parse Cabal files - The work for extensible snapshots accidentally fixed this, and then we added a test case in #3396 <#3396> to ensure it stayed that way - With GHC-8.2.1, the only problem was ghc.cabal, so we made a patch release Stack 1.5.1 to work around that limitation with PR #3304 <#3304> In other words, this will be fixed permanently with the upcoming release. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3624 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABP-u5dvG2U7rol3eavv9GdDe5Vvn1pHks5s9MoGgaJpZM4Q1gSA> .

[snoyberg]

Maybe I misunderstood. I thought this would only cause trouble if you try using a newer Stackage Nightly snapshot. Are you saying that Stack is broken for all cases, even if you aren't using a GHC 8.2.2 snapshot?

[TravisWhitaker]

Ah, it seems if you're on a snapshot with GHC 8.0.2 or earlier things are fine (makes sense seeing #3396); a colleague reported this breakage with LTS 9 but I wasn't able to reproduce it just now.

I do get this issue with the 8.2.1 nightlies (11/24 and previous). If integer-gmp-1.0.1.0 is meant for GHC 8.2.2, it's strange that it appears in the 11/24 snapshot, but perhaps I'm missing something.

If I understand correctly, the Cabal 2 syntax will make stack 1.5.1 effectively unusable with the GHC 8.2.* snapshots, is that correct? What a mess this whole ^>= business seems to have caused.

[snoyberg]

I'm confused about integer-gmp-1.0.1.0, I don't know how it would have affected the 8.2.1 nightlies without further research. But the basic idea is: if a package in a snapshot takes advantage of a new Cabal feature, then older Stacks will have no way of using it. We can try to work around it by blocking such packages in Stackage, but we can't do anything about packages shipped with GHC itself.

I've requested that some time be taken to allow tooling to upgrade before we start using new features, but that request was denied.

What a mess this whole ^>= business seems to have caused.

You don't know the half of it :( For example, see #3464.

[tfausak]

I just ran into this problem. I'm on Windows using Stack 1.5.1 and the nightly 2017-12-01 resolver. I understand that integer-gmp is at fault for using the new ^>= bounds, but I can't find a bug tracker for that package. I worked around this by switching to the Stack 1.6.1 release candidate, which handled this perfectly.

[austinvhuang]

We're hitting this problem with integer-gmp-1.0.1.0 in our CI as well. As with the OP we also seem to observe this even when reverting to the 11/24 8.2.1 nightly (which had previously been working).

[tfausak]

I created a ticket on GHC's Trac for this: https://ghc.haskell.org/trac/ghc/ticket/14558

[steshaw]

How do I subscribe to the GHC#14558 over on GHC Trac?

[snoyberg]

@steshaw Click on "Modify Ticket" and add your name to the "CC" field, then click "Submit changes." And for the record: I find that workflow really confusing :)

[steshaw]

Thought I'd pick up on @TravisWhitaker's note about broken reproducibility. We have a working build of our app on CircleCI using resolver: nightly-2017-12-01. The build was for testing our code against GHC 8.2.2. The earliest successful build was at Tuesday, December 5, 2017 at 3:12:14 PM UTC+10. We haven't experienced any build failures on CircleCI to date. The reason seems to be that we use CircleCI's caching facility to cache the directories between builds:

  - ~/.stack
  - .stack-work/downloaded
  - .stack-work/install

I only noticed this integer-gmp-1.0.1.0 problem when doing a local docker build in a fresh container. My local macOS build also works without error because — I guess — of cached metadata in ~/.stack. Today, I was able to reproduce the error on CircleCI by doing "Rebuild without cache".

So, I imagine that the metadata for integer-gmp-1.0.1.0 has been very recently updated on hackage. If that's the case, could this problem be solved by reverting that recent change? BTW, is there a way to view the history of metadata changes on hackage? It would help to confirm the theory.

If I'm not mistaken, the issue of reproducibility in the face of hackage metadata updates has been raised in #2217. Perhaps this problem "in the wild" with integer-gmp-1.0.1.0 can raise the priority of that issue. After all, "What makes stack special? The primary stack design point is reproducible builds".

[tfausak]

integer-gmp-1.0.1.0 has not been modified since it was uploaded at 2017-12-04T18:44:27Z. If there were revisions, you could see them here: https://hackage.haskell.org/package/integer-gmp-1.0.1.0/revisions/

I didn't think about this before, but the recent upload date (two days ago) made me think of something. I pinged @hvr on Twitter two days ago because I didn't see base-4.10.1.0 (i.e., GHC 8.2.2) on Hackage. I didn't get any explanation for why it was missing or confirmation when it was uploaded, but it showed up at 2017-12-04T19:37:03Z. That's a little less than an hour after integer-gmp-1.0.1.0 was uploaded.

I don't have an explanation for why the new version of integer-gmp broke Stack builds, but the new Hackage index must have something to do with it. It's just weird that the state of the Hackage index would affect a package that's wired in.