Sbt-github-packages: GitHub Packages doesn't seem to support sbt (yet)

As far as I can tell, -sources, -javadoc, -tests are not supported

using sbt-aether-deploy, I have successfully published to github packages from sbt

project/plugins.sbt

addSbtPlugin("no.arktekk.sbt" % "aether-deploy" % "0.23.0")
addSbtPlugin("com.dwijnand" % "sbt-dynver" % "4.0.0")

build.sbt

externalResolvers += "GitHub lokkju Apache Maven Packages" at "https://maven.pkg.github.com/lokkju/github-action-sbt"
publishTo := Some("GitHub lokkju Apache Maven Packages" at "https://maven.pkg.github.com/lokkju/github-action-sbt")
credentials += Credentials("GitHub Package Registry", "maven.pkg.github.com", "lokkju", "<GITHUB_TOKEN>")

// GitHub package repo isn't supporting javadoc and sources
publishArtifact in (Compile, packageDoc) := false
publishArtifact in (Compile, packageSrc) := false

sbt aetherDeploy

An additional aspect; if I use the Charles Web Debugging Proxy, everything works fine. Without it, I occasionally get errors about the server failing to response.

@lokkju Hmm, what about multi-module builds?

I haven't tried them; key here seems to be using aether-deploy, though it doesn't entirely fix it. I'm suspecting it's an issue with the SSL support in the client versions used by each tool.

I'm going to try a few different java http libraries, and see if I can replacate a simple working and not working configuration. I'll keep you updated.

So, after more exploration, I've found some interesting symptoms.

Just using curl to try to upload packages, I've found;

• The first file you PUT is treated as the main artifact, regardless of the filename or extension
• you can then upload as many (valid) POM files as you like, as long as they have unique names
• you can't upload any other non-POM files
• Files with JAR and POM file extensions are parsed by github, and invalid formats will report an error as the message of the HTTP 4xx error that is returned.

The key takeaway currently is that you must upload the JAR artifact first, then publish the POM.

I have an email into support, we'll see if they provide more information.

Hmm, so no artifact signing either, then?

It auto-generates the *.md5 and *.sha1 for any files uploaded.

Also, found this: https://github.com/sbt/librarymanagement/blob/d09f9f81b664baaac15054730fbcb51e1b240de2/ivy/src/main/scala/sbt/internal/librarymanagement/IvyActions.scala#L123

Artifactory deals with the publishing (and republishing) of SNAPSHOTs using a strict rule on the order of publishing. The ;build.timestamp=... suffix is the alternative.

The strict rule is:

publish the main artefact (which has no classifier)
publish the POM/ivy.xml file
publish additional artefacts which have a classifier

I can't seem to get classifiers deployed at all so far, which is bothersome, and part of what I have a question into github about.

I'll probably create a pure maven/java project and see if it works then

Just tried deploying a snapshot using the default publish and got this:

[error] stack trace is suppressed; run last publish for the full output
[error] (publish) java.io.IOException: PUT operation to URL https://maven.pkg.github.com/company/foo/bar/utils/3.24.0-SNAPSHOT/utils-3.24.0-SNAPSHOT.pom failed with status code 400: Bad Request

No mention of snapshots in the docs

In fact I have the same for normal packages:

java.io.IOException: PUT operation to URL https://maven.pkg.github.com/company/foo/bar/utils/utils/0.0.1/utils-0.0.1.pom failed with status code 400: Bad Request

[error] java.io.IOException: PUT operation to URL https://maven.pkg.github.com/company/foo/bar/utils/utils/0.0.1/utils-0.0.1.pom failed with status code 400: Bad Request
[error]     at org.apache.ivy.util.url.AbstractURLHandler.validatePutStatusCode(AbstractURLHandler.java:82)
[error]     at org.apache.ivy.util.url.BasicURLHandler.upload(BasicURLHandler.java:264)
[error]     at org.apache.ivy.util.url.URLHandlerDispatcher.upload(URLHandlerDispatcher.java:82)
[error]     at org.apache.ivy.util.FileUtil.copy(FileUtil.java:150)

I wonder what the body of that 400 looks like

Are there any updates on this?

No updates yet! @lokkju's experiments are the latest, I think. I want to play around with it some more, but honestly it sounds like GitHub Packages still doesn't support hosting files which are part and parcel to a standard Maven distributable (such as signatures, documentation, and classifiers). It's hard to do anything practical with it as an artifact host until they correct these deficiencies.

Some more info, testing with plain sbt 1.3.2 (no extra plugins)

You need to publish to the repository of your project:

publishTo := Some("GitHub Package Registry" at "https://maven.pkg.github.com/[username]/[project]")

credentails should be defined like this

credentials += Credentials("GitHub Package Registry","maven.pkg.github.com","[username]","[token]")

Your token needs the following scopes: read:packages, repo and write:packages

but once you publish you end up with these proposed package coordinates (where organization="io.test" and name="test"):

<dependency>
  <groupId>com.github.[username]/[project]</groupId>
  <artifactId>io.test.test_2.13</artifactId>
  <version>0.1</version>
</dependency> 

Snapshots can be uploaded but they probably have no special handling as you can't upload them twice

Pom upload works, but the rest fails:

[info] published test_2.13 to https://maven.pkg.github.com/[username]/[project]/io/test/test_2.13/0.0.3/test_2.13-0.0.3.pom
[error] java.io.IOException: Access to URL https://maven.pkg.github.com/[username]/[project]/io/test/test_2.13/0.0.3/test_2.13-0.0.3.jar was refused by the server: Forbidden response=Error: "test_2.13-0.0.3.jar" in version 0.0.3 of "io.test.test_2.13" has already been published.

@francisdb JAR upload fails because SBT uploads the files out of order; from what I can remember, it's a long standing bug in SBT (IvyActions.scala) where the publish function uses a map type instead of seq type. With the move to the Coursier dependency management system, this could self resolve.

My suggestion would be to override the upload process itself within this plugin, making sure to upload the pom first. Of course, this still doesn't help with classifiers, which I need support for.

looks like on sbt 1.3.2 Coursier is not in use for uploading artifacts, only for downloading

Also, I haven't had any response from GitHub on this issue

Hmmm... I just had a very quick look. sbt 1.2.8 is still used here everyhwere:
https://github.com/djspiewak/sbt-github-packages/blob/0839b97dea06100a202f2a105cbe83873784bf77/build.sbt#L28
https://github.com/djspiewak/sbt-github-packages/blob/0839b97dea06100a202f2a105cbe83873784bf77/src/sbt-test/sbtghpackages/publish/project/build.properties#L1
https://github.com/djspiewak/sbt-github-packages/blob/0839b97dea06100a202f2a105cbe83873784bf77/project/build.properties#L1

(Maybe missed some however)

I guess these need upgrades?

Pull request: #2

Coursier is only used by default starting with sbt 1.3.0

my testing was with 1.3.2

@francisdb Got it.

Looks like Github has changed some things, was able to do a full upload today. (vanilla sbt 1.3.3)

[info]  published test_2.13 to https://maven.pkg.github.com/francisdb/foo/com/example/test_2.13/0.0.9/test_2.13-0.0.9.pom
[info]  published test_2.13 to https://maven.pkg.github.com/francisdb/foo/com/example/test_2.13/0.0.9/test_2.13-0.0.9.jar
[info]  published test_2.13 to https://maven.pkg.github.com/francisdb/foo/com/example/test_2.13/0.0.9/test_2.13-0.0.9-sources.jar
[info]  published test_2.13 to https://maven.pkg.github.com/francisdb/foo/com/example/test_2.13/0.0.9/test_2.13-0.0.9-javadoc.jar

The web ui is still confused about the sbt suffix

<dependency>
  <groupId>com.example.test_2</groupId>
  <artifactId>13</artifactId>
  <version>0.0.9</version>
</dependency> 

This is exciting! I'm going to do some testing today and see if I can close this out. Very very exciting.

I can confirm that, with a few tweaks to the plugin to use @francisdb's experimental results, things are now working! The webui issues with cross-build suffixes do not appear to affect actual resolution, just the UI. So publication and resolution now work with the plugin. I haven't yet tested things with a private repository; my guess is that they aren't quite working yet, but things are good enough that I've published 0.1.0.

Thank you, everyone, for all your hard work and experimentation! The plugin is remarkably trivial thanks to all of you.

@francisdb

Snapshots can be uploaded but they probably have no special handling as you can't upload them twice

You _can_ upload them twice: You can delete a snapshot in the GitHub UI and then again upload a new snapshot (meaning same snapshot version). That worked for me. A bit cumbersome, but it works.