Do you have a repro that I can use verbatim? Having trouble reproducing this.

No, sorry.

It wasn't actually from my personal github though. It's "git+ssh://[email protected]/<org>/<package>.git"

The repo is private and I have read/write access. (This accesses it via a SSH key registered on my github account)

Is there additional log output I can get somehow for you?

I will mention this happens with more than one such identifier.

Minimum repro:

{
  "name": "x",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "devDependencies": {
      "eslint-config-radweb": "git+https://[email protected]/radweb/eslint-config-radweb.git"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}

The package isn't on the registry, if that makes a difference.

This also errors when specifying a git tag (which npm allows).

Example snippet:

...
  "react-quill": "git+https://[email protected]/alexkrolick/react-quill.git#v2.0.1",
...

I also get this #621

Just adding in case subtlety different/requires an additional solution to @BBB case of git tag:

I am trying to install a _specific commit hash_ with git+ssh. NPM default client supports this.

Looks like #573, #633 and #639 are related

For context, from NPM's docs:

npm install <git remote url>:

Installs the package from the hosted git provider, cloning it with git. First it tries via the https (git with github) and if that fails, via ssh.

<protocol>://[<user>[:<password>]@]<hostname>[:<port>][:][/]<path>[#<commit-ish>]

<protocol> is one of git, git+ssh, git+http, git+https, or git+file. If no <commit-ish> is specified, then master is used.

It should also be noted that <commit-ish> is a pretty wide array of resolvable values.

A commit object or an object that can be recursively dereferenced to a commit object. The following are all commit-ishes: a commit object, a tag object that points to a commit object, a tag object that points to a tag object that points to a commit object, etc.

Note: These git remote url installs also should be ensured to work across both public and private git server instances using SSH keys for server authentication, not just GitHub/GitLab/etc. You could imagine a scenario where a company uses a local git server in-house for all of their internally managed dependencies (or even private GitHub repos accessed via SSH). As of right now, yarn is not set up to accommodate these _relatively common_ use cases.

Easiest way to set up a repro case would be to try to install a package from a private GitHub repository using Yarn.

If you specify the following source string:

"devDependencies": {
    "license-builder": "ssh://github.com/<user>/<package>",
}

then it does attempt to clone the stated repository over SSH, but fails with "Permission denied (publickey)" because GitHub expects clients to login as the git user, and in this case the local account user was used by default.

When you do specify git@ to force git to login as git to GitHub, then it fails with the usual:

error Couldn't find any versions for <package> that matches ssh://[email protected]/<user>/<package>.

So, it almost works, but doesn't support specifying the username. If your local account user happened to be called git, then it would actually succeed.

If you add the following to your ~/.ssh/config file:

Host github.com
        User git

you can force all logins to github.com via SSH to use the user git by default, and this makes yarn able to clone from private repositories when using the ssh://github.com/<user>/<package> source format.

This is a strong no-go for us, using git referenced repos (pointing our local gitlab EE instance) is a strong part of our workflow :cry:
Also it's very usefull for forking and pointing to "before merge and npm publish" packages (e.g. http-proxy ...)

@milosivanovic

If you add the following to your ~/.ssh/config file:

But my config already has my ssh auth credentials for github so I can access the private repos. This workaround only works for public repos, right?

@milosivanovic @ntucker actually this worked in my particular case; I didn't had any ssh config file to start with.

@kblcuk ah, well @milosivanovic was going around to other issues and claiming his workaround worked for those cases, so I figured this was the issue for the general problems with git urls.

@ntucker the stated workaround is for private repositories. If you already have a Host github.com entry in ~/.ssh/config, then add User git to that entry and yarn will be able to clone when you specify a source string such as ssh://github.com/<user>/<package>, i.e. without "git+" and without any user specified.

@milosivanovic How does github know my username to do ssh auth then?

@ntucker when communicating over SSH, GitHub expects you to log in to their servers as user "git". If you try to log in with your usual GitHub username, authentication will fail. GitHub over SSH differentiates you by your public key, not username. Reference: https://help.github.com/articles/testing-your-ssh-connection/

Just thought to mention because many are probably not aware of this feature. For GitHub, you can also depend on the tarball URL which works fine with yarn. Installs faster too.

https://github.com/user/repo/tarball/branch

@milosivanovic unfortunately this workaround doesn't work for our internal urls in the format:
git+ssh://[email protected]:team-name/repo.git

If you change the initial repo to the format ssh://source.com/team-name/repo.git...

...then it will work for the initial one...but then of course all the other internal dependencies that the first internal dependency points to breaks it, since they're all in that format.

Without going through and changing all the URLs on all our repos and dependencies to the workaround format (then having to deal with it not working on npm normally), we're a bit blocked on this too.

As @131 pointed out, this is a major way teams use npm internally (that I'm aware of).

Looks great besides that though!

@brokenalarms the ssh://host.com/user/repo format is fully backwards-compatible with npm (as long as the expected user is specified in the SSH config file), but that's a fair point nonetheless.

I see....so they just know what user based on the ssh key then?

@ntucker yes

I had the same problem but ntucker's solution to add User git to ~/.ssh/config helped me. At least in development environment. Will try to deploy to AWS EB now:)

Can confirm, that using git+ssh://[email protected]:<org>/<repo> doesn't work in yarn and changing to https://github.com:<org>/<repo> does work, but that will fail on our CI server that still uses _NPM_.

This workaround helped me:

1. changed my private repository URL from

git+ssh://git@host/user/private-repo.git 

to

ssh://host/user/private-repo.git

1. Added User git to ~/.ssh/config:

Host bitbucket.org
    User git

Host github.com
    User git

Anyone checked if workarounds work with bitbucket?

@tgarbiak - yes, I'm using bitbucket.

Using BitBucket, I added this to my ~/.ssh/config:

Host stash.company.com
    port 7999
    User shawn

And used yarn to add this package:

yarn add ssh://stash.company.com:7999/~user/package.git

When I run npm install, it works fine, but when I run yarn install, I get this error:

error TypeError: Cannot read property 'endsWith' of undefined
    at removeSuffix (/Users/slooker/.nvm/versions/node/v6.4.0/lib/node_modules/yarnpkg/lib/util/misc.js:42:14)
    at Function.parseRefs (/Users/slooker/.nvm/versions/node/v6.4.0/lib/node_modules/yarnpkg/lib/util/git.js:447:55)
    at /Users/slooker/.nvm/versions/node/v6.4.0/lib/node_modules/yarnpkg/lib/util/git.js:376:24
    at next (native)
    at step (/Users/slooker/.nvm/versions/node/v6.4.0/lib/node_modules/yarnpkg/node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30)
    at /Users/slooker/.nvm/versions/node/v6.4.0/lib/node_modules/yarnpkg/node_modules/babel-runtime/helpers/asyncToGenerator.js:28:20
    at run (/Users/slooker/.nvm/versions/node/v6.4.0/lib/node_modules/yarnpkg/node_modules/core-js/library/modules/es6.promise.js:87:22)
    at /Users/slooker/.nvm/versions/node/v6.4.0/lib/node_modules/yarnpkg/node_modules/core-js/library/modules/es6.promise.js:100:28
    at flush (/Users/slooker/.nvm/versions/node/v6.4.0/lib/node_modules/yarnpkg/node_modules/core-js/library/modules/_microtask.js:18:9)
    at _combinedTickCallback (internal/process/next_tick.js:67:7)
    at process._tickCallback (internal/process/next_tick.js:98:9)

Yes, as previously indicated this workaround works.

The point is that going to change this in every one of our 50+
dependencies, and requiring that every future user now takes the additional
step of preparing their ssh config file to work with either yarn or the
existing npm setup is unfortunately not an option.

On Wed, Oct 12, 2016, 3:47 AM Sven Varkel [email protected] wrote:

This workaround helped me:

1. changed my private repository URL from git+ssh://git@host/user/private-repo.git
   to
   ssh://host/user/private-repo.git
2. Added User git to ~/.ssh/config:
   ``
   Host bitbucket.org
   User git

Host github.com
User git

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/513#issuecomment-253180666, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AHC8CnUaBP_B_FL_AX1xL5FUrEWR-rnPks5qzLrTgaJpZM4KO4Cm
.

@diorman & anyone who reads this: I highly recommend that you don't check in github token to your package.json file in source control.

@jsdnxx thanks for pointing this out. I just jumped into a big private project that had the token already on the package.json for private dependencies. Will follow your advice. Thanks again

For branches, the tarball workaround of https://github.com/yarnpkg/yarn/issues/513#issuecomment-253059522 doesn't seem to work, perhaps due to caching. I use Yaska/keystone#yaska-build as the to-install package name, and it uses a wrong commit, and when using https://github.com/Yaska/keystone/tarball/yaska-build it still uses the wrong commit. npm handles it correctly.

On a related note, if I have yarn linked a private repo dependency locally, yarn shouldn't even check to see if that dependency is in the npm registry, but currently yarn is failing in this case without any of the workarounds mentioned in this thread.

The proposed ~/.ssh/config config breaks package resolving, so not working. Hopefully the PR that fixes this will be merged soon, otherwise back to good ol' NPM.

Workaround for gitlab is using this format:

{
    "PROJECT": "http://gitlab.com/NAMESPACE/PROJECT/repository/archive.tar.gz?ref=BRANCH_OR_TAG"
}

Works too with private repository, use Gitlab Personal Access Token:

{
    "PROJECT": "http://gitlab.com/NAMESPACE/PROJECT/repository/archive.tar.gz?ref=BRANCH_OR_TAG&private_token=TOKEN"
}

@Webysther - as @jsdnxx said:

I highly recommend that you don't check in github token to your package.json file in source control.

The same obviously goes for GitLab or any other private tokens.

Private Token is just for param, newer version of Gitlab able manage multiple access token.
I'd love to use git+ssh in yarn...

From Gitlab Docs:

Personal Access Tokens

You can generate a personal access token for each application you use that needs access to the GitLab API.

Private Token

Your private token is used to access application resources without authentication.

I had a look at the code and it seems to me that once a git repo has been
gotten, its commit hash is no longer checked, so you can't track a branch.

Is that a correct assessment, and if so, does that belong in a different
issue?

On Fri, Oct 14, 2016 at 6:52 AM Webysther Nunes [email protected]
wrote:

Private Token is just for param, newer version of Gitlab able manage
multiple access token.
I'd love to use git+ssh in yarn...

From Gitlab Docs:
Personal Access Tokens

You can generate a personal access token for each application you use that
needs access to the GitLab API.
Private Token

Your private token is used to access application resources without
authentication.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/513#issuecomment-253709157, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AADWlhxxjS7Kl1wt_Wm6UG1Q_7X86D7oks5qzwqYgaJpZM4KO4Cm
.

@wmertens No, the version stay locked inside yarn.lock. Try yarn upgrade

@Webysther alas, yarn upgrade doesn't make a difference. It keeps using the old commit. I must note that the branch was force-pushed, but I don't think that matters, because yarn should look up the commit matching the given tag and install that, right?

I found a workaround to force yarn to fetch the correct commit:

Remove the package from ~/.yarn-cache and then run yarn upgrade.

It will fetch it again and things are as they should be. Is it wrong to expect yarn upgrade to check the commits of git repos?

I think it is correct that yarn upgrade should check for new git commits, however that should be a per project version, not cached in the user home directory. But that is also a separate bug I think

Our projects have hundreds of package.json entries of the form
"[name]": "[email protected]:[team]/[project].git"
Same error seen

Will this be fixed by PR #971 ?

@BryanCrotaz No, that doesn't appear to be a complete solution. Seems limited to GitHub. Private repos are still an issue (ex. git+ssh://[email protected]:user/project.git#d6c5789)

EDIT: As pointed out by @bdougherty below, git+ssh://[email protected]/user/project.git#d6c5789, with a / instead of : preceding the user, does work.

+1

I was able to work around this issue by changing the format of the urls from

git+ssh://git@host:org/repo.git

to

git+ssh://git@host/org/repo.git

Both formats are valid in npm and the only catch is that all dependencies need to use that format.

@kittens I think this (git+ssh://git@host/org/repo.git) works for me now?

yarn: v0.16.1
node: v6.9.1

I did not try git+ssh://git@host:org/repo.git but url.parse() does not seem to ignore the : entirely so it may need to be stripped:

> url.parse('git+ssh://[email protected]:org/my-repo.git')
Url {
  protocol: 'git+ssh:',
  slashes: true,
  auth: 'git',
  host: 'github.com',
  port: null,
  hostname: 'github.com',
  hash: null,
  search: null,
  query: null,
  pathname: '/:org/my-repo.git',
  path: '/:org/my-repo.git',
  href: 'git+ssh://[email protected]/:org/my-repo.git' }

Perhaps https://github.com/yarnpkg/yarn/pull/934 inadvertently fixed this?

@Fishrock123 I can confirm what with yarn v0.16.0 git+ssh package install seems to work.
With v0.13.0 it was failing consistently with error Couldn't find package "<package>" on the "npm" registry. for git+ssh packages.

@Fishrock123 Confirmed. Even here it works now.

Yes, #934 was intended to fix this :)

However the following format won't work: git+ssh://git@host:org/repo.git (with a : separator)

Is there any intel on how soon the : separator support might be coming?

I was working on something, but haven't had chance to first some edge cases. I'll try to get to it next week if no-one beats me to it.

Well I still do have a problem with this, but probably little bit different. I can install single package with yarn add git+ssh://[email protected]/group/foo.git#0.0.4 and it works greatly. Then I want to install another one to the same project yarn add git+ssh://[email protected]/group/bar.git and suddenly I am getting Couldn't find package "group-foo" on the "npm" registry.

I am using version 0.16. Shall I rather create new issue with this?

Edit: Might want to add that yarn.lock looks ok...

"git+ssh://[email protected]/group/foo.git#0.0.4":
  name group-foo
  version "0.0.4"
  resolved "git+ssh://[email protected]/group/foo.git#6e25bb42e1725b260d4f1c95582c18aea73e5f5c"

Edit2: Might be actually problem in package.json, it looks like this after first installation. Obviously it has dropped the protocol while in yarn.lock it was kept. So I guess it just cannot find it so it looks in npm instead.

"dependencies": {
  "group-foo": "gitlab.com/group/foo.git#0.0.4"
}

+100

Updating to v0.16.1 and using the git+ssh://git@host/org/repo.git syntax fixed the issue for me (note: still would not work with the git+ssh://git@host:org/repo.git syntax)

The point surely is to support existing package.json files otherwise migration is hard and dual running for testing impossible

Using yarn 0.16.1 I was able to use a private repository with the git+ssh syntaxis. Also, it used the git@ user correctly.

@fermuch And can you run eg. yarn ls after such installation? What does your package.json look like, is url to private repo same or changed somehow?

@FredyC my output of yarn add:

yarn add git+ssh://[email protected]/foobar/my-private-package.git
yarn add v0.16.1
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
warning Unmet peer dependency "whatwg-fetch@^1.0.0".
[4/4] Building fresh packages...
success Saved lockfile.
success Saved 1 new dependency
└─ [email protected]

This was run after deleting my-private-package from node_modules.
After the yarn add, I can see the files correctly inside node_modules.

Yarn ls outputs:

error Couldn't find any versions for my-private-package that matches github.com/foobar/my-private-package.git. Possible versions: 0.1.4

I'm not sure why it gives 0.1.4 since the package doesn't exist in the npm registry, and the github's package has the version 2.1.3.

EDIT

Also, worth noting this was added to my yarn.lock:

"git+ssh://[email protected]/foobar/my-private-package.git":
  name my-private-package
  version "2.1.3"
  resolved "git+ssh://[email protected]/foobar/my-private-package.git#99186dc139e13a1420e56288efd02fd0b3158aa7"

@fermuch Yea I have exactly same issue there. I am almost certain that if you try to add any other package there now (even one from npm), it will fail as well. I have already created separated issue for this ... #1312

I have the same issue here, but with the public url, not over ssh.

  "devDependencies": {
    "code": "2.x.x",
    "hapi": "10.x.x",
    "lab": "10.x.x",
    "k7": "[email protected]:thebergamo/k7.git#v1.5"
  },

Hi All,

I've found that the issue is also related to what node version you have.

I'm using the format "git+ssh://[email protected]//.git#". I do development on OSX and CentOS 7. I've found that with the latest versions of node 4 (v4.6.1) and node 6 (v6.9.1) yarn works with this format without issue. With an older version of node 4 (v4.4.5) it works on OSX but not CentOS 7. Specifically when yarn attempts to download the repo it just hangs forever. If you are experiencing the same problem, make sure you are running the latest version of node 4 or 6.

In reference to kcormier I tried both node 4.6.1 and node 6.9.1 and neither of them resolves the issue of yarn not being able to find specific tagged versions of a repo over SSH.

Format that fails is:

git+ssh://[email protected]:<username>/<project>.git#<tag>

It still gives an error that it can't find a version that matches the tag (works fine with npm).

I find it works fine if I change the colon after the domain to a slash. Weird, right?

@alanhogan I've also noticed it works fine if we change the colon after the domain to a slash, but if the package has other git+ssh dependencies you will have to change it in the package.json of the library/package being installed. Another problem is that even if you change the colon to a slash an error will be triggered if you trying to reference a specific commit or branch.

I've had success referring to branches and tags, myself. I was using node 6.9.1

But yes the recursive problem is real, although not too bad since in theory only our own private modules will be affected.

@alanhogan yeah i'm facing the same issue.

The fix above does not seem to work in my case. I've forked an official Npm package in my repo, and when I give the URL of my repo, even with a / instead of :, yarn is resolving the official Repo. (official: https://github.com/TheLarkInn/angular2-template-loader, mine: https://github.com/Krisa/angular2-template-loader). I could not find a workaround (beside using Npm the time being).

Switching from a versioned dependency to a tarball dependency requires a yarn cache clean before it will actually unpack the tarball as the new node_module (Node v6 LTS and yarn v0.16.1).

an army of developers has voted this up, is there a way we can assist ?

@f-sign is working on it on our team. Anything you'd like help with from an army, Flávio?

One essential part of this issue seems to be the inconsistency of package.json and yarn.lock as described by @FredyC : The package.json does not contain the git+ssh://git@ prefix, which is maintained in yarn.lock during installation. I was thinking, that yarn is prefering to look at the yarn.lock file rather than retrieving the resolution information from package.json

After editing the package.json by hand and setting the prefix, everything worked fine.

@maybeec That problem is actually already solved in master branch... https://github.com/yarnpkg/yarn/issues/1312#issuecomment-258230803

Well done, I am looking forward for the next release. I think it will fix a lot of issues.

Yea, I am totally doumbfounded what's the holdup with making new releases. This one is around like a month now? I guess it's some strict Facebook policy or what... 😢

1784 requests a new release. Please just leave a thumbs-up reaction!

My issue describes a problem, that is similar, even though not the same. I looked through the code a little, and found this interesting part, which is actually used on every git url:

static cleanUrl(url): string {
    return url.replace(/^git\+/, '');
}

Soooo.... Can anyone tell me, what's the reason for removing the _git+_ for each git url passed to yarn? I don't see a real reason, and the code is lacking documentation, so maybe someone could explain the intention :)

1816 may well fix this - take a look at the code changes - it certainly fixes the problem with colon after the domain

The problem seems to be fixed on yarn v0.17.0. I was able to get one of my private Github repository on a specific version.

Is this issue fixed? I m trying to migrate my project from npm to yarn but still facing this issue with [email protected] !

@viswanathamsantosh Seems to work on my side

looks like this is fixed here https://github.com/yarnpkg/yarn/pull/971 !replacing colon( : ) with slash( / ) actually worked. :')

Replacing the colon with a slash doesn't work in my case :( _git+ssh://git@private..._ still gets cut down to _ssh://git@private..._

Yes I'm still having this problem too, even with version 0.17.2 of Yarn. The git+ part gets removed and I end up with:

Permission denied (publickey).
fatal: Could not read from remote repository.

Given that it's working for some people though makes me confused. Any idea what we're doing wrong?

place this in ~/.ssh/config

Host github.com
        User git

Yes! That fixed it for me. Thanks.

It'd be nice though if cleanUrl were not getting run, so we could have this directly in the URL instead. Having to change a configuration file requires a devops change where we could otherwise have version control handling it. Not sure what the thought process behind this was...?

Same problem here. Urls of private repositories are not working as before (npm).

git+ssh://[email protected]:ORG/repo.git should work as it's needed to be compatible with npm during the migration phase...

@DominicBoettger Once you've added User git to your ~/.ssh/config you'll also want to change that colon to a forward slash. In my experience today, Yarn doesn't play well with the colon yet.

git+ssh://github.com/ORG/repo.git

dependencies of the form

git+ssh://[email protected]:myuser/repo.git#v1.0.0",

do not work for me with the latest yarn 017.2. The error is:

ssh: Could not resolve hostname bitbucket.org:myuser: Name or service not known

Haven't tested the work arounds yet but hoping yarn will eventually support the same syntax as NPM. Should this be a new issue or is this still applicable to this issue?

@sarus PR #1816 will fix this

Please merge PR #1816 and publish a new version 👍

Merge? Someone? NPM IS DRIVING ME NUTS!! Please merge and release :(

This issue remains in v0.18.0.

Calling yarn install in a clean project, without node_modules and yarn.lock file works. Calling it again straight after will result in the "Could not resolve hostname" error.

I discovered that removing the yarn.lock file works, so I assume that there is either something wrong in the lock file or the way that yarn clones when reading from a lock file.

Hope this helps!

This issue is the very one which stops many developers to use yarn.
We need take this issue seriously

@regou totally agree man... This is the only reason I cannot use Yarn...

People, instead of this constant yammering that nobody works on this, just watch that mentioned PR #1816 and you will see they are trying to merge it...

Please everyone we have a fix for this in #1816 that Flavio and I spent around ten days on.

However a different set of tests fails depending on where the tests are run.

Please run the tests on your machine and report on #1816 what results you got and what your OS and node version is

Thanks @FredyC & @BryanCrotaz for point everyone to #1816. I'm locking this thread for the time being.

Fixed via #2384