OpenShift 3 : unable to clone a private BitBucket repository - openshift

I'm trying to migrate from OpenShift 2 to OpenShift 3.
I have created a new app on OpenShift 3 but I'm struggling to clone my BitBucket private git repository to it. (I had no problem with OpenShift 2).
I have tried setting secrets (SSH or Basic Authentication) in Build/Advanced Options but without luck.
Here is the error message :
Cloning "git#bitbucket.org:(myusername)/(myrepository).git" ... error:
build error: Host key verification failed. fatal: Could not read from
remote repository. Please make sure you have the correct access rights
and the repository exists.

The steps if working from the command line are as follows:
1) Create a new SSH key pair for use with the repository. This cannot have a passphrase.
ssh-keygen -C "openshift-source-builder/repo#bitbucket" -f repo-at-bitbucket -N ''
This will generate files:
repo-at-bitbucket
repo-at-bitbucket.pub
being the private and public key files.
2) Go to Settings->Access keys for the repository on BitBucket, select Add key and in the popup window enter the key name openshift-source-builder and paste in the contents of the public key file. In this case repo-at-bitbucket.pub. Confirm creation by clicking on Add key on the popup window.
3) Create a secret in OpenShift for the key by running:
oc secrets new-sshauth repo-at-bitbucket --ssh-privatekey=repo-at-bitbucket
4) Enable access to the secret from the builder service account.
oc secrets link builder repo-at-bitbucket
5) In order that OpenShift knows the secret is for this specific private Git repository and automatically uses it, annotate the secret with the SSH URI for the repository.
oc annotate secret/repo-at-bitbucket \
'build.openshift.io/source-secret-match-uri-1=ssh://bitbucket.org/yourusername/private-repo.git'
Very important here is the form of the URI. In the BitBucket web interface it will show it as:
git#bitbucket.org:yourusername/private-repo.git
Do not use that. You need to use the SSH form of the URI here.
6) We can then deploy the application from the private Git repository.
oc new-app httpd~git#bitbucket.org:yourusername/private-repo.git --name mysite
Okay to use git#bitbucket.org:yourusername/private-repo.git here, or could also use the SSH form of the URI.
You can also do all this from the web console instead. Important if creating the secret as a separate step in web console to link the builder service account when doing that. If create the source secret when deploying, then it will automatically link the builder service account.
Note that if the OpenShift instance has a firewall between it and BitBucket and SSH connections are blocked, this will not work. In that case you need to fall back to using a personal access token (app password) over a SSH connection using HTTP basic authentication.
These details are now much better explained by the blog post series starting with:
https://blog.openshift.com/private-git-repositories-part-1-best-practices/

Related

What is github actions' equivalent to circle ci's add_ssh_keys

In CircleCI, we need to add_ssh_keys to access another private repo that is not in the organization.
We are using fastlane match in the jobs so it is not simple checkout.
e.g.
- add_ssh_keys:
fingerprints:
- "SO:ME:FIN:G:ER:PR:IN:T"
How do you do that in GitHub Actions?
You can use the install SSH key action. Just make sure you’re using a SSH git URL in your Matchfile, not a HTTPS URL.

Could not push code to the CodeCommit repository

I am trying to push my code to the beanstalk but I am getting an error when I hit the eb create command
WARNING: You have uncommitted changes.
Starting environment deployment via CodeCommit
Could not push code to the CodeCommit repository:
ERROR: CommandError - An error occurred while handling git command.
Error code: 128 Error: fatal: unable to access 'https://git-codecommit.us-west-2.amazonaws.com/v1/repos/origin/': The requested URL returned error: 403
I have already created an environment using aws beanstalk, how I should push to that.
This issue could be related to your AccessKey and SecurityKey. Perhaps AK or SK has expired/inactivaded.
In my case with the same error, the problem was an expired key.
After releasing a new key, make sure the git credential store have the new key (if used).
I fixed this by:
Attaching AWSCodeCommitPowerUser to my user group
Generating a CodeCommit credential for my user.
STEP 1
Sign in to the AWS Management Console and open the IAM console at https://console.aws.amazon.com/iam/.
In the IAM console, in the navigation pane, choose Users, and then choose the IAM user you want to configure for CodeCommit access.
On the Permissions tab, choose Add Permissions.
In Grant permissions, choose Attach existing policies directly.
From the list of policies, select AWSCodeCommitPowerUser or another managed policy for CodeCommit access. For more information, see AWS managed (predefined) policies for CodeCommit.
STEP 2
On the user details page, choose the Security Credentials tab, and in HTTPS Git credentials for AWS CodeCommit, choose Generate.
Use the Username and Password when prompt to enter credentials for the git repo
Take a look at this article for more information https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-gc.html

How to create app with credentials in OpenShift?

Try to create Node.js app in OpenShift in terminal, like this:
./oc new-app https://j4nos#bitbucket.org/j4nos/nodejs.git
Source code in BitBucket in a private account, how to set credentials? Once it asked for password, but not again. How can I set credentials?
Added annotated secret from GUI: repo-at-bitbucket
I have read Private Git Repositories: Part 2A tutorial, strange that for HTTPD app there is a Source Secret filed to select secret, but not when Node.js + MongoDB combo is selected. Why?
Ahh .. need to select pure Node.js app.
You need to authenticate to the private git repository. This can be done a few different ways. I would suggest taking a few a minutes and reading this blog series which outlines the different methods you can take.
https://blog.openshift.com/private-git-repositories-part-1-best-practices/
After reading first through initial few posts explaining concepts and doing it with GitHub, only then look at the BitBucket example.
https://blog.openshift.com/private-git-repositories-part-5-hosting-repositories-bitbucket/
Those GitHub examples have more explanation which will then make BitBucket example easier to understand.
The likely reason you were prompted for the password when running oc new-app is that you used:
oc new-app https://j4nos#bitbucket.org/j4nos/nodejs.git
Specifically, you didn't specify a S2I builder to use. As a result, oc new-app will try and checkout the repo locally to analyse it to try and work out what language it uses. This is why it would prompt for the password separately.
It is better to specify the builder name on the command as:
oc new-app nodejs~https://j4nos#bitbucket.org/j4nos/nodejs.git
This is an abbreviated form of the command and is the same as running:
oc new-app --strategy=source --image-stream nodejs --code https://j4nos#bitbucket.org/j4nos/nodejs.git
If you specify the builder, it already knows what to use and doesn't analyse the code so will not prompt for the password, plus you wouldn't need user in the URI.
Either way, when building in OpenShift you still need the basicauth secret and should annotate it so it knows to use the secret for that build.

Openshift- git clone gives Permission denied (publickey,gssapi-keyex,gssapi-with-mic)

I have created a new account with openshift online and have created my first app. I have rhc installed on my local machine. I setup ssh keys for the first time using rhc setup -l loginname
i used git clone to clone the remote repo and it worked fine. Then however after changes and tring to do a git push gave me the following error:
Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
I tried a couple of solutions provided in this forum like using ssh-add etc. but this did not work.I then made sure that all existing ssh keys under .ssh directory and from my openshift online account were deleted and all identities managed by ssh-agent were also deleted. I then launched rhc setup again This created the default ssh keys again and asked me if it could upload the public key which I selected yes to.
However then it gives me the following error:
An SSH connection could not be established to appname-domain-name.rhcloud.com.
Your SSH configuration may not be correct, or the application may not be
responding. connection closed by remote host (Net::SSH::Disconnect)
Kindly help.
I had the same problem while trying to clone from command line...
C:\> git clone GIT_URL DIRECTORY_TO_CREATE
Finally, cloning from the GIT UI solved the problem. Go to menu item All Programs --> GIT --> Git GUI and select "Clone existing repository".

How to setup Authorization Hudson /Jenkins to clone your mercurial repository

After installing and playing around with mercurial , I am trying to get Hudson to clone the repository so it can build my project.
At the moment the following task works.
I Can sync to my external host and the code shows up on that host.
Now I am trying to configure hudson / jenkins to access the code on my host.
But unfortunately I am rolling on a error:
Started by user anonymous
$ hg clone --rev default https://bitbucket.org/*/testproject "F:\Hudson\jobs\testproject\workspace"
abort: http authorization required
ERROR: Failed to clone https://bitbucket.org/*/testproject
[workspace] $ hg log --rev . --template {node}
java.io.IOException: Cannot run program "hg" (in directory "F:\Hudson\jobs\testproject\workspace"): CreateProcess error=267, The directory name is invalid
at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
at hudson.Proc$LocalProc.<init>(Proc.java:244)
at hudson.Proc$LocalProc.<init>(Proc.java:216)
at hudson.Launcher$LocalLauncher.launch(Launcher.java:698)
at hudson.Launcher$ProcStarter.start(Launcher.java:329)
at hudson.Launcher$ProcStarter.join(Launcher.java:336)
at hudson.plugins.mercurial.MercurialSCM.joinWithPossibleTimeout(MercurialSCM.java:298)
at hudson.plugins.mercurial.HgExe.popen(HgExe.java:191)
at hudson.plugins.mercurial.HgExe.tip(HgExe.java:171)
at hudson.plugins.mercurial.MercurialSCM.calcRevisionsFromBuild(MercurialSCM.java:254)
at hudson.scm.SCM._calcRevisionsFromBuild(SCM.java:304)
at hudson.model.AbstractProject.calcPollingBaseline(AbstractProject.java:1186)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1175)
at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:523)
at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:418)
at hudson.model.Run.run(Run.java:1362)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:145)
Caused by: java.io.IOException: CreateProcess error=267, The directory name is invalid
at java.lang.ProcessImpl.create(Native Method)
at java.lang.ProcessImpl.<init>(ProcessImpl.java:81)
at java.lang.ProcessImpl.start(ProcessImpl.java:30)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
... 18 more
Finished: FAILURE
What actions do i need to do to tell Hudson to use username x and password y to acces the data?
Edited => Found how to integrate ssh .
Used SSH instead of https
Download putty.exe, puttygen.exe, pageant.exe, and plink.exe from the PuTTY website.
Start puttygen and generate a key in OPENSSH FORMAT (hudsons format) (=> How to use Svn + SSH )
Click the Save private key button and save the .PPK file somewhere.
Click the Save public key button and save it.
Go to your website and enter the public ssh-key
Run pageant.exe. The pageant icon (a computer wearing a hat) will show up in the status tray.
Right-click the pageant icon and choose Add Key.
Choose the .PPK file you saved earlier and type in its passphrase.
The following (end part is copied) from Ted Naleid (Thank you!) blog witch can be found here : Hooking up hudson to your ...
Install the Mercurial plugin in Hudson
All that’s left to do now is install
the Mercurial plugin in hudson. In a
browser, go to
http://INSERT_YOUR_IP_HERE:8080.
Hudson should come up.
Click on “Manage Hudson” and go to
“Manage Plugins”. Go to the
“Available” tab, check “Hudson
Mercurial plugin” and hit the
“Install” button. Hudson will prompt
you to restart, and then it’s
installed.
After that, just create a new job and
you’ll have a new “mercurial” option
in the “source control management”
section. Select that and put the ssh
URL in the “Repository URL” field.
Then put “default” in the “branch”
field and set up the rest of the job
to build/test your code (an exercise
left to the reader).
and here it is the first succesfull build !
Conclusion : This is a summary of all the small blogpost scattered arround the internet. I hope this post helps you in starting hudson and mercurial.
I think the problem is not related to username and password. Your stacktrace tells you there's something wrong with the path F:\Hudson\jobs\testproject\workspace.
Cannot run program hg (in directory
"F:\Hudson\jobs\testproject\workspace")
The directory name is invalid
Anyway, you can specify the username and password in the URL like: http://user:password#mydomain.org.
To authenticate the Jenkins/Hudson Mercurial plugin with BitBucket I too found it useful to use the SSH protocol instead of HTTPS particularly since:
there doesn't seem to be a way to store your HTTPS credentials to BitBucket with the Mercurial Jenkins plugin, but with SSH you can safely and securely store your credentials
with SSH you can configure it to use compression, which Mercurial doesn't do natively.
Good instructions for setting up SSH access to BitBucket are available here: http://confluence.atlassian.com/display/BITBUCKET/Using+SSH+to+Access+your+Bitbucket+Repository
Notes:
If you are running Jenkins/Hudson on a *nix server, you will want to login as the user running the Jenkins process and perform these operations from that users home directory, so the configurations will be found by that user (e.g. on my Debian server installation of Jenkins standalone, the user 'jenkins' is created and the home directory is set to '/var/lib/jenkins' [not /home/jenkins] - where I performed the instructions provided at the above link).
I found it very helpful to assure the hg clone command worked from the command line before attempting to have Jenkins call it.
IMPORTANT: In order to get this to work, I had to generate a key ** without ** a passphrase.
You can add the following lines to jenkins .hgrc file (usually /var/lib/jenkins/.hgrc)
[auth]
bitbucket.prefix = https://bitbucket.org/your_user/...
bitbucket.username = your_user
bitbucket.password = ******
See http://www.selenic.com/mercurial/hgrc.5.html#auth
You can add your scm credentials in the 'Credentials` section of Jenkins:
Also change the job configuration to use the credentials: