Unable to install dev-master version of package from Satis repository - yii2

I'm developing a couple of composer packages (mainly related to the Yii2 framework) served by private Satis repository. Most of them is at the very beginning of development process, so they should be available as dev-master version due to master branch location within GIT repository.
The problem occurs when I'm trying to install dev-master of any package, i.e. foo/my-package:
./composer.phar require foo/my-package
Problem 1
- The requested package foo/my-package dev-master exists as foo/my-package[dev-master] but these are rejected by your constraint.
so, if dev-master exists, please install this version for me anyway:
./composer.phar require foo/my-package dev-master
[InvalidArgumentException]
Could not find package foo/my-package in a version matching dev-master
Now i'm confused. I've tried to install #dev version of foo/my-package and magically everything works, dev-master (!) version has been installed:
./composer.phar require foo/my-package #dev
Package operations: 6 installs, 0 updates, 0 removals
- Installing yiisoft/yii2-composer (dev-master 0933fd9): Downloading (100%)
- Installing yidas/yii2-bower-asset (2.0.12): Downloading (100%)
- Installing cebe/markdown (1.1.2): Downloading (100%)
- Installing ezyang/htmlpurifier (v4.14.0): Downloading (100%)
- Installing yiisoft/yii2 (2.0.12.2): Downloading (100%)
- Installing foo/my-package (dev-master ce0ac57): Cloning ce0ac5752b from cache
What is the main problem? my-package GIT repository have also feature/#1234 branch. When Satis repository generator is configured to serve all versions of all packages ({"require-all": true}) instead of configuration shown below, using #dev as package version during composer require operation will cause installation of source code from feature/#1234 branch (no matter if it's HEAD points to patch newer or older than masters' branch HEAD). I would like to setup Satis with require-all as true and install packages as dev-master in this case (but dev-feature/#1234 in other project); but first of all I need to understand why it's behave that way.
Composer version 1.10.25 2022-01-21 10:02:15 (it's required to use version prior to v2)
Satis configuration (main.json)
{
"name": "foo/bar",
"description": "foo",
"homepage": "https://foo.bar",
"repositories": [
{
"type": "git",
"url": "ssh://user#domain/path/foo-my-package.git"
},
...
],
"require": {
"foo/my-package": "dev-master",
...
}
"archive": {
"directory": "dist",
"format": "tar",
"checksum": true
}
}
foo/my-package composer file (composer.json)
{
"name": "foo/my-package",
"description": "description",
"type": "yii2-extension",
"license": "proprietary",
"keywords": [
"yii2",
"extension",
],
"minimum-stability": "dev",
"require": {
"php": ">=7.1.0",
"yidas/yii2-bower-asset": "*",
"yiisoft/yii2": "~2.0.12.2"
},
"autoload": {
"psr-4": {
"foo\\mypackage\\": "src"
}
},
"repositories": [
{
"type": "composer",
"url": "https://foo.bar"
}
],
"config": {
"preferred-install": {
"foo/*": "source",
"*": "dist"
}
}
}
Sandbox project configuration where I'm trying to install foo/my-package, in state before installing #dev version of the package (composer.json)
{
"name": "composer/sandbox",
"description": "description",
"keywords": [],
"type": "project",
"license": "proprietary",
"support": {},
"minimum-stability": "dev",
"require": {},
"require-dev": {},
"config": {
"process-timeout": 1800,
"allow-plugins": {
"yiisoft/yii2-composer": true
},
"preferred-install": {
"foo/*": "source",
"*": "dist"
}
},
"repositories": [
{
"type": "composer",
"url": "https://foo.bar"
}
]
}

Related

PhpStorm: what are php-cs-fixer files and how to ignore them

I keep getting "Multiple definitions exist for class..." warnings in PhpStorm, and upon inspection, I see these a bunch of these huge php-cs-fixer files (100K+ lines) with the comment "This file is part of PHP CS Fixer.".
I found that there are multiple copies of several other files named phploc, composer, php-scoper, etc., under vendor/library_name/tools/ in various libraries for some reason. They are all huge compiled files that PhpStorm detects.
I tried ignoring these files in PhpStorm one by one, and once re-indexing finishes, these files disappear, leading me to believe they're IDE-generated files. However, it makes no sense the IDE would generate them and in turn include them hinting code.
composer.json
{
"name": "magento/project-community-edition",
"description": "eCommerce Platform for Growth (Community Edition)",
"type": "project",
"license": [
"OSL-3.0",
"AFL-3.0"
],
"config": {
"preferred-install": "dist",
"sort-packages": true
},
"require": {
"magento/composer-root-update-plugin": "~1.0",
"magento/product-community-edition": "2.4.1"
},
"require-dev": {
"allure-framework/allure-phpunit": "~1.2.0",
"dealerdirect/phpcodesniffer-composer-installer": "^0.5.0",
"friendsofphp/php-cs-fixer": "~2.16.0",
"lusitanian/oauth": "~0.8.10",
"magento/magento-coding-standard": "*",
"magento/magento2-functional-testing-framework": "^3.0",
"pdepend/pdepend": "~2.7.1",
"phpcompatibility/php-compatibility": "^9.3",
"phpmd/phpmd": "^2.8.0",
"phpstan/phpstan": ">=0.12.3 <=0.12.23",
"phpunit/phpunit": "^9",
"sebastian/phpcpd": "~5.0.0",
"squizlabs/php_codesniffer": "~3.5.4"
},
"conflict": {
"gene/bluefoot": "*"
},
"autoload": {
"psr-4": {
"Magento\\Framework\\": "lib/internal/Magento/Framework/",
"Magento\\Setup\\": "setup/src/Magento/Setup/",
"Magento\\": "app/code/Magento/",
"Zend\\Mvc\\Controller\\": "setup/src/Zend/Mvc/Controller/"
},
"psr-0": {
"": [
"app/code/",
"generated/code/"
]
},
"files": [
"app/etc/NonComposerComponentRegistration.php",
"app/helper.php"
],
"exclude-from-classmap": [
"**/dev/**",
"**/update/**",
"**/Test/**"
]
},
"autoload-dev": {
"psr-4": {
"Magento\\Sniffs\\": "dev/tests/static/framework/Magento/Sniffs/",
"Magento\\Tools\\": "dev/tools/Magento/Tools/",
"Magento\\Tools\\Sanity\\": "dev/build/publication/sanity/Magento/Tools/Sanity/",
"Magento\\TestFramework\\Inspection\\": "dev/tests/static/framework/Magento/TestFramework/Inspection/",
"Magento\\TestFramework\\Utility\\": "dev/tests/static/framework/Magento/TestFramework/Utility/",
"Magento\\PhpStan\\": "dev/tests/static/framework/Magento/PhpStan/"
}
},
"version": "2.4.0",
"minimum-stability": "stable",
"repositories": [
{
"type": "composer",
"url": "https://repo.magento.com/"
}
],
"extra": {
"magento-force": "override"
}
}
PhpStorm ignored files (adding php-cs-fixer and phploc here somehow removed all/multiple copies from the vendor directories):
These files are in your vendor directory as you must have install your dependencies from sources, not distributions.
Let's use sebastian/code-unit as an example to see how it works:
the file obviously is in the repository: https://github.com/sebastianbergmann/code-unit/tree/1.0.8/tools
but excluded from being exported to distribution (the archive file you are actually downloading from GitHub when installing package): https://github.com/sebastianbergmann/code-unit/blob/1.0.8/.gitattributes#L6 - you can see that it's not present in file you download from https://github.com/sebastianbergmann/code-unit/releases/tag/1.0.8
So, you must have install your dependencies using composer install --prefer-source (or composer update --prefer-source as it's the other command with that option) and you want to use --prefer-dist or actually not using any of these flags as the latter is the default. You can find more information about it in official documentation: https://getcomposer.org/doc/03-cli.md#install-i
Remove your vendor directory and install dependencies once more without --prefer-source flag.
AFAIR there was also an issue when you did not have curl extension installed, then simply add this extension.

"UnexpectedValueException" on yii2 composer update

Got "UnexpectedValueException" Your github oauth token for github.com contains invalid characters:"" while composer update on yii2 project.
When i was installing jui datetimepicker via composer, I have asked to give git token to install it. I had generated the token from GitLab and provide the token to terminal.I also change the composer.json file with token. But it gives me an error of "UnexpectedValueException" Your github oauth token for github.com contains invalid characters.
I have tried following solutions :
1) I have deleted my .git folder and tried to update composer still same issue
2) I have deleted the vendor and reinstall the composer not worked.
3) I have deleted the vendor and composer.json and put the previous vendor and composer.json file from my backup. but still get same issue.
4) I have tried to change the token from the following code :
composer config --global github-oauth.github.com <TOKEN>
5) Also put the following code in composer.json :
{
"config": {
"github-oauth": {
"github.com": "<TOKEN>"
}
}
}
Still same issue.
My composer.json file is as follows :
{
"name": "yiisoft/yii2-app-advanced",
"description": "Yii 2 Advanced Application Template",
"keywords": ["yii2", "framework", "advanced", "application template"],
"homepage": "http://www.yiiframework.com/",
"type": "project",
"license": "BSD-3-Clause",
"support": {
"issues": "https://github.com/yiisoft/yii2/issues?state=open",
"forum": "http://www.yiiframework.com/forum/",
"wiki": "http://www.yiiframework.com/wiki/",
"irc": "irc://irc.freenode.net/yii",
"source": "https://github.com/yiisoft/yii2"
},
"minimum-stability": "stable",
"require": {
"php": ">=5.4.0",
"yiisoft/yii2": "*",
"yiisoft/yii2-bootstrap": "*",
"yiisoft/yii2-swiftmailer": "*",
"claudejanz/yii2-mygii": "dev-master",
"bower-asset/jquery": "#stable",
"bower-asset/jquery.inputmask": "#stable",
"bower-asset/punycode": "#stable",
"bower-asset/bootstrap": "#stable",
"3ch3r46/bootui-datepicker": "1.0.0",
"yiisoft/yii2-jui": "*",
"udokmeci/yii2-phone-validator" : "dev-master"
},
"require-dev": {
"yiisoft/yii2-codeception": "*",
"yiisoft/yii2-debug": "*",
"yiisoft/yii2-gii": "*"
},
"suggest": {
"codeception/codeception": "Codeception, 2.0.* is currently works well with Yii.",
"codeception/specify": "BDD style code blocks for PHPUnit and Codeception",
"codeception/verify": "BDD Assertions for PHPUnit and Codeception",
"yiisoft/yii2-faker": "Fixtures generator for Yii2 based on Faker lib"
},
"scripts": {
"post-create-project-cmd": [
"yii\\composer\\Installer::setPermission"
]
},
"config": {
"process-timeout": 1800
},
"extra": {
"writable": [
"backend/runtime",
"backend/web/assets",
"frontend/runtime",
"frontend/web/assets"
],
"executable": [
"yii"
],
"asset-installer-paths": {
"npm-asset-library": "vendor/npm",
"bower-asset-library": "vendor/bower"
}
},
"repositories": [
{
"type": "composer",
"url": "https://asset-packagist.org"
}
]
}
Please someone help.
I'm not sure what was the reason why it stopped working for you but as we discussed in comments the solution was to update the entry in auth.json that stores this GitHub token.
For everyone looking for this solution - auth.json is stored in your COMPOSER_HOME path. You can find this path by calling in the console
composer config -g data-dir
For more information about Composer configuration please refer to the Composer documentation at getcomposer.org.

Load composer package from private gitlab

In my composer setup I load packages from 2 repos. One composer repo and a private gitlab repo. When I try to do a composer install on my local windows machine, I get the following error messages:
GitLab: The project you were looking for could not be found. fatal:
Could not read from remote repository. Please make sure you have the
correct access rights and the repository exists. ... The requested
package sv-test/testextension could not be found in any version, there
may be a typo in the package name.
Authentification is done via lokal ssh-key and password, that doesnt seem to be the problem. Whats wrong with my setup?
The composer.json of the project looks like this:
{
"repositories": [
{
"type": "composer",
"url": "https://composer.typo3.org/"
},
{
"type": "git",
"url": "git#gitlab.xydevbox.de:sv-test/Testproject.git"
}
],
"name": "svdev/master-dev-box",
"description": "",
"type": "project",
"license": "MIT",
"homepage": "https://www.xydevbox.de/",
"authors": [
{
"name": "Sacha Vorbeck",
"email": "sacha.vorbeck#xydevbox.de",
"homepage": "https://www.xydevbox.de/",
"role": "Developer"
}
],
"require": {
"sv-testbox/testextension": "*",
"helhum/typo3-console": "^4.5",
"typo3/cms": "^8.7"
},
"config": {
"sort-packages": true,
"process-timeout": 2000,
"preferred-install": {
"typo3/cms": "source",
"*": "dist"
}
},
"extra": {
"typo3/cms": {
"cms-package-dir": "{$vendor-dir}/typo3/cms",
"web-dir": "web"
}
}
}
The composer.json from the package to be included from the gitlab private repo looks like this:
{
"name": "sv-testbox/testextension",
"type": "typo3-cms-extension",
"description": "",
"homepage": "https://www.xydevbox.de/",
"license": ["GPL-2.0+"],
"keywords": ["TYPO3 CMS"],
"version": "master",
"dist": {
"url": "git#gitlab.xydevbox.de:sv-test/Testproject.git",
"type": "git"
}
}
Maybe the latest composer version 1.5.2 is something for you if you take a look at the release notes
Fixed GitLabDriver looping endlessly in some conditions
Fixed
GitLabDriver support for unauthenticated requests
Fixed GitLab zip
downloads not triggering credentials prompt if unauthenticated
Fixed
path repository support of COMPOSER_ROOT_VERSION, it now applies to
all path repos within the same git repository
Fixed path repository
handling of copies to avoid copying VCS files and others
Fixed
sub-directory call to ignore list and create-project commands as well
as calls to Composer using --working-dir
Fixed invalid warning
appearing when calling remove on an non-stable package
https://github.com/composer/composer/releases
Thank you Georg and NextThursday. With some help on TYPO3 slack I finally got it running. The replace part was missing. I also learned that one should not edit composer.json files manually - always use the command line options to modify it. This example: https://github.com/TYPO3-Console/TYPO3-Console/blob/master/composer.json was also helpful.
project composer.json:
{
"repositories": [
{
"type": "composer",
"url": "https://composer.typo3.org/"
},
{
"type": "vcs",
"url": "https://github.com/svorbeck/masterconfig"
}
],
"name": "svorbeck/demo",
"description": "",
"type": "project",
"license": "MIT",
"homepage": "https://xydevbox.de/",
"authors": [
{
"name": "Sacha Vorbeck",
"email": "sacha.vorbeck#xydevbox.de",
"role": "Developer"
}
],
"require": {
"svorbeck/masterconfig": "dev-master",
"typo3/cms": "^8.7"
},
"config": {
"sort-packages": true,
"process-timeout": 2000,
"preferred-install": {
"typo3/cms": "source",
"svorbeck/masterconfig": "source",
"*": "dist"
}
},
"extra": {
"typo3/cms": {
"cms-package-dir": "{$vendor-dir}/typo3/cms",
"web-dir": "web"
},
"helhum/typo3-console": {
"install-extension-dummy": "0"
}
}
}
ext composer.json:
{
"name": "svorbeck/masterconfig",
"type": "typo3-cms-extension",
"description": "svorbeck master configuration",
"require": {
"typo3/cms-core": "^8.7"
},
"replace": {
"masterconfig": "self.version",
"svorbeck/masterconfig": "self.version"
}
}

Getting extension from private repo with composer

I have to create a private yii2 extension on BitBucket, and install it in my projects with composer.
The SSH is set up correctly as far as I can understand, it was not at first - and I got an error "repo not found or you do not have permission" (or something of the kind).
The composer finds the composer.json, I removed it to test and I got an error message that "valid composer.json not found".
Now when i run composer update in my Git Bash it just prints "Nothing to install or update" and nothing appears in my vendor/ folder.
I have no errors to go on here, any ideas are mostly welcome!
I've set up the repo and added the following files:
composer.json
{
"name": "ext. name",
"type": "yii2-extension",
"description": "My desc",
"authors": [
{
"name": "Jorgen",
"email": "jorgen#domain.com",
"homepage": "http://www.domain.com"
}
],
"minimum-stability": "stable",
"support": {
"source": "bitbucket.org/companyname/yii2-extensionname"
},
"require": {
"php": ">=5.4.0",
"yiisoft/yii2": "2.0.*"
},
"autoload": {
"psr-4": {
"companyname\\extensionname\\": ""
}
}
}
.gitignore
vendor/**
models/Test.php
<?php
namespace companyname\extensionname\models;
class Test
{
}
And I've added the following to my Yii2 project composer.json:
"repositories": [
{
"type": "vcs",
"url": "ssh://git#bitbucket.org/companyname/yii2-extensionname",
}
],
Update
So I figured out that if I add my repo in this format it's found and downloaded, but the autoloader-psr4.php file is not updated with my autoloading.
"repositories": [
{
"type": "package",
"package": {
"name": "vendor/yii2-extension-name",
"version": "master",
"source": {
"type": "git",
"url": "git#bitbucket.org:vendor/yii2-extension-name",
"reference": "origin/master"
}
}
}
When I have
"repositories": [
{
"type": "git",
"url": "https://bitbucket.org/vendor/yii2-extension-name"
}
],
I get the following error:
Problem 1
- The requested package vendor/yii2-extension-name could not be found in any version, there may be a typo in the package name.
Potential causes:
- A typo in the package name
- The package is not available in a stable-enough version according to your minimum-stability setting
see https://groups.google.com/d/topic/composer-dev/_g3ASeIFlrc/discussion for more details.
Read https://getcomposer.org/doc/articles/troubleshooting.md for further common problems.
There is no typos, its found and downloaded when I use "package". But then the autoloader is not updated.I've also tried all sorts of variations with git#, https and ssh. It always works as a package, never as a git or vcs.
Edit: oh, and minimum-stability is set to dev while troubleshooting.
You must use the correct name:
{
"name": "vendor/yii2-extension-name",
"type": "yii2-extension",
That is usually your Bitbucket username followed by the name of your repository.
That will work with your repositories Composer setting:
"repositories": [
{
"type": "git",
"url": "https://bitbucket.org/vendor/yii2-extension-name"
}
],

Configuring composer.json with private Bitbucket Mercurial repository

My project uses my own library which is in the private Mercurial repository placed on bitbucket.org. That library has no composer.json configured.
I try to make that library as a dependency to my project.
Firstly I wrote to composer.json the following strings:
{
"require": {
"php": ">=5.4",
"myname/mylibname": "dev"
},
"repositories":[
{
"type": "hg",
"url" : "https://bitbucket.org/myname/mylibname"
}
]
}
And running composer install I've got an error:
[RuntimeException]
Failed to clone https://bitbucket.org/myname/mylibname, could not read packages from it
abort: http authorization required
Than I changed "type": "hg" to "type": "vcs" and got another error:
[Composer\Repository\InvalidRepositoryException]
No valid composer.json was found in any branch or tag of https:/***/mylibname, could not load a package from it.
After additional reading of documentation I added description of my library to the composer.json of my project, and it began to look so:
{
"require": {
"php": ">=5.4",
"myname/mylibname": "dev"
},
"repositories":[
{
"type": "vcs",
"url" : "https://bitbucket.org/myname/mylibname"
},
{
"type":"package",
"package":{
"name":"myname/mylibname",
"version": "dev",
"source":{
"type":"vcs",
"url":"https://bitbucket.org/myname/mylibname",
"reference":"dev"
}
}
}
]}
The same error occured:
[Composer\Repository\InvalidRepositoryException]
No valid composer.json was found in any branch or tag of https:/***/mylibname, could not load a package from it.
I removed the part:
{
"type": "vcs",
"url" : "https://bitbucket.org/myname/mylibname"
},
and got an error:
[InvalidArgumentException]
Unknown downloader type: vcs. Available types: git, svn, hg, perforce, zip, rar, tar, gzip, phar, file.
I changed "type": "vcs" back to "type": "hg", composer.json looks like:
{
"require": {
"php": ">=5.4",
"myname/mylibname": "dev"
},
"repositories":[
{
"type":"package",
"package":{
"name":"myname/mylibname",
"version": "dev",
"source":{
"type":"hg",
"url":"https://bitbucket.org/myname/mylibname",
"reference":"dev"
}
}
}
]}
and an error:
[RuntimeException]
Failed to execute hg clone 'https:/***/mylibname' '/path/to/myproject'
abort: http authorization required
The structure of my auth.json, which lies besides of composer.json is:
{
"http-basic": {
"bitbucket.org": {
"username": "myusername",
"password": "mypassword"
}
}
}
Seems like bitbucket-oauth method is buggy in the current state as of 1.1 of composer. This means that either you must have setup the SSH key on the client or if you are like me and cant setup keys because of deployment server, you will have to use basic auth.
The only way I got this working was:
~/.composer/auth.json
{
"http-basic": {
"bitbucket.org": {
"username": "bitbucketUsername",
"password": "PasswordToBitbucket"
}
}
}
composer.json
"repositories": [
{
"url": "https://username#bitbucket.org/username/my-package.git",
"type": "git"
}
],
"require": {
"username/my-package": "dev-master"
},
Composer as of version 1.2.0 have sorted this with bitbucket oauth, this is a much better method than ssh-keys if multiple developers are working on a project as the auth.json can stay within the project repository (if it's private) and only has to be setup once.
auth.json
{
"bitbucket-oauth": {
"bitbucket.org": {
"consumer-key": "key",
"consumer-secret": "secret"
}
}
}
composer.json
"repositories":[
{
"type": "vcs",
"url": "git#bitbucket.org:path/to.git"
}
]
Just remove https://. Works for me :)
{
"require": {
"php": ">=5.4",
"myname/mylibname": "dev"
},
"repositories":[
{
"type":"package",
"package":{
"name":"myname/mylibname",
"version": "dev",
"source":{
"type":"hg",
"url":"bitbucket.org/myname/mylibname",
"reference":"dev"
}
}
}
]}
That didn't quite work for me, but it got me pointed into the right direction. Make sure you get your SSH key installed to access it via git#.
{
"repositories": [
{
"type": "package",
"package": {
"name": "myname/mylibname",
"version": "master",
"source": {
"type": "git",
"url": "git#bitbucket.org:myname/mylibname.git",
"reference": "master"
}
}
}
]
}
One comment on my end.
I have tested above scenarios I encountered on composer suggestion that repository needs to have at least one stable version.
https://getcomposer.org/doc/04-schema.md#minimum-stability
Due to this, I used "dev" TAG along with SSH connection and it works.
{
"require": {
"php": ">=5.4",
"myname/mylibname": "dev"
},
"repositories":[
{
"type":"package",
"package":{
"name":"myname/mylibname",
"version": "dev",
"source":{
"type":"git",
"url":"git#bitbucket.org:myname/mylibname.git",
"reference":"dev"
}
}
}
]}
I thought I had best contribute to the confusion and share what configuration worked for me. Firstly, I absolutely could not get the recommended setup from composer to work. However, the following did:
1.Edit ~.composer/auth.json and configure the http-basic key.
{
"bitbucket-oauth": {},
"github-oauth": {},
"gitlab-oauth": {},
"gitlab-token": {},
"http-basic": {
"bitbucket.org": {
"username": "USERNAME",
"password": "PASSWORD"
}
}
}
2.Use the following to define the package in your composer.json (i.e. private repository). Also bare in mind this is not a personal BitBucket account, I am a part of a team so the USERNAME#bitbucket.org is MY username and second instance is the company (https://{USERNAME}#bitbucket.org/{USERNAME||VENDOR}/{REPO}.git).
"require": {
"{USERNAME||VENDOR}/{REPO}": "dev-{BRANCH}
}
"repositories:" [
{
"type": "package",
"package": {
"name": "{USERNAME/VENDOR/REPO}",
"version": "master",
"source": {
"url":
"https://{URL}",
"type": "git",
"reference": "master"
}
}
}
]
Miscellaneous & noteworthy things to consider:
I did not use access keys
I do have my SSH key on BitBucket
I am an administrator on BitBucket
I am target the master branch
There are NO tags on the repository
Minimum stability is set to dev
You will need to add the autoload options to the package (see this)
I hope this helps anyone frustrated by this, it defintely seems as though everybody has issues. Thanks to #Tomasz Czechowski for providing the answer that eventually got this working for me!
Another example with bitbucket git repo using OAuth consumer authorization, which is actually working. Note that repo is starting with https, otherwise composer will fail with an error 'undefined offset: 1'.
auth.json
{
"bitbucket-oauth": {
"bitbucket.org": {
"consumer-key": "...",
"consumer-secret": "..."
}
}
}
composer.json
"repositories": [
{
"type": "git-bitbucket",
"url": "https://bitbucket.org/workspace/repo.git"
}
]