private add-on to a public project using Mercurial - mercurial

I have a small open-source C++ project, hosted on Bitbucket using Mercurial.
Now, I am developing a new feature which adds a couple of new files and new build targets; otherwise it does not change the existing files.
I have opened a new branch, but after I pushed it to the main repo I was told that I cannot make the new feature open .. so I closed the repo and started looking for a solution. I have two questions:
What would have been a good approach for this situation? I need something that allows me to synchronize fixes made to the common code between the public and private repo. I do not mind having the private code only locally. I found two things:
using private stage for the new branch; but I don't know how to get fixes I make in the secret branch to the common files over to the open repo
using subrepos; this would need some code restructuring, but might be cleaner .. it just bothers me that this is marked as "feature of last resort" in the documentation.
How do I fix the situation where I have already pushed the closed code the the repo? Would it help to strip the branch and push, or do I need to delete the repo from Bitbucket and create a new one?

Since I am happy with having the private branch only locally, I have done the following:
I stripped the private branch from the bitbucket repo as well as my local copy of that repo.
In my copy of the private repo, I merged the changes from the public to to private branch and also then copied any common files changed in the private branch into the public one.
Then I marked the whole private branch as 'secret', to keep it local.
For future work, I plan to using hg merge for moving changes from the public to the private branch and hg graft for the reverse direction.
As far as I can see, this should work...

Related

Get artifact from other repo

I have a private GitHub repository A that builds a library via GitHub Actions and uploads the built library as artifact at the end of the build workflow. This repository including the built library has to stay private due to license reasons.
Another public repository B should now link against that library during its GitHub Actions based build workflow. I only know the usual upload-artifact and download-artifact actions that can be used to manage actions within a workflow, but this does not seem like a solution to my problem.
Is there any official way to achieve what I want, is there any trick that could be used to make it work or is this simply not possible? If so, do you have recommendations for an alternative approach?
I realized it would be a lot easier to access release assets of repository A instead of workflow artifacts. There are multiple ready-to-use actions that allow uploading releases from within a workflow run.
I created a private and a public test repository where the private repository has a single release with a simple text file asset and the public repository has a simple Actions workflow that fetches the asset from the private repo. It uses GitHub CLI with a command like gh release download --repo <my/private-repo> <tag>. You find the repository here.

Error "Repository is unrelated" when pushing repository with sub-repository to bitbucket

There is repository (GameFramework) which I want to use as sub repository in another repositories.
I created a new main repository and then clone GameFramework into this repository and make GameFramework a sub repository. But when I'm trying to push main repository to bitbucket I'm getting error: repository is unrelated or repository is unrelated (in subrepo [path])
This is strange but when after error I clear bitbucket repository and then push again it works!
I taked the video https://youtu.be/WI86_3I2ZH0
Why is this happening?
Two repositories are considered unrelated by mercurial, if they do not have the same origin, thus were created independently.
Without the use of the --force option mercurial does not allow pushing to unrelated repositories.
In your case, you (or someone else) likely created a repository for the sub-project in each of the projects separately and independently - and not referencing the same repository as sub-repository.
Fixing the issue is a bit tricky. Likely the easiest approach is to change to one of the sub-repos. Pull from the other sub-repo and do a merge as needed. And also doing it the other way around.
You need to change .hgsub file.
By default this file has the next format:
[folder to sub repo] = [folder to sub repo]
You need to chane it on:
[folder to sub repo] = [sub repo url]

I cannot make my Morea repository private. How do I fix that?

When I try to make my Morea repository private, GitHub says that public forks cannot be made private. What do I do?
This was a bug in the original QuickStart instructions, which directed you to create a fork of the basic-template. They have now been fixed, but for those who have already set up a repo, please do the following:
(1) On your laptop, rename your morea directory (let's call it ics314f13/) to ics314f13-old/. Don't delete the directory, just rename it to something different.
(2) Go to GitHub, and delete your repository (i.e. ics314f13). You do this in the Settings page. While this appears scary, don't worry because you still have all your files in your local directory (i.e. ics314f13-old/). This step will mean that you've lost your commit history, but I assume that's not important. (If history is important, there are other more complicated ways to fix things, let me know.)
(2) Follow the revised QuickStart directions starting here:
http://morea-framework.github.io/quickstart.html#Createacourse
The new instructions direct you to create a new, empty GitHub repo. Use your original repo name, the one you just deleted (i.e. ics314f13).
You'll then download a new script (morea-vanilla-install.sh) and run it to initialize the contents of your empty repo with the contents of basic-template. The net effect is similar to a fork, but since you didn't actually fork, you'll be able to make your repo private if you want.
(3) Copy over the changed files from your "old" directory to your "new" directory.
You'll typically just copy over the file master/src/_config.yml and the entire directory master/src/morea/.
Advanced users like Cam might have created new navbar pages. He'll need to copy over these changes as well.
(4) Copy over scripts.
Copy over the morea-run-local.sh script from the "old" directory, and check to make sure everything looks OK.
Copy over the morea-publish.sh script from the "old" directory, and run it to push your changes to your new GitHub repo.
(5) Make your repo private.
You should now be able to go to the settings menu and make your repo private.

Mercurial: exposing a subset of history to the public

I'd like to publish a subset of an existing private repository into the public. Given two repositories, private and public, I want to do the following:
private contains the project's entire history, including confidential information.
public should contain a subset of private's history, minus the confidential information.
I can generate a new branch in private that takes the latest changeset and strips away all confidential information, but I don't want to share ancestors of this branch with public.
Question: How do I strip history from public while keeping the repositories related? Meaning, I need to be able to hg pull from public into private.
Update:
What makes this question different from https://stackoverflow.com/a/5516141/14731 is that I need to hide existing ancestors from public (versus hiding new heads).
https://stackoverflow.com/a/4034084/14731 might work, but I'm wondering if there is a better approach than merging against a disjoint head.
Upon further reflection, I think it makes sense that https://stackoverflow.com/a/4034084/14731 produces a disjoint head since the remote changeset really does represent a head without ancestors. On the upside, this approach has a minimal diskspace cost. The files are not duplicated on disk. You only end up paying a bit (85k on my end) for the extra metadata.
Here is how to implement this approach:
hg archive to extract the latest changeset from private's sanitized branch.
hg init to create a new repository from this changeset.
hg pull [private] --force to pull public (an unrelated repository) into private as a new disjoint branch.
At this point you have two options: Merging the disjoint head into private's sanitized branch, or not.
Option 1: Merged head
Advantages
The private repository can see a historical link between private and public.
Disadvantages
You can not push changes from private to public because doing so will push the ancestors you worked so hard to exclude. Why? hg push cannot exclude ancestors of a merge.
You need to interact with private directly in order to modify the sanitized branch.
Contributing patches from private to public becomes more difficult (since you cannot make use of the history metadata directly).
Option 2: Unmerged Head
Advantages
Ability to push changes from private to public without revealing private changesets. You can do this using hg push -b disjointBranch.
Disadvantages
You lose the historical link between public and its ancestors in `private.
I'm still looking for a more elegant solution. If you have a better answer, please post it.
If you real task is really "hide private data", not "show only small subset of history" (see difference) you can
Activate and use MQ Extension
Convert all changesets, which change non-public data, into mq-patches
Eliminate from patches all edits, non-related to private data handling
Replace in code all occurences of private data by some keywords
Edit related patches in queue (which now must replace keywords by values)
Push "polished" private-repo to public (with all mq-patches previously unapplied)
In order to have in future "safe push" add alias to private repo (which, when used, push only changesets without /if any/ applied patches), smth. like.
[alias]
spush = hg qpop -a && hg push
or, in more modern way, for Mercurial, which have support for Phases, always have mq-patches in secret phase (i.e unpublishable) and don't worry about applied|unapplied state before push
[mq]
secret = True
in private repo's .hgrc

Mirgrating from mercurial Bfiles to largefiles

I have used mercurial's bfiles extension for some time and it works fine. The only problems are installation and the special "hg bfadd" command.
Now that Mercurial 2.0 include the largefile extension I would like to switch.
Can't find any tools or guides on how to do it? Anyone tried it yet.
I have several repositories that all use the same store and have the following mercurial.ini.
[bfiles]
store=\\Someserver\Mercurial\bFilesStore
autostatus = true
autoupdate = true
autorefresh = true
autoput = *
You can find the documentation here : https://www.mercurial-scm.org/wiki/LargefilesExtension
To enable the extension add the following to your hgrc :
[extensions]
largefiles =
You can add a new large file with :
hg add --large thisfileislarge
About the migration, the readme.txt of the bfiles extension says something about a migrate.txt file ( https://bitbucket.org/gward/hg-bfiles/overview section "The future"). But I can't find the file anywhere on the repository, maybe he forgot to upload it.
There's also a mail on mercurial-devel about this : https://www.mercurial-scm.org/pipermail/mercurial-devel/2011-October/035161.html but nothing since then.
Maybe the better solution is to contact the author of bfiles about his status on the migration process and keep using the old extension until you have an answer ?
Either way, there's a lot of bug report about largefiles since the release of 2.0, so it's maybe a good idea to wait anyway :)
In the latest hg-bfiles version (from 2011-12-05), if you update to the migrate branch, you get this help file:
bfiles: migrating to largefiles
If you want to migrate from bfiles to the new largefiles extension in
Mercurial 2.0, you first have to decide: convert your repo or keep it?
Converting your repo
This is appropriate if:
you have a small repository
you know exactly where every clone of it is
you can replace every clone
It involves creating an new repo with .hgbfiles/ replaced by .hglf/.
This means that your changeset IDs will differ, so you cannot
pull/push between the old and new repos. You must replace every
existing clone with a clone of the converted repo.
The advantage of converting your repo is that you can say goodbye
forever to bfiles, and move into the future using largefiles alone.
The process is mostly automated by two shell scripts: convert-repo and
convert-store.
Use the convert-repo shell script to convert the repository itself. This is just a wrapper around "hg convert" that takes care
of all the fussy details needed to turn .hgbfiles/ into .hglf/.
It's easy to run:
./convert-repo SRC-REPO DST-REPO
The resulting DST-REPO is not yet ready to use: you still have
to convert the bfiles store to a largefiles store.
Use convert-store to turn the bfiles store into a largefiles store. You must have a local copy of the bfiles store -- so you
probably want to run this on the server where your bfiles store
lives. Again, it's easy:
./convert-store SRC-STORE DST-REPO/.hg/largefiles
Putting the store inside your DST-REPO is the easiest way to
make largefiles just work.
Keeping your repo
(Yes, that's the end of the file, there's no help on how to keep your repository)