Comment 1 Pierre Ossman 2024-08-21 14:03:12 CEST

For subversion, access is controlled based on group membership. This appears to be possible in GitLab as well, but is a paid feature:

https://docs.gitlab.com/ee/user/group/access_and_permissions.html#manage-group-memberships-via-ldap

That means we'll have to add and remove users manually, unfortunately.

On the bright side, you can add users to a GitLab group (a group of repositories, not users), which reduces the burden a bit.

Comment 2 Alexander Zeijlon 2024-08-21 15:58:41 CEST

We want to keep the hooks we currently have for SVN for external services, that we use to notify the developers when events occur related to development work.

Right now when working with SVN we have the following:

- Send email to shared email account when commits are done to the repo.
  - Contains the contents of a "git show" for the corresponding commit.

- Send message to a Slack channel.
  - Contains the commit message and the SVN reference number.

- Post a comment with the commit message to Bugzilla.
  - Only post to a Bugzilla bug if its ID can be found in the commit message.

There may be other ways of keeping developers up-to-date in GitLab, but let's at least move these three over to GitLab initially, to keep our workflow as familiar as possible.

Comment 3 Alexander Zeijlon 2024-08-21 16:51:16 CEST

For email, there seems to be a few ways to trigger these, but it is a bit dependent on how we want the workflow to be moving forward.

Notifications can be configured on a per-user per-project basis (by the user itself). By default, the setting is set to "Participate" which means that you are only going to get email notifications for issues, merge requests etc. that you have already participated in.

There are a few other options to choose from, and we will probably have to decide on a common setting for all developers.

I tested with the option "Watch" which enables notifications for all types of events on a project or groups of projects. What I then get is emails sent directly to my inbox with short status updates about the event. E.g. "Merge request created - View on GitLab". Each subsequent action in a Merge request is then sent in the same email thread as the first notification.

This looks like a good way to keep track of one's own workflow, but it is not really what we need to be consistent with the email hook we have in SVN.

Comment 4 Alexander Zeijlon 2024-08-21 17:22:30 CEST

We also have the option to send emails via the GitLab integration "Emails on push".

This integration can be configured to send email only when pushing to a specific branch, and it can be configured on a per-group basis.

This is much more in line with the SVN hook. For e.g. an MR, the email message contains each commit message and a single diff for all commits combined.

Having all commits represented by a single diff here is maybe not optimal when discussing individual commits in an email thread, but we should consider that these emails only will be sent after an MR has been reviewed and approved if we configure the integration to only send mails when merging/pushing to master.

Comment 5 Alexander Zeijlon 2024-08-23 10:25:44 CEST

For now, we have decided to not use the "Emails on push" integration, since it is not really something that we can incorporate efficiently in our imagined GitLab workflow.

The email-hook in our SVN repo is, for us, a necessary evil, that we have used as a means of doing code review. This is something we can likely be without since we are now migrating to GitLab, which enables code reviews through merge requests.

Comment 6 Pierre Ossman 2024-08-23 10:37:57 CEST

We will continue to use bugzilla for tracking tasks, and GitLab seems to have some support for delegating things there:

https://docs.gitlab.com/ee/user/project/integrations/bugzilla.html

Unfortunately, it doesn't seem to support the normal way of referring to bugzilla bugs ("bug XYZ").

I can find some discussion about it in their issue tracker:

https://gitlab.com/gitlab-org/gitlab-foss/-/issues/44658

It looks like they think you should do your own plugin for that. Perhaps we can fork the Bugzilla module that's included?

Comment 7 Pierre Ossman 2024-08-30 13:36:04 CEST

A git hook is now in place to update bugzilla whenever something is pushed to the master branch. Our subversion hook used the format of "svn log", so it felt natural that our git hook uses the format of "git log".

We've also added a link to the relevant commit in the GitLab UI for each commit, something we didn't have for subversion as we didn't have a web frontend for that.

Comment 8 Pierre Ossman 2024-08-30 13:37:23 CEST

Our goal is to use merges and merge requests. As such, code review will be done in the GitLab interface for merge requests.

That means we do not need any emails for each commit for code review, and we will not do any hooks or integrations for that.

Comment 9 Pierre Ossman 2024-08-30 13:43:07 CEST

There is a Slack integration already available for GitLab, which means we do not need to port over a git hook. It does behave a bit different, though.

Like the bugzilla hook, we are only interested in notifying what happens to the master branch. Fortunately, the integration can trigger on pushes to specific branches.

The difference is that a push will generally contain multiple commits. For the bugzilla hook, we examine and act on each commit individually. The Slack integration has no such option. It sends a single message with a shortlog of what is included. This means much less detail than our existing subversion solution.

Fortunately, the Slack integration makes links for each commit listed (and some other details). That means you can easily click on an interesting commit and get all the details about it.

Hopefully, this slightly adjusted workflow will be good enough. If not, we'll need to write a new integration or hook from scratch.

Comment 10 Pierre Ossman 2024-08-30 13:45:53 CEST

Did a check to make sure everything is included in our backups:

 * Repository data is included in git bundles, and I've verified they can be restored using "git bundle unbundle"
 * Repository hooks are included as a simple tar ball
 * Logos and avatars are separately included as a tar ball
 * Other settings are included in a database dump. The database fields are often encrypted, so I did not verify the contents. But I can see that the encryption key is also included in the backup.

Comment 11 Pierre Ossman 2024-09-03 17:15:56 CEST

Jenkins has now been moved to use git instead of subversion.

I've also configured subversion to not allow any more writes to the cenbuild tree three. I tested this by trying to lock a file, which is now blocked.

Comment 12 Pierre Ossman 2024-09-06 09:40:15 CEST

(In reply to Pierre Ossman from comment #11)
> Jenkins has now been moved to use git instead of subversion.

Apparently not. Our container for the cenbuild builds does its own checkout of the repo from subversion. We need an overhaul of how that thing works. :/

Comment 13 Pierre Ossman 2024-09-06 09:50:16 CEST

GitLab has a very broken way of resolving merge conflicts in the UI:

https://gitlab.com/gitlab-org/gitlab-foss/-/issues/35054

They've closed that with the very minimal adjustment of blocking that broken behaviour for protected branches (e.g. merge from "prod" to "dev"):

https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53463

No option to disable the button yet:

https://gitlab.com/gitlab-org/gitlab/-/issues/435082

And they are also very against allowing custom css to hide undesirable elements:

https://gitlab.com/gitlab-org/gitlab/-/issues/15199


I think we need to add an instance level git hook to block these broken merges, and see what we can do to help users not do the wrong thing.

Comment 14 Alexander Zeijlon 2024-09-06 16:20:57 CEST

I have been looking a bit into what alternatives we have to enforce our own policies for merging with e.g. the default branch.

There are a few things that we may want to have control over:

- Validation for commit messages.

  E.g. to enforce that all non-merge commits have tagged a Bugzilla bug.

  E.g. to validate correct lengths for commit titles and messages.

- More generally, allow us to turn off features that we are not able to do via
  the GitLab webui.

One way to achieve this could be to add pre-receive commit hooks that are run on the git server before commits are added. I found a good reference for how to write these, https://github.com/github/platform-samples/blob/master/pre-receive-hooks/README.md. See also the GitLab docs for git server hooks [1].

I also found that something similar to pre-receive hooks are configurable directly in GitLab, but only if you are using a Premium or Ultimate license [2].

[1] https://docs.gitlab.com/ee/administration/server_hooks.html
[2] https://docs.gitlab.com/ee/user/project/repository/push_rules.html

Comment 15 Pierre Ossman 2024-09-10 12:31:35 CEST

GitLab has a tonne of features, many of which we do not want and just clutter up the interface. Unfortunately, GitLab seem to have a general stance of insisting on showing everything. So they've been very reluctant to add the option of disabling things (e.g. see the CSS issue above).

Pretty much nothing can be disabled in the admin interface. Some things can be disabled in the system configuration (gitlab.rb), but not much. Many things can be hidden per project in the project settings. Not everything, though, but might have to be good enough for now.

Some things can be set to disabled by default, but again, not many. So we'll need to manually tweak the important repositories.

One annoying thing that remains are the "Add CHANGELOG", "Add CONTRIBUTING", etc. links on the right. You can hide this entire section in your user settings, but each user would need to do this manually. It also loses the other stuff in that section, mainly a shortcut to the README.

Comment 16 Pierre Ossman 2024-09-13 09:09:35 CEST

We don't care terribly much about the messages in our merge commits, as they will only be for internal branches. What we do want is:

 * Short messages, for people scrolling logs without --no-merges

 * Ideally consistent, no matter how you do the merge

 * Easy to find the associated merge request

The last point is trivially handled by GitLab. It knows which commits are part of a merge request. So you can simply search for the commit id, and it will include a link to the merge request just below the commit message.

For the contents of the message, GitLab allows setting a template (per repository, though, which is a hassle), but the git tool does not. So the easiest approach is to make GitLab follow git. We'll also mostly be merging local branches, so the format will be:

  Merged branch 'new-feature'

We've added a git hook that can enforce this.

Comment 17 Pierre Ossman 2024-09-13 09:13:09 CEST

GitHub backups and forked repositories unfortunately do not play nice. Whilst in GitLab, it uses git's alternates feature to share storage between forked repositories. But when doing a backup, it does a simple "git bundle" on each repository. This duplicates any shared objects for each fork.

They have an open issue for this, but no resolution yet:

https://gitlab.com/gitlab-org/gitaly/-/issues/3384

It's unclear how many forks we need, so let's keep an eye on it and see if it's an actual problem.

On the plus side, the "git bundle" process removes any duplication or unreachable objects that might be present in the live repos.

Comment 18 Pierre Ossman 2024-09-16 13:57:10 CEST

Since we only want to do merges to master, ideally only via merge requests, it sounds like GitLab's protected branches and "Allowed to merge" setting would be ideal. Unfortunately, that setting only allows merging via the web UI, which isn't always suitable.

I reported this problem here:

https://gitlab.com/gitlab-org/gitlab/-/issues/489827

For now, it looks like we'll either need to hope for the best, or write a git hook for this.

Comment 19 Pierre Ossman 2024-09-16 14:25:05 CEST

(In reply to Pierre Ossman from comment #6)
> 
> It looks like they think you should do your own plugin for that. Perhaps we
> can fork the Bugzilla module that's included?

Unfortunately, it doesn't look like anything decent came out of those discussions. I cannot find any system where you can add your own plugins. So the only option would be to modify the code of our GitLab instance.

Comment 20 Alexander Zeijlon 2024-09-20 14:40:10 CEST

> MUST:
>  * Developers should be able to still do all their current tasks after
>    migration
Since many of us have been using git-svn when working with the old SVN repo, the transition should be easy. What's new is that we are now requiring ourselves to work with merge requests. We have deployed git hooks that help us verify that we are not pushing commits in a format we do not want.

As a rule of thumb:
  - Always work with merge requests.
  - Never update feature branches via merge.
  - Never commit any changes directly on master.
  - Don't handle merge conflicts in the GitLab web UI (These will be denied by
    one of our hooks).

>   * Commit history in bugzilla
The commit history has been migrated.

>   * Commit notifications in Slack
Our Slack workspace has a slack bot that will notify when a merge commit and its parent commits are pushed to master.

>   * Code review
We are enabling code reviews via merge requests.

>  * Important non-obvious tasks for developers and sysadmins should be
>    documented
Our Wiki has been updated, See https://intranet.lkpg.cendio.se/DriftInfo/Gitlab

>  * Both the repository and gitlab configuration should have automatic backups
All Repos, their settings, Merge request comments, thumbnails etc. are backed up, see https://intranet.lkpg.cendio.se/DriftInfo/Gitlab#Backups

> SHOULD:
>  * Safety features that are inherent to subversion should have equivalent
>    protection in git (e.g. no changing history on the master branch)
Force pushing as a means of manipulating the master branch has been disabled in the GitLab web interface, though this can be toggled on if needed in the future. Note that if we need to manipulate the master history for some reason, we may also need to turn off some of our Git hooks.

>  * A git centric workflow should be easy
We have used an Git(Lab) centric workflow while e.g. working on the Git hooks for this bug. And it has worked well. The merge request interface is a good way to discuss and review code.

Note that since we are still aiming to use Bugzilla as our platform for discussions and bug reporting, GitLab issues have been disabled for the cenbuild repo.

>  * Dangerous git/gitlab features should be blocked
We think we have been able to disable most of the "dangerous" features we found directly in the web interface. E.g. force push to master, and squash commits when merging etc.

We have also deployed a few git hooks that run server side when we are pushing. These will stop a push if:
  - Non merge commits are pushed on the master branch.
  - Merge commits are pushed that merge master into other branches. This is
    what happens when GitLab tries to handle merge conflicts.

>  * Commits to the old subversion repository should be blocked
It is no longer possible to push to the SVN cenbuild repo.

> COULD:
>  * Unused gitlab features should be disabled
Everything in the cenbuild project settings that can be disabled has been disabled.

>  * Merge requests are automatically test built
For now, Jenkins will build only when changes are pushed to master (as it was with SVN). There is no CI yet.

>  * Bugzilla references should be usuable links
This could not be done in a robust way. Not done.

>  * Subversion revision references should be usable links
This could not be done in a robust way. Not done.

Comment 21 Alexander Zeijlon 2024-09-24 15:43:06 CEST

This bug has been verified continuously by the project members over the course of the project. Closing!

Comment 22 Alexander Zeijlon 2024-09-26 10:25:29 CEST

A CI pipeline has been created for the repo. This consists of a job in Jenkins that runs the rebuild script when a merge request is created or updated with new commits. It runs without the --no-repo option. This is much faster than building everything from scratch, which is what we want from a CI job that functions as a sanity check before merging changes to master.

The Jenkins job builds packages with the same docker image that we are using when changes are pushed to master, but it sets up a separate container.

See updated wiki on for the setup procedure, https://intranet.lkpg.cendio.se/DriftInfo/Gitlab -> Continuous Integration with Jenkins