Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gitlab 12.6

about.gitlab.com

Post reply on HN
Print view View original

21–30 of 30 posts

Re: Gitlab 12.6

#21

Post by sytse »

orf wrote: post #9
clon wrote: post #6

I am thankful for this release. It finally fixed two major issues that had plagued Gitlab for us for a long time: - The "Rebase" button now actually seems to work reliably with no cases of endless spinning to report in the few tens of button presses so far for me (we merge fast-forward only). Hoping this is not a fluke and will not regress in the future. - The "Hide whitespace changes" button works again after breaki…

Not all of those issues are bugs though. Only 4,000 are labeled as bugs, with many more being labeled as improvements or product direction discussions. Gitlab’s development runs through those issues, so there are a _lot_ that are related to that.

Correct. A lot of the issues are feature requests.

Re: Gitlab 12.6

#22

Post by Tainnor »

ipodopt wrote: post #18
orf wrote: post #15

Earlier quoted context omitted.

But... it just runs shell commands. It doesn’t encourage you to do more complex things, especially compared to other CI systems like Jenkins. My point is your complaint seems more general than Gitlab itself.

Well, let's see. I am going to exaggerate a touch here but here is what I see at my workplace: a. They will write an inline script. Solution: Rip it out into a file. b. Each Job is using a different container. Solution: use Docker with mapped volumes or we all have the same machines (this might not be possible for some workloads) and only use a single container. Why not run the shell directly you may ask? b.1 Each co…

But the things that add complexity at least to our gitlab config are precisely things you can't really test well locally, like:

- making sure you can access the AWS docker image registry, which may require particular runner configuration

- making sure deploy keys are set up correctly so you can access private dependencies

- testing caching behaviour

- testing improvements to pipeline duration (for us, just parallelising and reordering some things meant build times decreasing from 24 to 9 minutes)

- testing when exactly jobs should be triggered (e.g. stages, "only", "when", etc.)

- testing the predefined variables (such as the name of the branch)

- and so on

Re: Gitlab 12.6

#23

Post by Volundr »

orf wrote: post #10
ipodopt wrote: post #7

Earlier quoted context omitted.

I am particular about how I code my builds so they are portable. I like debugging locally, its faster. Gitlab-Ci does not encourage this and I usually end up refactoring my colleagues code all the time to this end. Not sure if having a runner is the proper solution to this problem. Off the cuff I feel like there are better ways to go about it if I where making a CI server from scratch. Like only allowing single Docke…

Gitlab executes shell commands, you can and should run those locally to debug. If your putting complex build logic inside gitlab definitions then perhaps you’re doing it wrong - “make test” or “make release” should be all that Gitlab runs.

I've frequently found that the way my script behaves in Gitlab CI is very different from running those commands one by one in a shell. In particular, environment variables do not carry over like you expect them to, but there are plenty of other surprised as well. I when working on the build pipeline I frequently find myself doing `git commit --amend`, `git push --force` over and over again to avoid having a billion commits tweaking my build logic.

Re: Gitlab 12.6

#24

Post by donmcronald »

ipodopt wrote: post #18
orf wrote: post #15

Earlier quoted context omitted.

But... it just runs shell commands. It doesn’t encourage you to do more complex things, especially compared to other CI systems like Jenkins. My point is your complaint seems more general than Gitlab itself.

Well, let's see. I am going to exaggerate a touch here but here is what I see at my workplace: a. They will write an inline script. Solution: Rip it out into a file. b. Each Job is using a different container. Solution: use Docker with mapped volumes or we all have the same machines (this might not be possible for some workloads) and only use a single container. Why not run the shell directly you may ask? b.1 Each co…

I agree with this and do the same for a, b, c. It wouldn't surprise me at all to see projects treating Docker based jobs / pipelines like they're deterministic. Unless you learn how everything interacts and are really careful it's very difficult to create repeatable builds with GitLab CI (and most other CI) because of what you say in d.

If you can't run a build with your internet disconnected, it's not repeatable and you don't own it. The thing is it's MUCH better for companies like GitLab and GitHub to get you locked in to their SaaS build systems, so simple, repeatable, offline builds will never be a priority. The harder it is, the better it is because they can charge you more money to "manage" it.

I have no idea how someone looks at a system like GitHub Actions and likes it. To me it seems obvious what's going on a GitHub. Eventually Microsoft is going to marry a bunch of tech together (ex: GitHub, Visual Studio Online, GitHub Actions, Azure Resource Manager) and everyone will be so hyped about how easy it is to click a button and have dev, build, deploy "just work" that they'll blindly give up control of their development and build environment. Just look at how easy it was for Apple and Google to convince developers to give up their ability to deploy apps without being blessed.

Re: Gitlab 12.6

#25

Post by jl-gitlab »

rumanator wrote: post #12
mroche wrote: post #4

Interesting addition of Conan support right into GitLab. > For any development organization, having an easy and secure way to manage dependencies is critical. Package management tools, such as Conan for C/C++ developers, provide a standardized way to share and version control these libraries across projects. > In GitLab 12.6, we are proud to offer Conan repositories built directly into GitLab. Developers can now publ…

Yes, that's very odd as it would be relatively trivial to extend packaging support to handle DEB or RPM packages, for example.

We are looking at adding many new kinds of packages, including potentially Linux package types such as thesehttps://about.gitlab.com/direction/package/package_registry/...

Re: Gitlab 12.6

#26

Post by jl-gitlab »

robbya wrote: post #2

> Streamline Audits with Release Evidence I've seen companies with overly painful audit requirements, and making the process of generating artifacts for audit as easy as possible is a great way forward. Box ticking exercises should take minimal engineering time. My current project doesn't use gitlab, but I love the constant innovation Gitlab pushes out.

Thanks, that's really great to hear. Release evidence, and where we plan to take that feature long term (https://about.gitlab.com/direction/release/release_governanc...) is really exciting to me, having spent a lot of my career in release management. Would be great to get your feedback on where we are headed - even if you aren't a GitLab user today, may you will be in the future.

Re: Gitlab 12.6

#27

Post by jl-gitlab »

majkinetor wrote: post #20
DogLover_ wrote: post #5

Earlier quoted context omitted.

Deprecated `exec` command is hell to use Curious what makes it hell to use? Unfortunatelly there is still no way to run CI file locally which is major turn down I don't think thats correct. As you mention there is `exec`. Do you know of better alternatives?

One example that comes to mind is codefresh.io. It allows local run of the pipeline (and even debug via breakpoints) and only requires docker to be installed locally. I currently use task system with Gitlab to make any job in 1-3 lines. Then I can easily replicate build locally. The task engine is either installed on runner's host itself or run inside docker container.

We are looking to add breakpoints in an upcoming release, building on our interactive web terminals feature. If you're interested in providing feedback the public issue is https://gitlab.com/gitlab-org/gitlab/issues/39527.

Re: Gitlab 12.6

#28

Post by jl-gitlab »

majkinetor wrote: post #3

Unfortunatelly there is still no way to run CI file locally which is major turn down. Deprecated `exec` command is hell to use. I should be able to install runner on developers machine and just run and debug pipeline there, with or without Gitlab server present.

We have a new PM for the runner and this is on his agenda to research and improve. The reality is that the job execution environment is complex - each job does just have a simple script section which is just shell commands, but there is a lot of context that impacts behavior (to which environment are you deploying, what environment variables are set specific to that environment, is it a protected environment with variables that are shareable to any random user, are artifacts being passed, and so on.) Making iterating on pipelines easier is definitely on our agenda, and adding your thoughts to issues like https://gitlab.com/gitlab-org/gitlab-runner /issues/2226 or making new ones for your specific use case will help. We are also working on other oblique solutions to make pipelines easier to iterate on like https://gitlab.com/gitlab-org/gitlab/issues/39527, which help solve the problem in a different way where you don't need to try to replicate the state of the world on your local environment but can still find and fix problems quickly.

EDIT: I should also mention, https://gitlab.com/groups/gitlab-org/-/epics/2072 is a big priority for us where we want to reduce the time to first green pipeline (not just for new users, but for new projects too). Feedback is welcome there as well.

Re: Gitlab 12.6

#29

Post by jl-gitlab »

orf wrote: post #10
ipodopt wrote: post #7

Earlier quoted context omitted.

I am particular about how I code my builds so they are portable. I like debugging locally, its faster. Gitlab-Ci does not encourage this and I usually end up refactoring my colleagues code all the time to this end. Not sure if having a runner is the proper solution to this problem. Off the cuff I feel like there are better ways to go about it if I where making a CI server from scratch. Like only allowing single Docke…

Gitlab executes shell commands, you can and should run those locally to debug. If your putting complex build logic inside gitlab definitions then perhaps you’re doing it wrong - “make test” or “make release” should be all that Gitlab runs.

One relatively easy way to debug is to set up a runner instance on your machine and then send jobs that you're working on there. In that way you are validating real jobs, but can also monitor and interact with what's happening in real time.

Features like https://gitlab.com/gitlab-org/gitlab/issues/39527 will make this easier and allow you to do similar troubleshooting in an interactive web terminal.

Re: Gitlab 12.6

#30

Post by deastman »

0xEFF wrote: post #16
rumanator wrote: post #11

Earlier quoted context omitted.

> Unfortunatelly there is still no way to run CI file locally which is major turn down. You can run CI files locally. Just install a GitLab runner and run it through the command line.

The functionality is no longer being maintained [1]. Notice: Not all features of .gitlab-ci.yml are supported by exec. [2] [1]: https://gitlab.com/gitlab-org/gitlab-runner/issues/2710 [2]: https://docs.gitlab.com/runner/commands/#gitlab-runner-exec

Hi, Darren here - PDM for Runner (deastman@gitlab.com). Yes - as we mention on this page, https://docs.gitlab.com/runner/commands/#limitations-of-gitl..., there are limitations with the current exec implementation. We also have this issue https://gitlab.com/gitlab-org/gitlab-runner/issues/2797#prop... open on this topic. In reading the threads here, I will update that issue on GitLab with our latest thinking regarding implementation challenges, ideas, etc. At a high-level today, the Runner is really just a job execution engine for want of a better description. The Runner is focused on taking a job and executing it. So as we look at this problem of executing a pipeline locally, we have to think about all of the logic that's happening in the GitLab CI Rails app - stuff like artifacts passing for example. Then we have to figure out how to parse the .gitlab-ci.yml file. Ideas like, do we do this in the context of a specific project, or a connection to any GitLab API. So tons of pros and cons to consider for sure. Given the in-depth discussion on this topic, its clear that I and the team need to refocus on this topic sooner rather than later. In chatting with one of our lead backend engineers, his point of view is that " I totally agree. I use exec quite often and I'd love to have this command a first class citizen again." So I am going to spend some time in the next couple of days capturing some of the thoughts here and updating the issue in GitLab with our latest ideas and proposals. We definitely need to get this done, so any help from the community is welcome, as we think solving this is going be a bit of work and make take a few release cycles.
Post reply on HN