How to solve a stuck Gitlab CI pipeline? - gitlab-ci-runner

We've been using Gitlab CI for some months, and in the last 1 week, we've been using the specific runner installed on a VPS. Currently, we are using "shell" as the executor.
Today our pipeline got stuck out of sudden, when we looked into the server free RAM, it's only 48MB out of 996 MB, FYI, we're using CentOS 6.
We've been struggling to get the answers, but we're stuck at the moment, and would like to know :
What's causing the pipeline from getting stuck?
is it true because of low free RAM?
Should we use another executor, perhaps SSH or even docker?
What is the best practices to deal with this kind of problem?
We would appreciate any kind of help or directions.

In my case the pipeline was stuck because the only available runner had the option "Can run untagged jobs" set to "No" and the job was really untagged. One can fix this issue by changing the "Can run untagged jobs" option or by adding a tag to the appropriate section of the ".gitlab-ci.yml" file in the repository. In my case it was section default:tags:.
(It seems that your case is much more complicated. However I've came across this issue twice a month, and I've forgotten the decision at the second time. Thus I've came to this page which looks appropriate to save the decision. Hope the answer will help someone else.)

In my case, the pipeline was stuck because of two things:
The tags specified in the .gitlab-ci.yml do not match those in the runner configuration.
If you specify the simulator in the build command, ensure that you write the right version of the simulator.
Once I did these changes, everything worked well!
Good luck.

In my case, It happened on a container that was off for a couple of days for maintenance reasons, I had to clear runner caches, and it worked!

In my case the windows gitlab-runner service was not running. Starting it solved it.

Related

netdata alarm on server login

so I'm pretty new to netdata. I have an instance running on my server and I wanted to setup a few alarms to my telegram.
The examples given by netdata worked pretty well (cpu load, disk usage, ...). I also wanted to add an alarm when someone logs in to the server even though I'm using SSH keys for login. Why? Just because I want to learn how to do it.
Anyway. I don't really understand their documentation since I'm mostly a developer and no Linux-sysadmin.
So far I added config files to /etc/netdata/health.d/. But I don't understand how the system_login.conf file should look like.
I tried using this blog post but I don't quite understand how to apply it: How to monitor and troubleshoot systemd-logind
What do I have to put into the alarm property? For the on property I'd guess it is systemd-logind. On further research I found another blog post (systemd-logind monitoring with Netdata
) which says to edit some python.d.conf file that I don't have. I'm pretty confused right now and have no clue how to write the config file.

How to debug in github CI?

It's very weired that my code passes all UT/IT in my laptop, but it encounters errors in github CI.
Would you mind helping to take up some methods to debug in github CI? Or to make code runs in local as same as github?
It's a project about timeseries database, Apache-IoTDB. The error looks like a trivial logical error among ordinary code. Hope it may help diagnose the bug. Thank you very much !
act is a local runner for GitHub Actions workflows and should run nearly identically to the real thing.
Alternatively, the debugging-with-ssh action uses upterm to open an SSH listener within a container to get a shell on a running workflow within GitHub Actions itself.
The question solved directly by merging master(the branch my pull request forward to) again.
The point is, github CI (actions) may be running on the code which is AUTO-MERGED when the pull-request accepted.
So if your code passes all tests locally but failed in CI with different result from your local debugging, try merge the branch which PR forward may solve the problem.
Hope this may hepl you and thanks guys under this question.

Sync chef's cookbooks on self-host gitlab

The other day i was looking if it is possible to sync my chef workstation/server for easy manage/visualization of all cookbook components. I already tried looking for one solution and I didn't found any good info about this topic. So my questions are:
Is it posible to do?
Is it a good solution? And if not recommend one better?
If it's viable explain how can i do it?
Normally cookbooks already live in source control so it's not really a normal request. You can use the knife download command to pull back cookbook data but probably not in a format you want. tl;dr go the other direction, git -> chef server.

How to build Chromium faster?

Following only the instructions here - https://www.chromium.org/developers/how-tos/get-the-code I have been able to successfully build and get a Chromium executable which I can then run.
So, I have been playing around with the code (adding new buttons to the browser etc.) for learning purposes. So each time I make a change (like adding a new button in the settings toolbar) and I use the ninja command to build it takes over 3 hours to finish before I can run the executable. It builds each and every file again I guess.
I have a decently powerful machine (i7, 8GB RAM) running 64-bit Ubuntu. Are there ways to speed up the builds? (At the moment, I have literally just followed the instructions in the above mentioned link and no other optimizations to speed it up.)
Thank you very very much!
If all you're doing is modifying a few files and rebuilding, ninja will only rebuild the objects that were affected by those files. When you run ninja -C ..., the console displays the number of targets that need to be built. If you're modifying only a few files, that should be ~2000 at the high end (modifying popular header files can touch lots of objects). Modifying a single .cpp would result in rebuilding just that object.
Of course, you still have to relink which can take a very long time. To make linking faster, try using a component build, which keeps everything in separate shared libraries rather than one big onw that needs to be relinked for any change. If you're using GN, add is_component_build=true to gn args out/${build_dir}. For GYP, see this page.
You can also peruse faster linux builds and see if any of those tips apply to you. Unfortunately, Chrome is a massive project so builds will naturally be long. However, once you've done the initial build, incremental builds should be on the order of minutes rather than hours.
Follow the recently updated instructions here:
https://chromium.googlesource.com/chromium/src/+/HEAD/docs/windows_build_instructions.md#Faster-builds
In addition to using component builds you can disable nacl, use jumbo builds, turn off symbols for webcore, etc. Jumbo builds are still experimental at this point but they already help build times and they will gradually help more.
Full builds will always take a long time even with jumbo builds, but component builds should let incremental builds be quite fast in many cases.
For building on Linux, you can see how to build faster at: https://chromium.googlesource.com/chromium/src/+/master/docs/linux_build_instructions.md#faster-builds
Most of them require add build argments. To edit build arguments, you can see GN build configuration at: https://www.chromium.org/developers/gn-build-configuration.
You can edit the build arguments on a build directory by:
$ gn args out/mybuild

Catching the dreaded Blue Screen Of Death

It's a simple problem. Sometimes Windows will just halt everything and throws a BSOD. Game over, please reboot to play another game. Or whatever. Annoying but not extremely serious...
What I want is simple. I want to catch the BSOD when it occurs. Why? Just for some additional crash logging. It's okay that the system goes blue but when it happens, I just want to log some additional information or perform one additional action.
Is this even possible? If so, how? And what would be the limitations?
Btw, I don't want to do anything when the system recovers, I want to catch it while it happens. This to allow me one final action. (For example, flushing a file before the system goes down.)
BSOD happens due to an error in the Windows kernel or more commonly in a faulty device driver (that runs in kernel mode). There is very little you can do about it. If it is a driver problem, you can hope the vendor will fix it.
You can configure Windows to a create memory dump upon BSOD which will help you troubleshoot the problem. You can get a pretty good idea about the faulting driver by loading the dump into WinDbg and using the !analyze command.
Knowing which driver is causing the problem will let you look for a new driver, but if that doesn't fix the problem, there is little you can do about it (unless you're very good with a hex editor).
UPDATE: If you want to debug this while it is happening, you need to debug the kernel. A good place to pick up more info is the book Windows Internals by Mark Russinovich. Also, I believe there's a bit of info in the help file for WinDbg and there must be something in the device driver kit as well (but that is beyond my knowledge).
The data is stored in what's called "Minidumps".
You can then use debugging tools to explore those dumps. The process is documented here http://forums.majorgeeks.com/showthread.php?t=35246
You have two ways to figure out what happened:
The first is to upload the dmp file located under C:\Minidump***.dmp to microsoft service as they describe it : http://answers.microsoft.com/en-us/windows/wiki/windows_10-update/blue-screen-of-death-bsod/1939df35-283f-4830-a4dd-e95ee5d8669d
or use their software debugger WinDbg to read the dmp file
NB: You will find several files, you can tell the difference using the name that contain the event date.
The second way is to note the error code from the blue screen and to make a search about it in Google and Microsoft website.
The first method is more accurate and efficient.
Windows can be configured to create a crash dump on blue screens.
Here's more information:
How to read the small memory dump files that Windows creates for debugging (support.microsoft.com)