SLES

Last week Antonios Pappas told me that if I am already chasing slow tests, I should look at tests/support_server/setup.pm. It is known for wasting time. He also gave me the rule of thumb that made me take it seriously: every wasted second in the support server costs about three seconds, because the parallel nodes wait for the master to finish its setup.

That is the difference between a slow test and an expensive test. A slow single-machine test wastes one job. A slow support server wastes the whole cluster, and it does it before the real test even starts.

The openQA test suite for openSUSE and SLE has about 2,400 test modules and 1,500 schedule files. Over a 12-month window they run a few million times and burn about 710,000 machine-hours of worker time across two instances, the public openqa.opensuse.org and SUSE’s internal openQA. For the last two months I have been trying to shrink that number, cheap wins first.

I kept finding fossils. A comment header in hostname describing code deleted in 2019. A network restart timeout that crept from 10 to 120 seconds over two years of bug reports. A make -B workaround from 2018 for a bug fixed upstream that same year. Each made sense when it was written. But they add up, and they run on every job.

The openQA test suite for openSUSE and SLE has a test module called tests/console/sudo.pm. It verifies that sudo works: passwords, shells, sudoers rules, environment isolation. Basic stuff. It runs tens of thousands of times per year and takes about 9 minutes each time. That adds up.

Where the time goes

There is no single bottleneck. The test uses expect to interact with password prompts. Every sudo call goes through credential cache reset, process spawn, password entry, and result verification. It does this 20 times because the test runs the full suite twice with slightly different sudoers configurations.

A few years ago I wrote a blog post about a small hackweek project called rdepends. The idea was simple: given a package, find out what other packages depend on it, recursively. It lived in my home project on the build service and it was a useful but rough tool.

Since then a few things happened that made me revisit it.

Zypper got the feature

Back when I started this project, zypper had no built-in way to ask “what depends on package X?”. You could ask the other direction easily with zypper info --requires, but not the reverse. I raised this with the zypper developers and Benjamin Zeller implemented the --requires-pkg flag in zypper 1.14.33. So now you can do:

Introduction

Let’s start by clarifying what we mean by cross-building and cross-packaging. Cross-compilation is the process of compiling source code on one platform, called the host, in order to generate an executable binary for a different target platform. The emphasis here is on the word “different”. The target platform may have a different CPU architecture, such as when we work on an x86 computer and want to build software for a Raspberry Pi board with an ARM CPU. But even if the target platform has the same CPU architecture as the host, there may be several other possible differences. For example, the host may be running Debian Sid, while the target may be running openSUSE Leap. Different Linux distributions may have different compilers, linkers, and run-time libraries. Even when using the same distribution as the host for the target, they may be different releases, such as openSUSE Tumbleweed and Leap. In short, nothing guarantees that the target system will have the same shared libraries as the host system.

As start let’s sort it out what is dependency and what is reverse dependency.

Dependencies and reverse dependencies in Linux distributions are important concepts to understand. A package dependency means that another package relies on it in order to function. For example, if package B requires package A to be installed in order to work, then package B is dependent on package A and is considered a reverse dependency of package A.

From time to time we should ask ourselves how are we doing. Are we successful, are we on the right track, are we heading to the right direction, are we fast enough, are we accelerating or slowing down?

This time I am talking about the openSUSE Linux Distribution and about the SUSE Linux Enterprise Server.

And here I quickly would like to note an important disclaimer with a short story.

I have heard way to often the question from Linux and specially SUSE Linux users that “How can I check the changelog of a package or new version of a package available on the repository, but not yet installed”.

There was no easy answer for that question, so I have decided to make a little tool for that.

How it is done

All the enabled repositories have a bunch of configuration files in a well structured directory tree under the /var/cache/zypp/raw/.