Building a Package on Solus for Installation by eopkg from a Custom Local Repository

Feb. 28, 2020, 3 p.m.

When I tried Solus 3.9999 in late 2018, all of the software I needed at that time was available in the distribution's default repositories. Now, having installed the recently released Solus 4.1 Fortitude Plasma Edison, one important program for me, Kile, built using KDE technologies, is not available in the distribution, and the developers refuse to package it. It is available in other distributions and I have been using it regularly recently in Arch, openSUSE Tumbleweed, Fedora, Manjaro, and even Sabayon; so it works in the latest versions of these distributions with the latest KDE Frameworks, (although the version number inconsistency on the part of Kile required workarounds in the package version number).

As it seems the developers will not add this software, I think this is a perfect opportunity to try one of the innovative and unique features of Solus -- its packaging system. This article describes this system and my experience packaging Kile, creating a custom local repository, and installing the system package manager.

Introduction

The Solus build system, centered around the solbuild program, allows the building of packages in an isolated environment, via cheroot, against an image that contains packages in either the Solus unstable or stable repositories. Like all other packaging systems it uses a "recipe" for building a package in the form of package.yml. One of the nice features of the system is a script that generates a skeleton of the file from the upstream tarball link with some values prefilled (mostly accurately).

A directory is created for packaging where packaging helper scripts are cloned from a Solus GitHub repository. Within this directory a subdirectory for each package to be built is created and the tools made available to run in each package's build directory through symlinks. solbuild is run in the package's directory which will process the package's package.yml and build the package.

solbuild is run using one of several available profiles which will alter the characteristics of the resulting package. These are the main, unstable, and unstable-local profiles. The first results in a package built against the stable image, the second and third in a package built against the unstable image. The difference is that the packages of the last profile are automatically transferred to /var/lib/packages/local/ which can be used as a local repository to build packages against.

In my experimentation with Solus packaging, the goal was to simply build a package not available in the Solus repository and install it from a local .eopkg file, so I didn't bother with the unstable or unstable-local profiles. I made a directory alongside the build directory and configured it to be used as a local repository instead of /var/lib/packages/local/, which by default is only for locally built packages using the unstable profile against which other packages can be built, although it may be possible to manually place packages built with the main profile in the default unstable local repository and install them using eopkg instead of building against them. I didn't try this; instead I copied the packages written to the package's build directory by solbuild to my custom local repository and installed my build of Kile from there.

I also didn't use another component of the Solus packaging system, arcanist, a simple -- for the user or contributor -- tool to push packages and package changes to the remote development infrastructure. I also didn't bother with setting up a global Git configuration which is required for setting up arcanist. In normal use of the Solus packaging system for contributors, a Git repository is created in the package's build directory and the Git tracking is used in the remote infrastructure when submitting a package using arcanist. I did, however, set up a Git repository using a local configuration to track the changes to the packaging sources, primarilypackage.yml.

Prepare System

In building Kile and adding it to the local repository, I used the process described below, which is a little different from the documented process and the process presented in the YouTube tutorials on Solus packaging. The differences are focused on

  • building against the main profile
  • using a different local repository from the default
  • using the local repository to install built packages and not to build against
  • packaging for local use and not for contributing
  1. Create the packager file at ~/.solus/packager

    brook@g5-solus ~ $ mkdir .solus
    brook@g5-solus ~ $ touch .solus/packager
  2. Edit the packager file to look like the following.
    [Packager]
    Name=First_Name Last_Name
    Email=username@second_level_domain.tld
  3. Install development component and the build system, solbuild.
    brook@g5-solus ~ $ sudo eopkg it -c system.devel solbuild
    Installing the Development Component and solbuild, the Solus Build System
    Components are similar to "groups" of packages in other distributions.
  4. Initialize the build system, solbuild.
    brook@g5-solus ~ $ sudo solbuild init
    Initializing solbuild
  5. Update solbuild.
    brook@g5-solus ~ $ sudo solbuild update
    Updating the Build System Image
  6. Create a folder for building packages. When building a package one will create a subfolder in this directory for each package.
    brook@g5-solus ~ $ mkdir Solus-LB 
    brook@g5-solus ~ $ cd Solus-LB 
  7. Clone Git repository containing Solus build scripts.
    brook@g5-solus ~ $ git clone https://dev.getsol.us/source/common.git 
  8. Create symlinks to makefiles.
    brook@g5-solus ~/Solus-LB $ ln -sv common/Makefile.common .
    brook@g5-solus ~/Solus-LB $ ln -sv common/Makefile.toplevel Makefile
    brook@g5-solus ~/Solus-LB $ ln -sv common/Makefile.iso .
    The Contents of the Main Build Directory, In My Case ~/Solus-LB
  9. Make an alias named fetchYml for "$HOME/repository/common/Scripts/yauto.py" in ~/.bashrc. Although the documentation uses the ~/repository name for the top level package building directory, we are using ~/Solus-LB, so the alias will be to "$HOME/Solus-LB/common/Scripts/yauto.py"
    alias fetchYml="$HOME/Solus-LB/common/Scripts/yauto.py"

Prepare to Package Kile

  1. Make a directory for packaging Kile inside the main packaging directory.
    brook@g5-solus ~/Solus-LB $ mkdir kile
    brook@g5-solus ~/Solus-LB $ cd kile
  2. Configure git local.
    brook@g5-solus ~/Solus-LB/kile $ git config --local user.name "My Name"
    brook@g5-solus ~/Solus-LB/kile $ git config --local user.email "username@second_level_domain.tld"
  3. Make the Makefile available for the current package build.
    brook@g5-solus ~/Solus-LB/kile $ echo "include ../Makefile.common" > Makefile
  4. Update the build system image, solbuild.
    brook@g5-solus ~ $ sudo solbuild update
  5. Fetch the Kile source tarball and generate the package.yml skeleton file.
    brook@g5-solus ~/Solus-LB/kile $ fetchYml https://github.com/KDE/kile/archive/v3.0b3.tar.gz
    Fetching the Source Tarball and Generating the package.ypkg
    The fetchYml alias to one of Solus's helper scripts configured previously conveniently downloads the source into the Kile build directory and generates a skeleton of package.ypkg in one step.
  6. Edit the package.yml file as appropriate to build the package. In the following section I describe the process of adding necessary components of the file to build Kile. And the section after that I describe the actual build commands which completed successfully after four attempts due to errors, also described in that section.

Generate .ypkg

The most difficult aspect of preparing the package.yml is in having the complete and correct set of build dependencies, and to a lesser extent the runtime dependencies. In the case of packaging Kile the dependencies are not as clear cut as the examples in the tutorial videos published on YouTube by the lead developer. In the videos many of the examples were either Solus projects or simple software in terms of building and developers spelled out dependencies.

It was helpful to use

sudo eopkg info package-name
to find the dependencies of a package once a certain package was determined to be relevant. The epcsearch.py was also useful in finding some useful information for populating dependencies in the package.yml -- used, for example, as follows from the package's build directory:
../common/Scripts/epcsearch.py Qt5Widgets
. For packaging Kile the biggest clue, by far, as to the needed dependencies was the CMakeLists.txt file in the Kile GitHub repository, more so than the repository's README or similar documentation which are used in the tutorial examples as sources of dependency information. This file includes the block:

find_package(Qt5 5.6 CONFIG REQUIRED
	Core
	DBus
	Widgets
	Script
	Test
)

find_package(KF5 5.31 REQUIRED COMPONENTS
	Config
	CoreAddons
	Crash
	DBusAddons
	DocTools
	GuiAddons
	I18n
	IconThemes
	Init
	KIO
	Parts
	TextEditor
	WindowSystem
	XmlGui
)

These are directly visible in the other biggest clue, the openSUSE .spec file for their build of the their .rpm package. CMakeLists.txt also mentions find_package(Poppler COMPONENTS Qt5) as well as something similar for Okular which is also explicitly mentioned in the GitHub repository README.

My initial build attempt didn't get far because apparently the %configure as part of the setup step of the automatically generated skeleton of package.yml was not necessary. The corresponding message was to the effect that ./configure was not found. So I removed the setup step from package.yml and the subsequent build attempt progressed further.

The failure this time, according to the build error messages, was that KF5 component were not found. Because I couldn't find these components by using the methods in the YouTube tutorial, I assumed they were included in the plasma-framework-devel package. I finally found these components by looking through the Kate package.yml on the its page at the Solus developer site. All of these packages have names that atypically don't include "KF5" but are something like "ksomething-devel".

After adding these I had another build failure. This time the build error messages mentioned "ECM", which, on second look, were mentioned at the top of CMakeLists.txt. The openSUSE .spec indicated indirectly that this may be cmake-extra-modules, which I added to the build dependencies of the package.yml. The next build was successful. The final package.yml is below.

name       : kile
version    : 3.0.03
release    : 1
source     :
    - https://github.com/KDE/kile/archive/v3.0b3.tar.gz : ddc277e1589563475376067ad317eef5a8cc9f5cb82f1338619d6b6d52ddf4e1
license    : GPL-2.0-or-later
component  : editor
summary    : Kile is a user-friendly TeX/LaTeX editor for the KDE desktop environment.
description: Kile is a user-friendly TeX/LaTeX editor for the KDE desktop environment.
builddeps  :
    - pkgconfig(Qt5Core)
    - pkgconfig(Qt5Script)
    - plasma-framework-devel
    - okular-devel
    - pkgconfig(poppler-qt5)
    - extra-cmake-modules
    - kcrash-devel
    - kdbusaddons-devel
    - kdoctools-devel
    - kguiaddons-devel
    - kiconthemes-devel
    - ktexteditor-devel
    - ki18n-devel
    - kinit-devel
    - khtml-devel
    - kio-devel
    - kparts-devel
    - kxmlgui-devel
rundeps    :
    - qt5-base
    - plasma-framework
    - okular
    - texlive
    - imagemagick
    - tar
    - zip
    - gzip
    - bzip2
    - kbibtex
    - jabref
build      : |
    %cmake
install    : |
    %make_install

The complete build dependencies for Kile are below. Note that all of these aren't explicitly included in the package.yml, but fortunately for me the ones that aren't included are dependencies of some of the ones that are, so they got installed as build dependencies into the span main profile image automatically.

Dependencypkgconfigpkgconfig provideror package
Qt5CoreQt5Coreqt5-base-develN/A
Qt5ScriptQt5Scriptqt5-script-develN/A
Qt5TestQt5Testqt5-base-develN/A
Qt5WidgetsQt5Widgetsqt5-base-develN/A
Qt5DBusQt5DBusqt5-base-develN/A
KF5ConfigN/A in SolusN/A in Soluskconfig-devel
KF5CoreAddonsN/A in SolusN/A in Soluskcoreaddons-devel
KF5CrashN/A in SolusN/A in Soluskcrash-devel
KF5DBusAddonsN/A in SolusN/A in Soluskdbusaddons-devel
KKF5DocToolsN/A in SolusN/A in Soluskdoctools-devel
KF5GuiAddonsN/A in SolusN/A in Soluskguiaddons-devel
KF5IconThemesN/A in SolusN/A in Soluskiconthemes-devel
KF5InitN/A in SolusN/A in Soluskinit-devel
KF5KHtmlN/A in SolusN/A in Soluskhtml-devel
KF5KION/A in SolusN/A in Soluskio-devel
KF5PartsN/A in SolusN/A in Soluskparts-devel
KF5TextEditorN/A in SolusN/A in Solusktexteditor-devel
KF5WindowSystemN/A in SolusN/A in Soluskwindowsystem-devel, dependency of plasma-framework-devel
KF5XmlGuiN/A in SolusN/A in Solusplasma-framework-devel
KF5I18nN/A in SolusN/A in Soluski18n-devel
okular-develN/AN/Aokular-devel
extra-cmake-modulesN/AN/Aextra-cmake-modules
poppler-qt5-develN/AN/Apoppler-qt5-devel

The complete runtime dependencies for Kile, some of which are optional, are below.

Dependencypkgconfigpkgconfig provideror packageUpstream Description
KonsoleN/Akonsole
TeXLiveN/Atexlive
ImageMagickN/Aimagemagick
KBibTeXN/Akbibtex
psutilsN/Apsutils

Build

Finally, build the package with:

brook@g5-solus ~/Solus-LB/kile $ sudo solbuild build package.yml -p main-x86_64
from within the Kile package build directory. The option -p main-x86_64 is necessary in order to build against the stable solbuild profile. Without it the package would be built against the unstable solbuild profile.
Click on any of the thumbnails to view a slideshow of the images.

Result

I enabled a local repository at a path I chose, and populateed it with the package I built using the Solus build system, using the following process.

  1. Create a directory for the local repository.
    brook@g5-solus ~/Solus-LB/kile $ mkdir ../../Solus-LR
  2. Copy the packages from the packages build directory to the local repository.
    brook@g5-solus ~/Solus-LB/kile $ cp ./kile* ../../Solus-LR/
  3. Index the packages in the custom local repository directory.
    brook@g5-solus ~/Solus-LR $ sudo eopkg index -o /home/brook/Solus-LR/eopkg-index.xml --skip-signing /home/brook/Solus-LR
    Password: 
    Building index of eopkg files under /home/brook/Solus-LR
    
    Adding package to index: kile-3.0.03-1-1-x86_64.eopkg                           
    Index file written
    
  4. View the configured repositories before adding the new custom local repository to the system. (This step is only for information.)
    brook@g5-solus ~/Solus-LR $ sudo eopkg lr
    Solus [active]
       https://mirrors.rit.edu/solus/packages/shannon/eopkg-index.xml.xz
    
  5. Add the new custom local repository location as a repository named Solus-LR.
    brook@g5-solus ~/Solus-LR $ sudo eopkg ar Solus-LR /home/brook/Solus-LR/eopkg-index.xml.xz
    Repo Solus-LR added to system.
    Updating repository: Solus-LR
    Package database updated.
    
  6. View the repositories known to the system again. This time the new local repository is shown. The order that the repositories is important because repositories that appear higher in the list produced by the eopkg lr command have higher precedence.
    brook@g5-solus ~/Solus-LR $ sudo eopkg lr
    Solus [active]
       https://mirrors.rit.edu/solus/packages/shannon/eopkg-index.xml.xz
    Solus-LR [active]
       /home/brook/Solus-LR/eopkg-index.xml.xz
    
  7. To lower the priority of the official Solus repository, re-add it.
    brook@g5-solus ~/Solus-LR $ sudo eopkg ar Solus https://mirrors.rit.edu/solus/packages/shannon/eopkg-index.xml.xz
    Repo already present with name Solus and same URL. Removing first.
    Repo Solus added to system.
    Updating repository: Solus
    eopkg-index.xml.xz.sha1sum     (40.0  B)100%    815.12 KB/s [00:00:00] [complete]
    eopkg-index.xml.xz             (2.3 MB)100%    125.07 MB/s [00:00:00] [complete]
    Package database updated.
    
  8. Confirm that the new custom local repository is appears above the default repository when repositories are listed. If it appears before the default repository, packages in that repository will have higher priority.
    brook@g5-solus ~/Solus-LR $ sudo eopkg lr
    Solus-LR [active]
       /home/brook/Solus-LR/eopkg-index.xml.xz
    Solus [active]
       https://mirrors.rit.edu/solus/packages/shannon/eopkg-index.xml.xz
    
  9. After this procedure I was able to search for and install Kile, but there was a problem with the build in that the dependency on KF5Init caused a kile-devel package to be produced which caused a dependency cycle between kile-devel and kile which PISI a core component of eopkg couldn't handle as pacman can after an error message. Of course, were it not for my ignorance, this was probably avoidable with a better package.yml file.

    Installation Error and Subsequent Successful Installation of Kile

    brook@g5-solus ~/Solus-LR $ sudo eopkg it --ignore-dependency kile-devel
    brook@g5-solus ~/Solus-LR $ sudo eopkg it kile
    The Locally Built Kile Running on Solus 4.1 Fortitude Plasma Edition
    Click on any of the thumbnails to view a slideshow of the images.

    In any case I was able to resolve this to the satisfaction of my needs, which was to have a properly working Kile on my installation of Solus by first installing kile-devel with the --ignore-dependency option to the eopkg install command, then installing kile using eopkg without any options.

Conclusion

The worst part of the experience was with the repository indexing command as the documentation doesn't mention the option -o which allows users to provide the path of the output file. In the documentation the provided command is

sudo eopkg index --skip-signing /var/lib/solbuild/local/

which doesn't require an output file path as it automatically places the output in /var/lib/solbuild/local/ where Solus would like the local repository to be. I mistakenly added the .xz, which as it turned out, is automatically added by the indexing command. This resulted in an error when adding the repository.

Other things which I thought were interesting or otherwise worth mentioning:

  • It would be nice if the scripts could also make a local package using the stable image main profile which automatically moves it to an automatic local repository instead of only for packages built with the unstable image unstable-local profile.

  • Apparently, Solus has no capacity of indicating that runtime dependencies are recommended or suggested, thus no way for the user to install or not install either of these types of package dependencies.

  • The packaging documentation emphasizes the use of pkgconfig, a system that allows consistent naming of dependencies in packaging scripts and a one to one mapping of requested packages in the source package and a distribution's available packages, however very few dependencies have pkgconfig names in Solus, at least compared to openSUSE.

  • The package and repository manager eopkg is still essentially PISI, Pardus Linux's package manager. This is evident in the eopkg dependency cycle error message which actually asks users to report the error as a bug to a Pardus email address. The Python 2.7 traceback that is displayed with the error contains paths topisi modules. The eopkg executable itself includes a copyright between 2005 - 2010 to a moniker of probably a Pardus developer.

But finally having a working Kile I can appreciate the Solus build system and the package manager. It is easy to use and eases building packages by users for contribution to Solus, but it is not as elegant, efficient, or flexible as Arch's Pacman and PKGBUILD system or as powerful and flexible as openSUSE's combination of rpmbuild and zypper. The best may be Gentoo's system because, by design from the ground up, the package manager is a build system and binary and source package manager.

References

The Solus Project has a series of articles on using the solbuild, which can be accessed at the main Packaging page on the Solus Help Center. A series of video tutorials is also available, the first of which is Learn Solus Packaging - Session 4 (Beginners).

blog comments powered by Disqus