It all started with a seemingly innocent question. A CVS commit email brought my attention to the existence of a Pkgsrc Work-in-progress Taskwarrior package and I decided it would be a good idea to switch my Taskwarrior install from source to Pkgsrc because Pkgsrc would still allow me to use local patches for my Daylight Saving Time fix and my NetBSD build fix had been merged in some time ago anyway.

I tried to build it and saw it was using GCC 4.5.3, the system GCC version of NetBSD 6, but knew that Taskwarrior needed at least 4.7. I had a quick search around and discovered GCC_REQD as a Pkgsrc variable that could be added to the Makefile. So I added GCC_REQD = 4.7, starting re-building and this time I saw it started trying to download GCC 4.7, but, hang on! I had GCC 4.9 installed already, why couldn’t it just use that?

And that dropped me straight off in the deep end of the Pkgsrc build system and trying to understand how /usr/pkgsrc/mk/compiler/gcc.mk works and also how to improve it to get the behaviour I, and others, want.

I’ve done my fair share of programming by now and dabbled in quite a few different languages (although I’m definitely no expert in any), but I’ve found the Pkgsrc Makefiles confusing. There is a guide for Programming in Makefiles, but that doesn’t seem to apply directly to the Pkgsrc ones, at least as far as I can tell. One very basic thing I’ve struggled with is the “print as debug” trick, but in my attempts to I’ve found something that doesn’t work, but does…

NOTHING!=echo ${VARIABLE_OF_INTEREST} >> /path/to/a/pkg/echo

This will moan that it can’t read the shell output for the echo statement, but at the same it will happily print out the value I’m looking for and also accumulate them in a file! Good enough for me. For example, here’s me inspecting GCC_REQD:

make: "../../mk/compiler/../../mk/compiler/gcc.mk" line 251: warning: Couldn't read shell's output for "echo 4.9 >> /usr/pkgsrc/wip/task/echo.

My first attempt at modifying gcc.mk added the following directly before a routine that “Distills the GCC_REQD list into a single _GCC_REQD value that is the highest version of GCC required”. Because I didn’t, and still don’t, understand everything in gcc.mk I didn’t want to alter anything too drastically.

# See if any of GCC_VERSIONS_ACCEPTED are available, this will get the first match
.if defined(GCC_VERSIONS_ACCEPTED)
    GCC_REQD=  ${GCC_VERSIONS_ACCEPTED}
    _GCC_PKG_SATISFIES_DEP=     NO
.   for _version_ in ${GCC_VERSIONS_ACCEPTED}
        _GCC_VERSION_ACCEPTED!= ${ECHO} ${_version_} | ${SED} -e "s/\\.//"
.       if empty(_GCC_PKG_SATISFIES_DEP:M[yY][eE][sS])
            _GCC_PKG_SATISFIES_DEP!=    \
            if ${PKG_INFO} -qE 'gcc${_GCC_VERSION_ACCEPTED}'; then \
                ${ECHO} "YES";                      \
            else                                \
                ${ECHO} "NO";                       \
            fi
.           if !empty(_GCC_PKG_SATISFIES_DEP:M[yY][eE][sS])
                GCC_REQD= ${_version_}
.           endif
.       endif
.   endfor
.endif

What this does (or is at least meant to do) is check for the presence of a GCC_VERSIONS_ACCEPTED variable in a package Makefile, which would be a list of acceptable GCC version (“4.7 4.8…”, etc). It then checks if any of these versions are actually installed (the ${PKG_INFO} -qE 'gcc${_GCC_VERSION_ACCEPTED}' bit) and picks the first it comes across by overriding the GCC_REQD variable, which itself a list of required GCC versions defined by various criteria in gcc.mk; I think a lot of this could be accumulated cruft from over the years as it sets versions such as 2 and 3.

The problems with this approach are numerous:

  1. Overriding GCC_REQD with a single version. Inefficient although works to a certain extent.
  2. Picks the first version it finds, rather than highest overall. I was thinking too simplistically for single independent packages, rather than ones with dependencies. I.e. this could lead to related packages building against different GCC versions if one had “4.7 4.8” and a dependent package had “4.8 4.9”.
  3. GCC_VERSIONS_ACCEPTED is perhaps a complicated approach. Better to specify a minimum and a list of incompatible versions (although this is being debated on the mailing list).

I am being even slower in writing up my ideas than I am in implementing them, and my implementation is already severely lagging the ideas on the mailing list, but I have my fingers in lots of pies at the moment what with TTYtter and Simplenote.vim maintenance. Part 2 will follow soon, but as a giveaway that is also a wrong approach.

Notes to self:

  • As far as I can tell everything should be space indented, the only exception is for “system calls” which must be tab indented (See the man page for Make), but then these must also come after a target. You can’t simply stick echo ${VARIABLE_OF_INTEREST} in gcc.mk and have it actually echo because there are no targets. Sometimes you’ll find things that are tab indented, such as sections like the _GCC_PKG_SATISFIES_DEP!= \ which contain system calls, but, as far as I can tell, these are tab indented for consistency only; they aren’t required to be tab indented as they are not plain system calls.
  • I don’t yet know or understand why the Pkgsrc makefiles use variables such as ${SED} instead of just sed. I presume it is because Pkgsrc can be used on multiple systems so somewhere a file must set these specific to the system it is being used on.
  • Similarly, why _GCC_PKG_SATISFIES_DEP:M[yY][eE][sS] has to match against lowercase and uppercase when the script is explicitly echoing uppercase, must be because of the different systems it can be used on. Perhaps some only work in lowercase (I recall OSX has or had different case sensitivity install options for the filesystem)?
  • See the Make man page for what the different variable assignment operators (=, !=, etc) mean.
  • The if !empty(VARIABLE:M[yY][eE][sS]) is confusing at first sight, but is matching if a variable is “YES”. :M matches only those words that match a pattern so will leave something in the brackets and then empty returns true if it is an empty string, hence why Not (!) is used.

This is likely to turn into a little bit of a series with each attempt I make. HOPEFULLY I’ll eventually develop an understanding of how gcc.mk works and if I’m really lucky I’ll actually get a decent patch for gcc.mk put together.