Downgrading packages in Ubuntu

So I needed to downgrade the version of my gzip package in Ubuntu 10.04 because it wouldn't work with my openembedded build (I was getting errors unpacking the expat tarball). What I ended up doing was adding the "hardy" packages to my list of sources through the software sources gui (deb http://security.ubuntu.com/ubuntu hardy-security main). Then I followed the instructions here to find the older version of gzip and downgrade it. Now my build works. Basically what I did was:

1. System->Administration->Software Sources
2. Go to Other software and click Add
3. Add the line above (deb http://security.ubuntu.com/ubuntu hardy-security main)
4. Click close and allow it to rebuild the database.
5. run apt-cache showpkg gzip
6. You'll see the versions. You want the 1.3.12-3.2ubuntu0.1 version
7. run sudo apt-get install gzip=1.3.12-3.2ubuntu0.1
8. Select yes when asked if you want to downgrade.

A quick note on conf files

So I wanted to have my own configuration file in OE that I could use to specify things like which packages to make source ipks for or what my PREFERRED_PROVIDERs were for things like the virtual/kernel package.

This let me to the question of how can I easily add a new configuration file? I figured that some files must just be picked up automatically but was it really just as simple as adding another line to something like my local.conf? The simple answer is YES.

You can just add a line like:

include /.conf

The only thing to be aware of is that you can either "include" a configuration file, which means that it will be used if it exists but if it doesn't exist that's OK. Or you can "require" a configuration file, which as the name implies means it has to be present or you will get an error.

As for adding a configuration file "automatically" I'm not sure how to do that and I have it on good advice that you don't want to do that. I guess I can understand that because it is nice to know explicitly how something got added and forget all the automagical stuff.

What does virtual/xyz point to?

Often I see things in recipes where they rely on things like "virtual/kernel". But how do I know what bitbake recipe virtual/kernel points to?

Usually you can find this information in the machine configuration file. This isn't always guaranteed, it could be stashed in some other configuration file like the distro configuration file (i.e. Angstrom defines the PREFERRED_PROVIDER_virtual/update-alternatives)

In the case of the dm3730-am3715-evm machine type its configuration file specifies:

PREFERRED_PROVIDER_virtual/kernel = "linux-omap-psp"

This tells me that if I were to "bitbake virtual/kernel" it would look for the latest linux-omap-psp recipe and build that with the exception of whether or not that recipe set DEFAULT_PREFERENCE to -1. If so then it would look for the next version of linux-omap-psp.

So in general the virtual/xyz values are set in the configuration files. You could of course always just "bitbake virtual/xyz" and see what it builds, but if you have a slow system like mine sometime you want to know without waiting on a build :)

What is SOC_FAMILY?

So I was playing around with adding new tasks for the dm3730-am3715-evm machine type (I'll be submitting patches for these soon) and I was getting really annoyed because I kept picking up packages that I didn't want. For example I had the following section in one of my tasks:

MULTIMEDIA = ""

MULTIMEDIA_omap3 = " \
ti-dmai-apps \
ti-codec-engine-examples \
task-arago-gst \
gstreamer-ti \
"

RDEPENDS_${PN} = "\
${MULTIMEDIA} \
"


I was expecting that since I didn't have a MULTIMEDIA_dm3730-am3715-evm variable defined that I would pick up the default MULTIMEDIA line and there would be no RDEPENDS setting. Instead I kept getting the omap3 settings which was trying to build packages I didn't want.

So I dug into the conf/machine/dm3730-am3715-evm.conf file and noticed that it included the omap3.inc file which sets SOC_FAMILY to omap3. This gave me my insight into what was happening. The omap3 lines were getting picked up because the dm3730-am3715-evm section was empty. Basically the order was MACHINE -> SOC_FAMILY -> DEFAULT.

I'm sure this is somewhat simplified, but it gets the point across. Basically as I see it the SOC_FAMILY is a good way to specify variables for an entire family of devices without having to say XYZ_dev1, XYZ_dev2, etc and set them all the same.

writing my source ipk class for OE

So as part of my work for building TI SDKs I needed a good way to distribute the sources for a package wrapped in an ipk package. This is because our newer installers actually use opkg to install the various components of the SDK onto the user's host system. Going forward this will allow users to select the components that get installed in their SDK with finer granularity.

But, if we are installing these packages we really need a good way to install the sources that correspond with them. This is where the sourceipk.bbclass comes into play. I should take a moment here and clarify that this source ipk is NOT intended as a method for building the binaries of a package like you might be used to when thinking of things like a source rpm. It is only for distributing the sources of a package.

So some goals:

• I don't want to have to modify a bunch of base recipes to add this feature. This causes divergence from the mainline.
  
• I want to distribute the patched sources as well as the patches and recipe
  
• I want to have control per package over whether a source ipk is built, what architecture it is built for, and where it will install on the host system.

Now what did I do?

I created a sourceipk.bbclass file in the classes directory of OE and added a "INHERIT += sourceipk" line to my configuration file (could be local.conf, angstrom.conf, whatever). This class adds a step between the do_patch and do_configure steps that will make a copy of the sources (which have now been patched) and generate an ipk in the deploy directory with the form (package name)-src_(version)_(architecture).ipk.

But just adding the INHERIT line is not enough. By default this class won't actually do anything unless you specify a non-zero value for CREATE_SRCIPK for the package you want (see the comments in the sourceipk.bbclass file for some more details). Basically you have to say per package whether you actually want a source ipk generated or not. I did this because there are more packages where I don't want/need the sources installed on the host system than packages that I do want the sources for. For example I may not need to have the dropbear sources on my customers system but I definitely need to give them the Linux kernel sources so they can rebuild their kernel.

So how do I specify which packages get source ipks built?

There are two ways to do this. For testing or if you are writing a new recipe and want people to get a source ipk by default you can add the following lines to your bitbake recipe:

CREATE_SRCIPK = "1"
SRCIPK_INSTALL_DIR = "where/do/you/want/it/installed"
SRCIPK_PACKAGE_ARCH = "architecture"
inherit sourceipk

The SRCIPK_INSTALL_DIR and SRCIPK_PACKAGE_ARCH lines are actually optional because the will default to "/usr/src/${PN}-src" for the SRCIPK_INSTALL_PREFIX and "all" for the SRCIPK_PACKAGE_ARCH if you don't specify them.

But remember that one of my goals was to NOT have to change a bunch of recipes to make source ipks. This is where the second method comes in. In the second method I can add the inherit of sourceipk into a configuration file and then define the values for each package in that file. For example I can modify my angstrom-2008.1.conf file and add the following lines:

INHERIT += "sourceipk"
CREATE_SRCIPK_pn-helloworld = "1"
SRCIPK_INSTALL_DIR_pn-helloworld = "demos/"

CREATE_SRCIPK_pn-linux-omap-psp = "1"
SRCIPK_INSTALL_DIR_pn-linux-omap-psp = "psp/kernel"
SRCIPK_PACKAGE_ARCH_pn-linux-omap-psp = "armv7a"

The above lines mean that I will generate a source ipk for the helloworld and linux-omap-psp packages when they are built. In the case of helloworld I want it installed on my target system in a directory called demos whereas I want the Linux kernel installed in a "psp/kernel" directory. I also specified a different architecture for the Linux kernel package (armv7a). This was just to show that it can be done. Usually you want to leave this as "all" so that the sources can be installed on any system.

Another quick note about the install directory. When installing the ipks for the TI SDK we specify a "-o" option to the opkg program. That way if I say that helloworld should install in the "demos" directory I can use a -o value of "/home/user/SDK" and it will actually install helloworld in the "/home/user/SDK/demos" directory. This is important to remember so that you don't install into the root of your development host.

Some interesting issues I faced:

• I originally had the sourceipk.bbclass adding a new package to the PACKAGES list so that it would just use the normal packaging system. That all seemed to be working well until I hit packages like the Linux kernel which decides to just reset/overwrite the PACKAGES variable. This meant that while my code would get executed and the sources made ready to go the default packaging system wouldn't make the package. So instead I decided to just call ipkg-build myself to create the package explicitly. I found the packaged-staging.bbclass sources helpful as well as the IPKG Howto site.
• When the patches were applied to the sources a "patches" directory was created in the sources directory containing all the patches, or so I thought. What it turned out to be was that they were symbolic links to the patches that were applied. So when I was doing a copy I was getting the symbolic link. When I would later install the source ipk I would get a bunch of messages about being unable to create the symbolic links because the files weren't there. What I needed to do was add the -L flag to my cp command so that I would copy the actual file pointed to by the symlink and not the symlink itself.

I still have an issue with trying to use the ipkg-build script. If I use that script I have to add a dependency on ipkg-utils-native to the sourceipk.bbclass. This produces a circular dependency because ipkg-utils-native will inherit sourceipk which in turn depends on ipkg-utils-native. For the mean time I have taken the pieces I needed of the ipkg-build script and put them into the sourceipk.bbclass directly. See the TODO comment in the sourceipk file for more details and if you know how to fix this issue please feel free to share.

OK, so that was a long blog post but I wanted to note what I had been working on, why I made it, and how it works. I also tried to document this in the comments and header of the sourceipk.bbclass code. I hope this was helpful.

My first post

OK, so now I have a blog going and I guess I have to do something with it. I've generally avoided blogging and twittering and things like that but I find that I should probably start writing down some of the things I am working on. I work for Texas Instruments focusing on open source platforms and methods. Currently I'm focused on things like the beagleboard and the future OSD3 device from Neuros. I'm also working a lot on how TI will build and distribute our software development kits going forward. This is going to be an OpenEmbedded based solution using an overlay called Arago. My blog is primarily going to focus on these topics.

Hopefully I'll keep up with my posting (remember I don't really like social media).