Anyhow, prior to installing CM 7.1 I’d carefully made a nandroid backup with ClockWorkMod recovery. Having set everything up I chose a ringtone + notification sound. Having lived with it for a bit I realised I miss the old sounds from the stock desire rom. To recover them I needed to mount extract the system.img file and then push the old audio files to the correct place.

Warning: Please follow these steps at your own risk!

Compiling unyaffs

To extract .img files you’ll need unyaffs which can extract the files from a yaffs file system image. First copy the .img file into a directory. (For me the Nandroid img files were found in clockworkmod/backup/<date>/<foo>.img on the sdcard)

To compile unyaffs grab the unyaff.c and unyaffs.h from http://code.google.com/p/unyaffs/downloads/list and compile it with:

gcc -o unyaffs unyaffs.c

Once you’ve done that and added it to your path (alternatively copy it to /usr/local/bin).

Navigate to the directory containing the yaff img and run:

./unyaffs system.img

This will un-pack the files into that directory.

Getting Files onto a Rooted Phone

Once you have the files the next step is to get them onto your phone. To do that you’ll need the Android SDK to be installed and set-up.

When the SDK is there, connect your phone with the USB cable (with USB debugging enabled – Settings->Application->Development) run the following from the sdk’s platform-tools directory.

./adb reboot recovery

When the phone reboots to recovery mount the system partition.

Next as an example here’s the command I ran to copy the ringtones from the desire rom onto the phone:

./adb remount
./adb push /home/bungle/desire-system/media/audio/ringtones /system/media/audio/ringtones

This will copy all the files in the ringtones dir onto the phone.

With the latest versions of Spotify on Linux (e.g: 0.6.1.309.gb871a7d0, 0.6.2.291.gcccc1f58) the DBus interface has changed a little which has resulted in the need to make some changes so that the playing status is tracked correctly.

The code can be found here on github as a Gist or the raw file.

I’ve recently updated the spotify notifier example I’d previously posted as an example along with my spotify screensaver script. First, I made an update to have it listen to the PropertiesChanged signal. Annoyingly for some unknown reason this signal isn’t visible in D-feet which had led to the erroneous assumption it wasn’t implemented.

Implementing Album Art

After getting it working again, I started to think about how it would be nice to add album art. So I rigged up the script to the last.fm API to grab album art. I’d have preferred to pull the album art from spotify or via open.spotify.com, but annoyingly there doesn’t seem to be a way to get the album art url without scraping, which just seemed too much of a kludge. If anyone knows how to get album art from the spotify cache (assuming it’s there) then I’d be interested to hear about it.

Listening for Spotify announcing itself on D-Bus

Once I had that done, I then thought about how this script would run. Ideally it needs to run before spotify is necessarily open. This causes a problem though because if you try and listen for D-Bus events when the app in question isn’t running you’ll get an error.

So I used the NameOwnerChanged signal and listened for Spotify joining D-Bus and leaving it again. This then provided all that was needed to turn on or off the listener for the PropertiesChanged signal that fires when the track changes.

Also I added a check to try and set-up the PropertiesChanged listener at start-up in-case spotify is already running when the script is launched.

The code

#!/usr/bin/env python
# -*- coding: utf-8 -*-

"""
Title: Spotify Notification Demo
Author: Stuart Colville, http://muffinresearch.co.uk
License: BSD

"""

import os
import dbus
import gobject
import pynotify
import httplib2
from urllib import quote
from cgi import escape
from xml.dom.minidom import parseString
from hashlib import md5
from dbus.mainloop.glib import DBusGMainLoop
from dbus.exceptions import DBusException

LASTFM_API_KEY = os.environ.get("LASTFM_API_KEY")
HTTP_CACHE_DIR = os.path.expanduser("~/.cache/spotify/http/")
IMAGE_CACHE_DIR = os.path.expanduser("~/.cache/spotify/art/")
BASE_URL = ("https://ws.audioscrobbler.com/2.0/?method=album."
            "getinfo&api_key=%s&artist=%%s&album=%%s" % LASTFM_API_KEY)

MISSING = os.path.realpath(os.path.join(os.path.dirname(__file__), "missing.png"))

if not os.path.isdir(HTTP_CACHE_DIR):
    os.makedirs(HTTP_CACHE_DIR)
if not os.path.isdir(IMAGE_CACHE_DIR):
    os.makedirs(IMAGE_CACHE_DIR)

class SpotifyNotifier(object):

    def __init__(self):
        """initialise."""
        bus_loop = DBusGMainLoop(set_as_default=True)
        self.bus = dbus.SessionBus(mainloop=bus_loop)
        loop = gobject.MainLoop()
        self.http = httplib2.Http(HTTP_CACHE_DIR)
        self.notify_id = None
        try:
            self.props_changed_listener()
        except DBusException, e:
            if not ("org.mpris.MediaPlayer2.spotify "
                    "was not provided") in e.get_dbus_message():
                raise
        self.session_bus = self.bus.get_object("org.freedesktop.DBus",
                                 "/org/freedesktop/DBus")
        self.session_bus.connect_to_signal("NameOwnerChanged",
                                        self.handle_name_owner_changed,
                                        arg0="org.mpris.MediaPlayer2.spotify")

        loop.run()

    def props_changed_listener(self):
        """Hook up callback to PropertiesChanged event."""
        self.spotify = self.bus.get_object("org.mpris.MediaPlayer2.spotify",
                                           "/org/mpris/MediaPlayer2")
        self.spotify.connect_to_signal("PropertiesChanged",
                                        self.handle_properties_changed)

    def handle_name_owner_changed(self, name, older_owner, new_owner):
        """Introspect the NameOwnerChanged signal to work out if spotify has started."""
        if name == "org.mpris.MediaPlayer2.spotify":
            if new_owner:
                # spotify has been launched - hook it up.
                self.props_changed_listener()
            else:
                self.spotify = None

    def handle_properties_changed(self, interface, changed_props, invalidated_props):
        """Handle track changes."""
        metadata = changed_props.get("Metadata", {})
        if metadata:
            if pynotify.init("Spotify Notifier Demo"):

                title = unicode(metadata.get("xesam:title").encode("latin1"))
                album = unicode(metadata.get("xesam:album").encode("latin1"))
                artist = unicode(metadata.get("xesam:artist").encode("latin1"))
                hash_ = md5()
                hash_.update("%s-%s" % (artist, album))
                hash_path = hash_.hexdigest()
                image_path = os.path.join(IMAGE_CACHE_DIR, hash_path)
                if not os.path.exists(image_path):
                    url = BASE_URL % (escape(quote(artist.encode("utf8"))),
                                       escape(quote(album.encode("utf8"))))
                    response, content = self.http.request(url, "GET")
                    dom = parseString(content)
                    images = dom.getElementsByTagName("image")
                    for image in images:
                        for attr, value in image.attributes.items():
                            if attr == "size" and value == "medium":
                                image_url = None
                                if image and image.firstChild:
                                    image_url = image.firstChild.nodeValue
                                if image_url:
                                    response, image_contents = \
                                                self.http.request(image_url)
                                    if image_contents:
                                        fh = open(image_path, "w")
                                        fh.write(image_contents)
                                        fh.close()            

                if not os.path.exists(image_path):
                    image_path = MISSING

                alert = pynotify.Notification(title,
                              "by %s from %s" % (artist, album), image_path)
                if self.notify_id:
                    alert.props.id = self.notify_id
                alert.set_urgency (pynotify.URGENCY_NORMAL)
                alert.show()
                self.notify_id = alert.props.id

if __name__ == "__main__":
    SpotifyNotifier()

Pre-requisites

• Spotify for Linux
• A Spotify unlimited or premium account
• Install httplib2: sudo apt-get install python-httplib2 (I think that’s the only dep that isn’t installed by default)
• Set-up the script as outlined below.

Getting the code

The code is available in a bzr branch here: code.projectfondue.com

To run it check it out and follow the notes in the README file. No doubt there might be some bugs so feel free to let me know of any possible improvements that can be made to it.

Pathogen

pathogen is a helper for modifying the runtimepath. This makes it possible to branch a plugin from github to .vim/bundle and then pathogen will ensure that it’s added to the runtime path.

To make this work first you need to download pathogen and place it in .vim/autoload

Next add the following to your .vimrc file:

call pathogen#runtime_append_all_bundles()

Once you have done this simple check-out the branches of your favorite vim plugins to .vim/bundle and that's it.

Scripting plugin updates

To make life even easier I knocked up a very crude python script and Makefile to clone all my favorite plugins when I run "make get" and update them when I run "make update".

It's crude because currently it only cares about git (currently all the plugins I care about are on github) and it doesn't currently allow revs to be specified. Though both of these features could be fairly easily added. Anyway, in-case it's useful as a starting point to for someone else, here's the script:

Also if you're using git to version your branch (I'm not). Then you could use git submodules to achieve a similar effect.

#!/usr/bin/env python

import os
import sys
import subprocess

BUNDLE_DIR = os.path.expanduser("~/.vim/bundle")

plugin_branches = {
    "git": [
        "git://github.com/tomtom/tlib_vim.git",
        "git://github.com/garbas/vim-snipmate.git",
        "git://github.com/scrooloose/nerdtree.git",
        "git://github.com/scrooloose/syntastic.git",
        "git://github.com/MarcWeber/vim-addon-mw-utils.git",
        "git://github.com/scrooloose/nerdcommenter.git",
    ],
}

def get():
    os.chdir(BUNDLE_DIR)
    for vcs_type, branches in plugin_branches.items():
        if vcs_type == "git":
            for branch in branches:
                subprocess.call(["git", "clone", branch])

def update():
    os.chdir(BUNDLE_DIR)
    for vcs_type, branches in plugin_branches.items():
        if vcs_type == "git":
            for branch in branches:
                subdir = os.path.split(branch)[-1][:-4]
                os.chdir(subdir)
                subprocess.call(["git", "pull"])
                os.chdir("../")

if __name__ == "__main__":
    args = sys.argv
    if args and len(args) > 1:
        if args[1] == "get":
            sys.exit(get())
        elif args[1] == "update":
            sys.exit(update())

If you save this somewhere as deps.py you can then run it as python deps.py get and python deps.py update. I use a simple Makefile to do this.

The code is deliberately structured to make it easy to modify if you wanted to add mercurial or bzr support. That is left as an excercise for the reader.

It's likely I will update this script in due course, to keep up to date with that and any other aspects of my vim conf you can get the whole thing from here: lp:~muffinresearch/+junk/dotvim

All that was required was something to pretend to be a real SMTP service as we didn’t need to view the messages at all.

Here’s the snippet I came up with using the smtpd module in Python:

#!/usr/bin/env python
"""A noddy fake smtp server."""

import smtpd
import asyncore

class FakeSMTPServer(smtpd.SMTPServer):
    """A Fake smtp server"""

    def __init__(*args, **kwargs):
        print "Running fake smtp server on port 25"
        smtpd.SMTPServer.__init__(*args, **kwargs)

    def process_message(*args, **kwargs):
        pass

if __name__ == "__main__":
    smtp_server = FakeSMTPServer(('localhost', 25), None)
    try:
        asyncore.loop()
    except KeyboardInterrupt:
        smtp_server.close()

To use this, save the above as fake_stmp.py and:

chmod +x fake_smtp.py
sudo ./fake_smtp.py

NOTE: Running this requires sudo to bind to port 25 as binding any port < 1024 requires superuser privileges.

Clearly you could easily adapt that to do something more useful if you want to see what's in the messages sent, either by writing messages to a file or printing them to stdout.

If printing them to stdout is what you need then there's also smtpd.DebuggingServer which is a subclass of SMTPServer which prints messages out to stdout. Using that is even easier and can be done with the following one liner:

sudo python -m smtpd -n -c DebuggingServer localhost:25

Again we use sudo in this case because we’re using port 25 if you don’t want that you can use a port higher than 1024.

To test this out we can use telnet like so:

$ telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.localdomain.
Escape character is '^]'.
220 localhost6.localdomain6 Python SMTP proxy version 0.2
HELO localhost
250 localhost6.localdomain6
MAIL FROM: test@test.com
250 Ok
RCPT TO: test@testing.com
250 Ok
DATA
354 End data with .
subject: Hello
to: test@test.com
This is my message
.
250 Ok
QUIT
221 Bye
Connection closed by foreign host.

Over on the terminal running the debugging server you should see the following:

$ sudo python -m smtpd -n -c DebuggingServer localhost:25
[sudo] password for moo:
---------- MESSAGE FOLLOWS ----------
subject: Hello
to: test@test.com
This is my message
------------ END MESSAGE ------------

I’ve recently found it really hurts my eyes when working late at night going from a terminal (which is black background and light text) to a browser window like google.com with a large expanse of white.

To help with this I’ve been looking for ways to reduce the screen brightness.

The solution I’ve come up with on Ubuntu is to use Compiz to invert the screen colours. If you open CompizConfig Settings Manager and look for Negative under Accessibility all you need to do is make sure it’s enabled and set a keystroke to toggle the inversion. By default it inverts everything within the window. However when you have terminals set to dark backgrounds like I do you don’t want those to be inverted and become glaring white backgrounds with dark text. Fortunately Compiz provides a feature to exclude certain windows.

Here’s an example for the “Exclude Windows” field to exclude the Desktop (a default), Guake and Gnome-terminal from being inverted.

(type=Desktop) | class=Guake.py | class=Gnome-terminal

If you use a mac you should be able to achieve the raw inversion with “Control Option Command 8″

Great. Now all I need is a silencer for my clacky keyboard!

New version: I’ve updated the script to use D-bus with spotify for Linux version 0.4.8 see Spotify Screensaver Toggle with D-bus for further information.

In a previous post, Lock Screen and Pause Spotify I provided a rudimentary script to pause spotify when your screensaver kicks in. Now with the Spotify Linux preview being available I’ve started using (and paying for) a Spotify unlimited subscription. I decided I wanted to update the script to be able to pause Spotify when the screen is locked and also start playing after I log-in.

Initially I was hoping to do all of this by listening to Dbus events for when the screensaver is activated/deactivated. However, for some reason it’s not possible to play/pause spotify based on the DBus event saying that the screensaver is active (Presumably because it’s not possible to focus a window after the screensaver is active). Because of this current limitation the workaround is use this script to lock the screen for you. This way it locks the screen after it has set-up the listeners for the screensaver Dbus events.

When the screensaver is de-activated the script fires off the function to get Spotify playing again but only if it was playing before the screensaver was activated.

When spotify has an api, it will then make sense to use that instead, which should mean that this script can then be completely run by listening to dbus events which would be much more elegant.

Dependencies

The deps are the same as before – here’s the incantation to install them for Ubuntu.

sudo apt-get install wmctrl xvkbd

Usage

To set it up, link the script to somewhere in your path and then create a keystroke in “System -> Preferences -> Keyboard Shortcuts” to lock the screen. Once that’s done, you should see the screen lock when that keystroke is executed and Spotify will be paused if it was playing.

Get the code

You can grab the code from here. Feel free to send me improvements or suggestions on how to make it better.

Known issue: I’ve noticed a bug in that it doesn’t pause if the window is hidden. This is tricky as it’s not visible to the window list. Suggestions how to workaround this are welcomed.

Additional thanks

Tommy Mattila’s Spotify Control which was based on my original script, provided useful insight into handling the active window.

DDMS stands for the Dalvik Debug Monitor Server. It provides various facilities to debug applications on you phone.

Enable USB Debugging

To use it first you need to, enable debugging via USB. To do that open the settings app. From there navigate to “Applications”, in there select the “Development” option and then tick “USB Debugging”.

Connect your phone

Next Connect your phone with the USB cable. Disk Mode is fine for this purpose.

Get a copy of the SDK

If you don’t already have a copy of the Android SDK you can get one from the SDK downloads page.

Once you have this unpack it.

Running DDMS

To run ddms you should be able to run from the tools directory ddms directly e.g:

./ddms

If you can see you phone connected then all that’s left to do is select it and then select Device -> Screen capture.

However if you just see unknown device with lots of question marks like I did then you’ll need to start the android debug bridge.

To do this run the following from the sdk tools directory after shutting ddms down:

sudo ./adb kill-server
sudo ./adb start-server

Once it’s running open ddms again:

./ddms

You should now see the device connected. Now you can select the device and then hit Device -> Screen capture.

The following window should open up:

With that open you can then open any apps that you need to and just press refresh to get the current screen. Once you have the shot that you wanted press save and that’s all there is to it. I’m looking forward to there being a simple app to do this that doesn’t require root!

Photo CC licensed by oedipusphinx

I’ve been working for canonical for about four and a half months now. One of the things that I’ve been really impressed with is the workflow for development and the process for how new features end up in trunk.

The workflow is as follows:

Feature Development

Our primary development focus is trunk as this is what’s deployed on a weekly basis. So the first step is to pull the local copy of trunk to ensure it’s up to date. Next, the developer makes a branch of trunk for the feature/bugfix.

cd trunk
bzr pull
cd ../
bzr branch trunk branches/my-awesome-feature

The feature is developed with TDD (tests first, followed by code to make the tests pass). Once the feature or bugfix is finished the branch is pushed up to launchpad under the project.

# hack hack hack
# Ensure tests pass
bzr push lp:~<USER>/<PROJECT>/<BRANCHNAME>

A nice feature of bzr that I’d not used prior to working at canonical is the way you can automatically set push targets for branches based on the directory that contains them. So for example we have a config in ~/.bazaar/locations.conf that looks something like this (obviously substituting the variables in angled brackets for your username and project etc):

[/path/to/branches/]
push_location = bzr+ssh://<USERNAME>@bazaar.launchpad.net/~<USERNAME>/<PROJECT>
push_location:policy = appendpath

So if I have a new branch at /path/to/branches/foo-bar-baz

This will automatically set the push location to:

bzr+ssh://<USERNAME>@bazaar.launchpad.net/~<USERNAME>/<PROJECT>/foo-bar-baz

Meaning all I have to do is use bzr push to send my branch into launchpad.

Note: At any time you can check the push location with bzr info

Code Review

The next stage is for the developer to make a “merge proposal” via the launchpad interface. This puts the merge request in a “to be reviewed queue” which provided a merge proposal page with a diff of the changes from trunk (the target branch is customisable). The rule is that every branch has to be approved by a minimum of two devs. Reviewers can make recommendations or assert that something needs to be fixed and the developer can then push subsequent changes to the same branch and those changes are made visible.

The focus of reviews are on coding style, standards e.g. adherence to PEP 8, and reviewers will run the test suite and run pylint to check for any glaring violations. Merging with trunk is also carried out to check that no conflicts occur.

Landing branches

Once the code has been successfully reviewed code is submitted via PQM (Patch Queue Manager) which then runs the test suite and only merges the code with trunk if the tests pass. We’re still using PQM but most projects have already begun the process to move to tarmac which is tightly integrated to launchpad.

I like the rigorous review process. By having code reviews a mandatory part of the process it becomes just part of a routine, everyone is used to it and so are happy to receive and give criticism. Reviews are definitely seen as a route to better code rather than making people feel small which is exactly how it should be.

The process might sound quite lengthy but because of the ease of branching in bzr (like any other DVCS) the process is very quick and easy and branches are continually landed in trunk every day. It’s actively encouraged that branches are kept around 500 lines of diff where possible, to make it easier for code to get reviewed. Branches aren’t worked on for weeks on end, they are developed, reviewed and merged within what’s usually at most a couple of days.

Future changes

One of the things we’re currently looking at is using hudson to automate the running of the test-suite for merge-proposals. This would mean that hudson could run the tests as soon as a merge proposal is made and provide an “approval” or “needs fixing” status based on the results of running the test suite. The net effect would be that we’d then require 3 approvals rather than 2.

Summary

Obviously our workflow revolves around the use of launchpad, but there’s plenty of ways to adopt a similar workflow without requiring the use of launchpad. For example on personal projects I’ve adopted pretty much the same process except branches are pushed to remote server instead of launchpad and rietveld has been used for reviews in place of the merge proposal feature. Rietveld is fairly vcs-agnostic however, if you’re using git as your vcs then something like gerrit might be worth looking at. (gerrit was originally a fork of rietveld but has more recently been completely re-written using java instead of python). Also review-board is another code review tool that works for all popular VCSs – review-board is django-based and is therefore relatively easy to set-up.

In summary:

• Use a DVCS (e.g: git, mecurial, bzr) so branching is easy. I remember at Yahoo when CVS was the VCS everyone had to use, branching was explicitly avoided as much as possible because merging was seen as such a pain. Worst case, if you’re stuck with something like svn be comfortable with merging so branching is not a hurdle.
• Branch as often as necessary.
• Review everything and make it a part of the code routine.
• Merge as early as possible.
• Keep Trunk pristine.

$ sudo puppetd -t
err: Could not retrieve catalog from remote server: undefined method `closed?' for nil:NilClass
warning: Not using cache on failed catalog
err: Could not retrieve catalog; skipping run

Certainly not the most enlightening error message. After checking over the syntax of my mainfests it still wasn’t obvious what was wrong.

First port of call to diagnose the problem is to try enabling debugging on the client:

On the client:

# Run the client with debugging enabled.
$ sudo puppetd -t --debug

If that yields nothing useful try the following:

On the puppetmaster run:

# If it's running stop the puppetmasterd service
# change this to suit your distro - the following is debian specific
$ sudo /etc/init.d/puppetmaster stop 

# start the puppetmasterd in debug mode without daemonizing it.
$ sudo puppetmasterd --no-daemonize --debug

Back on the client re-run the puppetd:

$ sudo puppetd -t

By looking at the debug output on the puppetmaster I was able to see my problem which was that a generate command didn’t have permission to write a necessary file. Once I’d fixed it everything worked as expected.

Once finished ctrl-c the foreground puppetmasterd and restart the service if you’d stopped it to run it in debug mode:

$ sudo /etc/init.d/puppetmaster start