Tim Hatch

Weblog | Photos | Projects | Panoramas | About

Earthquake Yesterday 30 Jul, 2008

Looks like we had a 5.6 earthquake in the greater L.A. area yesterday. We felt it at the office, but it wasn’t much since we are around 100 miles away (a bit like I was tapping my leg, only I was’t). So I have seen a wildfire, evacuation, a heat wave, and an earthquake. Now the only thing I need to be fully acclimated, the locals tell me, is a flood. Good thing I live on a hill!

Simulating a Rebase in Mercurial 26 Jul, 2008

I ran into this while doing some work with Pygments this morning — I continued working in a local repo without consciously realizing that a merge node had been created from my work yesterday. If I had just pulled that merge node before committing this morning, everything would be pretty. But it wasn’t, and I wanted to avoid merging the merge which would then cause another merge upstream. Forgive the ascii art.

       *   $PUSH1
       |\
       | * 
       |  \
       |   * $PUSH1
MERGE1 *--/ \
       |     \
       |\     * $CUR2, I don't want this.
       | \
       |  * $PUSH2, I want this, so I can...
MERGE2 *-/

Here’s how:

Background

I pushed $PUSH1, which got merged with $TRUNK, creating $MERGE1.

Some local changes (on top of $PUSH1) later, I am ready to push again (I have $CUR2 and want to end up with $PUSH2 on top of $MERGE1).

First, enable the transplant extension.

[extensions]
transplant=

Go ahead and pull so you have multiple heads in your local repo (don’t push to your personal remote, any of ($MERGE1:$CUR2] or you’ll have to go through an extra set of steps first, below).

Rebasing in HG

# we start with two heads, $MERGE and $CUR2, and no uncommitted changes
hg update -C $MERGE
hg transplant --branch $CUR2
# at this point, your $MERGE head is updated to what we'll call $MERGE2
cd ..

# this is the easiest way to remove $CUR2's head
hg clone -r $MERGE2 repo new_repo
mv repo old_repo
mv new_repo repo
cd repo
hg push

Extra steps if you pushed

If you did push something on $CUR2’s line, you’ll need to fix that first.

ssh remote

hg clone -r$MERGE1 remote new_remote
cp remote/.hg/hgrc new_remote/.hg/
# set permissions like they were before
mv remote old_remote
mv new_remote remote

A Good Profiling Decorator 20 Jul, 2008

To see where slowdowns are in your Python programs, Python provides some low-level hooks (sys.settrace) and a module to evaluate timings (profile and its faster cousin cProfile). It’s pretty easy to run these on your whole program (on Ubuntu, make sure you install python-profiler):

slow.py

def slow():
    x = "Hello world" * 100000
    y = ''.join(x)

slow()

Invocation

$ python -m cProfile ./slow.py

         6 function calls in 0.067 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.067    0.067 <string>:1(<module>)
        1    0.000    0.000    0.066    0.066 slow.py:1(<module>)
        1    0.001    0.001    0.066    0.066 slow.py:1(slow)
        1    0.001    0.001    0.067    0.067 {execfile}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        1    0.065    0.065    0.065    0.065 {method 'join' of 'str' objects}

This is a pretty powerful tool, so look into the options for sorting — in this case the slowdown is in str.join() which would be highlighted by sorting on tottime (add -s time before the file to run).

The built-in profile module doesn’t work so well on threaded programs (which I was last week) or when the entire app waits a lot on io (ditto) or when you’re running on a really slow cpu (ditto), so I came up with a good profiling decorator, which allows you to just see profiling stats on a function at a time:

profiler.py

try:
    import cProfile as profile
except ImportError:
    import profile

import os.path

__all__ = ['profiled']

def profiled(path, multi=True):
    """
    Decorator to allow individual functions to be profiled, without profiling
    the whole program.  This allows for much more targeted profiling, which is
    necessary for threaded programs or if performance becomes an issue.

    multi: if True, adds a sequential number to each profile run to prevent
                    name collisions.
           if False, the last invocation wins.
    """
    # This extra layer of indirection is so the decorator can accept arguments
    # When the user doesn't provide arguments, the first arg is the function,
    # so detect that and show an error.
    if not isinstance(path, (str, unicode)):
        raise Exception("This decorator takes a path argument")
    d = os.path.dirname(path)
    assert os.path.exists(d)
    p, q = os.path.splitext(path)
    i = [0]

    def decorator(func):
        def newfunc(*args, **kwargs):
            pr = profile.Profile()
            ret = pr.runcall(func, *args, **kwargs)
            if multi:
                fn = "%s.%d%s" % (p, i[0], q)
            else:
                fn = path
            i[0] += 1
            pr.dump_stats(fn)
            return ret
        # Be well-behaved
        newfunc.__name__ = func.__name__
        newfunc.__doc__ = func.__doc__
        newfunc.__dict__.update(func.__dict__)
        return newfunc
    return decorator

Invocation

Edit your source file that needs profiling:

from profiler import profiled
@profiled("/tmp/func.stats")
def slow(): ...

Now just run it as usual (python slow.py). /tmp/func.0.stats contains the first run, so let’s look at it:

$ python -c 'import pstats; pstats.Stats("/tmp/func.0.stats").sort_stats("time").print_stats(30)'
Sun Jul 20 10:38:42 2008    /tmp/func.0.stats

         3 function calls in 0.051 CPU seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.050    0.050    0.050    0.050 {method 'join' of 'str' objects}
        1    0.001    0.001    0.051    0.051 slow.py:3(slow)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

The argument to print_stats controls how many lines are printed, and there are alternate sorting keys (make sure you specify one, otherwise it autoselects “random”): See http://docs.python.org/lib/profile-stats.html

Update: Found a more complete (including coverage) decorator by Marius Gedminas whom I met briefly at Europython 2007, http://mg.pov.lt/blog/profiling.html.

Wireless Network Names 13 Jul, 2008

I’m amused by the names of wireless networks that people come up with. My first example is at my old apartment in Denton, and everyone was getting FIOS routers with random-looking 5-character SSIDs. “Ninjas > Pirates” I’m not sure if I agree with, but I’m glad that they’re at least opinionated enough to broadcast that one. Mine is minihub, on a WRT54GC so it’s small.

This one is at my new office, where there’s a Rails shop across the hall which actually has a scrum-style burndown chart on the wall. I’m assuming they also do pair programming, but don’t let people working alone on that network. (Given the existence of “pair”, “Pairs”, and “solo”, even the pair programmers must be separated somehow, maybe if they’re the top pair they get their own network?) nutricat’em (no apostrophe, just because), and gumnut are ours, and we don’t know who has the Apple one.

Deja Vu 07 Jul, 2008

We’re un-evacuated now, so I am home finally, safe and sound.

A bit like the first chapter of the Hitchhiker’s Guide to the Galaxy, some strange events are occurring to me and I don’t realize as early as I should that something is strange.

I have done the following almost every day in July. Here’s how it happened tonight:

  • I was minding my own business, reading my RSS feeds and checking my mail, when the music cut off. Pandora at work does that all the time so I don’t think anything of it. I was on the laptop.
  • I go to ssh somewhere, and get an error. Pinging 4.2.2.4 gives “Connection Refused” at which point I realize the Airport isn’t connected to anything.
  • Upon searching, approximately zero wireless networks show up. Yellow.
  • Powercycle the Airport card, no change. Bulldozers.
  • I go to flip on a light as I head toward the router, guess the bulb is burned out.
  • Guess I kicked the power strip, my laptop isn’t charging anymore. Oh well, I’ll deal with that after the router.
  • Doh! The power probably went out again. Look at the neighbors’ lights. Yeah. Time to read a book.
  • You know, I need to get a bigger UPS. In a browser I type newegg.com, which results in an error page. At which point I think maybe a nap would help more.

Resumr 0.1.1 Released 05 Jul, 2008

I am proud to announce an experimental piece of software I’ve written to help recovering broken rars. Say you've got a rar set for your favorite Linux iso, but some random part is bad and you have no recovery volumes. In this situation, you also have a torrent available, but don’t feel like downloading the whole thing. This tool will take your (broken) rars and reconstruct the full file for you, so the missing blocks can be recovered using the torrent.

Check it out in the projects section.

Evacuated Due to Fire 04 Jul, 2008

Today is the first time I've honestly wondered, "what is the proper whitebalance for a sky that's raining ash?" The Gap Fire which has simply been making it uncomfortable to breathe, threatened my neighborhood last night so an evacuation order was issued while I was out setting up a UPS to keep the network gear alive at the office. I'm staying with my coworker Roberto and his wife Karen now (thanks, guys).


The sky at the office last night


Cathedral Oaks near La Patera, two days ago

I wasn't conscious of the area covered by smoke until I left it last night — in the normal course of life I've been under it for three straight days. In the evenings it blows south, straight to the ocean; during the day, to the west. Fire map as of 8 this morning The fire is creeping around so instead of being just a mile west, it's also a mile north (see map at the side, pulled from Google Earth with the Forest Service's fire kmz overlay).

Goleta Post Office 02 Jul, 2008

I just went by the post office on Patterson and they have an area labeled “cycle parking only” with an actual place to lock your bike. Cool.

What Good is a PC that Won't Boot? 02 Jul, 2008

Among the things I’m getting around to unpacking is my desktop computer. I was planning a switch to the Athlon64 I have since the dual Athlon XP bit the dust just days before I left Denton (it was bridge modded and three years was longer than I expected to get out of it anyway). The Athlon64 no longer POSTs due to I believe a momentary short in the front panel USB ports of the case, which has fried the northbridge — it gets burning hot within seconds of applying power now, and never gives a post beep even with the motherboard out of the case).

Computer’s don’t travel well, it seems. Oh well, trip to Fry’s! They had a Q6600 combo on sale (and they’re real good about having boxed cpu deals now) so I made the trek last weekend. I don’t know why I still get CDs from Fry’s, since the last three I have all had broken jewel cases found after the fact... but I did. I think it’s the instant gratification as opposed to the value and relative quality of Amazon’s music selection.

The motherboard was an ECS G31T-M, which features some impressive onboard video (quite well supported on Linux, although no DVI connector) and good overall performance. FSB1333 is not supported, and the USB2 ports onboard use up a lot of cpu, but that’s easily worked around by a USB2 PCI card I had sitting around. Also the MS Wireless Optical Desktop 4000 has pretty good range (around 5-6 feet reliably) so I’m using it at home and taking to work so I have a consistent keyboard feel. It’s slightly ergonomic but not too goofy. (I hit H with the wrong hand so all the MS Natural keyboard slow my typing considerably unless I use them all the time.)

After I got all that working, I bicycled 5 miles and came back tired. Still not ready for long rides.

The Drive from Dallas to LA 01 Jul, 2008

The countryside in New Mexico, I think

It’s long. Even flying, it’s something like 5 hours and a pretty boring flight. That should be some sort of warning. Luckily my family and I managed to avoid finding $5/gal gas everywhere except in the middle of the desert (it’s still well under $5 here in SB right now). I got around 33mpg so it was around as cheap as flying if you just consider fuel costs, just took a bit longer (3 long days). Dealing with plant inspections is something new to me, so I had to leave my Ikea plant back with my grandparents in Lubbock. Now that I’m here I bought another from the Ikea in Carson (I was in Long Beach visiting my great aunt’s family).

I now have a California driver’s license, which entailed a trip to the DMV which is quite interesting. I got to queue to get a form to fill out, then queue again to turn in the form to get a number, which when called would let me go to a window to figure out the next window to queue at, take a test, and queue again to get it graded. The test is quite a bit different from the one I took in Texas (which was more concerned with right-of-way in different road size situations), and had topics like curbing wheels on hills and safety zones for trolleys.

The Foothills and a Fire 01 Jul, 2008

View from Old San Marcos Rd., in the hills

Last week was another World Wide Panorama event, and since I was still moving in, I wasn’t sure I’d participate. Curious for ideas, one of my housemates named Paige helped out and showed me how to get to Old San Marcos Pass which is a real windy road that heads up into the hills. The modern San Marcos Pass is highway 154 which has a speed limit of something like 55mph. Signage is a bit more sparse on OSMP and it’s not like you’re going to hit anything more than 25mph average anyway. The few signs warning of sharp turns were especially disconcerting after you’ve passed 10 really sharp turns with no sign and wonder exactly how much sharper this one must be to get a sign.

The photo to the side is from the location that I think I’ll submit, and is a view southwest from the location of the map above.

I read that there are something like 1000 fires burning at any given point in time in California which seems like a large number to an outsider. I saw an odd-shaped cloud on the way home from work today and realized that it is a fire burning on the side of the hills, visible from Goleta around two miles from home. I don’t know if any structures were involved or if it was just a forest fire, but nobody seemed much alarmed about it so I suppose it’s a regular occurrence. The location was around Wylie Canyon somewhere (which is about a mile rightward of the photo here).

Update: The Independent is reporting on the fire.

I’ve arrived 01 Jul, 2008

East Beach in Santa Barbara

I’ve arrived! Things have finally slowed down enough that I can get a weblog post in edgewise. First things first, I’m now moved to the Pacific coast in California, and have a new job as well.

Santa Barbara is literally pretty cool. The temperatures are almost always perfect (in the dead of winter the high is in the 60’s), and moving from a wooded part of Texas it’s a big shock how cool it can get at night, even when the days are hot. I’m renting a room, in a house with a view of the channel islands (on a clear day, over some other houses and power lines).

Another shock was that there is no FIOS available here, so I actually have to plan ahead if I have large data files I need to move around. I got a bit spoiled in Denton since if needed to move a 4GB VMWare image, it could transfer faster than I could drive home to burn it. 3-5Mbit DSL and Cable seem to be the norm here, with miniscule upload speeds by comparison.

The company I’m working for is called Nutricate, and I found out about the job from the Python Job Board so I can tell you that it really does work. I started there in June and it’s been a lot of fun so far. I get to write Python, primarily for an embedded ARM system, but a good part of it is interacting with serial peripherals, having only a partial spec for how the communication is to occur. Our product is to add nutrition information to receipts whenever you order food, which is currently a box (but we’re “not a hardware company”).