Looks like we had a 5.6 earthquake in the greater L.A. area yesterday. We felt it at the office, but it wasn’t much since we are around 100 miles away (a bit like I was tapping my leg, only I was’t). So I have seen a wildfire, evacuation, a heat wave, and an earthquake. Now the only thing I need to be fully acclimated, the locals tell me, is a flood. Good thing I live on a hill!
I ran into this while doing some work with Pygments this morning — I continued working in a local repo without consciously realizing that a merge node had been created from my work yesterday. If I had just pulled that merge node before committing this morning, everything would be pretty. But it wasn’t, and I wanted to avoid merging the merge which would then cause another merge upstream. Forgive the ascii art.
* $PUSH1
|\
| *
| \
| * $PUSH1
MERGE1 *--/ \
| \
|\ * $CUR2, I don't want this.
| \
| * $PUSH2, I want this, so I can...
MERGE2 *-/
Here’s how:
I pushed $PUSH1, which got merged with $TRUNK, creating $MERGE1.
Some local changes (on top of $PUSH1) later, I am ready to push again (I have
$CUR2 and want to end up with $PUSH2 on top of $MERGE1).
First, enable the transplant extension.
[extensions] transplant=
Go ahead and pull so you have multiple heads in your local repo (don’t push to
your personal remote, any of ($MERGE1:$CUR2] or you’ll have to go
through an extra set of steps first, below).
# we start with two heads, $MERGE and $CUR2, and no uncommitted changes hg update -C $MERGE hg transplant --branch $CUR2 # at this point, your $MERGE head is updated to what we'll call $MERGE2 cd .. # this is the easiest way to remove $CUR2's head hg clone -r $MERGE2 repo new_repo mv repo old_repo mv new_repo repo cd repo hg push
If you did push something on $CUR2’s line, you’ll need to fix that first.
ssh remote hg clone -r$MERGE1 remote new_remote cp remote/.hg/hgrc new_remote/.hg/ # set permissions like they were before mv remote old_remote mv new_remote remote
To see where slowdowns are in your Python programs, Python provides some low-level hooks (sys.settrace) and a module to evaluate timings (profile and its faster cousin cProfile). It’s pretty easy to run these on your whole program (on Ubuntu, make sure you install python-profiler):
def slow():
x = "Hello world" * 100000
y = ''.join(x)
slow()
$ python -m cProfile ./slow.py
6 function calls in 0.067 CPU seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 0.067 0.067 <string>:1(<module>)
1 0.000 0.000 0.066 0.066 slow.py:1(<module>)
1 0.001 0.001 0.066 0.066 slow.py:1(slow)
1 0.001 0.001 0.067 0.067 {execfile}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
1 0.065 0.065 0.065 0.065 {method 'join' of 'str' objects}
This is a pretty powerful tool, so look into the options for sorting — in this case the slowdown is in str.join() which would be highlighted by sorting on tottime (add -s time before the file to run).
The built-in profile module doesn’t work so well on threaded programs (which I was last week) or when the entire app waits a lot on io (ditto) or when you’re running on a really slow cpu (ditto), so I came up with a good profiling decorator, which allows you to just see profiling stats on a function at a time:
try:
import cProfile as profile
except ImportError:
import profile
import os.path
__all__ = ['profiled']
def profiled(path, multi=True):
"""
Decorator to allow individual functions to be profiled, without profiling
the whole program. This allows for much more targeted profiling, which is
necessary for threaded programs or if performance becomes an issue.
multi: if True, adds a sequential number to each profile run to prevent
name collisions.
if False, the last invocation wins.
"""
# This extra layer of indirection is so the decorator can accept arguments
# When the user doesn't provide arguments, the first arg is the function,
# so detect that and show an error.
if not isinstance(path, (str, unicode)):
raise Exception("This decorator takes a path argument")
d = os.path.dirname(path)
assert os.path.exists(d)
p, q = os.path.splitext(path)
i = [0]
def decorator(func):
def newfunc(*args, **kwargs):
pr = profile.Profile()
ret = pr.runcall(func, *args, **kwargs)
if multi:
fn = "%s.%d%s" % (p, i[0], q)
else:
fn = path
i[0] += 1
pr.dump_stats(fn)
return ret
# Be well-behaved
newfunc.__name__ = func.__name__
newfunc.__doc__ = func.__doc__
newfunc.__dict__.update(func.__dict__)
return newfunc
return decorator
Edit your source file that needs profiling:
from profiler import profiled
@profiled("/tmp/func.stats")
def slow(): ...
Now just run it as usual (python slow.py). /tmp/func.0.stats contains the first run, so let’s look at it:
$ python -c 'import pstats; pstats.Stats("/tmp/func.0.stats").sort_stats("time").print_stats(30)'
Sun Jul 20 10:38:42 2008 /tmp/func.0.stats
3 function calls in 0.051 CPU seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.050 0.050 0.050 0.050 {method 'join' of 'str' objects}
1 0.001 0.001 0.051 0.051 slow.py:3(slow)
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
The argument to print_stats controls how many lines are printed, and there are alternate sorting keys (make sure you specify one, otherwise it autoselects “random”): See http://docs.python.org/lib/profile-stats.html
Update: Found a more complete (including coverage) decorator by Marius Gedminas whom I met briefly at Europython 2007, http://mg.pov.lt/blog/profiling.html.
I’m amused by the names of wireless networks that people come up with. My first example is at my old apartment in Denton, and everyone was getting FIOS routers with random-looking 5-character SSIDs. “Ninjas > Pirates” I’m not sure if I agree with, but I’m glad that they’re at least opinionated enough to broadcast that one. Mine is minihub, on a WRT54GC so it’s small.
This one is at my new office, where there’s a Rails shop across the hall which actually has a scrum-style burndown chart on the wall. I’m assuming they also do pair programming, but don’t let people working alone on that network. (Given the existence of “pair”, “Pairs”, and “solo”, even the pair programmers must be separated somehow, maybe if they’re the top pair they get their own network?) nutricat’em (no apostrophe, just because), and gumnut are ours, and we don’t know who has the Apple one.
We’re un-evacuated now, so I am home finally, safe and sound.
A bit like the first chapter of the Hitchhiker’s Guide to the Galaxy, some strange events are occurring to me and I don’t realize as early as I should that something is strange.
I have done the following almost every day in July. Here’s how it happened tonight:
I am proud to announce an experimental piece of software I’ve written to help recovering broken rars. Say you've got a rar set for your favorite Linux iso, but some random part is bad and you have no recovery volumes. In this situation, you also have a torrent available, but don’t feel like downloading the whole thing. This tool will take your (broken) rars and reconstruct the full file for you, so the missing blocks can be recovered using the torrent.
Check it out in the projects section.
Today is the first time I've honestly wondered, "what is the proper whitebalance for a sky that's raining ash?" The Gap Fire which has simply been making it uncomfortable to breathe, threatened my neighborhood last night so an evacuation order was issued while I was out setting up a UPS to keep the network gear alive at the office. I'm staying with my coworker Roberto and his wife Karen now (thanks, guys).
The sky at the office last night
Cathedral Oaks near La Patera, two days ago
I wasn't conscious of the area covered by smoke until I left it last night
— in the normal course of life I've been under it for three straight
days. In the evenings it blows south, straight to the ocean; during the day,
to the west. 
The fire is creeping around so instead of being just a mile west,
it's also a mile north (see map at the side, pulled from Google Earth with the Forest Service's fire kmz overlay).
I just went by the post office on Patterson and they have an area labeled “cycle parking only” with an actual place to lock your bike. Cool.
Among the things I’m getting around to unpacking is my desktop computer. I was planning a switch to the Athlon64 I have since the dual Athlon XP bit the dust just days before I left Denton (it was bridge modded and three years was longer than I expected to get out of it anyway). The Athlon64 no longer POSTs due to I believe a momentary short in the front panel USB ports of the case, which has fried the northbridge — it gets burning hot within seconds of applying power now, and never gives a post beep even with the motherboard out of the case).
Computer’s don’t travel well, it seems. Oh well, trip to Fry’s! They had a Q6600 combo on sale (and they’re real good about having boxed cpu deals now) so I made the trek last weekend. I don’t know why I still get CDs from Fry’s, since the last three I have all had broken jewel cases found after the fact... but I did. I think it’s the instant gratification as opposed to the value and relative quality of Amazon’s music selection.
The motherboard was an ECS G31T-M, which features some impressive onboard video (quite well supported on Linux, although no DVI connector) and good overall performance. FSB1333 is not supported, and the USB2 ports onboard use up a lot of cpu, but that’s easily worked around by a USB2 PCI card I had sitting around. Also the MS Wireless Optical Desktop 4000 has pretty good range (around 5-6 feet reliably) so I’m using it at home and taking to work so I have a consistent keyboard feel. It’s slightly ergonomic but not too goofy. (I hit H with the wrong hand so all the MS Natural keyboard slow my typing considerably unless I use them all the time.)
After I got all that working, I bicycled 5 miles and came back tired. Still not ready for long rides.
It’s long. Even flying, it’s something like 5 hours and a pretty boring flight. That should be some sort of warning. Luckily my family and I managed to avoid finding $5/gal gas everywhere except in the middle of the desert (it’s still well under $5 here in SB right now). I got around 33mpg so it was around as cheap as flying if you just consider fuel costs, just took a bit longer (3 long days). Dealing with plant inspections is something new to me, so I had to leave my Ikea plant back with my grandparents in Lubbock. Now that I’m here I bought another from the Ikea in Carson (I was in Long Beach visiting my great aunt’s family).
I now have a California driver’s license, which entailed a trip to the DMV which is quite interesting. I got to queue to get a form to fill out, then queue again to turn in the form to get a number, which when called would let me go to a window to figure out the next window to queue at, take a test, and queue again to get it graded. The test is quite a bit different from the one I took in Texas (which was more concerned with right-of-way in different road size situations), and had topics like curbing wheels on hills and safety zones for trolleys.
Last week was another World Wide Panorama event, and since I was still moving in, I wasn’t sure I’d participate. Curious for ideas, one of my housemates named Paige helped out and showed me how to get to Old San Marcos Pass which is a real windy road that heads up into the hills. The modern San Marcos Pass is highway 154 which has a speed limit of something like 55mph. Signage is a bit more sparse on OSMP and it’s not like you’re going to hit anything more than 25mph average anyway. The few signs warning of sharp turns were especially disconcerting after you’ve passed 10 really sharp turns with no sign and wonder exactly how much sharper this one must be to get a sign.
The photo to the side is from the location that I think I’ll submit, and is a view southwest from the location of the map above.
I read that there are something like 1000 fires burning at any given point in time in California which seems like a large number to an outsider. I saw an odd-shaped cloud on the way home from work today and realized that it is a fire burning on the side of the hills, visible from Goleta around two miles from home. I don’t know if any structures were involved or if it was just a forest fire, but nobody seemed much alarmed about it so I suppose it’s a regular occurrence. The location was around Wylie Canyon somewhere (which is about a mile rightward of the photo here).
Update: The Independent is reporting on the fire.
I’ve arrived! Things have finally slowed down enough that I can get a weblog post in edgewise. First things first, I’m now moved to the Pacific coast in California, and have a new job as well.
Santa Barbara is literally pretty cool. The temperatures are almost always perfect (in the dead of winter the high is in the 60’s), and moving from a wooded part of Texas it’s a big shock how cool it can get at night, even when the days are hot. I’m renting a room, in a house with a view of the channel islands (on a clear day, over some other houses and power lines).
Another shock was that there is no FIOS available here, so I actually have to plan ahead if I have large data files I need to move around. I got a bit spoiled in Denton since if needed to move a 4GB VMWare image, it could transfer faster than I could drive home to burn it. 3-5Mbit DSL and Cable seem to be the norm here, with miniscule upload speeds by comparison.
The company I’m working for is called Nutricate, and I found out about the job from the Python Job Board so I can tell you that it really does work. I started there in June and it’s been a lot of fun so far. I get to write Python, primarily for an embedded ARM system, but a good part of it is interacting with serial peripherals, having only a partial spec for how the communication is to occur. Our product is to add nutrition information to receipts whenever you order food, which is currently a box (but we’re “not a hardware company”).