Whizzard

As I've mentioned a few times, I spend a reasonable amount of my free time (and sometimes bits of my not free time ("compiling!")) working on Sutekh, our card manager for VtES.

As I've also mentioned, I created a fork of Sutekh called Thufir, which does the same thing for the defunct Dune CCG. While I got Thufir to a point I was happy with, I never really kep it up to date with Sutekh's developments, so the two code bases diverged rather rapidly.

Now we add the recent Android: Netrunner LCG into the mix. While there are a varierty of web-based card managers for A:NR, I find them clunky, and there's a dearth of other options. Maintaining a third fork of Sutekh also looked really unappealing, so I hatched a cunning plan to refactor Sutekh considerably, pulling a lot of non-VtES specific code into a nice, reasonably reusable basis for writing card management applications. I also reworked Thufir to use this new framework, fixing a lot of bugs and gaining a whole lot of new, useful features there. Of course, this all took time, and, as a result, I didn't get around to actually working on my A:NR app (called Whizzard).

Over the weekend, I decided to knuckle down and actually start working on this. Although there is still missing data and a bunch of features to hook up, it's moved past the vapourware stage really quickly, which gives me a warm fuzzy feeling about the refactoring work done on Sutekh.

Sutekh's base infrastructure is still not quite as self-contained as I'd like, which would avoid needing to duplicate code in the various repos, but I still haven't come up which a way of plugging the last couple of leaks I'm happy with, so I'll live with the current state of affairs until inspiration strikes.

Gtk+ lessons

I learnt of a couple of interesting gtk+ quirks over the last couple of days while hacking on Sutekh. Both rather surprised me, so are thus worth noting for future reference.

The first quirk is something I still don't fully understand. For various reasons[1] Sutekh uses strings to display the numbers with markup in the interface. I recently added a tweak so that these would sort numerically, rather than lexically. Since this involves stripping off the markup and converting to an integer, the sort is not lightning fast, but still acceptable. Unfortunately, adding the tweak the obvious way doubled the length of time to run the tests on the model, despite the model being unsorted by default. I've been unable to track down why this affects the tests so badly, since we shouldn't be using that code path at all. Fortunately, a simple lazy work-around exists, which is to tweak the sorting in a slightly less obvious way, and avoid it impacting the test suit.

The second quirk is something I do now understand, although the behaviour did surprise me. Sutekh is capable of generating very large gtk+ models (using "show all cards, show expansions + card sets" and editable options, it's easy to end up with a gtk.TreeStore containing more than 100000 entries). Loading these was unacceptably slow, with most of the time spent adding rows to the model, despite using the usual recommended tricks of disabling notifications and such while adding the rows.

It turns out that the underlying data structure for the gtk.TreeStore is glib's n-ary tree, g_node.
g_node, however, represents each list of children as a singly linked list from the parent. Consequently, appends into long lists are slow, since you have to walk to the end of the list. There's the obvious work-around, which is to insert thing in reverse using prepend, and it is indeed significantly faster, but you'd think this would be mentioned prominently in the docs, rather than requiring one to grub around in the glib code.

[1] i.e. We haven't got around to fixing this the right way yet.

What I've done on my Xmas Holiday

As has been mentioned several times, I tend to spend a lot of time hacking on Sutekh, the VtES card manager.

I am, however, also somewhat addicted to the (now defunct) Dune CCG, so some time ago, I vaguely started working on a fork of Sutekh to deal with Dune's cards.

And, with minimal cheating [1], I now present the first evidence of Thufir's evolution to non-vapourware status [2]:




[1] It doesn't parse unmodified html files yet - I'm leaning towards just distributing the tweaked files as a better choice fixing the parser to handle the rather horrible html.

[2] git repo

Sutekh 0.6.0 released

After far too long, we finally got the current trunk to a state where we were happy to release. Unlike the previous stable releases, we decided that this one was worth a bigger splash, and announced it on the VTES newsgroup.

As such releases go, that seems to have been a success, and I've spent the last couple of days obsessively watching the sourceforge downloads counter tick up.

The increasing number of downloads, however, has not been accompanying by any comparable flood of queries or requests. As a measure of validation that people are using out work, this leaves a fair bit to be desired. Typical, really. Users are never around when you want them, and often there to complain when you don't.

Development style

A couple of weeks ago, I decided to make a concerted effort to fill in some of Sutekh's documentation, since we are planning to release fairly soon.

As a result, Sutekh now displays icons in the main tree view, and has a plugin for importing card sets from zip files.

I'm prone to this development pattern, and, given the amount of feature creep you see in modern software, I suspect that it's actually quite common.

(There has also been progress on the documentation, though, and the icons feature is really nifty)

Thoughts on test driven development

At the start of May. Simon and I redid the database layout for Sutekh, and, since then, have been working on fixing all the fallout of that. I recently spent a bunch of time working on finishing off some parts of the transition. Since we had a good idea of what the code should accomplish, I ended up doing most of this in a very test-driven fashion. Since most of my work is much more experimental in flavour, and doesn't thus doesn't lend itself to strongly test-driven development, it was an interesting exercise.

Some observations

1. Having the test available is nice, especially given the number of interactions involved
   
2. It tended to get sucked into "let just get this test working", and, as a result, the first version that successfully passed all the tests is a poster case for the difference between "working code" and "decent code". I'm still busy refactoring the code to amalgamate common bits, and there's still considerable work before the code is maintainable.
3. I frequently erred towards being overly concerned with not breaking the tests. I ended up duplicating code unnecessarily in places as a result.
   
4. Refactoring is made easier by the test, though.
5. Largely as a result of 2, I don't actually have that good a mental map of the code, which feels rather weird. Spotting repetitions between the different code paths is thus quite time consuming.
6. I do have quite a lot of confidence in the tests.

A fair bit of this list boils down differences in the manner in which code is produced. In my usual development model, by the time I'm satisfied that the code is doing what it should, I've spent a lot of time worrying about specific details and all sorts of implementation issues, and consequently, the code is vastly more polished. Here, because I was usually focused on short term gains, I have something which does what I want, but still needs considerable work before I'll regard it as complete. Also, as the list illustrates, I made the error of treating this as a bug-fixing exercise, rather than as development, so that's something I'll have to bear in mind in future. Point 5, though, is the real problem, though, and I'm not quite sure how to fix that, although things should hopefully be a lot better by the time I finish refactoring.

Still, despite my complaints about the resulting code, it did make the development a lot easier, and, for a part-time project, being able to deal with it in the smallish chunks of "let's get this test case working" did make it much easier to maintain momentum via visible results.

Saturday's Vampire tournament

I decided to give the Malkavian antitribu deck another run in a tournament - I tweaked the balance somewhat (I reluctantly removed Aristotle, since he tends to generate excessive table hate, and I replaced the confusions with more generic stealth cards). I managed a single VP with the deck, but, thanks to the scary efficiency of Val's anarch reckless agitation deck (1 table sweep, 1 near sweep), that was enough to qualify for Wednesday's final.

There were only 9 players, so the split was into two tables, one of 4, one of 5.

So, the rounds:
Round 1: I was bleeding Robert's guhuri farm deck bleeding Val's reckless agitation deck, bleeding Phillip's Aabbt deck, bleeding Garrick's Brujah equipment deck.

Robert started. I had a decent crypt draw, and was able to put decent pressure on Robert, but didn't draw enough bleed cards (I drew 1 kindred spirits in the game, and only 2 eye's of chaos's), so I was never quite able to finish him off. I erred badly in not stopping Val's Firebrand when I could have, which made it easier for Val to get the vote lock, and sweep the table. Phillip's deck got going, but wasn't fast, which left Val easily able to finish Phillip and Garrick in quick succession, before going on to sweep the table. Garrick's deck didn't really get going - he ended up spending a lot of time setting up with equipment, and a couple of bleeds by Phillip put him very much on the defensive, but he struggled to generate enough intercept. Robert didn't really get the full arm going, but a couple of well timed minion taps, and some other pool jiggling meant he survived my attacks - that he never blocked any of my actions also meant I was jamming on stealth cards a bit. Annoyingly, when I was ousted, the two of the next three cards in my deck were a Kindred Spirits and a Eye's of Chaos - given that I had stealth in my hand, and Robert was on only 4 pool at the time, had these cards come up a little earlier, I might well have been able to oust him.

The second table was Mike, playing an Imbued event deck , bleeding Verolin, playing and Akunanse deck, bleeding Dan, playing his Seraph multi-act Assamite deck, bleeding Marc, playing a Tzimisce bleed deck (no idea who started). The table timed out. Mike was on quite low pool, but Marc was never able to generate enough bleed to defeat the Imbued's farming. I'm not sure why Mike was unable to oust Verolin, since he was down to a single minion by the end of the game, and didn't seem to have much farm going, but the table indeed timed out.

Round 2:
Table 1: I was bleeding Phillip's deck, bleeding Mike, bleeding Robert. I started.

I didn't have an ideal crypt draw (two Dolphin Blacks, the General and Hannibal), and I drew a lot of Master's early on, so it took me a bit of time to get myself setup. Fortunately for me, both Phillip and Robert were bringing out 11 blood vampires (Sutekh and Eze respectively), so my slow start didn't really hurt me. Mike, on the other hand, drew an information highway in his opening hand, and, combined with the other acceleration tricks he had, had 4 minions out by the time Phillip brought out Sutekh. With Eze and Sutekh on the table, and Vox Senis down early, I didn't have the voter lock, but I was able to successfully whittle Phillip out - it was a close run thing though, as he had played the Kahabr towers the turn before I ousted him, and would almost certainly had burnt Sutekh for the 11 pool on his next turn. Thereafter, with the Unmasking down, I run into a drought of stealth cards, so wasn't able to put pressure on Mike, and with both Mike + Robert attacking me, I ended up losing minions fairly rapidly. Mike eventually ousted Robert, since timeout was looming, and then easily ousted me for the game win.

Since this game went close to timeout, I didn't see what happened on the second table - I know Verolin got 1 VP, which was apparently engineered by Val, and Val cleaned up the table thereafter.

So, reflections:

• I probably need to add a couple of Barren's to the deck - when people aren't blocking, it's prone to jamming on stealth
• The changed balance is better - despite the occasionally weird draws, I had fewer completely useless cards at any stage, and was generally able to go forward fairly well.
• I need more Mental Maze's, since the Malk's remain crunchy in combat.
  
• I still can't believe I only drew 1 Kindred Spirits in the first game - Sutekh's card draw probability calculator tells me that the odds of that are < class="blsp-spelling-corrected" id="SPELLING_ERROR_33">probability plugin, I'm naturally completely confident in the accuracy of the figure).
• Outer Limits had shiny Twilight Rebellion boosters. I had little self control, and now have a number of shiny Twilight Rebellion boosters (next on the agenda: somehow secure some Fee Stakes, so I can actually use the nice shiny tech)
  

We'll see how the final goes on Wednesday

Sutekh ramblings

(Sutekh is, of course, the python Vampire card management app Simon Cross and I have been working on)

Conceptually, the app is fairly simple, and I've been largely using it as a learning bed for pygtk. Consequently, it does things by hand, rather than the often recommended "use glade" approach. Personally, for something that now as a number of dynamic aspects to the gui (ability to rearrange panes, and so on), I think glade would have become quite painful, and straight pygtk coding is fairly easy once you wrap your head around the whole container paradigm. As an vehicle for learning pygtk, it has been pretty good.

But what has surprised me is it's ability to uncover database oddities. We have to date, discovered two issues with sqlobject (both fixed), had to work around sqlite's query optimiser by throwing in extra joins, triggered a segfault in sqlite (fixed in more recent sqlite's fortunately) and hit a bug in mysql query optimiser that makes certain queries run extremely slowly. WE ahven't triggered any postgresql bugs yet (although there have been a few weirdness's with certain versions of sqlobject talking to postgresql), but have had to fix several bugs in Sutekh's code because of postgres's much more stringent SQL implementation. For not a teribly complex database app, it's quite a impressive list.

So, do other people not trigger these bugs? Are we stretching corners of the databases that many other apps just don't hit (Sutekh has a number of tables, and can involve some quite complex joins)? Do other people just stick to one db, and work around it's quirks?

I dunno, but on the other hand, it's quite pleasing to see how some of this bugs have disappeared during Sutekh's lifetime, which is always one of the strengths of FOSS.

Weekend 29 June 2007

Having finally got my car back, the special seminar series was able to resume on Saturday. Kevin, alas, could not make it, and thus fell even further behind in Cowboy Bepop (which remains very cool).

After reaching our Anime quota, Simon and I finally merged the database rework branch into Sutekh. This went surprisingly painlessly, although I spent part of Sunday morning stamping out various little awkward bugs in a less than ideally tested part of the code.

The reason why I was working on Sutekh was that we'd agreed to play vampire on Sunday. I left for this slightly late (due to the bug fixing), but fortunately the N2 was comparatively sane - until the bit where they'd closed it off due to some or other race, anyway - so getting across wasn't too much of an issue.

The modifed close-range Osebo deck was able to successfully burn a vampire in combat each game. It arguably needs some combat rush, and definately needs some more ways of gaining blood. Soem more tastes (if I had any) and some blood dolls would probably do. More immortal grapples wouldn't hurt, so I an get around S:CE. The deck, like all potence decks, suffers from not having a way around Fortitude damage prevention, which, since both simon and Adrianna were playing decks using fortitude, made my life rather difficult.

Anyway, in the first game, Kevin's mixed crypt Presence S:CE and bleed deck, swept all before it. In the second, Simon's Tremere deck did quite well, although Adrianna's Salubri deck farmed extremely successfully and showed good defensive capabilities, although not really able to generate enough bleed to pressurize Simon.

Afterwards, I returned to fix some more bugs in Sutekh. By the end of Sunday, it was getting to be back in reasonable shape.