Additional suggestions welcome.

What they offered, instead, was a maintenance nightmare. The number of genuinely unique suffix formats is actually a relatively sparse ten. Suffixes break down into these formats:

• LPSuffix — Zero arguments
• Encodes: APPLIED, CAST_START, CAST_SUCCESS, CREATE, DESTROYED, DIED, DURABILITY_DAMAGE, DURABILITY_DAMAGE_ALL, INSTAKILL, KILL, REMOVED, RESURRECT, SHIELD, SPLIT, and SUMMON.
• LPAmountSuffix — One unsigned integer argument.
• Encodes: EXTRA_ATTACKS
• LPAmountStringSuffix — Two arguments. Unsigned integer followed by a string.
• Encodes: ENERGIZE
• LPDamageSuffix — Nine arguments, encoding the values of the DAMAGE suffix specifically.
• LPDrainSuffix — Three arguments. An unsigned integer, then a string, then an unsigned integer.
• Encodes: DRAIN, LEECH
• LPHealSuffix — Four arguments, encoding the values of the HEAL suffix specifically.
• LPSpellSuffix — Three arguments, making up an LPSpell.
• Encodes: DISPEL_FAILED, INTERRUPT
• LPSpellStringSuffix — Four arguments. An LPSpell plus an extra string.
• Encodes: AURA_BROKEN_SPELL, DISPEL, STOLEN
• LPStringSuffix — One string argument.
• Encodes: AURA_APPLIED, AURA_BROKEN, AURA_REFRESH, AURA_REMOVED, CAST_FAILED, SHIELD_MISSED, MISSED
• LPStringAmountSuffix — Two arguments. String followed by an unsigned integer.
• Encodes: AURA_APPLIED_DOSE, AURA_REMOVED_DOSE

The end result of all of this is a much more manageable code base, which is permitting me the opportunity to add some additional features (like pervasive string caching) to all of the suffixes instead of just a choice few. Once that’s done, the goal is to develop a hashing algorithm to override NSObject’s hash method and let me use a backing hash table to cache the suffixes as well.

The end goal, both in the code and in the parser’s RAM, is to eliminate redundancy. This was a good first step!

For instance, actors repeatedly causing the replenishment buff on the raid generate an enormous amount of identical or near-identical events to occur in rapid succession. There should be no reason why I should have to individually store all of these, if I can cache the identical potions in a hash table and place pointers in my LPEntry objects instead.

Work is ongoing! Adventure and excitement await!

What a dry post may look like.

It’s been a while since I wrote about anything at all, during which time I’ve been spending a lot of time working on other things, like raiding in Firelands! During this time, I’ve also taken a while to rethink some of the decisions I made about the structure of my log parser, especially the data flow between the log file on disk and the combat log data structure in memory. Thus far, I have yet to step back and reinvent the wheel on my prefix and suffix handling, which still remains the bulk of the (lines of) code, so I’m happy to write about them here!

In any given night of combat log, I typically see approximately 33 entry types, meaning 33 unique pairings of PREFIX_SUFFIX. There are more legitimate versions that can appear, but are exceptionally rare in practice due to them being cases that people don’t often do, like enchanting weapons during raids. Still, with 33 combinations, it’s necessary to find a way of handling these data structures without writing special case code for each individual type. That’s where the prefix and suffix structures come in.

The base classes for these objects are the LPPrefix and LPSuffix classes. The interface for LPPrefix looks like this:

typedef enum {
	PREFIXUNKNOWN,
	SWING,
	RANGE,
	SPELL,
	SPELL_PERIODIC,
	SPELL_BUILDING,
	ENVIRONMENTAL,
	UNIT,
	ENCHANT,
	PARTY
} LPPrefixFlag;

@interface LPPrefix : NSObject {
	@private
    LPPrefixFlag flag;
}

@property (readonly) LPPrefixFlag flag;

-(id)init;
-(id)initWithArray:(NSArray *)args forLog:(LPCombatLog *)log;
-(id)initWithFlag:(LPPrefixFlag)f;
-(void)dealloc;
-(NSString *)description;

+(NSUInteger)numArgs;
+(NSString *)nameForFlag:(LPPrefixFlag)f;

// Interface to subclass properties.
-(LPSpell *)spell;
-(NSString *)type;
@end

LPSuffix is similar. There’s not much to this, in practice. A simple type descriptor and interfaces to allow subclasses to all use a common interface despite having varying types of data contained within. I’m not terribly happy with the presence of the “spell” and “type” methods, as they break the OO model slightly by requiring the superclass to be informed about interesting attributes of subclasses, but I made a decision in my code to store pointers only to the superclass, which makes display in an NSTableView with cocoa bindings awkward without this pass-through interface. I wish I knew a better way of doing this, but I’m a novice when it comes to the cocoa GUI library.

A concrete subclass of LPPrefix of note is the LPSpellPrefix class, one of the few prefixes to actually contain any interesting data other than a type.

@interface LPSpellPrefix : LPPrefix {
@private
    LPSpell *spell;
}

@property (readonly) LPSpell *spell;

Also of note is the LPSpell class.

@interface LPSpell : NSObject {
	@private
		NSString *name;
		uint64_t spellid;
		uint8_t school;
}

@property (readonly) NSString *name;
@property (readonly) uint64_t spellid;
@property (readonly) uint8_t school;

-(id)init;
-(id)initWithName: (NSString *)inName idString:(NSString *)inID schoolString:(NSString *)inSchool;
-(id)initWithName: (NSString *)inName spellid:(uint64_t)inID spellSchool:(uint8_t)inSchool;
-(void)dealloc;

+(id)keyForIDString:(NSString *)idstr;

-(NSString *)schoolDescriptor;
+(NSString *)descriptionForSchool:(uint8_t)school;

-(NSString *)description;
@end

The LPSpellPrefix class, then, encapsulates the “SPELL”, “SPELL_PERIODIC”, and “SPELL_BUILDING” prefixes shown at Wowpedia. It’s also constructed in a way that makes it store only unique copies of this data, using references into a backing NSDictionary of LPSpells to consolidate duplicates, but that’s a story for another day!

So that I’m not writing about suffixes without providing any information on what they look like, here’s the interface to LPDamageSuffix, used in events like SPELL_DAMAGE or SWING_DAMAGE. It’s one of the most common suffixes in the log.

@interface LPDamageSuffix : LPSuffix {
	@private
	NSUInteger damage, overkill;
	uint8_t school;
	NSUInteger resisted, blocked, absorbed;
	BOOL critical, glancing, crushing;
}

@property (readonly) NSUInteger damage;
@property (readonly) NSUInteger overkill;
@property (readonly) uint8_t school;
@property (readonly) NSUInteger resisted;
@property (readonly) NSUInteger blocked;
@property (readonly) NSUInteger absorbed;
@property (readonly) BOOL critical;
@property (readonly) BOOL glancing;
@property (readonly) BOOL crushing;

-(id)init;
-(id)initWithArray:(NSArray *)dArgs forLog:(LPCombatLog *)log;
-(void)dealloc;

-(NSString *)description;

+(NSUInteger)numArgs;
@end

In making the parser’s back end, one of the first tasks was to define and code up all of these classes, so that each combat log event could have a valid, unique combination of them to properly describe any event that happens. Of course, this has also led to some possibly weird combinations. For instance, my log parser could probably handle nonsensical combat log events like SPELL_BUILDING_AURA_BROKEN_SPELL or ENCHANT_DURABILITY_DAMAGE_ALL. The prefix and suffix handling is not, in any way, checked to make sure that it would appear in the game. If the combat log provides a valid string of content, then it’ll get parsed just like a proper entry.

The construction of the entry factory is going to be my next topic in this series, if I don’t do another post to describe the LPCombatLog implementation. The entry factory largely uses a series of array slices and some runtime reflection (and the numArgs method on every prefix/suffix) to construct combat log entries without knowing anything about the specifics of the individual prefix and suffix types.

So, that said, I wrote everything after this with the presumption that the rumor is true. Consider it a “What If?”

A Return to Outland

Taking us back to Outland is a smart move on Blizzard’s part, not because of where we need to go to fight the future, but because of where we have to go to fight the past. The Outland questing experience is, at this point, terrible. Boring quests, flat stories, and a lot of outstanding lore tied up in level 69-70 endgame content in Netherstorm and Shadowmoon Valley sum up to a frustrating and poorly executed gaming experience in the current day. By placing the next expansion in Outland, it would provide the 60-68 leveling zones with the needed economic and administrative backing to get the same sort of overhaul that the old world did for the release of Cataclysm.

Beyond that, WoW’s increasingly powerful engine and graphical technology could do wonders with the very beautiful Outland vistas. Zones like Nagrand and Netherstorm have some incredible potential that hasn’t necessarily been realized with the 2006-era technology used in The Burning Crusade.

Tying Up Lore Loose Ends

A lot of lore threads were left dangling in Burning Crusade that are crying out to be wrapped up. The fate of the demon hunters obviously factors into the leak, and was well-prepared for with the Illidan quests in Felwood that were introduced in Cataclysm. I’m particularly looking forward to a revivification of the neglected Draenei and Naaru storylines, especially in regard to the ruined Draenei settlements across outland in Shadowmoon Valley and Terokkar Forest.

While I’m most definitely an Alliance player, seeing the rumored expansion of the shadow priest lore would do wonders in giving some depth to the Forsaken, who are currently suffering from a bit of “Zombie Nazi Menace” syndrome. Presuming that at some point Lady Sylvanas will lose her plot armor and see some sort of comeuppance, it may even be possible for a delegation of the Cult of the Forgotten Shadow to take power in the Forsaken society, significantly changing the dynamic of both that race and the Horde in general.

Likewise, a return to Outland would likely pick open some scabs for the Alliance in the form of the unseen remnants of the original Outland expedition (*cough* Alleria *cough*) and the Sin’dorei in the form of Kael’thas’s remnants, still potentially pervading the Netherstorm and the battered, broken manaforges. That’s interesting plot fuel, and it’s being wasted currently.

Evil A’dal? I guess the light DOES forsake its own, eh Bridenbrad?

I don’t have a whole lot to say about A’dal being an antagonist except that hearing dozens of conspiracy theories about it all back in the days of Burning Crusade got old then, and it will still be old now. I suppose this answers some of the questions about why we were tasked with saving Millhouse Manastorm when he was, in reality, a Twilight Cultist. Also, a redemption of Illidan? Fun glowy blindfolds for everyone!

Ethereals and Nagas

To me, having the Protectorate join the Alliance feels very forced in much the same way that the Draenei were shoehorned into the Alliance narrative when, in reality, they would probably be too peaceful and wholesome to not be a neutral faction in another game. The Protectorate should have no beef with the Horde, which seriously makes me question what the motivation could be for attempting to generate enough of a rivalry between the two groups to push the Protectorate into the Alliance.

The Coralbreach naga joining the Horde seems equally unlikely, mostly because the Horde has evolved as a relatively xenophobic and distrustful entity. The orcs, and Garrosh in particular, have grown wary of outsiders polluting their way of life. The Forsken, Sin’dorei, and Darkspear Tribe have grown reclusive and withdrawn, turning inward to marshal their forces for the specific betterment of their own people. Meanwhile, the Tauren have developed into a strongly tribal society that practically looks ready to me to split off from the Horde and return to a nomadic lifestyle with fewer enemies to worry about. The Horde don’t seem to really be doing each other many favors, especially in terms of mutual support and defense. It’s hard to see how there’s going to be room in that milieu for members of a race that have attempted to flat-out destroy substantial regions of two different worlds.

Racial and and political ties are highly correlated in the WoW universe. There are no Alliance tauren or trolls, no Horde humans or night elves. All dwarves are now Alliance, and all orcs identify as Horde (welcome or not in the New Horde). How would that work with naga in the Horde? Would they take the Bilgewater Cartel approach and try to pretend like a very small niche population of a specific race is large enough to make a fully contributing member of the Horde? Maybe, but I think that’s wearing a little thin in the narrative, with no attempt to even hand wave the massive population difference that should exist in game terms between the sizes of the Alliance and Horde.

Game Mechanics Are Fun

Five new levels, refurbished zones, and no hint of new serious mechanics like stat changes or the like. To me, that says that we’re gearing up for a straight forward content expansion that won’t alter the status quo too much. I am concerned, however, that Cataclysm has shown us that a shallow endgame can be incredibly frustrating for certain players who end up feeling starved for interesting and diverse instance content. All of Cataclysm’s endgame content thus far — except for Bastion of Twilight — has been in the form of nostalgic re-imaginings of old settings (Blackwing Descent and Firelands) or small instances (Throne of the Four Winds). Blizzard hasn’t even been able to keep up with the development of what new, interesting content we were promised in the form of the Abyssal Maw, and we haven’t heard whisperings lately on the development of the anticipated War of the Ancients raid. What do, Blizz? Can you take another five-level expansion and give it enough legs in the endgame to keep us interested? I don’t want another seven month wait for seven bosses.

Too much of this! Time to play some WoW!

Has it been an entire raiding tier already? Tomorrow, we get to venture into the elemental plane of fire to face Ragnaros! Again! Personally, I’m more interested in his lieutenant, Majordomo Frandral Staghelm, who has the dubious distinction of being one of the few genuinely hated lore figures, despite having been a closeted villain for more of his career, and who drops an awesome fire cat staff! Yeah!

Other things I’m looking forward to? The extension of the Hyjal lore is very appealing to me as a night elf and druid lore wonk, and there are several new pets that I’ll be picking up from the dailies there. I’m also hoping to find time this tier to start doing some rated battlegrounds with my guild, which never seemed very appealing to much of us during tier 11 — We never had a proper guild group, and those who wanted to do RBGs were pugging with other, more pvp-oriented guilds on Zul’jin.

I’d also like to actually finish an entire tier of content on every difficulty level for the first time since tier six. In the past, I’ve always had one or two bosses hanging loose at the end of a tier. I never finished sunwell at level, nor did I get that kill of Sarthartion-3D on ten (we did on 25)… we didn’t get Firefighter or Algalon 25, and then when I joined BRM, we never got our Tribute to Insanity 25, though we did score a 49-attempt Mad Skill, I think. Never did Light of Dawn at level, either, though not for lack of trying. This tier, we went 9/13 including a heroic Al’Akir 10 kill. Years of almost there, but never quite scoring that final check mark on any tiers since Burning Crusade.

The game has changed since then, though, and I’m looking forward to getting my ducks in a row and joining this tier with a clear head and fresh motivation. After spending 9, 10, and sometimes 14 hour days learning about Cocoa and programming my brains out for some time, setting up a blog and attempting to learn how to use GIMP and Inkwell to pretend to make graphics, and only logging in to WoW to do fire festival dailies and truegold cooldowns, I’m ready to play some more.

It’s time to marshal my cadre of alts, farm a ton of valor points to make my craftables on my druid, and knock down bosses like dominoes.

This took me longer to make than the post. So worth it.

I had intended to continue my series on writing a WoW log parser this weekend, but a fairly substantial refactoring of the code has thrown a wrench into that. Thankfully, I decided to dip my toes into Git support in XCode to branch this refactoring, so now I have something new to write about!

I’m relatively new to the modern world of version controlling software. Years ago, I used CVS and dipped my toes into the then-young Subversion when writing some code in college, but for my own personal projects since then, I’ve not really bothered. I certainly never tied it into a modern IDE like XCode, having done most of my previous coding with vi, SubEthaEdit, and TextWrangler/BBEdit. On my kick recently of, “I’m going to stop programming like I’m in college!”, I spent a few hours this weekend trying to get XCode 4 to work properly with a Git repository that I created for my parser project. It didn’t go well, unfortunately.

XCode 4′s Git support seems simple enough. You have an option of creating a new project with a Git repository thrown in already, which is kind of neat, but my project is old and I didn’t want to try recreating it, so I had to manually create a repository from the command line and hook my project into it. That went fine, following Apple’s guide to doing exactly that. What I wasn’t prepared for is that Apple’s instructions leave you with a very awkwardly set up Git repository, because they don’t teach you how to exclude files that are constantly changing, like the pervasive Mac OS .DS_Store metadata files, or XCode’s own project metadata files that store, seemingly, information like “Timestamp of last user inhale.” that is constantly in flux. This causes situations like what I experienced today. Commit my project, look at window and blink thinking, “Not bad!”… two seconds later, the XCode metadata is dirty again and wants to be recommitted. Repeat ad nauseum.

The answer to this is to set up a Git ignore file, which would have been fine, except you also have to find the files that have been ignored and manually remove them from git. Fine, that’s more work than this was worth, but I learned how.

Then I went to go push my local working copy of my project onto my AFP-mounted NAS volume (Named The TARDIS because it’s bigger on the inside) and I found out that XCode hates that, too, throwing me errors saying that the repository cannot be found while I’m staring at it in Finder.

Long story short: screw that noise! I’ll use snapshots until Apple decides to provide a more effective UI for their tool or a set of documentation that doesn’t constantly leave me scrambling for blog posts and stack overflow pages and radar listings to teach me the basic concepts via Google.

1. Build an array of arguments that consumes the first nine strings in the input array (the timestamp is two strings as described in my previous post). This produces an array with four objects contained within: The the timestamp NSDate, type NSString, and two LPActor pointers that encapsulate three strings each for name, id, and flags. I also maintain a dictionary of actors that is searched here, so that each LPActor is only stored once.

What the baseArgs data structure looks like, transformed from the input.

1. Parse the “type” NSString to determine the following values:
   1. pFlag = Prefix flag
   2. pArgC = Prefix argument count
   3. sFlag = Suffix flag
   4. sArgC = Suffix argument count
2. Slice the input array starting at index 8 with length pArgC, and hand that slice to a method that builds an encapsulated LPPrefix subclass.
3. Slice the input array starting at index (8+pArgC) with length sArgC and hand that slice to a method that builds an encapsulated LPSuffix subclass.

The construction of the (roughly fifty) LPPrefix and LPSuffix classes created for this program will be the subject of my next programming post, but for right now we’ll just use a conceptual forward declaration and say that a combination of runtime reflection and dynamic method dispatch lets that work pretty easily.

The resulting LPEntry object finally contains only the things constructed here. The base arguments, prefix, and suffix. In this structure, we can semantically encapsulate all of the possible events that can occur in the WoW combat log.

Three asides as I wrap this up. First a note on versioning: This post is for WoW version 4.1 and prior. As I understand it, next week in 4.2 we’re getting two more elements in the base arguments to represent the raid icons on the source and destination actors.

Secondly, the Wowpedia page I linked earlier for reference is sometimes wrong on argument counts. I can’t check at the moment to find which one because the whole curse network is down and Wowpedia along with it.

Finally, whoever at blizzard decided that some numbers (i.e. spell school flags) in some contexts should be in hex code and in others should be in decimal needs to check out a book on “Things someone parsing this should not have to pay attention to.”

I was thinking about the choices we have for gear in firelands this morning, and as Borsk so commonly says, the answer was a spreadsheet. I spent some time doing my prep for loot council this tier by reading Hamlet’s awesome post over at Elitist Jerks. I made a document of every piece of gear I could find for this tier and went about picking obvious winners on the basis of, “More intellect is better, spirit is nice, meet haste goal, prefer mastery.”

The linked spreadsheet is the result for the normal-mode gear. Several choices will have to be rethought moving into heroics, though. The gloves, cloak, ring #2, and boots selected do not have heroic mode equivalents unless they become upgradable via the sunmote-2.0 system. I didn’t check that out, yet.

Blizzard seems to have learned their lesson about bracers this tier, as there are several wrist choices to pick from. Keeping it real, they’ve apparently stuck us in belt hell by way of recompense, as the only leather intellect belt that I could find in the 4.2 lists was a rep reward with crit/haste on it. I suppose hoping for a spirit/mastery, spirit/haste, or master/haste item was too optimistic! Also, we’ll be wearing a mismatched color scheme for the whole tier because there’s no heroic-colored belt. Fix it, please, Blizzard!

Sometimes I envy warriors and their ability to get a heroic-level item in every slot. I don’t think offhand that any tier of content has offered that for a druid, mostly owing to relics. This tier, the relics are supposedly upgradable, but we’re getting shafted on a belt!

The one on the left is 210 MB and 1.35 million lines.

The WoW combat log, specifically its on-disk incarnation, is a hairy subject to address in my second blog post, I’m sure. I can’t imagine it has very mass-market appeal as a topic, but it’s been a major focus of my programming efforts over the last month or so, and what else am I going to write about, anyway? New spider pets for hunters in 4.2? Puh-leez. Fair warning, if you’re educated in the ways of computer science or basic string programming, this is going to look like child’s play. I don’t mean to over-elaborate on a simple subject, but it doesn’t matter because who reads this stuff anyway? Sheesh!

So, the combat log is, in theory, a pretty simple thing. It’s plain, UTF-8 encoded text that represents one action per line. Those actions are serialized as a (mostly) comma-delimited list of string tokens. The meaning behind each token depends on its position and on the values of a special type-token near the beginning of the line.

Having the meanings of each token dependent on value-decoding earlier tokens belies the modern day combat log’s roots as having a pseudo-English sentence structure. It also means that it requires a relatively intense tokenizing and parsing effort compared to more modern file formats, like XML, that are guaranteed to be machine-readable without writing lots of task-specific code.

But I digress. The important thing is that a line that used to look like a pretty unassuming English sentence now looks like this:

5/19 00:00:00.960  SPELL_PERIODIC_HEAL,0x02000000011E5E3D,"Eranthe",0x511,0x0200000001EBB45B,"Tlazolteotl",0x512,64801,"Rejuvenation",0x8,3541,0,0,nil

That’s me (Eranthe) healing Tlazolteotl with the pre-hot tick of Rejuvenation from Gift of the Earthmother for 3541 healing.

There are a few features that stand out. One is that the string isn’t actually comma-delimited like it pretends to be, because the timestamp at the beginning is space-delimited instead. Commas can also appear inside the quoted strings for names at several points. Another is that some numbers are in hex (designated by an 0x-prefix) and some others are in decimal, but that’s a topic for a very sarcastic paragraph in the upcoming post on the semantic parsing of this. Still, it’s easy to tokenize once you start doing it the way we learned in school and using a bunch of c-style switch statements while iterating over each character in the string. Out comes an ordered NSArray of NSStrings, each containing an un-delimited token.

If you’d like to tokenize a combat log line in Objective-C, well am I the girl for you! This method will take an NSString and give you back its tokens as an NSArray, as described above. It operates in O(n) for the string length. The first two tokens in the array encode the date and time for the timestamp separately. I glue them together later on when converting them into an NSDate. This will break if some wisecracking raider finds a way to include a double-quote character in his or her name.

+(NSArray *)tokenizeString: (NSString *)input
{
	NSRange tokRange = NSMakeRange(0,0);
	NSUInteger length = [input length];
	BOOL escaped = NO;
	NSMutableArray *chunks = [[NSMutableArray alloc] initWithCapacity: 12];

	while ((tokRange.location + tokRange.length) < length) {
		unichar here = [input characterAtIndex:(tokRange.location + tokRange.length)];
		if ((here == ' ' || here == ',') && !escaped) {
			// We found the end of a token.
			NSString *token = [input substringWithRange: tokRange];
			if (here == ',' || (here == ' ' && !([token isEqualToString: @""]))) {
				[chunks addObject: token];
			}

			// We broke off this token, so move the pointer forward.
			tokRange.location = tokRange.location + tokRange.length + 1;
			tokRange.length = 0;
		} else if (here == '"' && !escaped) {
			// Found the beginning of a quoted string.
			escaped = true;
			// Also we don't want to capture the beginning quote.
			tokRange.location = tokRange.location + tokRange.length + 1;
			tokRange.length = 0;
		} else if (here == '"') {
			// Found the end of a quoted string.
			escaped = false;

			[chunks addObject: [input substringWithRange: tokRange]];

			// Broke off this token, move the pointer forward.
			tokRange.location = tokRange.location + tokRange.length + 2;
			tokRange.length = 0;
		} else {
		    ++tokRange.length;
		}
	}

	// After the last delimiter, we need to chop off the last token.
	NSString *lastChunk = [input substringFromIndex: tokRange.location];
	NSString *trimmed = [lastChunk stringByTrimmingCharactersInSet: [NSCharacterSet whitespaceAndNewlineCharacterSet]];
	if (![trimmed isEqualToString: @""])
		[chunks addObject: trimmed];

	return chunks;
}

From here, the question becomes, “What on Earth can I do with all of these tokens that I just made?” To answer that, it’s necessary to look into the semantic format of the line, and how you determine what each token means. Next time!

Before I forget, I should mention that there is a step 0 to building a log parser, which is efficient reading of a 200+ megabyte text file. I haven’t gotten to that part yet in my code, and instead read the whole file into memory and then deal with it from there. It’s inefficient and bad, but it’s one line of code compared to many!