Not paying OnTheHub to redownload Windows ISOs

If you’re looking to re-download Microsoft Windows 10 Education edition ISOs and have a valid key from the university licensing program already, grab the education edition ISO images directly from Microsoft instead of paying OnTheHub/Kivuto protection money.

Context

UC Berkeley, like many other universities, has a volume licensing deal with Microsoft for its operating system products. In particular, it offers free downloads of Windows 10 Education edition to all current students.

The irritating thing about this is that they offer this deal through a shady vendor called OnTheHub or Kivuto. Instead of allowing the ability to re-download the ISO image, it holds you hostage if you want to access the image file again after 60 days.

When you first download, it offers you a discounted “extended access guarantee”:

To ensure that your download and/or key(s) remain accessible to you, you can extend your coverage to 24 months with the Extended Access Guarantee for $4.95. With this service, Kivuto will back up your download and/or key(s) on their servers, allowing you to access this information at any time under the “Your Account” section of the WebStore.

If you declined to take advantage of this oh-so-generous offer, and find yourself needing the ISO again after 60 days:

Access Guarantee Retrieval (60 days)
Purchase this service if you wish to recover your download(s) and/or key(s) after access has expired.
You will gain another 60 days of access to any expired product in your order.Learn more
$11.95

In my case, I backed up the wrong ISO — the generic install ISO, instead of the education edition ISO. The product key Kivuto issues you is for the Education edition, so the generic ISO you can get online won’t let you install with the key. It’ll report an error of “The product key entered does not match any of the Windows images” at install time.

So when it came time to reinstall, suddenly I’m faced with the OnTheHub protection racket.

Solution

For Windows 10, at least, you don’t have to pay the protection money. Education edition ISOs are available directly from Microsoft after product key verification. The download page is rather well-hidden, under the “more options” link from the software download page.

Unsurprisingly, OnTheHub makes no mention of this download source.

I wonder how much money OnTheHub makes off people who didn’t realize there was a free, official source for Windows 10 ISO images. Bandwidth costs money, sure, but you are a *education software download vendor* in 2015, with a target market of underpaid students and faculty. Nickel-and-diming poor student users for software downloads, in this era of cloud-driven computing, is an obsolete and despicable business practice.

A wishlist for a native Mac email client

As the tech-savvy sort, I often do some things that the typical consumer might not care about. However, it is strange that I simply cannot find a native Mac email client on the market that fulfills what I consider to be very basic features for decent email management. While everyone is gushing over the latest social network to be jammed into an email client, I just want my email to work in a sane way.

The features I’m looking for:

  • POP3/IMAP + SSL support
  • Full-text search, or indexable by OS X’s Spotlight
  • Multiple account support
  • Archival — if I delete an email account, it should not wipe out all emails from local storage that belonged to this account, especially if said account was POP3
  • Plaintext composition — I’m taking the side of format=flowed in plaintext composition. Manually formatting hard line breaks at char 78 is an insane holdover from a bygone age when dumb ASCII terminals were still the primary user interface. At least allow this as an option, or support the quoted-printable content type and soft line-breaks during composition.
  • Filtering rules — Basic filters that let me decide where to put messages based on mail headers and subject is enough
  • mbox import/export — I need to be able to import mail from my previous client. Similarly, in the scenario that this app is no longer supported, I need the option of moving to another client.
  • Bonus: auto-bcc to an arbitrary email address — I’ve never solved my problem with archiving sent mail. All you really need to do is auto-populate the BCC field on any “New Mail” composition window with this address. That’s all I really need to be happy here.

Amazingly enough, there is not a single Mac mail client that fulfills all of these basic conditions. Especially egregious problems for the top three mail clients:

  • Microsoft Outlook Mac 2011 — mangles all outgoing plaintext emails by inserting hard line breaks. No support for flowed plaintext, despite promises to the contrary, and no support for quoted-printable content-type and soft line breaks. No provision for auto-bcc.
  • Apple Mail (Snow Leopard) — deleting an old email account deletes ALL mail belonging to this account, even for downloaded POP3 mail. WTF. Can only auto-bcc “myself”, which is a fixed email address corresponding to the sending account; if you allow autobcc, why not let the user pick the email to autobcc to?
  • Mozilla Thunderbird 8 — deleting an account deletes all downloaded mail belonging to this account, unless messages are stored to the “local folder” rather than its inbox abstractions.

For fear of accidental data loss, I’ve stuck to the Microsoft offering, despite its incredible inability to keep plaintext mail intact. The newer social clients are all Thunderbird-based and leave little else worth examining — they seem far more interested in integrating ever more social network APIs than to support more basic email features. The rest of my friends are all sworn Gmail webmail users and think I’m a strange luddite for even considering native clients. I’ll leave the webmail vs native client debate for fear of going on a page-long rant; suffice to say that the user experience between the two options is not comparable at all.

It almost makes me want to start writing a new email client for myself, or hire someone to start some skeleton code at least, because obviously no one else is going to scratch this itch of mine. That kind of spare time and money, however, would probably never come unless I manage to actually sell a company or two (rather than just founding unsuccessful ones).

A Brief Guide to Golems of Amgarrak with a New Character

So I’ve been playing Dragon Age: Origins in the few spare hours I get every week. Having never finished Baldur’s Gate 2, this is redemption for my neglected gamer soul, I think. In general, it’s pretty easy. However, several of the most annoying battles in all of DAO are from the Golems of Amgarrak DLC. So far, all of the FAQs and guides I’ve seen (including videos on YouTube) are about absurdly min-maxed level 35 characters imported from Origins and Awakening, with elite gear, hundreds of poultices and potions, and the most overpowered skillsets.

This note is not about trouncing the Harvester with level 35 ubergeared supermen. Any halfwit can do that — just go get the gear, follow any YouTube min-maxing guide for DAO, and start whacking away. (My personal favorite: 100% dodge build: nothing can land a hit on you with physical attacks. Eat that, Harvester. But I digress). You can do this DLC easily with any properly geared character. This note is about how to complete Golems of Amgarrak on Nightmare difficulty with a newly created, level 20 mage, with stock gear and sneaky tactics. This is not to claim it’s the best way to accomplish the feat — just the way I managed it. If you have better ideas about how to use a newly created level 20 character to win, contribute them in the comments.

This note only applies to DA:O as played on the PC. On some console-based versions, the pet bronto cannot be controlled, making the final battle even more annoying than usual.

Background

If you don’t know yet, completing Golems of Amgarrak on Nightmare (or Hard, actually) difficulty awards all of the other characters in other campaigns (even existing campaigns’ characters) some really nifty items — but only nifty for lower level characters. This includes the amulet The High Regard of House Dace, which gives +6 cunning so Daveth can open up all those annoying chests in Ostagar, and The Reaper’s Cudgel, which is worth well over 300 sovereigns. The early cash allows you to buy the Blood Dragon Armor, the Spellward, and the Arcane Knowledge and Physical skill tomes from Bodahn immediately after Lothering before his store restocks; you can then buy another copy of these items after the restock.

With my time constraints, I will most likely play DAO only once. So I’d like to have these items in my Origins campaign, without having to complete the game first to get an absurdly well geared character to do Amgarrak. I thought I can just start a new Amgarrak campaign with a new character, and finish it to get my gear. Boy, was I wrong. People call this the hardest DLC for Origins, and for good reason. I played a warrior and simply could not win an initial fight against 4 elite golems, and had to delete, restart the campaign, and rethink my strategy.

General Setup

Despite claims, the module is not impossible, even with a new level 20 character. It is simply very poorly balanced for new characters, and forces you to proceed carefully and methodically. This is Dragon Age, not Diablo — as a new character, you cannot just rush into every battle, aggro’ing all the mobs at once. Separate mobs from each other, killing them one at a time, withdraw backwards to avoid being surrounded, pause frequently to issue new orders as the situation requires and don’t rely on the Combat Tactics scripts, and you should have very little difficulty dealing with most creatures in the module.

To win, I strongly suggest playing a mage. In fact, the best character for completing the DLC with is probably the mage, regardless of level. The main reason: there are no merchants, not nearly enough healing poultices, and most unfortunate of all, no other playable mage in the entire module. The objective is to preserve all potions and consumables until the final battle with the Harvester, which means the party will need to rely on mage healing spells.

The party at the start consists of a duel-wielding rogue, a bronto (with a charge attack and a Taunt), and the PC. The party at the end adds a sword-and-shield warrior (without Taunt!) and a Runic Golem with some typical golem abilities, plus a weak Group Heal, a Cleansing Aura, and Chain Lightning/Tempest spells. All of these characters are poorly configured with mediocre equipment, and the party only has one respec tome (Memoirs of the Grey Wardens).

Save the tome for the rogue. He’s going to be the main DPS character against the Harvester, but only if he’s an archer and standing far away from the action.

Attributes and Specializations

If starting this without having beaten Awakening, then the mage is restricted to three base specializations: Spirit Healer, Arcane Warrior, and Shapeshifter. Since Shapeshifter is worthless, the two specializations are decided already. If you have started Awakening, you might consider the Battle Mage instead of Arcane Warrior for additional utility, but Spirit Healer is an absolute must as a new character.

As a mage, 60+ magic is necessary for spells to have decent effect. The rest should be spent in willpower to pad out the mana supply. Usually, when you build a Mage for Origins or Awakening, you don’t want to put any points in Willpower. However, remember here, there are very few, very precious lyrium potions and no way to buy more; there are no special attribute bonuses that boost the mana pool for free; and the PC’s starting equipment is subpar and does not provide a large mana boost (and again, there is no way to buy better equipment). If you don’t take additional Willpower, you will not be able to cast during the long endgame battle against the Harvester, and that would be very unfortunate.

The mage should have maxed Combat Training to ensure that spells aren’t interrupted by ordinary attacks. Survival is a plus for the nature resistances, and the first level Vitality and Mana skills from Awakening. This is a pure combat module, with no ingredients for crafting and a grand total of two conversations — no need for Coercion or any of the crafting skills.

Talents

One might be tempted to build an armored arcane warrior, since this is a popular “I win” character build.

This would be … unfortunate. Normally, the arcane warrior is very overpowered: all the gear of a tank and the firepower of a mage. In this module, with a new character, you have neither of these advantages. The small module does not offer a merchant or any sort of decent armor, and a stock character does not have enough willpower to have a deep mana pool after putting up the arcane warrior defensive abilities. Wearing a robe, swinging a Warden’s Companion longsword with a mediocre shield, while not being able to cast healing spells — this is a recipe for disaster. Further, Brogan, the warrior in the party, is configured as a tank — a poor tank, but he can’t be respec’ed as a DPS due to lack of tomes, while the PC can be a DPS. Therefore, playing a support + nuker mage is probably a better choice. Some of the advice that follows will assume a mage with an assortment of crowd control and attack spells.

As for specific talents, since it’s a stock character, there aren’t that many points available. Here is my advice:

  • The Healing tree up to Regeneration is a must. As Spirit Healer, also take at minimum Group Healing, because both the golem enemies and the Harvester at the end have area-of-effect attack abilities that can damage an entire party. Remember that this character will be the only healer until the Runic Golem is fully upgraded, and even so, the Golem is a terrible healer. These healing spells are basically critical to finishing the module, and is why stock level 20 Warriors or Rogues have a hard time surviving.
  • Make sure to get Force Field, for several possible reasons. Crushing Prison is nice, but only if there are points for it.
  • It would be wise to take Mana Clash, at the top of the Mana Drain tree. In several battles (unfortunately, not the end battle), the party will face several spellcasters along with hard-hitting warriors. Mana Clash will immediately remove the casters from the scene, which trivializes the encounter.
  • For me, I took Tempest and Blizzard (from the Lightning and Cold trees), since I already have Spell Might from the Mana Drain tree. This sets up the Storm of the Century combination that was useful in one instance, but this is optional. I strongly recommend Cone of Cold in any case as it is an absurdly overpowered spell with a freezing effect that applies even to elites and bosses.
  • If taking Storm of the Century, or not taking Mana Clash, then Glyph of Neutralization is desirable. As for the other glyphs, since the main enemies are almost always elite or boss level golems and undead, they are often resisted — except the Paralysis Explosion combination, if set up correctly.
  • I did not use any other spells. If you have a favorite set, use the remaining points to take them. Remember that practically all enemies worth using mana on are Elite or above, so some spells that have minimum effect on elites are not worth taking.

Remember that only one party member will get a chance to respec, and the rogue should use it to reconfigure as a survival-minded DEX-based archer. This is because this underpowered party simply cannot stand toe-to-toe against any of the major enemies, and will rely on kiting (and not even simple kiting either, given the setup of the final battle) and off-tanking to survive many fights.

Jerrik, the Archer

Jerrik will be the main non-magical DPS. Unfortunately, he is setup as a frontline fighter. Because the tank is terrible, and can’t even manage threat effectively (no Taunt!), plus both enemy Golems and the Harvester have area-of-effect attacks to damage everyone in their immediate vicinity, this is a bad configuration. He must respec into a DEX-based DPS archer (enough STR to wear his armor, then all DEX), with Master Archer or Scattershot, Arrow of Slaying, and Feign Death (for the many times when the tank loses threat and the enemy rushes the archer). Optionally, he could take a few cunning and take Stealth to lose threat, though the final battle involves an elite boss and hordes of minions.

Major Battles

On Nightmare difficulty, in general, proceed slowly and cautiously, withdraw backwards into chokepoints when needed, and there should be minimal pain. In particular, when faced with multiple spellcasters, hit Mana Clash to remove most (if not all), and then rush them. Play this as a normal RPG and use the pause key frequently to strategize and adjust tactics. Try not to use any consumables until the final fight. For a general walkthrough, follow Gamebanshee’s Amgarrak guide.

There are two battles that takes some strategy to pull off, however. Some people are stumped here and can’t proceed further. They’re not hard, but you can’t use the normal tank-and-spank tactic with a level 20 Warden and a crappy party.

The ancient hall of golems

As soon as the party enters Amgarrak, there are three directions to go, two of which are sealed off at the start. In the third direction, there is a room with four stationary golems. Two of them are called Watchers, the other two are elites called Sentinels. When the party proceeds far enough into the room, these four will activate.

This is one of the toughest fights because the party does not have its warrior tank yet. Further, these four golems will open the battle by hurling rocks — which deals heavy area-of-effect damage and knocks down any character beside the Runic Golem. They will then follow by using Slam on the incapacitated character, and then Hurl Rock again, basically going into a stunlock until the character is dead.

Careful pulling is impossible, because all four activate at the same time. However, there is one good way to deal with these golems: The Hallway of Doom.

There is a chokepoint hallway going into the hall of golems. The party could fight the golems one-on-one at the doorway, but they’ll jut use Slam to knock the blocker out of the way and rush into the room. Then again, they won’t be able to do that …if the blocker is force-fielded.

Have the party hold position. Use the Runic Golem to trigger the four enemy golems, and then run back to the door. At the doorway, use the Runic Golem to block the door and force field the golem before he gets Slammed. Have the ranged DPS open fire. Force Field your own golem, because the enemy golems are elite and will have shorter Force Field durations.

For even better effect, throw down a Blizzard to slow the enemy as the Runic Golem approaches. Then, trigger Storm of the Century using Tempest, and then Force Field your Runic Golem before it gets seriously damaged. Now the enemy will stand in the massive electrical storm, be drained of stamina by the lightning storm’s side effect (so they can’t Hurl Rock or Slam anymore), and run around uselessly. When the Force Field expires, withdraw the Runic Golem and slaughter the heavily injured enemies (or heal the Golem until you’re able to Force Field again — the enemy should no longer have enough stamina for another Slam).

Make sure the mage is standing off to the side as the Runic Golem runs toward the party, because the enemy could see the mage and interrupt the casting with a hurled rock. If you cast the Force Field too late, the enemy will Slam the blocker out of the way, which will probably cause a party wipe.

I really wish I had two mages in this module, which would be able to keep this little trick going forever. Alas, that is not to be.

NOTE: If you didn’t take Storm of the Century Force Field, or if you can’t get the blocking trick to work, there is another way. You need to have respec’ed Jerrik with at least Combat Stealth. Have the party hold position far away from the action, have Jerrik walk into the hallway and trigger the golems. When the golems activate, Stealth immediately and walk away. Using Jerrik, carefully pull the now activated golems, one or two at a time, and kill them individually as they follow you down the hallway. This is arguably easier, if less impressive, than the hallway blockade trick.

The green room

Again, a horde of golems.

all golems hostile in the golem room
This time, the party needs to flip a switch in the green phase, which triggers an encounter with four boss golems and two elite golems. If you didn’t feel enough hurled rocks and full-bodied rock slams in that previous golem encounter, this one will surely give enough. Even with Brogan the warrior, no one in a level 20 party can tank through that. There is a doorway chokepoint, but boss golems have huge resistances, and even a full Storm of the Century will only take off 1/3 of their hitpoints on Nightmare. The blocker golem won’t survive too long after the Force Field expires; certainly not long enough to get another Force Field up. Once again, they all activate at the same time, so you can’t pull one away by itself…or can you?

This one needs an underhanded trick. Remember, there was a room next to the starting chamber of the Amgarrak level with a blue and a purple switch. Both of these switches will throw the party out-of-phase with the golems, meaning that they disappear from the map. When this happens, the hostility chase trigger is also reset. If you can somehow kite the golems to other rooms, then run to the switch to go into blue or purple phase, walk away while phased out, and then switch back into normal phase when you’re far away enough…these golems will lose sight of you and stand still. Furthermore, now they’re isolated from each other, and you can pull each one independently instead of triggering all six at once! This means you can pull each golem, kill it, heal, and repeat.
Run back into this room and hit a switch to avoid the chasing golems
So the strategy is as follows: after you hit the green room switch to activate the golems, immediately run back out of the room, down the stairs, and toward the starting chamber. The faster you run, the more the enemy golems aren’t going to keep up, and will be strung out along the way. As you arrive in the switch chamber, switch phase. All the golems disappear as you go out of phase with them.

Walk back to the green room, switch phase again, rob the chests if you need to, and return to normal phase. Slowly go down the stairs again, pull each golem one at a time along the way, kill them one by one until you arrive at the switch room again. Problem solved.

The Final Battle

This is the fight that few people seem to win as a poorly geared level 20. This is because they are going toe-to-toe with the Harvester flesh golem. There are two major problems with this.

  1. The Harvester hits for 90+ damage, and crits for 100+ damage on Brogan or the tank, due to the crappy equipment setup of the stock party. There are not enough healing poultices or spells to keep the tank alive at that rate.
  2. There is a second problem — the Harvester in the Flesh Golem phase periodically summons a skeleton, either with his Vile Disembowelment skill, or they just wander into the room. As the flesh golem takes damage, the skeletons spawn faster. The skeleton is usually elite, which hits really hard, or boss, which can seriously damage even the tank. This isn’t even the main problem — the real problem is that the skeletons go after the DPS’ers. Because Brogan has terrible threat management, the skeletons will wander into the room, go immediately for the caster or the archer, use Riposte to stun them, and hit for 50+ on leather and even higher on cloth. If the mage and/or the golem goes down, it’s a guaranteed wipe.

Since the tank can’t hold up, the only strategy is to kite the Harvester around the room. There is plenty of space to maneuver, but there is not enough DPS with a stock party to take it down quickly enough. Jerrik hits for 80 to 100 per arrow on the Harvester, and crits for a bit higher. The mage hits at around 30 or 40 with the staff and more with spells if not resisted, but she can’t cast too many attack spells as healing is very needed and there are about 10 to 20 lyrium potions in all. The rest of the party is fairly useless in terms of DPS, since they’re melee and can’t get a good hit in on the Harvester as it walks around. The room is going to be filled with skeletons. They all have stun skills. If the kiter take a stun or two, he will get swarmed and probably die.

Preparation

The party needs to set some basic tactics.

  • Brogan needs to take the least powerful health poultice at 25% health or less
  • Jerrik needs to set typical archer tactics, and feign death at health < 10%
  • The PC’s mage needs to drink a least powerful lyrium potion at 25% mana, cast Regeneration on an ally when the ally’s health is < 50%
  • The Runic Golem should turn off his Hurl Rock and Quake tactics, and cast Group Heal when an ally’s health < 50%

Have the Runic Golem set Flame Weapons. When the battle begins, coat weapons with the highest level poison you have available.

Phase One

In Phase One, the Harvester will jump inside a flesh golem and open with a Vile Disembowelment, an area-of-effect attack that immediately spawns a new skeleton.

The initial setup against the Harvester

As the fight begins, set the party to Hold Position, and manually move the party members into position. The mage and the archer should go stand by the switch that appears, but far enough apart that they don’t get nailed by the same AOE attack at the same time. Brogan needs to stand close by as the off-tank. The Runic Golem should stand at the center of room with his Cleansing Aura. The bronto (which actually has Taunt and runs faster than any other character in combat) should attack the Harvester, throw down a Taunt, and then start kiting the Harvester around the Runic Golem’s aura radius.

If a normal skeleton appears, Brogan should go tank it immediately, and the mage can help freeze it with a Cone of Cold. If an elite or boss skeleton appears, or if there are more than 3 skeletons, have the mage pull the switch immediately to destroy all the skeletons. Jerrik should always focus on the Harvester. If either the archer or the mage is under attack, have the bronto run over and Taunt, then run the mage and/or archer away. The skeleton should lose interest and focus on the bronto. If not, Brogan needs to pick it up and tank it while the DPS’ers run far away.

Heal the bronto constantly, as it’ll inevitably get hit by the Harvester and its skeletons. If the bronto is stunned, or if it is killed, Brogan must pick up threat and resume kiting immediately until Jerrik is able to resummon the bronto. Switch back once the bronto is up again.

(If this is starting to feel like playing a MMO 5-man dungeon boss encounter, except you have to control every party member by yourself, well…yes. Use that Pause function. The Pause key is the one major difference here that lets you control the multiple things going on in this encounter.)

Keep an eye on the archer and mage while you’re kiting with the bronto. Make sure no skeleton has wandered into the room and started attacking them. Pause a lot if you need to.
Nearing the end of the First Phase of the Harvester Fight
If this is executed correctly, eventually the Harvester will fall. When its health gets very low, try to avoid hitting the switch and save it for just before Phase Two. The switch will not appear again at that point, and there may be several skeletons that need killing at the end.

There is a bit of luck involved. If the switch is hit too early, and elite skeletons emerge while the switch is inactive, then this is probably bad. If it’s a boss skeleton, you might consider using Brogan to kite it around until the switch is available again. Controlling two kiters is very difficult, so pause a lot.

Phase Two

When the flesh golem is killed, the Harvester will jump out and attack. If the party made it this far, and the switch is still available, then the battle is more or less won. The Harvester lizard-form is far, far weaker than its previous flesh golem body. It hits for about 60 on Brogan, but hits slowly and misses a lot, so it can be tanked. It also disengages a lot to channel a skeleton summoning spell, so it doesn’t have a lot of DPS to hit the party with. The best news: skeletons will no longer wander into the room in large packs — at this point, the Harvester has to summon one with his spell.

When the game screen comes back from the cutscene, hit the switch one last time to clear out all remaining skeletons if it’s still available. If not, resume the kiting strategy and quickly remove the remaining skeletons. Then, have the party resume normal attack stance from hold position and focus fire on the Harvester’s lizard form. If it uses the jump ability, cast Crushing Prison (it won’t resist nearly as much now) or the Runic Golem’s Slam attack to slow it down while the melee attackers catch up. It can also be frozen solid with Cone of Cold, though it can’t be shattered obviously. Jerrik and the mage are still going to be the main DPS. If the Harvester finishes channeling a normal or elite skeleton, tank it with Brogan and blast it down quickly. If it’s a boss skeleton, Force Field it or have the bronto kite it around the room while the remaining party members continue to blast the Harvester. It is only a matter of time before the Harvester dies.

The Grim Reaper achievement should hit the screen after a short cutscene. Woot!

The first time I did this, it took me three tries and about an hour of playing, but is definitely a reproducible strategy on Nightmare difficulty. If playing on Hard, this is actually really easy, as most skeletons will be normal skeletons and can be frozen and shattered. It does take a lot of micromanagement, but since you can pause the game and issue new orders, this management shouldn’t be too hard — kinds of reminds of me Baldur’s Gate 1, where I auto-paused after every combat round and barely took any damage.

As a final word: the builds and strategies outlined in this note are *not* how you should typically play DAO. Given an entire campaign to work with, the builds here are suboptimal. These builds are designed only to complete this specific module, within the constraints that it imposes.

musings on self and search

So as you may be aware, I developed SelfSolved in order to avoid cluttering up this blog with “I fixed this!” posts. I’m trying to get into a habit of writing at a higher conceptual level, instead of this repository being just yet another technical problems blog. It turns out that without writing about random things that I fixed, I have little else to say. When I wrote up problems for this blog, I had monthly posts. Now I’ve gone 3 months without writing anything.

It’s odd, as Ph.D. students (and former startup founders) can hold forth via very long and extensive blog posts, especially on technical topics they care about (and sometimes on topics that they don’t). It may speak to my lack of aptitude for Ph.D. work; I hope not.

When I implemented SelfSolved, I created a sitemap for search engine crawlers as a matter of course, but did not do so immediately. Lots of content were present on the site before I completed the sitemap feature. It turns out that search engines like the GOOG seem to ignore certain content in the sitemap — so much for indexing all the world’s knowledge. I thought the crawler would get around to it eventually, but it doesn’t appear to be the case after nearly a year.

It’s interesting, because there doesn’t to be any rules. For example, “turning caching off completely in Pylons” has a very short problem statement and solution. Other posts, however, are fairly normal: “SVN directory is viewable through Apache” is by all accounts a normal sized SelfSolved problem, with references and a full solution writeup. As of this post, neither are indexed.

As soon as I wrote this blog post on this WordPress installation, the post became available in GOOG’s main index. If I search on the phrases above, I get an entry back immediately. But for a site like SelfSolved, these indexes aren’t available for days or weeks, even though SelfSolved publishes an Atom feed of its content as well, and pings the same GOOG notification URIs. This does suggest that if you build your site on a well-used CMS, rather than custom-designed software, it will get you into indexes in almost real-time vs in days. If you run a for-profit content-driven site (which SelfSolved is not), that could mean a fair bit of money.

It’s no wonder that SEO consultancies are flourishing. There are certain hidden rules with crawling that go beyond the simple “create good content” and “get a sitemap” guidelines that the GOOG wants you to believe.

Musings on the state of popular science fiction

Received an email from a good friend on the “new and improved” Star Trek:

I gave this some thought, and perhaps one could say that blowing up Vulcan is a symbolic “fuck you” to the core values of Trek, to wit: “Logic, reason, humanism–fuck these things! Do what feels right regardless of the consequences. Or you’ll get sucked into a black hole.” Vulcan, I think, represented a key component of the Federation’s philosophy, and now they expect us to accept its destruction so they could thrill some casual viewers?

To my mind, the interesting tension in the Trek universe prior to this travesty resulted from Federation principles being tested against competing internal and external forces. However, in Trek: Rebooted land, there’s one hand clapping–it’s all barbarism all the time; there were no discernible Federation principles in the first place. You know, in Star Trek VI, Spock said “Logic is the beginning of wisdom, not the end.” New version: “Fuck all that shit man, party on the Enterprise!” Logic is apparently a trifle to be jettisoned whenever your adrenaline glands act up.

I think this has become a fairly general problem in what is considered “mainstream” sci-fi — the shows that “ordinary people” would accept. The so-called more real vision of the future espoused by these things can roughly be generalized into “darker, edgier, younger”. Reality, as it seems, must be full of angst, petty problems, and on the occasion, explosions.

A far cry from when the thoughtful visions of TNG garnered mainstream viewers by the millions.

There’s a quote attributed to AI pioneer Marvin Minsky, of the MIT AI Lab:

“General fiction is pretty much about ways that people get into problems and screw their lives up. Science fiction is about everything else.”

The point of science fiction is to imagine what is possible beyond the confining frameworks of everyday reality and drudgery. It is supposed to illustrate what humanity is capable of and what they should aspire toward.

TNG, for example, aimed to explore a post-scarcity human society where the driving force is no longer personal economic gain. Of course it would not be “real”, so much as matter replication and warp drive are also “not real”. An average man of today would have to have to stretch his imagination to understand the motivations of those characters and their society — but that’s precisely the point. If we all spun in place forever, mired in the minutiae of what is (rather than what should and will be), there would never be any progress — only more of the same.

This is also part of a rather annoying trend in popular culture to treat technology as something to be feared, and how curiosity should be punished. How many recent sci-fi successes can be boiled down to: “Don’t research/develop technology X, because it’ll just destroy all humans/blow up the world/nuke us and all of our colonies!”. It is trivially easy to spin a cautionary tale of fear for the new and unknown, to play to the most primitive parts of the human mind — such fearmongering undoubtedly holds TV ratings and sells movie tickets. It takes a true visionary to look into what might be the next step for human progress, and instead of fearing the unknown, embrace it.

The Original Series inspired a generation of young people to become scientists and engineers. The Next Generation taught the value of deliberation, diplomacy, and teamwork, creating a vision of a humanity pulling away from the basest parts of its past — and moving onto bigger and better things. While they are still quite entertaining, they are part of that grand tradition of science fiction, opening the minds of ordinary people and pushing them, if only a little bit, toward exploring “final frontiers”.

What do the latest and leading “sci-fi” do for that vision?

getting my act together: papers and projects

Honestly, this was one of my more productive years. While I still haven’t gotten any further on either of my private projects (and in fact abandoned one of them, having been beaten to the punch by a startup this summer. You ambitious entrepeneurs should leave me some hobbies) , I did manage to get some reasonably meaningful things out.

Worked on some pretty awesome stuff at Y!RB. Programming, data analysis, and social media research – it doesn’t get any better than that. Now I just need to figure out what I want to be doing after this year, which may in fact mark the end of my affiliation with graduate school and possibly academia in general. Scary thought. Where to go from here…

Information Services and Design symposium, brief retrospect

Participated in the Information Services and Design symposium last week. Gave a short talk on intellectual property rights in context of economic development and the knowledge economy, as well as submitted a small paper that is now part of the UCB iSchool Report Series, 2007-011 (PDF).

In truth, I’m not terribly confident about the paper. The domain was too large to be effectively covered in either a 5000-word paper or a 15-minute talk. There are significant implications in IPR in development that falls more into international political economy. The problem is very multifaceted, and there are a number of IPE perspectives that can be used as a lens on this. I think my conclusions here derive from a realist position that national interest has trumped any pretenses to free market liberal economics in this case, but that would be a fairly gross simplification of the factors at work.

Eh, but it’s a tangible output that’s seen more publicity than anything else I’ve done lately.