Skip to content

Outlook 2011 for Mac still adding arbitrary line breaks into plaintext emails

Posted by on 25 December 2011, 2:00 am

Outlook 2011 on Mac OS X, v14.1.3, for whatever reason, still does not properly support “format=flowed” content-type or “quoted-printable” extensions for plaintext emails. This causes plaintext emails to be sent as mangled messes, full of arbitrarily inserted linebreaks. This appears to be a regression from Entourage, as far as I recall, which never handled plaintext quite this badly, and this is also despite Microsoft’s promises to have “implemented format=flowed”.

This is the last straw. I’ve been a loyal MS Entourage / MS Outlook user since the days of Outlook Express for Mac and Office 2001. But at this point, this software has actively impeded my communications with my friends and colleagues. We’re done.

The Problem

Here’s a really simple illustration of the problem, from the receiver’s end:

See how the URL, which was composed as one plaintext line, gets split up into two lines?

Here is another example, purely from the editor UI (and not even being sent yet). I start with a perfectly good reply saved as a draft:

I make a small wording change and resave:

See that third line? Thanks to the hard line breaks inserted by Outlook (even at composition stage), the line wrap has been mangled. This draft has to be re-wrapped manually, by the tedious process of deleting the newline-based hard line breaks from every line following in the paragraph. That was a short paragraph. Imagine doing that in a long paragraph, from the first line.

To add insult to injury, there is not even a “re-wrap” functionality in the editor, to at least solve this user-interface level problem (as opposed to the protocol level problem). Obviously no one at Microsoft sends plaintext emails anymore.

The Issue

Back when email was first devised, servers didn’t have a lot of memory, and people had pretty tiny terminals with fixed line widths and not a whole lot of processing power to deal with it. The Internet standards for email messages http://www.ietf.org/rfc/rfc2822.txt, RFC2822 Section 2.1.1, defines recommendations for email body text transferred over SMTP:

There are two limits that this standard places on the number of
characters in a line. Each line of characters MUST be no more than
998 characters, and SHOULD be no more than 78 characters, excluding
the CRLF.

The 998 character limit is due to limitations in many implementations
which send, receive, or store Internet Message Format messages that
simply cannot handle more than 998 characters on a line. Receiving
implementations would do well to handle an arbitrarily large number
of characters in a line for robustness sake…

The more conservative 78 character recommendation is to accommodate
the many implementations of user interfaces that display these
messages which may truncate, or disastrously wrap, the display of
more than 78 characters per line…

…it is encumbant upon implementations which display messages
to handle an arbitrarily large number of characters in a line
(certainly at least up to the 998 character limit) for the sake of
robustness.

Basically, the SMTP server can count on messages that come in 80 characters per line (and always less than 1000 characters per line), and email clients can trust that they only have to render up to the 78th column of text. This limitation is hardly useful in the modern age, but persists since it’s part of the standard. And it’s a fine, conservative design model. But now we write some pretty long lines without linebreaking ourselves, so something magical has to happen in the email client itself, like Outlook 2011.

The naive solution, of course, is to slap arbitrary line breaks into the user’s email message at every 78 characters, which is what ye olde email clients from yesteryears did (and Outlook 2011 still does). It’s a matter of personal preference whether this is a reasonable solution. Proponents argue that the email will “always look the same” on all devices, including those limited to 78 chars per line.

I (and many others), on the other hand, think the spirit of the RFC is to allow the actual handling client to decide where to break lines. With the exception of source code, it is almost always better for the email client to use the full width of their display, however many characters that might be. Even in the case of source code, it should also not be mangled by the insertion of arbitrary line breaks in them — what if newlines are meaningful in this language, and the author used more than 78 characters per line? The example with the URI is illustrative of this problem — the URI got an arbitrary newline in the middle, destroying its meaning. Users who copy-paste the two lines will end up getting a 404, due to that stupid inserted newline in the middle of it. This should not be allowed to happen.

Because this naive solution was not perfect, an extension was proposed as RFC 2646. In the email header, if the Content-type was set to have the format=flowed parameter, such as

Content-type: text/plain; charset=US-ASCII; format=flowed

(emphasis mine), the sending and receiving email clients are allowed to reflow the text based on user linebreaks. It follows some simple reflowing rules, but in short it will preserve user-inserted hard line breaks while adjusting the rest of the message for the proper line length while the message is “on the wire”, and recombining the lines on receipt and display. Modern email clients like Thunderbird, designed for user comfort and the generous system limitations of the year 2011, implement this standard.

Guess what header Outlook 2011 sends?

Content-type: text/plain; charset="US-ASCII"

Not even an option to change that behavior. It does not appear that Outlook 2011 deals with any of this. It just inserts some line breaks and calls it a day.

An alternative, implemented by Apple’s Mail.app, is to send messages with the Content-Transfer-Encoding header set to “quoted-printable”, as per RFC 2045. In this model, soft line breaks are sent explicitly with the character “=” representing it, breaking at the usual 70-odd character column. On the receiving end, the client processes this character as a no-op and concats the line back together for display.

Outlook doesn’t do that either. It just wants to mangle your emails.

Conclusion

The world moved on and adopted HTML emails, which doesn’t have this newline problem. For those of us who do think HTML emails are an atrocity to be used sparingly, if at all, the idiosyncrasies of plaintext email have to be addressed. Outlook 2011 appears to do even worse than Entourage 2008 at this problem, by not dealing with it at all. And apparently getting a bunch of Microsoft “MVPs” on their forums to cloud the issue with promises of support and unrelated commentary.

Given the sad state of email clients on the Mac, I believe Thunderbird is now my only option for sane plaintext messaging.

Filed under Software. Tagged , , , , , , ,
Comment

A wishlist for a native Mac email client

Posted by on 21 August 2011, 1:46 pm

As the tech-savvy sort, I often do some things that the typical consumer might not care about. However, it is strange that I simply cannot find a native Mac email client on the market that fulfills what I consider to be very basic features for decent email management. While everyone is gushing over the latest social network to be jammed into an email client, I just want my email to work in a sane way.

The features I’m looking for:

  • POP3/IMAP + SSL support
  • Full-text search, or indexable by OS X’s Spotlight
  • Multiple account support
  • Archival — if I delete an email account, it should not wipe out all emails from local storage that belonged to this account, especially if said account was POP3
  • Plaintext composition — I’m taking the side of format=flowed in plaintext composition. Manually formatting hard line breaks at char 78 is an insane holdover from a bygone age when dumb ASCII terminals were still the primary user interface. At least allow this as an option, or support the quoted-printable content type and soft line-breaks during composition.
  • Filtering rules — Basic filters that let me decide where to put messages based on mail headers and subject is enough
  • mbox import/export — I need to be able to import mail from my previous client. Similarly, in the scenario that this app is no longer supported, I need the option of moving to another client.
  • Bonus: auto-bcc to an arbitrary email address — I’ve never solved my problem with archiving sent mail. All you really need to do is auto-populate the BCC field on any “New Mail” composition window with this address. That’s all I really need to be happy here.

Amazingly enough, there is not a single Mac mail client that fulfills all of these basic conditions. Especially egregious problems for the top three mail clients:

  • Microsoft Outlook Mac 2011 — mangles all outgoing plaintext emails by inserting hard line breaks. No support for flowed plaintext, despite promises to the contrary, and no support for quoted-printable content-type and soft line breaks. No provision for auto-bcc.
  • Apple Mail (Snow Leopard) — deleting an old email account deletes ALL mail belonging to this account, even for downloaded POP3 mail. WTF. Can only auto-bcc “myself”, which is a fixed email address corresponding to the sending account; if you allow autobcc, why not let the user pick the email to autobcc to?
  • Mozilla Thunderbird 8 — deleting an account deletes all downloaded mail belonging to this account, unless messages are stored to the “local folder” rather than its inbox abstractions.

For fear of accidental data loss, I’ve stuck to the Microsoft offering, despite its incredible inability to keep plaintext mail intact. The newer social clients are all Thunderbird-based and leave little else worth examining — they seem far more interested in integrating ever more social network APIs than to support more basic email features. The rest of my friends are all sworn Gmail webmail users and think I’m a strange luddite for even considering native clients. I’ll leave the webmail vs native client debate for fear of going on a page-long rant; suffice to say that the user experience between the two options is not comparable at all.

It almost makes me want to start writing a new email client for myself, or hire someone to start some skeleton code at least, because obviously no one else is going to scratch this itch of mine. That kind of spare time and money, however, would probably never come unless I manage to actually sell a company or two (rather than just founding unsuccessful ones).

Filed under Personal. Tagged , , ,
Comment

musings on self and search

Posted by on 5 March 2010, 2:48 pm

So as you may be aware, I developed SelfSolved in order to avoid cluttering up this blog with “I fixed this!” posts. I’m trying to get into a habit of writing at a higher conceptual level, instead of this repository being just yet another technical problems blog. It turns out that without writing about random things that I fixed, I have little else to say. When I wrote up problems for this blog, I had monthly posts. Now I’ve gone 3 months without writing anything.

It’s odd, as Ph.D. students (and former startup founders) can hold forth via very long and extensive blog posts, especially on technical topics they care about (and sometimes on topics that they don’t). It may speak to my lack of aptitude for Ph.D. work; I hope not.

When I implemented SelfSolved, I created a sitemap for search engine crawlers as a matter of course, but did not do so immediately. Lots of content were present on the site before I completed the sitemap feature. It turns out that search engines like the GOOG seem to ignore certain content in the sitemap — so much for indexing all the world’s knowledge. I thought the crawler would get around to it eventually, but it doesn’t appear to be the case after nearly a year.

It’s interesting, because there doesn’t to be any rules. For example, “turning caching off completely in Pylons” has a very short problem statement and solution. Other posts, however, are fairly normal: “SVN directory is viewable through Apache” is by all accounts a normal sized SelfSolved problem, with references and a full solution writeup. As of this post, neither are indexed.

As soon as I wrote this blog post on this WordPress installation, the post became available in GOOG’s main index. If I search on the phrases above, I get an entry back immediately. But for a site like SelfSolved, these indexes aren’t available for days or weeks, even though SelfSolved publishes an Atom feed of its content as well, and pings the same GOOG notification URIs. This does suggest that if you build your site on a well-used CMS, rather than custom-designed software, it will get you into indexes in almost real-time vs in days. If you run a for-profit content-driven site (which SelfSolved is not), that could mean a fair bit of money.

It’s no wonder that SEO consultancies are flourishing. There are certain hidden rules with crawling that go beyond the simple “create good content” and “get a sitemap” guidelines that the GOOG wants you to believe.

Filed under Personal. Tagged ,
Comment

fixing a scrambled IPython command history on stock OS X 10.6

Posted by on 22 December 2009, 1:42 am

So I started over with a fresh install of OS X 10.6 recently, and wanted to restore my Python development environment. In doing so, IPython is absolutely essential if you want a sane interpreter environment to test out code. I had a bit of trouble with it though.

The Problem

The stock Python 2.6 shipped with OS X 10.6 Snow Leopard has a readline module linked to libedit, the BSD alternative to the GPL’ed readline. The readline module, if you are not aware, is (among other things) responsible for keeping command history in the IPython interpreter. This causes command history in the IPython 0.10 interpreter to behave in very odd ways. When backtracking through the command history buffer using the up-arrow key, for example, the previous command is only partially recalled, and appears completely scrambled. Indents, too, seem off — in a whitespace-sensitive language like Python, this is annoying. (See first figure)

IPython command interpreter is broken when using libedit with command history

IPython command interpreter is broken when using libedit with command history

Fixing IPython’s bugs are beyond my ability. While I certainly don’t want to delve into the quagmire that is GPL vs BSD licensing, I do understand why Apple would want to avoid the viral nature of the GPL and ship libedit instead. However, using a genuine Readline library is going to be the best recourse for this problem. I already have a copy of readline compiled and ready to go, and just need a new version of readline.so, the library that links Python to readline.

The easy solution

Sifting through my records, I came across a SelfSolved problem record from my good friend Hannes who had issues with his IPython command history.

The solution: sudo easy_install readline, which uses setuptools to install a precompiled package of readline.so statically linked to genuine GNU readline. Restart your IPython console and everything should work. (See second figure)

IPython with readline

IPython with readline

The hard solution

Being the inquisitive sort, I also wondered how I would be able to reproduce this work from scratch. readline.so ships with the Python source package, but surely I would not be required to compile a whole new copy of Python for one measly module library?

I documented this process in SelfSolved again: building readline.so for Python. At some point I should write an interface between SelfSolved and WordPress so that I don’t have to reproduce a lot of my work here manually.

Compiling readline.so

This is actually fairly easy.

  1. Get a copy of the Python source code. In OS X 10.6, it ships with Python 2.6.1.
  2. Unpack it and go into its directory. You should find a Modules subdirectory. In it is readline.c, the source file for readline.so.
  3. Compile the source file. The appropriate incantation is:
    gcc -O2 -arch x86_64 -arch i386 -shared -o readline.so readline.c -I/usr/local/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -L/usr/local/lib -lreadline -ltermcap -framework Python

    where the -arch flags should be whatever processors you wish to support, the -I arguments should point to the directories that contain header files for the readline library and the Python framework, and the -L argument should point to the path for the readline library. Use whatever optimization flags you feel comfortable with, instead of -O2, if you wish.

Replacing readline.so

So now we have a readline.so that’s properly linked to readline.dylib. The thornier question is how to override the system-provided readline.so. The system version is located at /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload/readline.so, and the naive would simply overwrite it with their new readline.so. This is a bad idea.

As I have mentioned in the past, overwriting system libraries in OS X is an unhealthy thing to do. The problem is that Apple furnishes no official package management system — anything you personally change is considered fair game for the next official system update. On the next system update, if the Python component is affected by the update, the Apple updater will happily clobber your compiled files with its own, leaving you suddenly back at square one. You don’t know how many times I’ve had to recompile emacs (for X11 support) on OS X 10.4 because of this little annoyance. Leave things in the /System/Library directory hierarchy alone, for your own sanity.

However, in this case /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload comes ahead of the user-modifiable /Library/Python/2.6/site-packages directory on Python’s sys.path. So if you just drop readline.so into site-packages, the system version still takes priority.

There are a few ways to do this. For one, you can create a sitecustomize.py in /Library/Python2.6/site-packages. In this file, arbitrary Python statements can be written, and the interpreter will automatically execute them at runtime. So, you can add a sys.path = ["/dir/here"] + sys.path statement and point it to a directory containing your readline.so file. Alternatively, you can abuse the technique used in the easy_install.pth file. It turns out that if you ever used easy_install, directories pointed to by the easy_install.pth file takes priority over the system paths. They use an interesting way to accomplish this, which you can copy. Or, you can just insert your directory containing readline.so into easy_install.pth. In any case, this will force the readline-based readline.so to take precedence over the libedit-based readline.so, without overwriting anything.

Discussion

So for any sane person, the easy solution should be enough. For the rest, the hard solution is an interesting exploration of how some of Python’s built-in modules can be compiled and inserted individually.

Filed under Solutions. Tagged , , , , ,
2 Comments

Upgrading the Seagate Barracuda 7200.11 to firmware SD1A

Posted by on 8 July 2009, 10:44 pm

So perhaps you have heard of Seagate’s little manufacturing issue with its internal 3.5-inch Barracuda 7200.11 1TB drives a while back — namely, that some drives shipping with SD15 firmware are dying horribly. I had the unfortunate experience of buying such a hard drive — the ST31000340AS — as a scratch disk for my main machine, a MacBook Pro with a mere 240 GB internal drive (a pre-unibody revision, where the HD is insanely difficult to replace).

Seagate did in fact issue a firmware update — SD1A — that supposedly addressed this issue, but of course, there’s one catch: you can’t install the firmware through an external drive enclosure. In communication with Seagate support, a representative confirmed that for those of us without a desktop tower that has a SATA bay, we’re hosed:

Unfortunately, due to the nature of firmware updates and the way external drives work, the firmware update program cannot directly communicate with the drive in the manner it needs to in order to be able to upload the new firmware to the drive. It must be plugged into an internal SATA controller in order to update the drive.

Fair enough. That makes technical sense — but of course, it doesn’t work for me. I asked whether they would handle a mail-in repair, given that I have no easy access to such a desktop. The answer, of course, is No.

I have to find a desktop, open it up, jam this baby in (possibly in place of the existing drive if there’s only one bay), update the firmware, and put everything back together. Sadly, most of my friends who still own desktops would not trust me that far.

Half a year passes, and I finally find a sucker good friend who’s gullible awesome enough for me to try this procedure on his machine. The fellow owns a nice if aging Dell Precision T5400, which comes with two SATA bays (so I don’t have to inflict undue harm onto the existing system). Since this thing can run two drives at once, I can use the first method (a Windows-based firmware updater), though I burned a boot CD for the second method just in case. I popped in the drive, fired up Windows XP, downloaded the Windows-based Firmware Update Utility, double-clicked, and thought it was the (triumphant) end. In fact, it took 3 hours of my life to find out just how deep this rabbit hole goes.

Problem 1: The lying updater

The firmware updater will give a bunch of scary warnings and then reboot the machine. It will automatically reboot to a Seagate Loader screen, which attempts to apply the patch to all eligible SATA drives. To its credit, it’ll skip the non-qualifying (i.e. non-Seagate, non-Barracuda, etc.) drives, but it’ll still try them out first. At the end of the process, it will report “firmware downloaded” and “SUCCESSFUL” or some variant thereof, and automatically reboot back into Windows.

At this point, I advise you to use the SeaTools utility to verify that the firmware update actually applied. Despite its claims, if you were on a stock setup Dell T5400 (or perhaps other models as well), this will prove that the updater is a lying scumbag. And in fact, this particular drive still reported firmware SD15, the broken one.

Problem 2: The broken Boot CD

To save both me and my gracious host (who’s starting to suspect my computer-fixin’ skills now) some time, I decided to try the boot CD method, rather than pounding my head trying to see why the updater was lying. I downloaded the boot CD from the same Seagate Support site above, burned it to disk, and tried it out.

The result is a new SelfSolved posting: SelfSolved #59: getFatBlock error when upgrading Seagate Barracuda 7200.11 firmware. In essence:

The FreeDOS boot CD reports a number of ” error reading partition table drive 01 sector 0 ” errors. This is followed by ” get Fatblock failed:0x000000e8 ” or some variant of ” getFatBlock failed : ” The FreeDOS boot process appears to stall at this stage, and does not continue to the firmware flasher program.

That was lovely.

The Solution

I chased some red herrings. I came across postings about failures in various FreeDOS-based Seagate tools. One such post mentioned that it took a long time for the boot disc to get over the “error reading partition table” errors, but I waited forever (well, 15 minutes) and the boot process did appear to be frozen / stalled. I reformatted the drive via diskpart clean, thinking that the getFatBlock and error reading partition errors were related to a non-MBR partition table (I had it set to GPT). I should have realized, of course, that the errors were completely unrelated to filesystems, despite the “fat block” to which it refers.

The actual solution is deceptively simple — the boot disc & flasher appears to handle AHCI-based SATA mode badly. The Dell I was using was set to AHCI mode, out of the three possible Legacy, AHCI, and RAID options for SATA. Apparently the boot disc simply doesn’t handle this mode correctly on the Dell machine (and may also be related to why the Windows-based updater lies). When the machine switches on, use F12 to enter the boot menu, and select Setup to enter the BIOS. Then, on the list of Drive Options, skip past the SATA drives and down to SATA options. Pick the Legacy option to use ATA mode, instead of AHCI. Once this is done, the boot disc will function correctly, and the updated firmware will be applied without incident. Remember to switch the mode back to AHCI — it’s default for reason, no doubt.

The “error reading partition” messages were completely red herrings. They appear whether you are in the right SATA mode or not, and does not appear to affect the operation of the firmware updater or the boot process. It should not take very long to get to the flasher on this particular setup, so don’t wait on that message too long — it’s a good sign something’s not quite right.

In the end, I did recover my $100 hard drive, and the confidence of my peer in my mad hardware skillz (actually, quite non-existent).

Discussion

In the end, I’m quite appalled at Seagate. This sort of failure shouldn’t have happened, of course. Once it did, Seagate should have offered to take back and replace broken drives — the data I had on there was non-critical. I would have been perfectly willing to pay shipping costs to get a fixed replacement through mail-in service. I should not have been forced to search my social network for a person willing to let me tear his desktop computer apart, for a dubious and unsure firmware update procedure that fails mysteriously. I spent an additional 3 hours tracing mystery failures, for which the error messages were rather useless. Without my trusty iPhone and access to the Internet, I would not been able to solve this problem. How should I have known what “getFatBlock failed” means?

This little episode has convinced me to never buy a Seagate drive again — I simply cannot afford the time and energy for these sort of firmware upgrade adventures. While I was looking for a desktop to tear apart, I bought a Western Digital Caviar Black 1TB drive instead. Another $100, but at least I had a scratch drive for my work.

The moral of the story: Seagate, you are the worst storage vendor I’ve had to work with so far. I hope this record is not broken in the future.

Filed under Solutions. Tagged , , , , ,
11 Comments

Subversion 1.6.2 runtime error on network access on OS X 10.5

Posted by on 27 May 2009, 10:45 pm

A new SelfSolved solution is up for perusal. The problem I tried to solve:

After compiling Subversion 1.6.2 from source on OS X 10.5 Leopard, the compilation is apparently successful, but svn dies when it tries to connect to the network for the first time. Crash log reports that symbols are missing from libneon.dylib.

Crash report from shell:

dyld: lazy symbol binding failed: Symbol not found: _ne_set_connect_timeout
Referenced from: /usr/local/lib/libsvn_ra_neon-1.0.dylib
Expected in: dynamic lookup

dyld: Symbol not found: _ne_set_connect_timeout
Referenced from: /usr/local/lib/libsvn_ra_neon-1.0.dylib
Expected in: dynamic lookup

Check out the places that I googled and my final solution writeup … at SelfSolved #49: Subversion 1.6.2 explodes on first network access.

The problem is very similar to a previous compilation issue I solved for PHP. In essence, the -L library search path passed to GCC at compilation time has /usr/lib in front of everything else. This means whatever library path you might have given to it at configure time, it’ll always look for the library in /usr/lib first, picking up the old system libneon in the process. Since the bad libneon dynamically linked, the problem doesn’t manifest itself until runtime — and only at runtime with network access involved.

As with the PHP issue, change the very first -L/usr/lib to -L/usr/local/lib (or wherever your newer libneon is located), and it’ll link correctly.

Out of curiosity, I checked MacPorts first. The MacPorts solution of disabling libneon version checking is odd — it also works, but I dunno if it’s linking to the right thing or not.

Filed under Solutions. Tagged , , , , , ,
Comment

finding a fault-tolerant HTML parser for iPhone SDK

Posted by on 15 May 2009, 2:16 pm

A new SelfSolved problem is ready for perusal:

A couple of my iPhone projects require a decent HTML/XHTML parser. On OS X, Cocoa ships with NSXMLDocument, which includes dirty HTML parsing functionality from libtidy. Unfortunately, NSXMLDocument is not part of the actual iPhone 2.2 SDK (though it is part of the 2.2 Simulator — so it’ll compile just fine at dev time but break when deploying — a big gotcha if you never tested against a real iPhone).

NSXMLParser is a part of the iPhone SDK…This is not a reasonable alternative.

Check out my writeup at SelfSolved #42: HTML or XHTML Parser for iPhone SDK 2.x

Finally, all out of all the potential alternatives I found (all referenced at the SelfSolved writeup — including one that requires a license fee to use), this one seems to be the most promising and requires the least amount of pain (read: interaction with the libxml C API — god knows I’ve done enough of that while building prototypes at Yahoo! Research Berkeley)

Filed under Solutions. Tagged , , , , ,
Comment

Musings on the state of popular science fiction

Posted by on 10 May 2009, 9:32 am

Received an email from a good friend on the “new and improved” Star Trek:

I gave this some thought, and perhaps one could say that blowing up Vulcan is a symbolic “fuck you” to the core values of Trek, to wit: “Logic, reason, humanism–fuck these things! Do what feels right regardless of the consequences. Or you’ll get sucked into a black hole.” Vulcan, I think, represented a key component of the Federation’s philosophy, and now they expect us to accept its destruction so they could thrill some casual viewers?

To my mind, the interesting tension in the Trek universe prior to this travesty resulted from Federation principles being tested against competing internal and external forces. However, in Trek: Rebooted land, there’s one hand clapping–it’s all barbarism all the time; there were no discernible Federation principles in the first place. You know, in Star Trek VI, Spock said “Logic is the beginning of wisdom, not the end.” New version: “Fuck all that shit man, party on the Enterprise!” Logic is apparently a trifle to be jettisoned whenever your adrenaline glands act up.

I think this has become a fairly general problem in what is considered “mainstream” sci-fi — the shows that “ordinary people” would accept. The so-called more real vision of the future espoused by these things can roughly be generalized into “darker, edgier, younger”. Reality, as it seems, must be full of angst, petty problems, and on the occasion, explosions.

A far cry from when the thoughtful visions of TNG garnered mainstream viewers by the millions.

There’s a quote attributed to AI pioneer Marvin Minsky:

“General fiction is pretty much about ways that people get into problems and screw their lives up. Science fiction is about everything else.”

The point of science fiction is to imagine what is possible beyond the confining frameworks of everyday reality and drudgery. It is supposed to illustrate what humanity is capable of and what they should aspire toward.

TNG, for example, aimed to explore a post-scarcity human society where the driving force is no longer personal economic gain. Of course it would not be “real”, so much as matter replication and warp drive are also “not real”. An average man of today would have to have to stretch his imagination to understand the motivations of those characters and their society — but that’s precisely the point. If we all spun in place forever, mired in the minutiae of what is (rather than what should and will be), there would never be any progress — only more of the same.

This is also part of a rather annoying trend in popular culture to treat technology as something to be feared, and how curiosity should be punished. How many recent sci-fi successes can be boiled down to: “Don’t research/develop technology X, because it’ll just destroy all humans/blow up the world/nuke us and all of our colonies!”. It is trivially easy to spin a cautionary tale of fear for the new and unknown, to play to the most primitive parts of the human mind — such fearmongering undoubtedly holds TV ratings and sells movie tickets. It takes a true visionary to look into what might be the next step for human progress, and instead of fearing the unknown, embrace it.

The Original Series inspired a generation of young people to become scientists and engineers. The Next Generation taught the value of deliberation, diplomacy, and teamwork, creating a vision of a humanity pulling away from the basest parts of its past — and moving onto bigger and better things. While they are still quite entertaining, they are part of that grand tradition of science fiction, opening the minds of ordinary people and pushing them, if only a little bit, toward exploring “final frontiers”.

What do the latest and leading “sci-fi” do for that vision?

Filed under Personal. Tagged , ,
Comment
« Older Entries