Tag: Projects

DocPreview – browser plug-in to view Microsoft Word documents in Safari

Posted on by yiming
9/26/2016
Needless to say, DocPreview 2 is now deprecated and nonfunctional, given Safari 10’s changes to how extensions work. Back to the drawing board.
5/12/2015
Four years later, Safari on OS X still does not have native .doc preview capability.

But I have made a breakthrough! I have managed to make DocPreview 2, a Safari extension + NPAPI-based plugin that allows in-browser previews of Microsoft .doc and .docx files, for Safari 8.0.6.

I will be writing another blog post of this as soon as I clean it up enough for release, as DocPreview 2. For all the good it’ll do until NPAPI support is removed.

UPDATE:
7/21/2011 – DocPreview DOES NOT work with Safari 5.1 on OS X 10.6+. This is because the WebKit Plugin APIs it depended on are deprecated by Apple. This has broken a number of .webplugin extensions, including some of my favorites like the current (as of today) version of XML View. If you have upgraded to Safari 5.1, on OS X 10.6 or 10.7, then please uninstall DocPreview 0.1 (by removing it from the /Library/Internet Plugins or ~/Library/Internet Plugins directories, depending on where you initially installed it). For those still on Safari 5.0 and below, this will still work.

Only NPAPI plugins are apparently allowed from 5.1 forward. This API does not give native access to the browser window as before; therefore the old methods of converting .doc files to HTML no longer works. Because I still need Word preview capability for myself, I have a few ideas on how to make DocPreview work for 5.1+. However, this requires a complete re-write of the entire code. Apple obviously already has .doc preview capability on iOS but hasn’t shared that with us on OS X. I’m not sure it’ll be worth it to make these modifications, only for Apple to release .doc preview as a native capability to Safari a few months later.

12/1/2010 – As promised, DocPreview is now open sourced at Google Code.

01/24/2010 – DocPreview updated for 64-bit Safari and 10.6. Still seems to work.

early 2009 – Schubert|IT’s Word Browser Plugin has been made a Universal binary. Therefore, DocPreview is no longer needed, as I originally wrote this plugin to fill in the gap when Word Browser Plugin was PPC-only. Word Browser Plugin offers a more advanced user interface than DocPreview. You should try it out first. I will continue to tweak DocPreview, more as a challenge to myself. I will probably open source the package — at soon as I fix a hack or two and no longer feel ashamed of my code.

—

DocPreview is a lightweight WebKit browser plugin I wrote to display a text-only preview of Microsoft Word .doc documents, inline and within Safari 3.x (and, apparently, 4.x) on both Intel and PowerPC Macs. This behavior is much like the functionality provided by the PPC-only Word Browser Plugin from Schubert|IT. DocPreview, of course, is a universal binary plug-in — since Word Browser’s lack of Intel support is/was really why I wrote this plug-in.

DocPreview only supports WebKit-based browsers that can use .webplugin files. Therefore, this includes Safari, Shiira, etc., but not Chrome (as of the time of this update). I am currently unable to support other browsers, since DocPreview is built against the WebKit API instead of the Netscape plug-in API. Frankly, the Netscape API is a mess, and I haven’t had the time to figure it out. A NSAPI guru who could point me in the right direction would be much appreciated.

Download
DocPreview.zip (Safari 5.0 and below on 10.5, 10.6) – v0.1
DocPreview.zip (10.4) – v0.1

This is a work-in-progress. It passes the “dogfooding” test; as in, I’m using this plug-in daily (eating my own dogfood, as they say). While I believe it functions correctly — at least on my machines, I make no guarantees as to stability and usefulness, and am not liable for blowing up your browser or any harm that might befall you through your adventurous use of this plug-in. I do want this thing to work for everyone, so please leave me a note here if it doesn’t work.

DocPreview features & limitations:

  • Universal binary support, for use on both PPC and Intel Macs. No Windows Safari support — and plus, there are already good solutions for inline doc browsing on the Windows side. If you’re on PPC, I suggest you use Word Browser Plugin instead unless you desperately need find-on-page using Safari’s built-in facilities with Word docs.
  • Tested extensively on Safari 4.x, 3.x, and somewhat with Shiira 2.x. May also work on earlier Safari versions, but I simply do not have the ability to test the plugin against them. Does not work on Chrome, but I’m sure there are better solutions using Chrome extensions.
  • Uses OS X’s internal engine for opening and processing Word files. DocPreview performs as well (or as badly, depending on your opinion) as OS X itself for the same task.
  • Supported on OS X 10.5 for .doc, .docx, and .odt files. On OS X 10.5, the plugin can parse Microsoft’s new OpenXML (.docx) and OO.org‘s ODF Text (.odt) documents. On 10.4, the plugin will still work, but only for .doc files.
  • Support for full document view mode or embedded mode (if the page author uses object and embed tags, like with Flash objects — although, who actually does that with .doc files? )
  • In full document view mode, DocPreview uses Safari’s built-in Find and Text Zoom abilities. Any command that can be run on a normal Safari webpage can be performed on a DocPreview rendering.
  • In embedded mode, Find on Page and Text Zoom support are implemented separately. I’m still thinking about how I can hook into Safari’s built-in system from within an embedded plugin. Since no one ever uses embedded .doc files anyway, the point is fairly moot for most users.


To install, drag DocPreview.webplugin into
/Users/<your name>/Library/Internet Plug-ins
or
/Library/Internet Plug-ins.

Screenshots of the plug-in in operation can be seen to the right of this post. The first pair of images are .doc files in full document mode in the browser and in Word 2008. The second pair of images are two documents embedded inline using object tags, and one of them shown in Word 2008. As you can see, the conversion fidelity is fairly decent — this is the same level of fidelity that you would have gotten by importing Word data into TextEdit, or using textutil on the command-line.

DocPreview serves the same purpose as Word Browser. It’s intended as a quick preview (much like how Google indexes the text in .doc files), so you don’t blindly download any Word files that actually don’t interest you.

( As an aside, I cannot understand people who want to disable built-in PDF support in Safari :p . That feature has singlehandedly improved my productivity/research output by a magnitude. )

I expect Apple to support .doc previews with the next version(s) of Safari, very soon (ok, so not in 4.x, but surely in 5.x. Anyone on the Apple Safari team reading this: come on, this feature is trivial — even I can do it ). MobileSafari on the iPhone already provides .doc viewing support (and .xls, if I remember correctly). On Windows, MS provides read support for .doc files within the browser. DocPreview is a temporary solution for Safari users until official Apple support (or better yet, better Microsoft Office integration) arrives.

If the thing doesn’t work for you, let me know via the comment thread. If it does work, let me know too. Any suggestions and comments welcome.

redhillonrails_core and broken MySQL empty string defaults

Posted on by yiming

While hacking on a side project in Ruby on Rails, I ran across this weird error when trying to insert new data:

ActiveRecord::StatementInvalid: Mysql::Error: Column 'attr2' cannot be null: INSERT INTO `foo` (`attr1`, `attr2`) ... VALUES ('1', NULL)

where attr2 is a varchar (or t.string, in Rails lingo) and set to not null default '' (or, in other words, :null=>false, :default=>''). Strangely enough, instead of the default value of ”, ActiveRecord was setting the value to nil instead, which translates into a NULL. Since the schema explicitly forbids NULLs on that column, the statement explodes.

After an hour of poking around and hacking up a spike solution, it turns out a plugin was to blame. I’d pulled in the foreign_key_migrations plugin (a highly recommended add-on) to automatically install foreign key constraints (in this day and age, the foremost web framework still can’t automatically handle FOREIGN KEY constraints, the most basic tool for ensuring data integrity in relational databases, for its migrations? Bah!).

This plugin has a dependency on redhillonrails_core, which has a known bug: Incorrectly overwrites mysql empty-string default with nil for string/text/binary types.

The bug is apparently not being worked on as of the time of this writing. The dev doesn’t consider this a bug, as he claims that “[he considers] empty strings to be semantically identical to NULL”.

This position, unfortunately, is not supported by the SQL standard. Wikipedia has a section on common SQL NULL mistakes documenting some of the potential problems involved in making such an assumption. Philip Greenspun has further notes on this. Using NULLs trigger the all the arcane annoyances of three-valued logic, and you must be prepared to consider True, False, and NULL values as comparison outcomes. Someone not very versed in three-valued logic can easily cause a number of subtle mistakes trying to compare values.

To workaround this bug, you will need to comment out the initializer function in

vendor/plugins/redhillonrails_core/lib/red_hill_consulting/core/active_record/connection_adapters/mysql_column.rb

Alternatively, you can delete the file, and remove relevant references to it..

If you agree with dev’s position that NULLs are “semantically identical” to empty strings, remember to pay attention when you formulate your SQL queries (and when Rails formulates those queries) — your results may not be what you expect, if implemented naively. Get your three-valued logic truth tables out 🙂

Getting custom HTTP variables out of PHP

Posted on by yiming

PHP 5.0 stores HTTP headers in the $_SERVER variable as key-value pairs. It mangles their field names, however, by:

  • prepending “HTTP_” to the key
  • replacing “-” with “_” in the key
  • uppercasing all letters

Say that your custom HTTP client sends X-Hello: World as a header. To retrieve the value (e.g. “world”) from PHP, the correct key to use is $_SERVER["HTTP_X_HELLO"].

This does fit with the existing access pattern (User-Agent: is retrieved by $_SERVER['HTTP_USER_AGENT']). But it was not well documented in corresponding page for reserved variables (as of today, October 7, 2007). Took a bit of trial and error for me to figure this out.

I’m sure that amongst the insanely numerous and ill-organized set of functions that PHP provides, there is one to do this exact task without reverse-engineering its key-mangling algorithm. But this way works too.

CVS, Cygwin, and error code 0xc0000022

Posted on by yiming

In short, if your project crashes at library load time after a round trip through CVS, you might want check your NTFS execute permissions on the DLLs that the project depends on. Also, if your application mysteriously blows up with error code 0xc0000022, you’d do well to make sure that:

  • all DLLs that your program depends on are valid and locateable.
  • Check all its DLL dependencies for permission problems. As in, permissions on the DLLs that your program depends on should be set to be executable for your user.

In one of my Windows projects, I wrote some code that relied a number of DLLs. To save myself sometime, I compiled these DLLs and checked in the compiled binaries into the CVS repository.

On another machine, I checked out the project via the cvs utility, under Cygwin, to work on it. As a Unix-y kind of guy, I prefer the tools that I’m used to. Everything compiled fine, but at runtime the application crashes before it gets to main(). ” The application failed to initialize properly (0xc0000022) … ” After some dependency tracking to find out if I lost a DLL somewhere, first via Dependency Walker, then via gflags, nothing unusual turned up.

Then I noticed that replacing the checked out libraries with fresh copies of the same DLLs fixed the issue. The problem was that upon checking md5 sum against the old and new libraries, they were exactly the same. There was no damage or corruption.

Turns out, of course, that Execute permissions were off on all those DLLs that I checked out. Apparently Cygwin’s cvs does not set execute bits on DLL files, and since you’re usually using ntsec settings with Cygwin, this causes a security/permissions problem on the Windows side. As a result, the project compiles just fine, fails at runtime, and gives you a completely obtuse error message that means very little unless you’ve done this sort of thing before. Cygwin and cvs’s role in this was also not a very obvious thing to deduce.

Two hours of my life, right there.

duplicates and invoking LaunchServices

Posted on by yiming

If you deal with duplicate files a lot, then fdupes is a very quick command-line solution. Among other things, it also does an MD5 comparison of file signatures, can recurse directories and following symlinks, and can be used for scripting as well as directly from standard input.


Bonaparte-Prime:/tmp $ fdupes -r -d ./
[1] ./foo.mp3                 
[2] ./bar/baz.mp3

Set 1 of 4, preserve files [1 - 2, all]:

However, sometimes I forget what the hell foo.mp3 and bar/baz.mp3 really were. Then I have to fire up the Finder (or another Terminal window) and go chase down the file. What I really wanted would be if there were an “open one of these files and let me see what it is, then I’ll make a decision” option.

The beauty of open source is that I can patch fdupes myself and scratch this itch.

Finding the right app to hand off to might be annoying. On OS X, if you want to use the nice automagical way that Finder opens files (meaning that it generally knows what applications are appropriate to open an arbitrary file), you’ll have to link against LaunchServices, a part of the ApplicationServices framework. The code to do that is actually quite trivial. Simply add the ApplicationServices.h header, and then a function such as:


OSStatus open_target_file(const char* filepath)
{
    FSRef ref;
    OSStatus err;
    Boolean isDirectory;
    
    err = FSPathMakeRef((const UInt8 *)filepath, &ref, &isDirectory);
    
    err = LSOpenFSRef(&ref, NULL);
    
    return err;
}

will do the job. Obviously this is simplistic and without error-checking, but you get the idea. LaunchServices will then handle the tedium of finding the right system application to launch the file.

The other half of the patch involves modifying fdupes’s command parser to take an additional variant. I chose a scheme by which if I precede the choice numbers I make with the character ‘o’, it should then hand off the filepath to the aforementioned open_target_file function and return to its previous state. Since the parser already has such a case to handle mistaken input, the patch is also quite simple. Simply check for ‘o’ in the first byte of the preservestr array at the appropriate place, and then should it match, have it execute open_target_file() rather than preserve[number] = 1; If you’re the portability stickler type, wrap these new lines under #ifdef __APPLE__ statements to check for OS X at compile-time.

Source code to simple utilities like these are really quite helpful. They let someone with just a modicum of programming ability tweak the behavior and address some common annoyances, without having re-invent the thing from scratch or to pester the original dev, who probably doesn’t care nearly enough about little features for specific platforms that only specific people might find useful. Write the extension code, create a patch, and keep your patch around in case the utility is ever updated. Scratch your own itch as free software intended.

Exchanging objects between PHP and Python

Posted on by yiming

So over the course of my various projects, personal or otherwise, I’ve collected an assortment of information that may or may not be of interest to others or to myself in the future. What does end up happening is that I would make notes about it in a file or (dear god) on a random piece of paper, post a message to some forum or mailing list, or just plain put it in that lossy storage medium of my own mind…and then promptly forget all about it. For a would-be information specialist, this ironic lack of information organization has caused many problems and continues to do so, especially at retrieval time.

Now in 2007, as New Year’s Day draws to a close, I am putting my laziness to the test again by resolving to begin this project to document Random Things That I Somehow Know About. Some of this is trivial, some of this is not. But for one reason or another, I intend to keep track of it.

Starting with something recent. To workaround a problem when deploying a Python-based service on a server that disallowed Python CGI execution, the Python driver program had to be wrapped around a PHP frontend (which the server did allow). However, the driver needed to accept a number of parameters, and the PHP wrapping must conveniently pass these parameters via a call to system() and print the resulting output from the Python driver to stdout. In a previous, similar project, I engineered a set of subroutines in the Python driver to parse options on the command line, and had the PHP script put those options on the commandline at invocation time. It was tedious, error-prone, and remarkably insecure, even with Python variants on getopt() to help

Since this was a proof-of-concept project in any case, there had to be a faster, friendlier hack. It would be great if the wrapper and the driver could exchange data objects directly – in this case, PHP associative arrays and Python dicts. Then the Python driver can simply ask for the necessary values bound to fixed options/keys.

Enter Armin Ronacher’s phpserialize.py, conveniently under BSD license. It exposes two functions, serialize()and unserialize(), which encodes and decodes data created by PHP’s own serialize() function.

The solution comes together as follows. On the PHP side:

//something to ensure that we've got a correct parameter object
$params = array('phpArgs' => 'yes'); 
$params['foo'] = 'bar'; //check and populate $params
$s = serialize($params);
//so that if we're passing on cmdline
//things don't blow up if weird bytes encountered
$s = base64_encode($s);
//...and so on

Once the $s in base64 text is passed to the driver script, the Python side will simply call this:

def getParameters(php_base64_str):
    php_parameters = base64.decodestring(php_base64_str)
    parameters = unserialize(php_parameters)
    return parameters

 

and parameters will be a Python dictionary with all the values in place, mapped to their original keys.

>>> params = getParameters(s) # retrieve s from argv first
>>> params['foo']
'bar'

This makes exchanging data between frontend and backend a lot less headache inducing.