Category: Software

Change IP address via (ab)use of DHCP client id

Posted on by yiming

Nowadays customers using cable modems tend to be re-assigned the same IP address once they get one. This is apparently done based on the client MAC address — unless that IP was already taken from the pool when the client connects, the DHCP server will tend to re-issue the old IP. Fairly nice, in that even for “dynamic IP addresses”, the actual IP address assigned rarely changes unless you go offline for an extended period of time.

Problem comes when you actually want to change to a new IP, and can’t wait around for days without a network connection. Usually this is accomplished by changing the MAC address on your machine or router, in a process called spoofing or cloning. Most routers will also have a MAC cloning feature. Swap out the MAC address, and you will be assigned a new IP the next time you connect to the DHCP server.

Except, of course, the Apple AirPort and AirPort Express routers don’t actually have this feature. This is understandable, considering that MAC addresses are supposed to be globally unique so as to avoid network problems — allowing the user to change it willy-nilly will probably not serve that goal. Nevertheless, it is quite annoying in this case, as you won’t be able to change IP with an AirPort router using the MAC cloning method.

There’s one trick that might work for you here, though. A few ISPs’ DHCP servers are set up to accept DHCP Client IDs — an optional field that identifies a DHCP client. While they default to use MAC address as a client identifier by default, they will treat you as a different DHCP client for IP assignment if you start using or change a Client ID. All that is required to obtain a new IP address, then, is to add a Client ID or change it. There is an interface for this in the AirPort Utility, as seen in the screenshot. This is far easier than trying to clone a MAC address on AirPort routers.

Not all ISPs support use of the DHCP Client ID, so this may not work for everyone. Since there are uniqueness requirements involved with these client IDs, it is a good way to screw up DHCP assignments if two clients claim the same ID. If your ISP does support this, make sure to pick a unique client ID.

This trick appears to work for Cox Cable at my current location. It does not appear to work for Comcast.

Bad Google cookie kills Safari

Posted on by yiming
03-10-2010: I believe this is fixed in latest Safari versions. The contents of this post remain for historical purposes only.

In a bizarre case of digital food poisoning, I experienced a series of mysterious, persistent, reproducible crashes with Safari 3.2.1 this morning, traceable to a bad Google cookie.

The symptoms

Google has a nifty query suggestion feature that is turned on by default on its homepage search box. Whenever I typed in a phrase query (e.g. +"query suggestion" +"Google features") with the suggestion feature turned on, the browser crashed with a SIGSEGV around 30% of the time.

Excerpt from the crash log:

Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x000000001bdca240
Crashed Thread:  0

Thread 0 Crashed:
0   ???                            0x16619e75 0 + 375496309
1   com.apple.WebCore              0x94325ea0 WebCore::AutoTableLayout::fullRecalc() + 704
2   com.apple.WebCore              0x9432581a WebCore::AutoTableLayout::calcPrefWidths(int&, int&) + 26
3   com.apple.WebCore              0x943252b8 WebCore::RenderTable::calcPrefWidths() + 56
4   com.apple.WebCore              0x9431c04b WebCore::RenderBox::minPrefWidth() const + 27
5   com.apple.WebCore              0x9432507c WebCore::RenderTable::calcWidth() + 124
6   com.apple.WebCore              0x943241a8 WebCore::RenderTable::layout() + 392
...

In the remainder of the cases, when it does not crash immediately, a JavaScript error is logged to the browser error console (to access, go to Develop -> Show Error Console)
SyntaxError: Invalid regular expression: nothing to repeat
http://www.google.com/extern_js/f/CgJlbhICdXMrMAc4AiwrMAo4EywrMA44AywrMBg4Ayw/nMD0sKnpeG0.js (line 21)
for every letter that I type into the search box. During this time, no query suggestion is made.

Diagnostics

  • I have never used an InputManager or “plug-in” to Safari
  • The same crash does NOT happen under a fresh new user account created for diagnostic purposes
  • Clearing the browser cache, temp files, hidden cache files ( getconf DARWIN_USER_CACHE_DIR ), etc. did not help.
  • Deleting Safari preferences did not help.

Solution

After applying a divide-and-conquer strategy to the entire ~/Library directory (not made any easier by Finder’s obstinate resistance to my attempt to move subdirectories within the Library directory, despite having the appropriate permissions — had to drop to Terminal for this), I traced it to the ~/Library/Cookies directory. Moving away the Cookies.plist file contained within cured the crash, the lack of query suggestions, and the Javascript error. More specifically, deleting all Google-related cookies within the Cookies file also accomplished the same thing.

Remarks

Some combination of a bad cookie and bad regexes appears to have triggered a crash bug in this version of WebKit / WebCore. You wouldn’t think a bad cookie could take down a browser. But apparently it does.

I dearly hope this is not a potential buffer overflow or other security problem within WebKit.

Email servers and the MAIL FROM syntax

Posted on by yiming

If you’re into chatting with SMTP servers via telnet ( only for debugging purposes, I swear :p ), be aware that some new email servers strictly interpret RFC 2821 and appears to reject MAIL FROM and RCPT TO addresses of the form [email protected]. Instead, it will return a 555 5.5.2 Syntax error. The proper solution is to enclose the address brackets, as in <[email protected]>.

A redacted transcript:

HELO example.com
250 Hello
MAIL FROM: [email protected]
555 5.5.2 Syntax error.
MAIL FROM: <[email protected]>
250 2.1.0 OK
RCPT TO: [email protected]
555 5.5.2 Syntax error.
RCPT TO: <[email protected]>
250 2.1.5 OK
DATA

....

I’m sure this is intended so that clients that send “Joe Schmoe <[email protected]>” are correctly interpreted. I’m not sure if the RFC asks for this strict enforcement (on my admittedly cursory reading), though and yet a fair number of servers are rejecting the alternate address-only syntax. Wacky.

redhillonrails_core and broken MySQL empty string defaults

Posted on by yiming

While hacking on a side project in Ruby on Rails, I ran across this weird error when trying to insert new data:

ActiveRecord::StatementInvalid: Mysql::Error: Column 'attr2' cannot be null: INSERT INTO `foo` (`attr1`, `attr2`) ... VALUES ('1', NULL)

where attr2 is a varchar (or t.string, in Rails lingo) and set to not null default '' (or, in other words, :null=>false, :default=>''). Strangely enough, instead of the default value of ”, ActiveRecord was setting the value to nil instead, which translates into a NULL. Since the schema explicitly forbids NULLs on that column, the statement explodes.

After an hour of poking around and hacking up a spike solution, it turns out a plugin was to blame. I’d pulled in the foreign_key_migrations plugin (a highly recommended add-on) to automatically install foreign key constraints (in this day and age, the foremost web framework still can’t automatically handle FOREIGN KEY constraints, the most basic tool for ensuring data integrity in relational databases, for its migrations? Bah!).

This plugin has a dependency on redhillonrails_core, which has a known bug: Incorrectly overwrites mysql empty-string default with nil for string/text/binary types.

The bug is apparently not being worked on as of the time of this writing. The dev doesn’t consider this a bug, as he claims that “[he considers] empty strings to be semantically identical to NULL”.

This position, unfortunately, is not supported by the SQL standard. Wikipedia has a section on common SQL NULL mistakes documenting some of the potential problems involved in making such an assumption. Philip Greenspun has further notes on this. Using NULLs trigger the all the arcane annoyances of three-valued logic, and you must be prepared to consider True, False, and NULL values as comparison outcomes. Someone not very versed in three-valued logic can easily cause a number of subtle mistakes trying to compare values.

To workaround this bug, you will need to comment out the initializer function in

vendor/plugins/redhillonrails_core/lib/red_hill_consulting/core/active_record/connection_adapters/mysql_column.rb

Alternatively, you can delete the file, and remove relevant references to it..

If you agree with dev’s position that NULLs are “semantically identical” to empty strings, remember to pay attention when you formulate your SQL queries (and when Rails formulates those queries) — your results may not be what you expect, if implemented naively. Get your three-valued logic truth tables out 🙂

APR and 32-bit/64-bit universal binary compilation

Posted on by yiming

When compiling APR, the Apache Portable Runtime 1.3.3 (as a part of Subversion 1.5.3 as I am doing here, or not) on OS X 10.5 Leopard, you may encounter the following error at compile time.

/bin/sh /tmp/subversion-1.5.3/apr/libtool --silent --mode=compile gcc-4.2 -Os -arch i386 -arch x86_64 -DHAVE_CONFIG_H -DDARWIN -DSIGPROCMASK_SETS_THREAD_MASK -no-cpp-precomp -I./include -I/tmp/subversion-1.5.3/apr/include/arch/unix -I./include/arch/unix -I/tmp/subversion-1.5.3/apr/include/arch/unix -I/tmp/subversion-1.5.3/apr/include -o strings/apr_snprintf.lo -c strings/apr_snprintf.c && touch strings/apr_snprintf.lo
strings/apr_snprintf.c: In function ‘conv_os_thread_t’:
strings/apr_snprintf.c:500: error: duplicate case value
strings/apr_snprintf.c:498: error: previously used here
strings/apr_snprintf.c: In function ‘conv_os_thread_t_hex’:
strings/apr_snprintf.c:671: error: duplicate case value
strings/apr_snprintf.c:669: error: previously used here

This will most likely happen when you are configured to build a dual 32-bit / 64-bit universal binary, whether it be ppc / ppc64, or i386 / x86_64, or any permutation thereof. This ticket over at MacPorts documents a particular instance of this problem, with no apparent solution.

The symptom is easy to explain. Somehow, two case labels in the relevant switch statement in strings/apr_snprintf.c:500:

switch(sizeof(u.tid)) {
    case sizeof(apr_int32_t):
        return conv_10(u.u32, TRUE, &is_negative, buf_end, len);
    case sizeof(apr_int64_t):
        return conv_10_quad(u.u64, TRUE, &is_negative, buf_end, len);
    default:
        /* not implemented; stick 0 in the buffer */
        return conv_10(0, TRUE, &is_negative, buf_end, len);
    }

have evaluated to the same value. In particular, it believes that sizeof(apr_int32_t) and sizeof(apr_int64_t) are the same value. As we all know in C, you cannot have two identical case labels in the same switch statement. However, the root of the problem is a bit more subtle.

In $SRCDIR/include/apr.h, you’re likely to see this fragment of code.

typedef  long       apr_int64_t;
typedef  unsigned long  apr_uint64_t;

Notice that it has typdef’ed apr_int64_t as a long and apr_uint64_t as unsigned long. This is because at configure time, the script detected that long values are 64-bit on this system, so it assigned the apache 64-bit types to longs. However, this only holds true for half of the compilation – because you are building a universal binary for a 32-bit architecture as well. Remember that in 32-bit GCC on OS X, longs are 32-bit rather than 64-bit. Your run-of-the-mill autoconf script, done by a non-OS X programmer, isn’t going to be able to detect this subtlety – if the 64-bit part worked, it’ll keep thinking longs are 64-bit, end of story – and happily generate the incorrect typedef expressions. When you apply sizeof to these types in apr_snprintf.c, both evaluate to 4 bytes under 32-bit compilation, thus blowing up the compile run.

To truly fix the root of the problem requires rewriting the autoconf script to detect Mac OS X and its universal binary building, which can potentially throw quadruple architectures at the same compilation script. However, a quick hack to make this particular problem go away is to change apr.h such that:

typedef  long long       apr_int64_t;
typedef  unsigned long long apr_uint64_t;

Now that we ensure in either 32-bit or 64-bit compilation, apr_int64_t and apr_uint64_t are always typedef’ed to appropriate, guaranteed 64-bit types. The compilation of APR (and Subversion) will proceed normally.

Note that long long is not an standard C type. As a GCC extension, this fix is a kludge. A kludge that works (for me), though.

UPDATE:
There may also be an issue with sizeof definitions that may cause the library to crash. In particular, there may be occurrences of

#define APR_SIZEOF_VOIDP 8

that were generated by configure. To fix this, you will need to remove the define and have the compiler check for 64-bit at compile-time:

#ifndef __LP64__
    #define APR_SIZEOF_VOIDP 4
#else
    #define APR_SIZEOF_VOIDP 8
#endif

In general, any predefined sizeofs need to be changed. I am not sure why the APR developers do hard-coded defines like this, given that the point of having sizeof() calls is to avoid such issues.

using ffmpeg for cutting media files – and the gotchas involved

Posted on by yiming

So here I was, trying to complete some ordinary transcoding / media file cutting tasks with ffmpeg. Turns out there are some weird gotchas when using some versions of ffmpeg (in my case, SVN-r10571) on the commandline.

Problem 1: transcoding to mp3 insists on using 64kbits/s or 128kbits/s bitrate

So a simple task is to transcode an audio file of some arbitrary format to MP3. In my case, I only wanted a 30-second piece of the original file, converted to MP3. The original was in 256kbits/s, and I decided to use the same bitrate for the output, just for kicks. Reading the man page, -b is for video bitrate, and -ab is for audio bitrate. So I executed:

wrong:  ffmpeg -ss 00:00:30.00 -t 25 -i foo.mp3 -ab 256 foo-new.mp3

The output, of course, was this:

Input #0, mp3, from 'foo.mp3':
...
mdb:109, lastbuf:0 skipping granule 0
size= 393kB time=25.2 bitrate= 128.0kbits/s
video:0kB audio:393kB global headers:0kB muxing overhead 0.007950%

128 kbits/s. That was…not what I wanted.

Turns out, as the ffmpeg man page hints but does not specify clearly, a “k” is required to label the units of the new bitrate. As in:

right:  ffmpeg -ss 00:00:30.00 -t 25 -i foo.mp3 -ab 256k foo-new.mp3

This time, the output was:
Input #0, mp3, from 'foo.mp3':
...
mdb:109, lastbuf:0 skipping granule 0
size= 782kB time=25.0 bitrate= 256.0kbits/s
video:0kB audio:782kB global headers:0kB muxing overhead 0.003996%

Much better. It’s very strange how that if I don’t give it the ending “k” but do give it a higher value than 64, it always bumps up the bitrate to 128kbits/s, from the default of 64k (but not to the number I actually wanted).

 

Problem 2: cutting media file without re-encoding

So ffmpeg can be used to cut a media file, without reencoding the media stream. You simply pass the raw copy codec to the -acodec (or the -vcodec for video) at encode time. For my MP3, I thought it was a pretty trivial problem, so I issued:

wrong: ffmpeg -i bar.mp3 -ss 00:00:30.00 -t 25 -acodec copy bar-new.mp3

This seemed to process correctly, except it created a 555-byte empty file with no content in it. What’s even weirder, if you issued:

wrong: ffmpeg -i bar.mp3 -ss 00:00:10.00 -t 25 -acodec copy bar-new.mp3

That is, -ss 00:00:10.00 to seek to the 10th second, and -t 25 to record 25 seconds worth of audio. Strangely enough, the output file had 15 seconds of audio, the subtraction of 25 by 10. Curiouser and curiouser. Now,

wrong: ffmpeg -i bar.mp3 -ss 00:00:10.00 -t 25 -acodec mp3 bar-new.mp3

does in fact create a 25-second MP3 file. Only, of course, a re-encoded one at 64kbits/s. Something that we are not looking for. But strange.

A post on ffmpeg-devel finally provided enough hints to clue me into the problem. Unlike many command line apps, the order of arguments passed to ffmpeg seem to be silently significant. The correct incantation is:

right: ffmpeg -ss 00:00:30.00 -t 25 -i bar.mp3 -acodec copy bar-new.mp3

Note the switched order for the arguments -i and the args -ss and -t, where the -i must follow the other two. Now, the desired 25-second file, cut from the original, is correctly produced.

When arguments are missing some arbitrary text or in the wrong order, ffmpeg doesn’t sanity-check or warn you of these…it just silently proceeds and does some very strange things. Things that make you scratch your head and wonder, “wtf? did I mistype an argument somewhere?”

Ah, and of course, you make me have to track down all these little idiosyncrasies, wading through blog posts and mailing lists and forums. Agh.

Subversion and undefined symbols

Posted on by yiming

This is fast turning out to be a blog about compiling open source software. Maybe I should change the title.

You may get this error while trying to compile Subversion 1.5.2:
ld: Undefined symbols:
_svn_fs_txn_root_base_revision referenced from libsvn expected to be defined in libsvn
_svn_fs_change_txn_props referenced from libsvn expected to be defined in libsvn
_svn_fs_get_mergeinfo referenced from libsvn expected to be defined in libsvn
_svn_fs_recover referenced from libsvn expected to be defined in libsvn
_svn_fs_upgrade referenced from libsvn expected to be defined in libsvn
_svn_fs_node_origin_rev referenced from libsvn expected to be defined in libsvn
/usr/bin/libtool: internal link edit command failed
make: *** [subversion/libsvn_ra_local/libsvn_ra_local-1.la] Error 1

Turns out there may be a bug in the libtool script shipped with the source tarball, under certain circumstances with OS X 10.4, that causes it to fail to link. Ah, libtool, you make my life so much harder sometimes.

If you copy /usr/bin/glibtool over $SRCDIR/libtool (that is, into the directory for the subversion source code, replacing the one that is placed there by the package itself), the package should compile with no further complaints. make test also shows success on all tests, so this seems a satisfactory solution.

UPDATE 1/31/2009
I’ve been informed in the comments that MacPorts may rely on source compilation from tarball, and thus have this issue. If you’re having this issue with the MacPorts SVN package, please check out the advice in the comment section from farkinga, who has additional notes.

GNUMake 3.80 and failed malloc

Posted on by yiming

Wacky compilation error:

Creating config.mak and config.h...
make(4162) malloc: *** vm_allocate(size=4272971776) failed (error code=3)
make(4162) malloc: *** error: can't allocate region
make(4162) malloc: *** set a breakpoint in szone_error to debug
make[1]: *** virtual memory exhausted. Stop.

This error has nothing to do with the particular code you’re compiling. Turns out GNU Make 3.80 has a bug, in which it sometimes sends a ridiculous requested block size to malloc(), which then fails to allocate the space.

The solution is to upgrade to GNUMake 3.81. Particularly pernicious on OS X 10.4 and Xcode 2.4.1.

AVCodecContext, Perian, and the Xcode dylib fetish

Posted on by yiming

The Context

So recently I was trying to interact with an avi file with a video stream encoded in H.264 and an audio stream encoded in MP3. To my surprise, my trusty Perian 1.1 failed to even play the video and gave me a black screen (though the audio played just fine). With some Google-fu, it was found in a forum post that the SVN version of Perian fixed this error.

So compile my own Perian 1.1.1b1 from SVN, right? No big deal. Heh. And thus begins our story. (Note that this procedure is accurate as of the time of writing. OSS projects change rapidly as bugfixes are checked in. You may not have this problem at all. But it’s still useful as a reference in the future, in case similar problems arise).

Problem 1: AVCodecContext.bits_per_sample

The first problem you hit is probably:
/tmp/perian-orig/ff_private.c:181: error: ‘struct AVCodecContext’ has no member named ‘bits_per_sample’
…
/tmp/perian-orig/FFissionCodec/FFissionDecoder.cpp:289: error: ‘struct AVCodecContext’ has no member named ‘bits_per_sample’

Turns out that the upstream project FFMPEG, who is responsible for libavcodec, has very recently changed the API in one of its latest revisions. bits_per_sample has become
bits_per_coded_sample in revision 15262. Because Perian’s SVN does an external SVN checkout of the HEAD of libavcodec, it pulled these API-breaking changes without considering whether that breaks its own compilation or not.

The fix is simple enough: global search and replace the member variable to bits_per_coded_sample.

Problem 2: Xcode’s unnatural fondness for dynamic libraries

At this point, libavcodec compiles, so you think you’re home free. For most users, that is probably the case. For some of us, though, there’s one more hurdle.

While compiling Perian.component, you might get something like:
ld warning: in /Developer/SDKs/MacOSX10.4u.sdk/usr/local/lib/libavcodec.dylib, file is not of required architecture
Undefined symbols:
"_avcodec_close", referenced from:
…and so on

Closer examination reveals that at some point, you might have compiled FFMPEG or MPlayer or any of the many media projects that depend on FFMPEG libraries (for me, that was transcode, whose own compilation idiosyncracies I documented in an earlier post). When you installed these libraries, they went into your library search path. Note the Perian compilation line immediately preceding the compile failure:
/Developer/usr/bin/g++ ... -read_only_relocs suppress -lavcodec -lavformat -lavutil -lebml -lmatroska ... /tmp/perian/UniversalDetector/build/Release/libuniversaldetector.a -lbz2 -o /tmp/perian/build/Deployment/Perian.component/Contents/MacOS/Perian

As you can see, the -lavcodec linker argument will cause the linker go on the search path and try to find the library. It of course first finds your pre-installed library in /usr/local/lib or some other system lib path, and tries to use it. If you compiled that library as Intel only or PPC only and you tried to build a universal Perian, that would obvious fail and generate an architecture error. If you, like me, compiled only for x86_64 or ppc64, that would also trigger the same architecture problem (as Perian builds as x86 32-bit).

This problem stumped a user “vs”, in this Perian-discuss Google groups thread. Notice how the devs didn’t get the root of the problem, and answered snarkily and defensively to the user bug report.

The devs aren’t wrong, though. Didn’t we just see a statically compiled target, libavcodec.a, in the target list for the Perian Xcode project? Why yes, yes we did! In fact, there are static targets for all of the -lav* libraries in the Xcode project. So why did the linker go off and use -lavcodec rather than, say, $PERIAN_DIR/build/Universal/libavcodec.a, as instructed?

The workaround

In short: Xcode is screwing things up. Xcode 3.1 has a distinct (some say, unnatural) preference for dynamic libraries. When you add a library to the “Link Binary With Library” listing under Targets, it simply adds a -l<libname> to the link line to pass to ld at link time. When this happens, it will look for lib<libname>.dylib (in this case, libavcodec.dylib, libavformat.dylib, libavutil.dylib, etc in your library path. If it finds one, it will use that one rather than the static library you provided in the Xcode project file. Obviously, this ends up linking the wrong library, perhaps with the wrong architecture or the wrong version. Compilation explodes.

The workarounds are not very clean. There seems to be no good way to instruct Xcode to forget its dynamic library fetish and use the static versions you provide. You could remove the dylibs from the system library path entirely — when the linker fails to find the .dylib version, it will fall back to the .a.

Alternatively, you can explicitly instruct it to use static linking as an environment flag and remove the built-in Xcode linker relationships:

  1. Right-click on your Target (in this case, Perian) and Get Info to bring up the Target Info box.
  2. Go to the Build tab
  3. Select your Build configuration – in this case, probably Deployment
  4. Find or locate OTHER_LD_FLAGS, also called “Other Linker Flags”
  5. Manually add the paths to the .a files in this field, one at a time. If you don’t know the path, right-click -> Get Info on the desired libraries under Link Binary With Library, and it should show you a field called Full Path on the General tab. That’s the path to use. Should look something like /tmp/perian/build/Universal/libavcodec.a
  6. Do the same for all offending libraries – this may include libavcodec, libavformat, libavutil, libmatroska, etc.
  7. Remove the libraries you added to Other Linker Flags from your Xcode’s Target -> Target Name -> Link Binary With Library section, to stop Xcode from auto-generating broken linker flags. This is important.

If done correctly, this should no longer find the wrong type of the library. Your compilation of Perian should proceed as normal.

The Moral of the Story

Xcode has some weird idiosyncratic ways of building its projects. It’s useful to poke at things and find out about these behaviors. Its unnatural preference for dynamic libraries over your explicit instructions is disturbing, though.

Don’t attack your users when they pose a problem, and drop the attitude of: ooo, we’re doing this for free, so fsck off, users. In both problems, the exact same attitude surfaced amongst a few of the devs. Arrogance (“we don’t have the responsibility to do anything for you”) and snarky dismissal (‘Well, there’s your problem. We don’t build libavcodec.dylib nor do we place it in /usr/local/lib.’) helps no one. No matter what you thought your build script did, the linker is heading off toward /usr/local/lib, and you better damn well find out why.

It’s obvious that there’s something peculiar going on with the link process, and the user tried to make it clear multiple times (“You missed my point. We build a .a file and we don’t put it in /usr/local/lib.”), but no one seems to pay attention. Following that thread, the user is increasingly frustrated and expresses that, and now the dev’s hammer of arrogance comes down hard ( “You looked like a prick here, to be honest.” — really? not your obtuse-ness or your blindness to a user need?)

When you have a “works-for-me” situation, it’s especially important to communicate effectively with the user. In most cases, you have no idea why it blows up for the user and not for you. Admitting it is fine — could be a very localized issue for that user, or could reveal a subtle systemic problem in your code – either way, it’s useful to have that established. I realize we’re all very busy people, but at the end of the day it’s still your project. Either acknowledge you have no idea what’s going on and solicit help and cooperation with the user who’s having this issue, or man up, put on your detective hat, and fix your own damn problems. The rest of us are busy with our own issues.

The Free Software culture seems to breed this sort of contempt for the user in some devs. Might be a paper in analyzing how corrosive this can be.

UPDATED: I certainly do not mean this last section as a criticism for all Free Software devs. The vast majority of people I’ve talked to in the various communities are really knowledgeable, helpful folks. There are just a few people in some cases, though, who seem to think that if the user can’t make the software work, it’s somehow the user’s fault. It’s a joint problem to be solved together, not “you’re an idiot if you can’t make it work”.

UPDATED 2 (9-29-2008): Perian 1.1.1 has been released. This should fix the original problem with H.264 and .avi files that prompted this post.

transcode 1.0.5 invalid immediate error on Intel OS X

Posted on by yiming

If you’re compiling transcode 1.0.5 from source on OS X Intel (for some odd reason, as I’ve just had to do on 10.5.4), your compile run may blow up in aclib with:


tcmemcpy.c:30:missing or invalid immediate expression `0b111' taken as 0
tcmemcpy.c:30:suffix or operands invalid for `and'
tcmemcpy.c:42:missing or invalid immediate expression `0b1000' taken as 0
tcmemcpy.c:42:suffix or operands invalid for `test'
tcmemcpy.c:52:missing or invalid immediate expression `0b1111' taken as 0

The problem here appears to be that the code in tcmemcpy.c under x86 and x86_64 relies on some inline assembly features not supported by all compiler/assemblers. Namely, the use of 0b (binary immediate) operands in assembly instructions is not apparently supported during compilation under Apple gcc and gcc-4.2.

Setting the CCAS flag did not seem to help. A quick fix, then, is to convert all binary immediates in the asm instructions to hex or decimal.

Thankfully, these operands only existed in tcmemcpy, so there’s not too much work. Look for 0bxxxx immediates and convert them to their hex or decimal equivalent. For example, 0b111 is 0x7, 0b1000 is 0x8, 0b1111 is 0xf. Thus, a line like:
and $0b111, %%eax # ... which is the number of bytes to copyn

becomes
and $0x7, %%eax # ... which is the number of bytes to copyn

Once these operands are modified, the compiler should no longer complain about invalid immediate expressions.

In any case, the latest CVS version of transcode should have resolved these problems already. The workaround should only be necessary if you still must compile the release tarball for 1.0.5.

Updated: There is a second potential problem if you were using ffmpeg-SVN and transcode 1.0.5, in which its hard-coded include lookup for avcodec.h always looks for ffmpeg/avcodec.h. The SVN version of ffmpeg has moved these headers to libavcodec/avcodec.h. A path patch for transcode 1.0.5 and ffmpeg is available, which basically boils down to changing the #include headers and the configure.in file to look for the new path.

Updated: A third problem has now cropped up, with the ffmpeg API change that moved AVCodecContext’s bits_per_sample to bits_per_coded_sample. A global search and replace seems to work for now. See this also in my post on compiling Perian.