Linux brew won't turn your dependency hell into a package management heaven, but it might turn it into a tolerable purgatory.

Installing software on Linux without root : managing packages in user space

First step : don’t. Reconsider. There really isn’t an alternative ? Using something else that is already installed ? Sweet-talking your sysadmin into doing the installation ? Giving up that particular task ? Giving up Computer Sciences altogether and moving to the country to raise pigs ?

Ok, so you really don’t have an alternative. May the gods have mercy on your soul, because Linux won’t. By necessity, this won’t be a step-by-step guide, because each system has its quirks. I’m not promising you heaven, just a tolerable purgatory instead of a living hell.

Take a deep breath.

(And as always : follow those instructions at your own risk. If you turn your Linux box into a Linux brick, or into Skynet, I’m not liable.)

The problem : dependency hell

There’s a reason why installing software from sources is so painful : dependencies. Sure, you just want to install Caffe, or Torch7, or Theano. But Theano needs python, python needs openssl, openssl needs… it’s endless.

High-level package managers like apt-get and yum are so popular because they deal with those. When installing from source, you’re on your own.

But here’s the catch : when installing from sources, you can almost always relocate the software to your ~home, bypassing the need for root access. High-level package managers, at least the current generation, can’t relocate.

Except for Homebrew.

The strategy : Linuxbrew

Homebrew was created as “the missing package manager for OS X”, and is required to do anything interesting on a Mac. It was designed around two principles : installation at the user home, and installation from sources.

Say that again ? Installation at the user home, without need for root. From sources. Wow. If only there was a version for Linux ! Enter Linuxbrew. Homebrew concept was so successful that, in an ironic turn, it’s now becoming “the missing package manager for Linux”.

So, case closed ? Hardly. To start, Linuxbrew has dependencies of its own, and you have to take care of those by hand. Then, the software you want to install has to be available as a brew “formula” (but the list is quite comprehensive, and growing). Finally, it doesn’t always goes smoothly. Linuxbrew is a much bumpier ride than Homebrew/OS X, at least for now. Most formulas will install without issue, but a good 20% will require tweaking, googling, and deciphering forum posts.

The strategy is most definitely not user-friendly. But contrarily to installing each and every package by hand it is just user-bearable enough to be feasible. If you really need the software. (Are you sure you don’t prefer the pig farm ?)

Okay, you are sure.

Our strategy will be :

  1. Ensuring Linuxbrew dependencies ;
  2. Installing and configuring Linuxbrew ;
  3. Using Linuxbrew to install the desired software… ;
  4. …or if you’re unlucky, using Linuxbrew to install the desired software dependencies, and then installing the desired software by hand.

The tatics

Installing Linuxbrew dependencies

Linuxbrew has, fortunately, few dependencies : Ruby, GCC, Git, and… Linux (duh !). It runs on x86 or 32-bit ARM platforms. If you’re running Linux in other architectures this is your cue to break down sobbing.

Most systems will, fortunately have those dependencies already installed. You can check the minimal versions currently required by Linuxbrew, and then check the versions installed at your system (and whether they are installed at all) calling the commands with --version :

$ ruby --version
ruby 1.8.7 (2011-12-28 patchlevel 357) [x86_64-linux]

$ gcc --version
gcc (SUSE Linux) 4.3.4 [gcc-4_3-branch revision 152973]
Copyright (C) 2008 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ git --version
If 'git' is not a typo you can run the following command to lookup the package that contains the binary:
command-not-found git
-bash: git: command not found

Linuxbrew won’t turn your dependency hell into a package management heaven, but it might turn it into a tolerable purgatory.

Linuxbrew won’t turn your dependency hell into a package management heaven, but it might turn it into a tolerable purgatory.

As you see, I got almost lucky. Ruby and GCC are good to go, but I’ll have to install Git (and its dependencies).

The trick is being minimalist. Many packages have optional features : extra commands, documentation, GUI interfaces, etc., that will fire zillions of extra dependencies. Get rid of those optionals as much as possible.

I recommend a top-down plan of attack. Start with the package you want to install and try to configure–make–install it. If it breaks (and it will, it will), find out which dependency/configuration is missing and correct it. Do this recursively, depth-first, until you get everything installed.

Resist perfectionism. You might spend a lot of time smoothing out every wrinkle of package-1.73.03 just to find a bit later that it breaks your installation and has to be removed to make room for package-1.73.04. This is war, kid : not a time for bells and whistles. Once you get a dependency working, move on to the next.

In more detail, each cycle will consist of :

  1.  Finding, downloading, and unpacking the source package ;
  2.  Configuring the package to work with a $HOME prefix ;
  3. Building the package ;
  4. Installing the package.

Step 1 is usually trivial after bit of googling. If your Linux distribution is Debian-based, you might be able to use a single command-line operation :

apt-get source git

There are similar facilities for other high-level package managers.

Otherwise, you might download either a compressed source file, or even the bleeding edge version from the source repository (sourceforge, github, etc.) In the case of Git, this would be at https://github.com/git/git. (Be careful, because those hyper-up-to-date versions might be unstable.)

Step 2 varies a bit from package to package, but usually consists in calling a ./configure script. Sometimes pre-configuration is involved : a call to make configure or make config, or another script, e.g., ./buildconf. Sometimes it involves cmake (cross your fingers for having autoconf/automake already installed). Sometimes there’s no step 2, all options being passed directly to make during step 3. It varies.

How will you know ? Try to locate a INSTALL.* or README.* file. Usually the instructions are there. No luck ? Try browsing the official website of the package for installations instructions. Googling <package> installation instructions usually will point you to the right direction.

For git, this will work :

cd git-2.1.4/
./configure --prefix=$HOME

Well, sort of. It will probably break, because one or more dependencies will be missing. Install those (and their recursive dependencies) and try again.

Step 3 is almost always :

make

or sometimes :

make all

Sometimes this is the moment when things break down for lack of dependencies (or wrong versions, or wrong settings, or the universe showing its lack of care). Sometimes the --prefix=$HOME option comes here instead of Step 2.

Step 4 is almost always :

make install

If you set the prefixes right, that will automagically put everything in the right place, under your ~home directory. And you won’t need root permissions.

Got it ? Good. I hope you enjoy typing command-line commands : you’ll be doing it all day. For extra enjoyment, get a second monitor and close the shades.

Installing Linuxbrew

Once you have all dependencies working, installing Linuxbrew itself is a breeze :

git clone https://github.com/Homebrew/linuxbrew.git ~/.linuxbrew

Aaand… that’s it. It won’t work immediately because you have to set the paths (see below). After you do it you can simply type :

brew install $WHATEVER_YOU_WANT

And it should take care of everything.

Before you do it, however it is a good idea to call

brew doctor

and check if everything is ok. Again, be minimalist : you don’t have to correct every tiny issue. Take a good look and make the smallest needed intervention.

Linuxbrew comes ready with a lot of recipes for installing packages, or as they call, formulas. You can keep them up do date by typing

brew update

Depending on what you want to install, however, you’ll need extra formulas. In Homebrew/Linuxbrew parlance this is called tapping. For example :

brew tap homebrew/science

will install a lot of new formulas related to science, data analysis, etc.

PATH configurations

Both phases (manual dependency installations; Linuxbrew operation) won’t do you much good if your paths aren’t configured. There are at least four important paths, maybe more depending on your setup : executables (PATH), static libraries (LIBRARY_PATH), dynamic libraries (LD_LIBRARY_PATH), and include files (CPATH).

The usual place to set up those is your your shell configuration file. The examples below assume you’re using bash. If that’s your case, decide whether .bashrc or .bash_profile is better for you (usually it’s the former).

During the manual installation of dependencies add the following lines :

# Manually installed packages
export PATH="$HOME/bin:$PATH"
export LIBRARY_PATH="$HOME/lib:$LIBRARY_PATH"
export LD_LIBRARY_PATH="$HOME/lib:$LD_LIBRARY_PATH"
export CPATH="$HOME/include:$CPATH"

During Linuxbrew operation put those additional lines :

# HomeBrew / LinuxBrew
export HOMEBREW_PREFIX="$HOME/.linuxbrew"
export PATH="$HOMEBREW_PREFIX/bin:$PATH"
export LIBRARY_PATH="$HOMEBREW_PREFIX/lib:$LIBRARY_PATH"
export LD_LIBRARY_PATH="$HOMEBREW_PREFIX/lib:$LD_LIBRARY_PATH"
export CPATH="$HOMEBREW_PREFIX/include:$CPATH"

Remember that shell configurations are not effective immediately, only on the next start. You don’t have to reboot the system : simple closing and reopening the terminal, or logging out and back in suffices.

An ugly ugly ugly workaround

During my installations, I faced an issue with CA certificates that I could not bypass. Many formulas would refuse to proceed, stopping during download with the error : “cURL error 60: SSL certificate problem: unable to get local issuer certificate”.

Yes : I tried downloading updated certificates from Mozilla Corp. Yes : I checked my curl-config --ca. Yes : I tried reinstalling cURL. And Git. And OpenSSL. I spent, litteraly, hours trying to solve the problem in an elegant way.

I concede defeat. Here’s the very inelegant solution. Be aware that it opens your package manager to man-in-the-middle attacks. That is more than a theoretical risk : it has been done. This is a huge security hole. If you decide to apply it, don’t do it preemptively, wait to see if you’ll actually get the SSL certificate problem.

So you got the error, and you’re willing to expose your neck to the wolves ? Sucks to be you. Open the file download_strategy.rb at ~/.linuxbrew/Library/Homebrew and find the lines below

# Curl options to be always passed to curl,
# with raw head calls (`curl -I`) or with actual `fetch`.
def _curl_opts
  copts = []
  copts << "--user" << meta.fetch(:user) if meta.key?(:user)
  copts
end

Change line four to

# Curl options to be always passed to curl,
# with raw head calls (`curl -I`) or with actual `fetch`.
def _curl_opts
  copts = ["-k"] # Disable certificate verification
  copts << "--user" << meta.fetch(:user) if meta.key?(:user)
  copts
end

And that’s it. You’re ready to proceed installing source packages. And to be a victim of cyber mafias, and of terrorists, and of tyrannical governments.

(Note to security people : if your watertight security solution makes a system unusable, guess what will happen ?)

Extra tips

First and foremost, source code relocation is not a panacea. Some things require root access, for example, driver installations, kernel recompilations, boot sector modifications, etc. You might want to check if your software require one of those before you start this whole adventure.

You can learn a lot about a formula with the info option

$ brew info python
python: stable 2.7.10 (bottled), HEAD
Interpreted, interactive, object-oriented programming language
https://www.python.org
Not installed
From: https://github.com/Homebrew/homebrew/blob/master/Library/Formula/python.rb
==> Dependencies
Build: pkg-config ✘
Required: openssl ✘
Recommended: readline ✘, sqlite ✘, gdbm ✘
Optional: homebrew/dupes/tcl-tk ✘, berkeley-db4 ✘
==> Options
--universal
  Build a universal binary
--with-berkeley-db4
  Build with berkeley-db4 support
--with-poll
  Enable select.poll, which is not fully implemented on OS X (https://bugs.python.org/issue5154)
--with-quicktest
  Run `make quicktest` after the build (for devs; may fail)
--with-tcl-tk
  Use Homebrew's Tk instead of OS X Tk (has optional Cocoa and threads support)
--without-gdbm
  Build without gdbm support
--without-readline
  Build without readline support
--without-sqlite
  Build without sqlite support
--HEAD
  Install HEAD version

Take a good look at the --without-* options because they are sometimes a lifesaver. Some packages have optional GUI extras. They might fire hundreds of useless extra dependencies — especially if you are installing on a headless server.

Sometimes Linuxbrew breaks down for the lack of a dependency, refusing to install it, but will gladly do it if you explicitly ask for it. For example : brew install package1 breaks for the lack of package2, and all it takes is typing brew install package2 and retrying package1. Mysteries.

Installation time is highly unpredictable. Sometimes a small innocent little package will require a precise version of GCC… that Linuxbrew will then have to install from the sources. Time for a tea.

If your installation becomes so corrupted with conflicting packages that you have to restart from scratch (nooooooo !), it can be — small consolation — accomplished easily :

rem ~/.linuxbrew -rf

For extra OCD cred, clean-up the download cache as well :

rm ~/.cache/Homebrew/ -rf

If the whole thing becomes so messed up that you have to scratch even the manual dependencies (two words : pig farm), it is also easily done :

rm ~/bin/ ~/lib/ -rf

You might also consider :

rm ~/include ~/etc -rf

but be careful because that might erase innocent third parties.

You might be forced to install multiple versions of the same package. That adds another nightmare layer to the ongoing nightmare, but it’s doable. Linuxbrew will usually be friendly enough to tell you what to do.

For example, when I had to install both opencv2 and opencv3 I got this :

opencv3 and opencv install many of the same files.

Generally there are no consequences of this for you. If you build your
own software and it requires this formula, you'll need to add to your
build variables:

    LDFLAGS:  -L/home/valle/.linuxbrew/opt/opencv3/lib
    CPPFLAGS: -I/home/valle/.linuxbrew/opt/opencv3/include

Those little displays of care are the reason why I like Homebrew/Linuxbrew so much. Love breeds love : a truism even for human–computer interaction.

"Upgrade to Yosemite," they said.
"It will be fun,", they said.

Back in the developer’s saddle in Yosemite ; Installing Maven on OS X

When Mavericks launched, I scheduled doing a clean reinstall over a blank, reformatted HD. (Due to the degradation of configurations, permissions, and other metadata, a system may suffer to something akin to a long-term aging effect. A reinstallation from scratch is a way to freshen it up.) The task, however, was marked “low priority” in my To Do list. The result : last week I was forced to upgrade to Yosemite, and still no reformatting.

As I explained in that post, I’ve noticed a trend of CS/IT professionals being the most reluctant users in updating to the latest hardware or software. Yosemite justified that reluctance, by breaking my HomeBrew installation. The reason : HomeBrew explicitly links to Ruby 1.8, which is obliterated by Yosemite in favor of Ruby 2.0. (Hey, Apple, word of advice : it’s no use having a sophisticated system of coexisting Framework versions if you decide on a whim to delete the older versions.)

I had experienced some  minor inconveniences before I encountered this problem. In the text that follows, I assume that you have already dealt with the following :

  1. Updating Xcode on App Store (Menu Apple … App Store…; tab Updates) ;
  2. Re-accepting the terms and conditions of Xcode : neither Xcode nor its command-line tools will run before will sell your soul to Apple again. And even if you have administrator permissions, you have to sudo a command-line tool to be able to do it. You’ll see an ugly message like : “Agreeing to the Xcode/iOS license requires admin privileges, please re-run as root via sudo.” Either re-execute the command with sudo (e.g., sudo make), or accept the agreement in the Xcode graphical app ;
  3. (Possibly ?) Reinstalling Java VM from Oracle. This might just be an issue for web browsing ; maybe the VM works on the command-line out of the box : I didn’t check it. But if you type java on Terminal and nothing happens, chances are you’ll need to get it before being able to do anything interesting.

The bad news : the only way I could get HomeBrew back to work was reinstalling Ruby 1.8.

The good news (if you have a Time Machine) : doing it is a breeze. Just restore the folder 

/System/Library/Frameworks/Ruby.framework/Versions/1.8/

to its rightful place.

If you don’t have a Time Machine (how do you even survive on OS X without one ?!), maybe you have an old MacBook stored in a cupboard ? Or an upgrade-averse friend who has not yet moved to Yosemite ? (Hint : do you know anyone who works on CS/IT ?) Get a copy of that folder and put it back where it belongs.

If you can’t get your hands on that folder anywhere, you’re probably out of luck. You might be able to fish the framework out of the installer packages of an older OS X version, but just thinking about it makes me want to cry. Maybe you can wait for HomeBrew to issue a patch ?

With Ruby 1.8 back in place, things become very straightforward. Just to be sure, run the following commands :

brew update

brew doctor

And check if there are any remaining issues to be solved. (By the by, you don’t have to try and solve every minor problem : in computing as in medicine, minimal intervention is often wise.)


All this marathon started when I needed to install Maven on my system.

With HomeBrew working, this takes an one-liner :

brew install maven

The installation worked without issues, but for some reason, Maven kept complaining that the JAVA_HOME environment variable was broken:

Error: JAVA_HOME is not defined correctly.
  We cannot execute /usr/libexec/java_home/bin/java

Naïvely setting JAVA_HOME to /usr let Maven run, but with an irritating warning :

Unable to find a $JAVA_HOME at '/usr', continuing with system-provided Java...

What solved the problem completely was adding this line to ~/.bash_profile :

export JAVA_HOME=`/usr/libexec/java_home`

“Upgrade to Yosemite,” they said.
“It will be fun,”, they said.

…but if you experience the same problem, you’ll first need to check where the java_home util (it prints the path of the Java VM on stdout) actually is in your system. If /usr/libexec/java_home runs, the solution above will probably work.

Upgrade cascade : iPhone, Yosemite, iPhoto, iMovie

I’ve noticed a consistent trend of my colleagues and I, Computer Sciences / Engineering faculty, being way less eager than the general public in updating to the latest hardware or software. There is, maybe, a component of the shoemaker’s son going barefoot, but most importantly — I suspect — it’s the knowledge on sausage-making impairing our appetites. When you know the reality of system design intimately, you become very reluctant in disturbing whatever metastability you might have reached.

But all systems have a service life, and eventually even the most reluctant user will be forced to upgrade. After skipping 2 generations, I thought it was time to abandon my iPhone 4S for a new iPhone 6.

(Which was an adventure in itself : amazingly, after almost 2 months, there are still queues for buying an iPhone on the States. So far, ok — supply and demand, etc. — but for some unfathomable reason, Apple has instructed their clerks to outright lie about the non-contract T-Mobile iPhone, in saying that it is not unlocked.  After some googling and whatsapping with friends, the truth emerged : it is unlocked. Still, at the first Apple Store I tried, the clerks where very non-cooperative, and one of them positively adversarial, like he’d rather not sell anything to me. I am really not the type of person to buy into this “privilege to be a customer” attitude, so I just went to another store. Long story short : two days and 830 bucks later, I had an iPhone 6 in my pocket. It is indeed unlocked, I had it working with my Vivo telecom nano-SIM immediately, still inside the store.)

But as often it happens, one upgrade leads to another in cascade effect : the iPhone rejected my old iTunes, forcing me to upgrade old faithful Mountain Lion to Yosemite.

Update Unavailable with This Apple IDAs if to confirm that upgrading is a messy business, Yosemite got me a great welcoming surprise : it disabled my old iPhoto (“incompatible with new OS version, must be updated”), and made it impossible for me to update it (“Update Unavailable with This Apple ID”). For some strange reason, the App Store utility insisted on that message, no matter which Apple ID I used (I only have two).

Apparently this is not a rare situation, and the causes and solutions are exasperatingly diverse. What solved the problem in my case, was closing the App Store, deleting iPhoto altogether (dragging the disabled application to the trash), opening the App Store again, and doing a fresh install. The procedure itself is not very painful, I concede : the annoyance is having to find out what exactly to do.

For upgrading iMovie, the solution was not so simple. It is not a mandatory upgrade (the Mountain Lion version still works with Yosemite), but since I had gone so far, I now wanted to go all the way. Deleting iMovie made available a fresh install on App Store… for 15 bucks. No good. I’ve tried, as some suggested by some users, reinstalling the original (from the Snow Leopard CDs in my case), but to no avail. In the end, I just moved the old Mountain Lion iMovie from the trash back to the Applications folder.

Curiously, XCode, which is normally a trouble-maker, updated without further ado.

Edit 19/11 : upgrading to Yosemite 10.10.1 solved the iMovie Apple ID issue. I’m guessing it would have solved the iPhoto issue as well. This is another golden rule of upgrading — never move to the version with a round number, always wait for the next minor patch.

Withdrawal Syndrome

Ceremonial tea whiskAlthough I have my personal opinion, the effort of ranking things like Ubuntu Linux, Microsoft Windows and Apple OS X from ‘better to worse’ is clearly Lilliputian. Therefore I shall abstain.

But I’ll report that I was recently given a Windows 8 laptop, and an Android ‘Jelly Bean’ tablet. Much to my embarrassment, I am finding both barely operatable.

Whilst I’ve been under the pain and delight of OS X and iOS, Windows went through several transformations, and is now barely recognizable from the last time we’ve met (XP, if you believe). Meanwhile, Android is for me completely new (I went directly from Simbian to iOS) and equally alien.

As a wide-eyed Computer Science student, I was put off by the design philosophy of OS X and iOS : the ‘my way or the highway’ appearance, the monastic simplicity, the detachment between ends and means achieved through an extreme cloaking of the mechanisms. The system reeked of coercion and… insincerity.

Now, as an overworked Engineering professor, the same dreaded features became desirable. I no longer want to choose one from 50 color schemes for my window manager, or worse, design my own color scheme. I want, out of the box, one beautiful color scheme that works. I know : it is kinda sad, and totally against the hacker ethos. But nowadays, the investment of the scarce time I have for hacking is decided very judiciously.

Once upon a time, the Rube Goldberg design of Windows and Android would make my delight for weeks — no checkmark would go unexplored, no deep dialog box would remain unopened. It was a given that I would have to reinstall the system from scratch a few times, for I would “customize” (i.e., torture) it to the point of hopelessness.

I wouldn’t mind the updates. I wouldn’t mind the nagging confirmations. I wouldn’t mind having to reinstall the system from scratch to get rid of all ‘free !’ useless/noxious third-party bundled software. I wouldn’t mind the inconsistent GUI design. I wouldn’t mind that the system, fresh out of the box, would let a driver do a “fatal exception”, or an essential application “execute an illegal operation” — either demanding hours of troubleshooting.

All that mattered was that once everything was sorted out — days or weeks later — I would get a system customized and optimized to my exacting tastes. A system that kept no secrets from me. A system of my own.

Magic BulletNowadays, I want to easily unwrap a beautiful package — at each step feeling that my gestures follow a rhythm etched in the very fabric of the continuum — and have the hardware satisfyingly fall in the void of my hands, heavily filling the negative space between my fingers, caressing my skin with the touch of finely crafted rich materials. The software comes alive with a barely conscious command. A few discrete questions and the device and I are one, its assimilation into my collective accomplished, my diffusion into its anatomy complete and unnoticed.

For this privilege, I am willing to go as far (the horror !) as actually buying an Apple product.

There must be a 12-steps program for that.

EDIT 17/01 : I was complaining about my Windows 8 experience on twitter, and this dialog happened. I have to recognize they are charming.

IT Industry and Users — a lesson in advanced BDSM

Case study :  you have an hour-long education video that you want to upload to your YouTube channel. The video needs minor editing : (1) trimming and eliminating a few portions; (2) splicing-in another short video; (3) maybe normalizing the sound.

On a 2011 MacBook Pro using OS X, your options are :

Use iMovie (bundled with OS X) — it will take 30′ importing the clips, then you’ll have editing capabilities much more advanced than you need. But it will take 10 hours for it to reencode the final video, and of course, there will be a compounded quality loss due to the double reencoding ;

Use Quicktime X Player (bundled with OS X) — contrarily to what “player” would suggest, Quicktime X gives you recording abilities, and very minor editing abilities, like trimming and gluing takes, without having to reencode the video. Sounds perfect, no ? The trimming feature is, however, practically useless, since it has very poor resolution (what is the use of  editing if you’ll end up having people being cut in the middle of a utterance ?). And you cannot remove inner portions of a video, just trim the edges : although you can go around this by trimming and gluing several times, for very large videos, it quickly becomes annoying.

Use Quicktime 7 Pro (US$ 30 on Apple Store US) — the old version of QuickTime has much finer trimming abilities, and it will allow you to remove inner portions of videos. Free trumps bought any day, but the price is not bad,  considering the time it could save. (What really ruffles my feathers is that Apple is basically selling you abandonware 30 bucks-a-piece, but Apple’s greediness shouldn’t really surprise anyone by now.) After you go through all the hoops to find out how to install QT7 on OS X Mountain Lion or Mavericks  (it does not work if you don’t get exactly the right version), you find the deal breaker : it is not available in Apple Store Brazil, and Apple Store US does not accept foreign credit cards.

(This is getting ridiculous.)

Use Avidemux (Free as in speech, and as in beer) — Jackpot ! This open-source project is meant to solve exactly your problem : make minor editing on videos without having to reencode the whole thing (although it has advanced reencoding abilities as well). Installing it is a bit of a hassle : you do all the clicks and have no sound on output. Some googling tells you that the problem is a preferences setting, but you can’t find the “Preferences” panel because a freaky bug has blanked out most GUI elements, including most of the menu items. One hour later, you finally have sound (having found out which blank menu item is “Preferences”, and which blank tab is “Sounds options”, and which blank field is “Sound device”). All that to find out that the preview window has a time delay on the play/pause/ff/rev buttons that makes it completely useless for precise editing purposes.

(If it weren’t four in the morning I would scream.)

Use ffmpeg (Free as in speech, and as in beer) — Video editing on the command-line : you may be an Apple user by choice, but even for you there is such a thing as too much masochism.

Use Final Cut Pro (US$ 300 in Apple Store US), Adobe Premiere CC (US$ 30 per month for Teachers and Students in US) — In order to cut and paste video pieces. Are you kidding ?

(Okay, you see where this is going —  I’ve tried and discarded many other solutions. For sake of curiosity my browser history shows :  “H.264 Cutter”, “Why am I doing this to myself”, “MPEG Streamclip”, “Mp4Split”, “Can a MacBook Survive a 4 story fall”, “Xilisoft Video cutter”, etc.)

And I didn’t find a solution ! In the end, I sucked it up and went with iMovies. Grudgingly.

Computer Users, we have to do something about this play : this is neither safe, nor sane. But Computer Industry, if you are hearing, this is definitely not consensual.

Update 13/dec : A typo in the link for my YouTube channel was misdirecting people to the fake tiniurl URL shortener, instead of the real tinyurl. My bad.

Update 18/dec : The story is, unfortunately, not over. Now iMovie refuses to generate an output with a mystifying “error 49” that, according to a Google Search, might mean everything and the kitchen sink. I’ve reverted to a rough editing with Quicktime X and got an output… without audio. I am arriving at the conclusion that I should give up IT entirely and move to the country to raise pigs. This whole “computing” idea should just be filed as “tried, did not work”.

Time Machine menu on notification areaUpdate 19/dec : apparently, iMovie and Time Machine don’t play nice with each other — which is one of the possible causes of the error 49. Unless you are rendering very short videos, it is better to disable Time Machine (click on the Time Machine icon on the notification bar — upper left corner — then “Open Time Machine Preferences”, then switch the big “On/Off” button. Remember to turn it on again once the Finalization or Export are over !). This worked for me, but it is no guarantee to make the pestering error disappear.

Excuse me while I go wrap my head in tin foil

I’ve never imagined that this blog would degenerate into so much rant about technology — maybe is it something about the 13 in 2013 ? Or am I becoming grumpy as I age ? (My detractors would reply that I’ve always been grumpy).

I have Google Chrome, Apple Safari and Mozilla Firefox installed on my MacBook, since more than twenty years have passed since the inception of the Web, and still designers can’t agree which browser to support (and browser makers can’t agree to which standards to commit). I switch defaults every few months, as I get fed up with the inconveniences du jour. Safari is the current default.

Now the horror story: whenever I open Chrome, it asks me to have access to the keys of one of my blogs. A blog whose existence it is not supposed to know. It is not on my bookmarks, it is not on my Google Account, it is not on the browsing history (supposedly — I’ve cleared it “from the beginning of time” — but that probably means that only I don’t have access to it anymore). Yet, every time I open Chrome, it asks me for the keys to the private content of that blog. Twice. So far, I’ve resisted.Google Chrome requesting access to OS X keychain

I’ve looked around, and there’s a general procedure to solve Chrome keychain indiscretions in OS X, which consists in opening Utilities / Keychain Access, and deleting the Chrome Safe Storage item. The procedure actually works, but it results in Chrome not syncing with the Google Account anymore. If I type my password to link Chrome to the KGB, the interrogatory starts again, meaning that the Central Agency already knows about my secrets.

I might need a double layer on that tin foil heat.

EDIT April, 1st : Victory at last ! I’ve found out deep inside the bowels of Chrome how to delete the information. Hit Chrome’s tools menu (in the upper right corner) and hit Settings (or in a Mac just hit Command+, ). Go to the bottom of the page and click on Show advanced settings. On the Passwords and forms section, click on Managed saved passwords. There you’ll find Chrome’s stash of sites whose passwords it wants to follow. Delete the offending entries (by clicking on the little “x” that appears when you hover), and Chrome you’ll leave you alone.

Mac OS X, Word and the quest for the unbloated PDF

Hard-science scholars are strange people, who insist on using TeX because it “typesets beautifully”, but then forget to check badness warnings, letting the lines spill beyond the right margin. I have resisted TeX as much as I could, until I finally caved to peer pressure. Still, I only use it for cooperative work : when all by myself, I want something, let’s say frankly, less Jurassic.

Still, I am forced to envy my frozen-in-1975 colleagues, when I find out that saving to PDF, an operation that the industry should had gotten right by now, turns my 1 MB Microsoft Word file into an 80 MB PDF-zilla.

I’ve spent a good part of my morning solving that problem, considering both the official solution, and more independente initiatives. The official solution flunked when I’ve found out that Adobe had no trial of Acrobat for Mac (am I really willing to spent US$ 500 just to find out whether or not it’ll do what I want?). I tried a PDF compression solution, PDF Shrink, which reduced my PDF… from 89 MB to 87 MB, while mangling horribly all the images: not exactly worth the US$ 35. I’ve also tried recreating the PDF from scratch, but PDF Studio, at US$ 125, just refused to open the Word file with a cryptic ‘error reading’ message. I was glad both were in trial.

In despair, I continued searching the Web. Lots of users crying “Large PDF !”, “Word PDF too big !”, “Huge PDFs on Mac !”, but very few answers. Industry, why u no listen ?

They say that we should never attribute to malice that which can be explained by incompetence. But Hanlon’s razor notwithstanding,  I couldn’t avoid drifting into conspiracy theories. What if that horrible implementation of PDF conversion was not completely accidental ?

Conspiracy theories are unfalsifiable, of course, but I’ll tell you what finally solved the problem and you’ll tell me if it doesn’t make you itsy bitsy suspicious :

  1. On Word, instead of saving to PDF, save to PostScript (using File… Print…, and then, on the print dialog, the PDF button on the lower left corner. The Save to PostsScript is one of the options);
  2. Open the PostScript file (double-click its icon) and let Preview make the automatic conversion;
  3. On Preview, save the file as PDF (using the menu File… Export… — or on older OS X versions File… Save as…)

And that’s all. Now lets check the sizes :

  • Original PDF file (using Save as PDF or Print to PDF from Word) : 89 MB
  • PostScript file (Using Print to PostScript) : 94 MB
  • Final PDF file (using the steps above) : 5 MB

That is, using only tools already present in OS X, and three small steps, I’ve got an almost 18x smaller file. Risking joining the ranks of the ‘moon hoax’ lunatics, I smell something rotten in the current state of PDF conversion implemented by Word–OS X.

* * *

Incidentally, I’ve found something I also needed : how to password-protect PDFs. I was ready to buy a solution, but I’ve found that unnecessary.

When creating a new one, on OS X, you can click on the “Security Options” menu of the “Export as” (“Save as” in older versions) dialog — how come I’ve never remarked that one ?.

If the PDF exists already, you can open it with Preview, go to File… Export… (File… Save as… in older OS X versions), and check the box “Encrypt”. Two textboxes below let you put a password. Save the file and it will be only visible after the password is entered.

EDIT 28/02/12 : I am finding out that the above method is by no means foolproof, i.e., it doesn’t work for every kind of PDF. In particular, I tested it for PDFs generated by PowerPoint, and it backfired (PostScript conversion generated a file much bigger). For image loaded PDFs from PowerPoint, contrarily to the mainly textual ones from Word, I’m finding that the usual tip of using the Quartz Filter (open the PDF file with Preview, then File… Export… [File… Save As… in older OS X versions], then select “Reduce File Size” on the Quartz Filter field in the dialog) works quite well.

EDIT 23/07/13 : I’ve never dreamed this blog entry was to become my most popular one. (Apple and Microsoft, aren’t you listening ?) I’ve edited the procedure above to reflect the change in the Save As… logic introduced in OS X Lion, when it became Export…

In a Flash

What is happening to the Firefox Flash plugin, those days ? The critter has created a life of its own. Nowadays, it’s always the same story: I am minding my own business, and all of the sudden the CPU fans start to spin, the computer gets warm, the battery starts to drain noticeably faster. I don’t bother anymore to check the processes list: the culprit is always the same.

This has lasted for the last few months, so it’s not just a rotten update: it’s a whole series of rotten updates. It has become so serious that before I found the problem I considered abandoning Firefox altogether. Nowadays I just keep Flash disabled, except when absolutely needed.

Interestingly, living without Flash has been much less nightmarish than I anticipated. Half of the time, even embedded YouTube videos will play (once you sign up for the HTML5 beta). Often, the only thing lost are the more aggressive forms of advertisement. (To which I say good riddance, of course.)