On MacOSX 10.7.5 [still, sorry], after updating Safari several months ago (was it to 6.1.3?), I started getting very very frequent alerts asking me whether I would like to allow or deny incoming network connections to Except that the alert would more or less instantly disappear, as if Allow had been clicked.

I checked System Preferences ‣ Firewall ‣ Firewall Options, and confirmed that was listed to allow incoming connections. The alerts continued.

What I didn’t realize (until today) was that the app listed in the firewall rules was this one:


while the app actually run by the current Safari [6.1.5] has the same name but is located in a different directory:


[How do you determine this? Hold the pointer over the list entry for a few seconds until a tooltip pops up; this shows you the file path.]

So, the fix was clear: Add the latter as Allow in the firewall rules.

There. Fixed. I don’t know how this got broken.

I am quite concerned about security from malevolent incoming connections; however, I think my fix is ok. If you know otherwise, please let me know.

Mathematica – UTF-8 – MacRoman – BOM

I experienced a weird situation in Mathematica yesterday. I have wholeheartedly embraced Unicode for several years, and for several years have been including Unicode characters in .m files. (In notebooks, whether characters are entered as single keystrokes (possibly modified by shift and/or option), as \[codes], or as \.xx or \:uuuu in hex, they end up as Unicode in your session. When a notebook is saved, they they take on their 7-bit ASCII representations.

I wrote a very simple bit of code to implement a truncated power function, defined with the exponent having limited range:

v_~tpf~g_ := Piecewise[{
    {0, v ≤ 0},
    {v^g, True}
}] /; 10*^-1 ≤ g ≤ 10

Note the ≤ characters. I saved this code in UTF-8 into a file that I subsequently read into a Mathematica session using << (a.k.a. Get[ ]) not mentioning any character encoding. Upon running my code, it broke. Examining the expression, I discovered that the less-than-or-equal-to symbols (≤) had been interpreted as superscript 2 (U+00B2) [which, somewhat surprisingly, is treated by Mathematica as a letter-like form]. My “condition” was not a sensible condition at all. It turned out to be difficult to debug this situation. I discovered that \.B2 is the MacRoman encoding of ≤ – that is, my .m file had been read in MacRoman code. All that time that I’ve been using Unicode it never occurred to me to ask whether Mathematica was capable of reading my old, old files in MacRoman coding – that is, had I used ≤ in files saved in MacRoman code, would they have worked? I think not. But in any event, before adopting Unicode, I made a habit of coding .m files in straight 7-bit ACSII, so ≤ should not have appeared in any of my files. But clearly in yesterday’s situation ≤ was interpreted in MacRoman, without any explicit action on my part. I never did figure out how the .m file came to be interpreted in MacRoman, though. In the course of debugging, I discovered that Unicode’s “byte order mark” (BOM, U+FEFF) is not elided by Mathematica when found as the first character of a text file being imported. This strikes me as a problem. Although I don’t use BOM to mark a UTF-8 text file, many people do. Worse, nonsensibly, \:FEFF is interpreted my Mathematica as a letter-like form (though it makes no mark on the screen or the page). Try it:     \:FEFF = 5

This can only be a “bug.” It seems to me that if CharacterEncoding is specified explicitly, it should be honoured; failing that, if BOM is found at the start of a file it should be deleted, while constraining any inferred encoding to either UTF-8 or UTF-16; and if no explicit encoding is specified and there is no BOM, then Mathematica should do whatever it currently does. Also, Mathematica should reclassify BOM so that it is NOT interpreted as a letter-like character, or at the very least disallow its presence in a symbol name.

Roozz/TransType is a bust

I’m still engaged in a long slog toward getting a fully functional FrameMaker installation [Fm 11 under Win 7 under Parallels under Mac OS X 10.7.5 on MacPro4,1; also, Fm 11 under Win 7 on native Intel hardware with Boot Camp Mac drivers]. The goal is to replicate Fm 7 under OS 9 as of a decade ago, or Mac Classic [Fm 7 under Classic 9.2.2 under 10.4.11 on PowerMac PPC] of half a decade ago!

Among many other things, I have to convert fonts.

FontLab’s TransType is alleged to convert Type 1 Mac fonts to modern PS-flavoured OTF. In addition to TransType being available as a conventional licensed app, it’s supposed to be available for rent. Rental is a business model suitable for one-off conversions such as I need. The rental version is hosted on Roozz, which is supposed to be useable on Windows.

I have no comment on TransType, because I didn’t get that far. After many attempts and several interactions with the company, I find Roozz to be unuseable across Explorer, Chroma, and Firefox, all on a vanilla, near-empty, un-customized Win 7 system. Don’t waste your time.

If “Roozz” appears at the upper left-hand corner of your paycheck, you’re welcome to give me a call and I’ll explain. (You could first check your company’s email.)

Display black setting & validation

Here’s a thought experiment. On an 8-bit computer display (that is, a display nominally operating to the sRGB spec, where code 0 is black and code 255 is white), set black level (commonly, “BRIGHTNESS”) to what you think is correct. Put up an image that is predominantly solid black (code 0), but having a patch at code value 0 upon which is rendered the digit 1, a patch at code value 1 upon which is rendered the digit 2, a patch at code value 2 upon which is rendered the digit 3, and so on up to a patch at code value 31 upon which is rendered the numerals 32.

Should the digit “1” be visible?

What if you were using a 10-bit display, where the numbers were separated by one LSB (that is, four times finer than the 8-bit case; still going up to 32, but the 32 numerals are now darker). The digit “4” has luminance equal to “1” in the 8-bit case. Should the digit “1” now be visible, or would you be happy if 1, 2, and 3 merged into invisibility and 4 was perceptible?

(If you like, you can think of this exercise in the video domain; the 8-bit example would have a background field at code 16, and the patches would run from numerals 17 through 48. If you prefer 10-bit, the background field would be at code 64, and the patches would run from numerals 65 through 96.)

What if you had a 12-bit display? Are you ok with “1” being invisible, that is, indistinguishable from “0”?

What about a six-bit interface?

If you think like me, after mulling over this exercise you conclude that black-setting – and the perceptibility of a one-LSB step up the code scale toward white – should not be dependent upon the number of bits across the interface. Maybe the “1” (in the sRGB example) is visible, maybe not; it all depends upon the display contrast ratio, the ambient light, the average luminance of the image upon which the black-setting element is displayed, and upon your own vision.

SMPTE black-setting procedures are expressed in a confusing manner, but lead you to the conclusion that black level is set to make the negative-going element just barely below the threshold of perceptibility. The positive-going element is not actively used in the procedure, but is expected to be perceptible once black is set correctly. If you reflect upon the use of the negative-going element, I think you’ll conclude that it does not really matter whether it is placed at -2%, -4%, or even -10%: the same setting results. The positive-going element (for HD, in RP 219) is at +2%; referenced to 8-bit sRGB, that’s equivalent to code 5. So, we expect a large code 5 patch to be distinguishable from a large code 0 (reference black) background, but no conclusion is evident concerning codes 1, 2, 3, or 4. The sRGB interface permits no element blacker than the code 0 of reference black – no negative-going PLUGE element can be conveyed – but -2% PLUGE black would be at code -5 if it could be represented. If you’re convinced that BLACK LEVEL/BRIGHTNESS control operation is reasonably linear in that range, you display patches 0, 5, and 10; then set a trial black to make 5 and 10 indistinguishable, set another trial black to make 0 and 5 indistinguishable, then adjust black down further by the amount that separates the two trial settings.

Please share your thoughts by email! – Charles

Mac service to convert aliases to symlinks

Finder Alias files on Mac, while useful, have bloated to 5 MiB on a typical Mac. Crazy. As far as I can tell, the bloat is due to huge icon images being embedded into every alias file. Ok, maybe that makes an alias file self-contained, but I don’t want hundreds or thousands of 5 MiB files all containing copies of the same icons. You would think that [at least for aliases to files without custom icons] the Finder would simply consult its own desktop icon database when required, instead of carrying the huge overhead of the icons in every alias file. If I move an alias file to a different Mac that does not have any files of the same kind, ok, just give me a generic document icon! Perhaps there’s some subtlety of Finder engineering that I’m missing. Anyway, I’m tired of the bloat.

Assuming that your files are stationary, instead of a Finder alias you can use a UNIX-ish facility called a symbolic link (symlink). In the process of whipping up an Applescript to make these, I discovered a decade-old Mac OS X Hints post entitled “A script to convert aliases to symlinks” that provides a shell script to do it: You pass one or more filenames of alias files as arguments; the script converts each alias file to a symlink. The Finder then subsequently treats symlinks as you would expect: A symlink behaves like an alias (and displays an icon badge like an alias).

I zipped-up the shell script: alias2ln.

Shell scripts are really only useful if you know how to obtain a command-line prompt, you know roughly what /bin/sh means, and you know how to set your path. Most Mac users don’t know these things. So, I packaged the shell script up into an Automator action: Alias to symlink.workflow. Unzip it. When you double-click, it will offer to install itself (into ~/Library/Services, as it turns out).

Once it’s installed, select one or more alias files, then choose Finder > Services > Alias to symlink. Watch the reference go from 5 MiB to a hundred bytes or so.

One thing you have to know: A Mac alias incorporates a mechanism to “follow” a file even if it is moved. A symlink has no such feature; when created, a symlink references a certain path fixed. (In fact, that path is precisely and exactly what the symlink file contains, along with one bit [in the inode, if you must know] that distinguishes it from an ordinary file.) If you move the original file, the symlink won’t function (unless you put some file or other into the original location).

On a closely related topic, Nick Zitzmann has created a Service, SymbolicLinker. Once you install this, select a file, then access the service through the Finder’s Services menu or through the Finder’s contextual menu. A symlink to the selected file[s] will be made and placed into the same folder. You can then move the [absolute] symlink anywhere you like. I have used the Keyboard preferences panel to make a keyboard shortcut key ⌘L for this service – that is, I have replaced the built-in “Make Alias” shortcut with a “Make Symlink” shortcut, so when I want to make a symlink, I just hit Command‑L.

Modern typesetting

I first used Microsoft Word in 1984, about three decades (a generation) ago, on a Hewlett-Packard 110 laptop computer, one of the very first. I wish I could tell you which version of Word – definitely, one-point-something. (I’m ok with not resurrecting the computer itself. It had lead-acid batteries; its mass was 4 kg, fairly heavy for a lap. It ran MS-DOS 2.11. It had a 480 by 128 bilevel LCD display, but ordinarily – well, ok, always – you ran the display as 16×80 characters, yes, 6×8 dot matrix, fixed-pitch type.)

In 1985, I bought my first Mac, and I bought Word for Mac version 1.0.

It simply AMAZES me that, 30 years later, Word still ships with nonsensical defaults on new documents, the same defaults as on my HP 110 thirty years ago.

First, by default a new Word document has no page numbers. Admittedly with Word for Mac 2011 it is reasonably easy to insert page numbers (Insert → Page Numbers…); prior to about 2011, it was insanely difficult for a naive user to figure out how to view headers and footers, establish a footer, and insert an auto number [ideally, right-justified]. This failure of the default settings explains why SO MANY documents on the web have no page numbers. Amazing.

Second, by default there is no paragraph indentation. The default settings act just like a typewriter: Hit “RETURN”, go to the left margin. A decent fraction of users can figure out how to indent: They hit TAB after RETURN. Or a few spaces. A much smaller faction of users figures out how to change paragraph formatting to set a first-line indent. A VERY small faction of users can manage to define an appropriate style, and an even smaller fraction (now we must be down to tenths of percents of users, several users in a thousand) can figure out how to redefine the default style.

Microsoft: Would you consider updating Word to (a) have page numbers, by default, and (b) have a quarter-inch indent on the first line of a paragraph, by default? Thanks.

Linotype Syntax (Next, SC/Small Caps, etc.)

I have written, illustrated, laid-out, and typeset three books. (I have not typeset books written by anyone else.)

In 1996, when I was polishing up my first book for publication, I invested several hundred dollars in a typeface (colloquially, font): Syntax, from Linotype.

By 2003, as I was polishing up my second book, I really really wanted one or two additional weights. Linotype by then had released a new version of Syntax (called, confusingly, Linotype Syntax, or LTSyntax). There was no upgrade option. I purchase a new licence outright, for USD 1200 (including all of the weights, and the small caps variants).

Around a year ago, I started the process of migrating to FrameMaker on Windows (under Parallels, under Mac OS X, on Mac Pro Intel hardware).

I had lots of difficulty in converting my typefaces, particularly Linotype Syntax, into a form useable on Windows. Font management on Windows is poor (in my opinion), and there ware major challenges in naming the font, its variations (roman, italic, bold, bold italic), its weights (light, regular, medium, bold, extra bold, heavy), and the small caps (SC) variant. A related problem was configuring Windows so that Adobe Illustrator EPS files (illustrations) would open both and print identically on the Mac side and the Windows side. Again, lots of naming issues – and not filenames, because font names are embedded within font files. Finally, to future-proof my workflow, I really wanted to convert from Type 1 to OTF format.

Around this time, Linotype released another version of the typeface, now called Syntax Next. So, I thought, the thing to do is to just buy it.

However, this turns out (a) to be a big cost (Linotype offers no upgrade from a previous version), and (b) not a complete solution anyway, because apparently Syntax Next has no small caps variant. Linotype may say, “Well, Charles, Syntax small caps didn’t sell very well, that’s why we didn’t port it to OTF for Syntax Next.” I would’t call it poor marketing, it’s deeper than that: I’d call it poor product definition. [There is now a Greek version, nice for me to have lambda, gamma, just USD 89 for the roman typeface. But: additional USD 89 for italics, USD 89 for bold, and USD 89 for bold italics! Sheesh! I guess Linotype would argue that you shouldn’t have to pay for variants that you don’t use, but it seems crazy that a typeface company would promote the idea that applying italics is a luxury.)

Email to Linotype is answered by front-line support staff who follow their scripts but are generally completely unhelpful. (Even for very carefully posed, simple questions, it is a surprise when a support rep is able to sensibly answer exactly the question that is asked.)

Rhetorical question: Why is Linotype unable to figure out that the company is in the software business?

“Mid”-grey, why not 50%?

If you visit your local photography store, they will happily sell you a mid grey (gray) test card. Many brands are available; let’s take Kodak’s R‑27 as an example.

Such cards do not have 50% reflectance. They typically have reflectance close to 18%. This information can be found in the relevant Wikipedia articles (Middle gray, Gray card). If you do a Google search on “kodak mid gray”, nearly all of the results on the first page mention 18%. None of these sources is definitive, of course, but you could measure a card yourself with a variety of techniques.

Why does a “mid gray” card not have 50% reflectance? Because human perception does not interpret lightness as proportional to optical power. Similarly, human audition does not interpret loudness as being proportional to sound pressure level. In both cases, the perceptual quantity is loosely related to the logarithm of the physical power. Over the range of reflectance percentages that can easily be created using ink or paint on a piece of cardboard – say, 3.6% to 90% – you can approximate lightness (the perceptual quantity) as about the 0.42-power of reflectance (the physical quantity).

Why does this matter to photography? Because photographic film did not respond linearly to light power. Its response is complicated, but Kodak picked 18% gray because that value is the middle of the perceptual response between black (say 3.6%) and white (say 90%). Take the geometric mean: sqrt(0.0362 + 0.092) = 0.18. Or, take the 0.42-power approximation that I mentioned above: 0.180.42 = 0.486, very close to 0.5 in the perceptual domain. In digital photography, things work similarly.

Or, take photographic stops, each stop being a power of two in the physical domain (and assuming that your calculator has base-2 logarithm capability): log2(0.036) = ‑4.8, log2(0.90) = ‑0.15, difference is 4.64 stops. Take the average, (‑4.8 + ‑0.15) / 2 = ‑2.32; 2‑2.32 = 0.2 (which is 20% reflectance relative to 90% white, that is, 18% reflectance relative to 100% white). The zone system of Ansel Adams and Fred Archer is based upon such calculations. (They simply added and subtracted the numbers reported by their light meters, which had already silently performed the logarithm calculation – which, by the way, Adams and Archer apparently understood quite well.) If your calculator hasn’t got base‑2 logs, log2(x), then use 0.3·log10(x).

Photoshop works in the perceptual domain: By default (in the absence of ICC profiles and such), Photoshop assumes that colour image data (pixel) values represent perceptual quantities, not physical quantities. Pixel components are expressed to the user as numbers very roughly 255 times reflectance to the power 0.42. For a mid gray card, 255·0.180.42 is about 124 (not quite 128, but several variables are implicated including photographic exposure). By the way, Photoshop implements LAB colour coding directly, and the lightness that I refer to here is exactly the L* component of that space.

Such issues are discussed in my Gamma FAQ (248 KiB PDF; somewhat dated now, but still reasonably correct and useful).

One way for a photography professor to deal with these issues is to ignore them and speak just in words, not numbers. However, if he or she chooses to speak in the numbers of physics (“50% reflectance”) or computing (“Photoshop code 128”), then it seems to me that he or she has a duty to get the numbers and the explanations reasonably correct.

Sending short text files to colleagues

I routinely write text files having nonbreaking spaces (e.g., 35 mm), curly “quotes,” and sometimes – as surrounding this phrase – en dashes. On my own machine (Mac), it is my custom to save such files in UTF‑8 (no BOM) character code and with “UNIX” [nl, that is, lf] line ends suitable for modern Mac OS. UTF‑8 is set with a nonbreaking hyphen; I don’t want UTF‑ at the end of a line and 8 at the start of the next. Entering text in this way is easy, even as I’m writing this blog entry: Safari and WordPress both use UTF‑8 unless instructed otherwise, and the only challenge [a minor one] is finding the keystrokes.

My favourite text editor (BBEdit) doesn’t care about line-ends; it will open a file having any convention, save the state while editing, and (unless instructed otherwise) save the file with the same convention as it was opened. You can change the character encoding of a file. That action has no effect but to save the preference while editing; the encoding change is effected when the document is saved. BBEdit has excellent character code support: It runs native UTF‑8 for editing, and two dozen or so character encodings are available for opening and saving.

If I have a note containing significant typographic elements (or layout), I transmit it to my colleagues in PDF form. However, I frequently transmit text files to colleagues – biographical data, paper abstracts, short technical summaries – where there is a sprinkling of typographical characters (as above) but no layout. I expect my correspondent to see a display that respects my conventions (for example, not breaking 35 and mm). Sometimes, my correspondent will import into Word (or some other tool), and edit and/or reuse. My correspondent may or may not edit using nonbreaking spaces, curly quotes, and en dashes; however, in many cases he or she will pass my file on to someone who does know how to typeset. I want to ease that process, and not destroy my typographical preferences.

Sometimes I resort to sending such files by appending as text the bottom of an email message, because I trust that my mail client (user agent) will tag the message with the appropriate character code and translate to Internet/MIME line ends, and that the receiver’s mail client will translate as necessary.

However, in cases that I want to send such a text file as a mail attachment, what’s the safe method? If I attach the file in my preferred form [UTF-8 code/LF lineends], do I risk a Windows-equipped recipient opening (by default in Notepad, I guess) and finding garbled characters (because Windows doesn’t default to UTF‑8) and one long line (because notepad even in 2013 Notepad ignores the LF lineends)? Should I resave to a temporary file having Windows Latin‑1 and CR/LF lineends, just for mail transmission purposes? (Will I risk Mac recipients seeing garbage?)

Am I off my rocker? I can’t see how all these things could possibly just work automatically, given that text files in both Mac and Win are stored in the filesystem carrying little or no metadata.