Archive for programming

What is the “hacker ethos”?

Posted in Uncategorized with tags , , , , on February 27, 2010 by spinoza1111

* Running code (and the “rough” consensus of the Lynch law) as
opposed to correct software: the deliberate use of inferior algorithms, anti-intellectualism and hatred of culture masquerading as a pseudo-scientific asceticism

* Treatment of artifacts such as computers and abstractions as more
important than human beings

* Autism

* A fundamental lack of decency

* A mythos in which the hacker fantasizes himself as uniquely
valuable to his company when he’s in fact a dime a dozen

* Majoritarian tyranny

* Disrespect for midlevel authorities that make safe targets

* Contempt for intellectual production masquerading as a critique of intellectual property, as in the case where almost the sole criterion of value is the servile haste of the producer

* Digital Maoism (cf Jaron Lanier‘s book You Are Not a Gadget): the cynical use of elites of what Adorno calls “domination of the dominated by the dominated” (Adorno)

An exemplary string replacement function in C

Posted in Uncategorized with tags , , , , , , on February 9, 2010 by spinoza1111

I don’t much like the C programming language, but as part of my Internet anti-bullying campaign (including the campaign against the anti-Schildt swift-boating campaign mentioned elsewhere here in my blog), I am relearning it in order to make my points more effectively at the usenet/Google Group comp.lang.c.

This is where the instigator of the anti-Schildt swift boating (Peter Seebach) resides. He’d posted a piece of code purporting to replace %s by strings, but I noticed that in using the C library string.H, he’d been blinded by the poor implementation of strings in C.

He’d used a character find to locate the percent sign in two different places, and hadn’t bothered, by his own admission, to check for the following s. This meant that correcting the code involved making the same clumsy change in two separate places, guaranteeing further problems.

I discussed a real string replacement program, but actually thinking about “strings” independent of “character arrays with a Nul at the right” is beyond the day to day ken of many C programmers due to the subtle ways in which string.H blinds the soul.

I then demonstrated a straightforward replace() function, working collaboratively online with a few well-intentioned posters who were willing to find bugs.

In Swift Boat mode, Seebach of course interpreted this as “posting buggy software”. Since I don’t take shit, the fun was on at the link below.

It amused me to get back up to speed in C after many years (having helped John “A Beautiful Mind” Nash with C at Princeton in 1991) in order to prove my point: that today’s corporate programmers rarely program, are unqualified, and tend to be bullies, who see in others “the secret contour” of their own weakness, incompetence and fear.

See this link for the Google Group discussions. Warning: drink deep or drink not from this Pieran spring, for the discussion is a real free for all, and it is a big mistake to make conclusions after reading only a few posts. Indeed, the Thugs rely on superficial readings absent any sort of moral grammar, so that their favorite argument, the criminal’s tu quoque, can be used.

Here is the code of a replace() function. It is raising for me some interesting questions of theory and method which I shall pursue primarily in the thread from hell with updates here. In particular, C is strongly biased towards Western languages read left to right, and the VERY interesting problem of a string like “banana” and a replacement of ana by omo (bomona or banono?) needs to be addressed.

Even more interesting would be the relationship of the straightforward-if-crude algorithm used below and the Boyer Moore algorithm.

Also, an argument, germane to some possible real-world applications, could be made that “banana” contains TWO partly overlapping occurences of ana, and could be converted to bonono! Boyer Moore may anticipate this: I need to recheck this source.

But: the Google Groups discussion is why I’m delighted to be a teacher today. My father asked me what would happen when all the code was written. The answer seems to me to be that a eunuch priesthood of machine tenders, without qualifications, courage, spirit, or heart, would then tear at each other for sinecures maintained only because someone needs to hold down the null function in the corporation.

Here is the code.

// *************************************************************** 
// *                                                             * 
// * replace() demo                                              * 
// *                                                             * 
// * Demonstrates how to replace non-NUL-defined strings in C    * 
// * using a simple function, and bypassing string.h.            * 
// *                                                             * 
// * C H A N G E   R E C O R D --------------------------------- * 
// *   DATE     PROGRAMMER     DESCRIPTION OF CHANGE             * 
// * --------   ----------     --------------------------------- * 
// * 02 07 10   Nilges         Version 1.0                       * 
// *                                                             * 
// * 02 07 10   Nilges         Bug: partial matches not handled  * 
// *                           correctly: need to iterate search * 
// *                           for match.                        * 
// *                                                             * 
// * 02 07 10   Nilges         1.  Santosh suggested tests       * 
// *                           2.  Heathfield put the boot in re * 
// *                               including malloc              * 
// *                                                             * 
// * 02 07 10   Nilges         1.  Remove string.h use and code  * 
// *                               strlen by hand                * 
// *                           2.  Add comment block and comments* 
// *                               inline.                       * 
// *                           3.  free() storage                * 
// *                           4.  Use macro for testing         * 
// *                           5.  Bug: calculation of           * 
// *                               lngNewLength used incorrect   * 
// *                               index (intIndex3 instead of 2)* 
// *                               which caused a memory leak.   * 
// *                                                             * 
// * 02 07 10   Nilges         1.  Bug: Ike Naar test failed.    * 
// *                               At end of scan loop, main     * 
// *                               string pointer was not        * 
// *                               correctly updated from index3 * 
// *                                                             * 
// * 02 07 10   Nilges         Added new Santosh test            * 
// *                                                             * 
// * 02 07 10   Nilges         Added new Ike Naar test           * 
// *                                                             * 
// * 02 08 10   Nilges         1.  Added some new comments       * 
// *                           2.  Make "check for a complete    * 
// *                               match "structured" by means   * 
// *                               of a tested assignment        * 
// *                           3.  Get rid of "replace at end"   * 
// *                               evilness: the only time this  * 
// *                               flag is meaningful is in the  * 
// *                               LAST segment.                 * 
// *                           4.  Return replace count          * 
// *                           5.  TESTER macro assertion added  * 
// *                                                             * 
// * 02 10 10   Nilges         1.  Bug fix: in a partial match,  * 
// *                               the main scan index is set to * 
// *                               one past the end, which misses* 
// *                               full matches that start       * 
// *                               between the first character of* 
// *                               the partial match and its end.* 
// *                                                             * 
// *                               No longer updating the main   * 
// *                               scan index (ptrIndex1) to the * 
// *                               point of partial match        * 
// *                               failure: setting it one past  * 
// *                               the first character of the    * 
// *                               partial match.                * 
// *                                                             * 
// *                           2.  Line up expected & actual     * 
// *                               results per excellent         * 
// *                               suggestion (who made this?)   * 
// *                                                             * 
// *                           3.  After a partial match, update * 
// *                               the main handle search index  * 
// *                               (ptrIndex1) to the first      * 
// *                               occurrence of the handle      * 
// *                               character after the start of  * 
// *                               the partial match, or to the  * 
// *                               index of the unmatched char.  * 
// *                                                             * 
// * 021310     Nilges         Bug: failed to handle a one-char  * 
// *                           replace (of a by x in b) since    * 
// *                           we set ptrIndex2 to point to NUL  * 
// *                           which made it appear that there   * 
// *                           was a match. When the main index  * 
// *                           to the master string (ptrIndex1)  * 
// *                           goes off the end of a cliff, we   * 
// *                           needed to break out of the search * 
// *                           loop.                             * 
// *                                                             * 
// *                           This also entailed setting the    * 
// *                           target index ptrIndex2 to 0 at the* 
// *                           beginning of each search so that  * 
// *                           it is not erroneously used to     * 
// *                           indicate that there's an insert   * 
// *                           needed at the end.                * 
// *                                                             * 
// *                           I have also added additional tests* 
// *                           to verify that the code works     * 
// *                           when the string and target end    * 
// *                           at the same time.                 * 
// * ----------------------------------------------------------- * 
// *                                                             * 
// * "In the near future we shall have to live with the          * 
// *  superstition that programming is 'so easy that even a      * 
// *  Republican can do it!'"                                    * 
// *                                                             * 
// *                   - E. W. Dijkstra                          * 
// *                                                             * 
// *                                                             * 
// *************************************************************** 
#include <stdio.h> 
#include <stdlib.h> 
// ***** Segmentation ***** 
struct TYPsegmentstruct 
       { char * strSegment; 
         long lngSegmentLength; 
         struct TYPsegmentstruct * ptrNext; }; 
// --------------------------------------------------------------- 
// Calculate string length 
// 
// 
long strLength(char *strInstring) 
{ 
    char *ptrInstring; 
    for (ptrInstring = strInstring; *ptrInstring; ptrInstring++); 
    return ptrInstring - strInstring; 
} 

// --------------------------------------------------------------- 
// Replace target by replacement string in master string 
// 
// 
// Caution: the string returned by this function should be freed. 
// 
// 
char * replace(char * strMaster, 
               char * strTarget, 
               char * strReplacement, 
               long * ptrReplacements) 
{ 
    char * ptrIndex0; 
    char * ptrIndex1; 
    char * ptrIndex2; 
    char * ptrIndex3; 
    char * ptrIndex4; 
    char * strNew; 
    char * strNewStart; 
    long lngNewLength; 
    long lngCount; 
    long lngReplacementLength; 
    struct TYPsegmentstruct * ptrSegmentStructStarts; 
    struct TYPsegmentstruct * ptrSegmentStruct; 
    struct TYPsegmentstruct * ptrSegmentStructPrev; 
    lngReplacementLength = strLength(strReplacement); 
    if (!*strTarget) 
    { 
        printf("Error in calling replace(): target can't be null"); 
        abort(); 
    } 
    ptrIndex1 = strMaster; 
    ptrSegmentStructPrev = 0; 
    lngNewLength = 0; 
    *ptrReplacements = 0; 
    while(*ptrIndex1) 
    { 
        ptrIndex0 = ptrIndex1; 
        ptrIndex2 = 0; 
        while (-1) 
        { 
            // --- Check for (one character) handle 
            for(; 
                *ptrIndex1 && *ptrIndex1 != *strTarget; 
                ptrIndex1++); 
            if (!*ptrIndex1) break; 
            // --- Check for complete match while remembering the 
            // --- last position of the handle 
            ptrIndex4 = 0; 
            for(ptrIndex2 = strTarget + 1, 
                ptrIndex3 = ptrIndex1 + 1; 
                *ptrIndex3 
                && 
                *ptrIndex2 
                && 
                *ptrIndex3 == *ptrIndex2; 
                ptrIndex3++, ptrIndex2++) 
                { 
                    if (*ptrIndex3 == *strTarget 
                        && 
                        ptrIndex4 == 0) ptrIndex4 = ptrIndex3; 
                } 
            // End test: check complete match, update main ptr past 
            // partial match while checking for end of loop 
            if ((!*ptrIndex2 ? ((*ptrReplacements)++, -1) : 0) 
                || 
                (!*ptrIndex3 ? (ptrIndex1 = ptrIndex3, -1) : 0)) 
                break; 
            // Update the main search pointer 
            ptrIndex1 = (ptrIndex4 == 0 ? ptrIndex3 : ptrIndex4); 
        } 
        // --- Create new segment 
        if (!(ptrSegmentStruct = 
              malloc(sizeof(struct TYPsegmentstruct)))) 
            abort(); 
        ptrSegmentStruct->strSegment = ptrIndex0; 
        ptrSegmentStruct->lngSegmentLength = 
            ptrIndex1 - ptrIndex0; 
        ptrSegmentStruct->ptrNext = 0; 
        if (ptrSegmentStructPrev != 0) 
            ptrSegmentStructPrev->ptrNext = ptrSegmentStruct; 
        else 
            ptrSegmentStructStarts = ptrSegmentStruct; 
        ptrSegmentStructPrev = ptrSegmentStruct; 
        // --- Update mallocation length 
        lngNewLength += ptrSegmentStruct->lngSegmentLength + 
                        (ptrIndex2 && !*ptrIndex2 
                         ? 
                         lngReplacementLength 
                         : 
                         0); 
        // --- Get past end of target string & iterate 
        if (*ptrIndex1) ptrIndex1 = ptrIndex3; 
    } 
    // --- Allocate just enough storage for the new string 
    if (!(strNewStart = malloc(lngNewLength + 1))) abort(); 
    // --- Build the new string whilst freeing the list 
    strNew = strNewStart; 
    ptrSegmentStruct = ptrSegmentStructStarts; 
    while (ptrSegmentStruct) 
    { 
        for (ptrIndex1 = ptrSegmentStruct->strSegment, 
             lngCount = 0; 
             lngCount < ptrSegmentStruct->lngSegmentLength; 
             ptrIndex1++, lngCount++, strNew++) 
             *strNew = *ptrIndex1; 
        if (ptrSegmentStruct->ptrNext 
            || 
            ptrIndex2 != 0 && !*ptrIndex2) 
            for (ptrIndex1 = strReplacement; 
                 *ptrIndex1; 
                 ptrIndex1++, ++strNew) 
                 *strNew = *ptrIndex1; 
        ptrSegmentStructPrev = ptrSegmentStruct; 
        ptrSegmentStruct = ptrSegmentStruct->ptrNext; 
        free(ptrSegmentStructPrev); 
    } 
    *strNew = '\0'; 
    return strNewStart; 
} 

// --------------------------------------------------------------- 
// Statement-format test macro 
// 
// 
#define TESTER(resultPtr, master, target, replacement, expected, 
expectedReplacements, replacements) \ 
{ \ 
    printf("Replace \"%s\" by \"%s\" in \"%s\"\n", \ 
           (target), (replacement), (master)); \ 
    printf("Expect \"%s\":\n       \"%s\"\n", \ 
           (expected), \ 
           resultPtr = replace((master), \ 
                               (target), \ 
                               (replacement), \ 
                               &(replacements))); \ 
    printf("Replacements expected: %d: replacements: %d\n", \ 
           (expectedReplacements), \ 
           (replacements)); \ 
    if (!(strLength(resultPtr) \ 
        == \ 
        strLength(master) \ 
        + \ 
        (strLength(replacement)-strLength(target)) \ 
        * \ 
        replacements)) \ 
        printf("Assertion failed\n"); \ 
    printf("\n\n"); \ 
    free(resultPtr); \ 
} 

// --------------------------------------------------------------- 
// Main procedure 
// 
// 
int main() 
{ 
    char *ptrResult; 
    long lngReplacements; 
    printf("\nReplace\n\n\n"); 
    TESTER(ptrResult, 
           "1111123bbb1111123bbb11123bb11111231111112111111123", 
           "111123", 
           "ono", 
           "1onobbb1onobbb11123bb1ono1111112111ono", 
           4, 
           lngReplacements) 
    TESTER(ptrResult, 
           "bbb1111123bbbbb", 
           "111123", 
           "ono", 
           "bbb1onobbbbb", 
           1, 
           lngReplacements) 
    TESTER(ptrResult, 
           "a stupid error", 
           "stupid error", 
            "miracle", 
           "a miracle", 
           1, 
           lngReplacements) 
    TESTER(ptrResult, 
           "a stupid error", 
           "stupid", 
            "miracle", 
           "a miracle error", 
           1, 
           lngReplacements) 
    TESTER(ptrResult, 
           "the stupid error", 
           "the stupid error", 
            "a miracle", 
           "a miracle", 
           1, 
           lngReplacements) 
    TESTER(ptrResult, 
           "the miracle", 
           "the", 
            "a", 
           "a miracle", 
           1, 
           lngReplacements) 
    TESTER(ptrResult, 
           "a miraclsnirpKamunkle", 
           "snirpKamunkle", 
            "e", 
           "a miracle", 
           1, 
           lngReplacements) 
    TESTER(ptrResult, 
           "a miraclesnirpKamunkle", 
           "a miracle", 
            "", 
           "snirpKamunkle", 
           1, 
           lngReplacements) 
    TESTER(ptrResult, 
           " a miraclesnirpKamunkle", 
           "a miracle", 
            "", 
           " snirpKamunkle", 
           1, 
           lngReplacements) 
    TESTER(ptrResult, 
           " a miraclesnirpKamunklea miraclea miracle", 
           "a miracle", 
            "", 
           " snirpKamunkle", 
           3, 
           lngReplacements) 
    TESTER(ptrResult, 
           "a miracle a miraclesnirpKamunkle a Miraclea miraclea 
miracle", 
           "a miracle", 
           "", 
           " snirpKamunkle a Miracle", 
           4, 
           lngReplacements) 
    TESTER(ptrResult, 
           "a stupid errord", 
           "stupid error", 
           "miracle", 
           "a miracled", 
           1, 
           lngReplacements) 
    TESTER(ptrResult, 
           "a stupid errod", 
           "stupid error", 
           "miracle", 
           "a stupid errod", 
           0, 
           lngReplacements) 
    TESTER(ptrResult, 
           "a sstupid error", 
           "stupid error", 
           "miracle", 
           "a smiracle", 
           1, 
           lngReplacements) 
    TESTER(ptrResult, 
           "a stupid errorstupid error", 
           "stupid error", 
           "miracle", 
           "a miraclemiracle", 
           2, 
           lngReplacements) 
    TESTER(ptrResult, 
           "a stupid error stupiderror", 
           "stupid error", 
           "miracle", 
           "a miracle stupiderror", 
           1, 
           lngReplacements) 
    TESTER(ptrResult, 
           "bbbbbbbbbb", 
           "b", 
           "a", 
           "aaaaaaaaaa", 
           10, 
           lngReplacements) 
    TESTER(ptrResult, 
           "In the halls of R'yleh great %s lies dreaming", 
           "%s", 
           "Cthulu", 
           "In the halls of R'yleh great Cthulu lies dreaming", 
           1, 
           lngReplacements) 
    TESTER(ptrResult, 
           "%s%s%s%s%s%s", 
           "%s", 
           "Cthulu", 
           "CthuluCthuluCthuluCthuluCthuluCthulu", 
           6, 
           lngReplacements) 
    TESTER(ptrResult, 
           "banana", 
           "ana", 
           "oat", 
           "boatna", 
           1, 
           lngReplacements) 
    TESTER(ptrResult, 
           " a stupid errorstupid errorHeystupid errors", 
           "stupid error", 
           "+", 
           " a ++Hey+s", 
           3, 
           lngReplacements) 
    TESTER(ptrResult, 
           "foo barfoo barf", 
           "foo bar", 
           "bas", 
           "basbasf", 
           2, 
           lngReplacements) 
    TESTER(ptrResult, 
           "abab", 
           "ba", 
           "ba", 
           "abab", 
           1, 
           lngReplacements) 
    TESTER(ptrResult, 
           "abab", 
           "bab", 
           "boop", 
           "aboop", 
           1, 
           lngReplacements) 
    TESTER(ptrResult, 
           "banana", 
           "ana", 
           "ono", 
           "bonona", 
           1, 
           lngReplacements) 
    TESTER(ptrResult, 
           "a", 
           "x", 
           "b", 
           "a", 
           0, 
           lngReplacements) 
    TESTER(ptrResult, 
           "x", 
           "x", 
           "b", 
           "b", 
           1, 
           lngReplacements) 
    TESTER(ptrResult, 
           "egregious", 
           "egregious", 
           "egregious", 
           "egregious", 
           1, 
           lngReplacements) 
    TESTER(ptrResult, 
           "egregious", 
           "egregious", 
           "x", 
           "x", 
           1, 
           lngReplacements) 
    printf("\n\nTesting complete: check output carefully: \"Assertion 
failed\" should not occur!\n\n"); 
    return 0; 
} 

Is Open Source theft of intellectual production and virtual slavery?

Posted in Uncategorized with tags , , , , on June 27, 2009 by spinoza1111

Is “open source” theft of intellectual production and virtual slavery?

“Open source” is the making-available of software source code and other media freely with permission to modify source code and original media. Examples include the Linux and Ubuntu operating systems and the content of wikipedia. A related term from Richard Stallman of the Free Software Foundation is “freeware” which is software provided free of charge in source form. In effect and according to Stallman, most Open Source is freeware and most freeware is O.S.

Open Source is “intellectual production” and not “intellectual product”. However, it appears to me that the concept, together with the architecture of the Web, obscures who writes it and in significant cases enables a virtualized, but not victimless, theft.

It did allow developers to escape the harsh corporate discipline of software shops, one manifestation of which was the constant necessity for overfocus on overdefined goals. That is: designing an effective software system for either a large or basically hard problem is designing a language and Wittgensteinian “form of life” for solving problems in an environment subject to rapid change, but it’s been a common experience that managers would prefer programmers not to do so, but instead deliver, at a quick pace, specific hard-coded modules that would appear to work, and “satisfy” an “end user”.

Whereas because much of Open Source and “open content” is written, modified and debugged by unpaid volunteers/interns, it can be more generally and reliably developed as seen in the case of Linux, which is of higher quality than the closed source monster, Windows.

Open Source was prefigured, in my experience, in the closed shops of the mainframe. Basically, early programmers (of which I was, in a sense, one) lavished time on what seemed at first to be silly goals, often their own personal time, only for the software they created to be key/strategic. For example, noticing that I was maintaining fifty small assembler programs to handle requests for selecting alumni of Roosevelt University, I, working on my own time, designed a language for specifying these requests and laying out the output report, from envelopes to (ta da) mailing labels.

Working happily, and oblivious to the hours or the complaints of my girlfriend, I produced a replacement for the previous programs which appears to have saved a lot of time, even after I left the university’s employ. I encountered the same type of program buried in libraries on Princeton’s mainframe years later, written in assembler as well.

The fraction of time I spent after hours was “open source” in a sense.

This continued, and I was never loth to work extra time to be able to solve a genuine, and genuinely interesting, problem, and I found that many such problems, falsely renarrated as specific “business” problems, lurked behind many vanilla applications. Reinsurance turned out to be a recursive application which needed a stack. Billing for pbx calls of any complexity turned out to be the need to simulate the operations of the pbx based on the only records available, which were of atomic telephony events.

A rational internal compiler for use of field engineers to control PBXs turned out to need a rational installation process itself controlled by software. And in many cases, the users either needed or wanted to enter logic (criteria) as data, and I either needed to write small compilers or, when I was more fortunate in the chosen language, to present the user’s logic to the language for runtime compile and execution…this was duck soup in Rexx and Quick Basic, not so easy in compiled languages.

Software turned out to be, as the creation not of code but of Wittgenstein’s forms of life, an activity that in capitalist relations always had a large “margin”, a “margin” that could not be spoken of for fear of the suits.

Consider a “real” square, drawn “perfectly” either with Microsoft Paint or a compass and straightedge. In all cases it isn’t a square, just a picture of a square whose “edge” is never Euclidean because its edge always has some thickness > 0. In computer graphics and presentation, the programmer has to learn the difference between what is called the “size”, say of a Windows “form” (box on the screen) and its “client size” for this reason. Only in special cases (a borderless box colored differently from its background) is client size == size.

[Would the only "real" square, isomorphic to the Platonic-Euclidean be either a square in a Mondrian painting or a Microsoft Paint square with no border but a different color from the background? The Mondrian-painted square always has bumps and painterly gestures that disqualifies it. This leaves the Microsoft Paint square which has no painterly "edge" as the closest to the Platonic square. But it's not perfect if space is curved, is it? There could also be micro-differences in height and width if pixels, whose size determine both, are not perfectly square.]

Mondrian and a red square

In general the problem is of the margin, and the fact that great programmers have always, in closed source and open, stayed up past their bedtimes. I know of no exception. This may be the result of the fact I identified in “Computer Software as a Philosophical Pathology“: computer memory is always finite and there’s always one more “bug” as a result.

Open source does not of course eliminate the theoretic problem, but it does free corporations from overseeing the creations of forms of life. Instead, they can legally dicker with the open source .org people to obtain commercially viable copies, and in so doing, I do not believe that the interests of the original creators are protected at all.

In fact, Open Source makes the whole process so opaque and so renders actual creators of content so anonymous as to resemble the takeover of Russia’s aging but huge industrial plant in the 1990s by gangstas and “biznezmienie” who were, in many cases, favorably situated sons of the Communist *nomenklatura*, men and women of the Stalinist bureaucracy who’d used prestige educations and inside information to essentially steal the people’s shares in industrial plant.

Programmers regard themselves as creative, professional “engineers”, but were never accorded this dignity from outside the paraprofession. Managers, lawyers, and other truly independent professionals had nothing but contempt for programmers, and as a condition of employment, the programmers had to agree to this low status, and allow their intellectual product to be underpaid and alienated in Closed Source regimes.

Open Source merely mystified and compounded this problem.

The most egregious example is wikipedia. Here, a sharpie and arbitrageur of other people’s use values, Jimmy Wales, was unable to manage people circa 2000 because of his personality problems. He therefore created a tax-exempt .org circa 2002 where “anyone” could change his encyclopedia and add content. The idea wasn’t his, it was Dr. Larry Sanger’s, but what the hell.

In 2004, the spirit of “be bold” and a certain collegiality reigned: but circa 2006, strange editors started to appear. Ignorant of actual scholarship and very rude to others while oversensitive to slights in the lower middle class register, these characters proceeded to hound and persecute contributors who did not conduct themselves with the right sort of subervience…a subservience last seen in the ante-bellum South of the USA.

Reputations were permanently ruined, and good contributors were driven out by “editors” who in many cases turn out to be convenience store clerks and religious obsessives, pre-Enlightenment characters not qualified to create a dictionary let alone an encyclopedia.

Mere thinking became “original research”, and in the manner of vile little clerks throughout history, rules were interpreted with a soul-destroying literality by people who, owing to their very lack of internal self-discipline, need rules at all time.

I quickly became a monstrum horrendum of course, having very low tolerance for this sort of thing, and the nastiness of the process became unbounded. For example, posters seeking to curry favor transformed my patronym (Nilges) into a racist slur as had my racist fellow students (including the racist, right wing rock star Ted Nugent) in high school. It was lynchin’ time, and I believe it became so because Jimbo Wales needs money.

The work of the eradicated was retained and their eradication ensures now that Wales shall be able to turn wikipedia into a profitable paper edition, which appears to be his real goal after all.

This provides Massah Wales (an American southerner) with some of the advantages of slavery (free labor product) without the ugly image of slave-driving, and I think it may generalize to Open Source.

Open Source has a “liberal” reputation because of the unreflective binary opposition in the computer field between the corporate fuddy duddy and the bearded, and therefore presumably free, hippie coder…an unreflective binary opposition which neglects the way in which IBM salesmen received a modicum of dignity and respect in the depths of the Depression by putting on white shirts and ties, and singing the IBM song, and the way in which Thomas J Watson opened up employment to people of color before EEO legislation.

Basically, the innocent American assumption is that “out there” there is a land of unstructured freedom, owning a house for no down payment, free of negative bringdown signifiers such as Hegel’s state and the corporate state, and this American dream keeps people in line, and subject to a much harsher regime than they’d otherwise accept…including the theft of their intellectual production.

“Intellectual production” is my own term of art, and it is meant to contrast with “intellectual property”. Our society has an excessive respect for the latter and none at all for the former, the work product of the forgotten man.

In 1976, Bill Gates called a spade a spade. He told the hobbyist-hippies of the Altair generation that they weren’t “free”: they were, he said, pirates and slaves because he and Paul Allen earned two dollars an hour net for producing the first useful Basic compiler for microcomputers. It had been stolen and widely disseminated by early computer thugs, some of whom bothered to claim to be “anarchists”.

But it was Microsoft which made millionaires out of secretaries and programmers from inferior universities as had the old IBM, and it did so by protecting its source code…religiously. Whereas if access to the means of production is made obscure, insiders with networks obtained at expensive universities have, whether post-Communism in Russia or in software, a built-in advantage.

Return to barbarism starts with this sort of theft. Corporate America never really figured out how to treat programmers with basic decency and respect, which destroyed a port of entry to the middle class open to talents in the 1960s. Open source, far more than offshore development, replaced paid employees with time slices of hidden slaves; in my experience, offshore creates jobs!

The result is a world of pirates and slaves, and a lack of interest in software careers.

Regular expressions and “the horror!”

Posted in Uncategorized with tags , , , on May 17, 2009 by spinoza1111

Jeff Atwood has long blogged about “coding horrors”, but reading this post crystallizes for me why I’ve left the programming field.

This is because like most folklore about programming, the sage advice ignores the essentials.

Regular expressions were originally purely mathematical expressions which could be ambiguous (such as “*A”, anything followed by A, where “anything” might include A). They could be ambiguous because their function was to declare a class of languages.

“*A” (“anything A”) doesn’t preclude A from appearing more than once in a string in the language it defines because it is silent about what follows the last A. Unlike the corresponding regular expression code in a “real” regular expression used by a programmer, “*A” is silent about what follows A.

For the mathematician, “anything A” is implicitly followed by indeterminancy. He will interpret “anything A” as possibly including the letter A in anything since it’s not followed by “end of signal” or “nothing”.

Mathematically, “anything A” is perfectly fine, or, if you prefer, well-formed mathematical expressions are beyond good and evil and must be taken on their own terms. For example, the “limit” of calculus, were it a programming tool or concept, would be unacceptably vague. Science deals with partial knowledge, but in programming we try to control everything.

Which means the programmer has to translate the math to “*A$” where the dollar sign is some character meaning “end”.

But “end of input” never occured, as far as I know, the John von Neumann or Claude Shannon, two of the original mathematicians to work with mathematical regular expressions. This is because end of input has no special mathematical meaning.

Consider this. The programmer defines $ (end of input) to mean “what happens at the end of an input string”, and he is thinking about phenomena without mathematical significance, such as some user pressing the Enter key.

But the mathematician is not at liberty to delimit his input (and it is a programmer failing to see how the latter’s ability to set norms and conventions for his work makes him a technician, not a scientist). If the mathematician is told, $ means end of input, he will ask, what come after that? To him, $ is just another symbol, without the power to forbid other symbols to follow it.

John Cusack, in the 1989 film Say Anything, is asked by his girlfriend’s father’s criminal tax accountant what he wants to do with his life. Cusack says, I dunno, I just want to hang out with Mister So and So’s daughter. When pressed, Cusack says, well I don’t want to “process” anything.

Likewise, the mathematician, unlike the programmer, is uninterested in “processing data”, so to a mathematician, $ have no meaning…just as Cusack seems to the criminal tax accountant to be uninterested in Holy Money.

“Anything A” is mathematically meaningful but lousy “code”. This is because it cannot be a subexpression in a larger regex without trouble.

Consider “*AB”. As code it doesn’t specify whether AB can appear as part of “anything”. Worse, different regular expression systems in common use will treat this situation differently, and are documented in ways that imply that there is more than one way of interpreting “*”. Here, the programmer is just kidding in using asterisk: he means “anything except, of course, AB, silly”, and worse, some regexp systems pander to this sloppy thinking.

The problem is that sequencing (one basic RE subexpression such as “*” followed by another such as “AB”) is not thought of consciously as a regular expression operator of equal status with alternation (*|AB). But concatenation, sequencing is a genuine operator, with three possible interpretations:

1. If the handles of the sequenced pair share a non-null common set of characters, refuse to proceed.
2. If the handles share this non-null intersection, prefer the left hand side
3. If the handles share this prefer the rhs.

Programmers are if Prometheus clumsy, for they pervert mathematics.

Which brings me to Jeff’s article. The credit crisis alone, which has a large software component, indicates that there’s something very wrong with programming as folklore.

Programming-as-folklore consists of pseudosage advice meant to avoid difficulty. Now, there’s nothing wrong with artisanship and craft, but programming folklore replaces mathematics.

In mathematics, “anything A” means “anything followed by the letter A, including AA or AAA”. In programming it might or might not be a bug, because no independent effort has been made to “standardize” regular expressions.

But “standardization” brings me to the real horror…the Heart of Darkness, if you will, or, if you prefer, Apocalypse Now.

“Standardization” is a bureaucratic ersatz for something hero computer scientist Edsger Dijkstra said he wanted in his 1972 Turing Award lecture and didn’t get. Dijkstra had been horrified, like Mistah Kurtz, by the IBM 360.

He was horrified by the IBM 360 because after several years of work which had shown the importance of being able to save the overall state of a computation quickly, safely, and in the general case, the designers (led by Gene Amdahl) decided that there needed instead to be 16 [sic] “general-purpose registers” named R0..R15…and no stack in sight. As a direct result, in the 1970s, I knew several bitter, twisted, and prematurely aged assembler programmers who each had come to terms, each in his own way, with what you do when you call a subroutine.

Amdahl’s anti-intellectualism treated the actual normal case, one of almost overwhelming complexity in real workplaces, as something that would almost never obtain: his reasoning resembled that of Columbia space shuttle engineers who reasoned that because by 2003 large chunks of foam had not dropped off Columbia, things would continue as before. But in reality, there were all sorts of complex applications for the 360 even in the “ordinary” business world.

For example, graphs, lattices, trees and recursion are thought to be “highly theoretical” by “ordinary business programmers”: but I encountered recursion implemented foolishly and by accident in software at a re-insurer (an insurance company that insured other companies against loss). The difficulty of “unwinding” securitized mortgages in today’s credit crisis is caused by the fact that in the large, a securitized instrument is a vast tree of financial relationships, encoded in software. In some cases, this branching tree may be a graph, for nothing in many systems prevents a derivative from being linked to itself through an indirect chain.

Dijkstra felt that some sort of tribunal of academic people should vet new computers for their mathematical safety and correctness. Suffice it to say, he was ignored, with the ersatz being “standardization” being driven by manufacturer greed, as in the absurd case of the “standardization” of C.

But this is a special case of something so general as to constitute our world: for the fact is that people, in an administered world, are so administered, and so fucked-over, from cradle to grave as to not be able to think outside the real box. In the 1950s, in Britain especially, in the USA to a degree, this was the welfare state, but the welfare state was of course entirely too much the carrot and not the stick, so the carrot is also the stick.

As such, when Dijsktra mentions a tribunal of academics, we get a standardization committee. Such committees, however, are unimpressed with anything like mathematical elegance.

“Regular expressions” retain for this reason a meaning only in mathematics. In programming, they are whatever Perl, or regex, chooses them to be.

I bailed happily out of the cynical world this created in programming.

“Verbosity”: a typical exchange

Posted in Uncategorized with tags , , , , on April 5, 2009 by spinoza1111

Here’s a typical comment on language from the lower middle class zone of programming. In Chicago, I worked for a “consulting” firm located in the same building as a porn studio, and as a tool of control, the management enforced that notion of “verbosity” where “verbosity” is the unexpected ability, coming from a mere programmer, to construct a complex sentence: the comment is typical of remarks made by the managers of that firm. The comment appears in a discussion at the Amazon site of my book, “Build Your Own .Net Language and Compiler”.

What a silly comment! You’re the author of this book? Good Christ. At least you shoud have kept quiet, not making your incompetence and even illiteracy obvious.

“Parr uses a compiler generator which is OK, although with modern editing tools and understanding it’s straightforward to “write” a manual parser, and, such assisted writing assists understanding.” — this is an example of the accursed gibberish that you should have avoided posting.

Here is my reply:

No, the example of my writing you provide is a correctly formed complex sentence: the first sentence of your reply above, on the other hand, contains four separate errors:

1. Since “Good Christ” is an exclamation, it needs to be terminated with an exclamation point.

2. “Shoud” is misspelled and it “should” be spelled as per.

3. Since “making” is in the scope of “should have”, it “shoud” have been, it shuddha have bin, a participle and not a gerund. In a verb phrase, including this one, in which the helper verb distributes over two opposed alternative main verbs (kept/make), the main verbs must follow a universal rule: have, has and had are followed by a participle in all cases. In fact, a subtler error, made by many pompous fools like you, is to use the present tense in the second main verb, but the regular participle “made” is the choice of literate people: “At least you should have kept quiet, not made your brilliance obvious: that is pearls before swine”.

4. To the ordinary slob, such as yourself, “even” is *just* an adjective. It’s not. This is evident from the fact that unlike an adjective, it can front pronouns: adjectives cannot: “Even he was aghast at Ghost’s idiot post”. The OED specifies that as an adjective, it means only “level”, thus a literate, but senseless, interpretation of your foolish post is that my illiteracy is “level”. Careful writers would write “even your illiteracy” since an adjective never fronts a possessive pronoun in a noun phrase, and this makes “even” into an adverb that modifes your solecism “making” (which, as I have said, should be “made”).

The ability of many people, including people with fancy degrees from expensive schools, to either write or comprehend such syntax as mine is in rapid decline. But it is not always the case that the thought could have been expressed more simply. The fact is that with modern tools and good understanding of recursive descent parsing, it is easier to write a manual parser than to hack lexx and yacc. Furthermore, it is a bubble-butt American illusion to think that working, thinking, and learning are disjoint. The act of writing and with restraint commenting a manually-written parser is a form of understanding, as you should be able to see (but probably won’t be able to see) if you download the code associated with my book.

As it is, the decline of understanding across the board, whether of programming or the English language, may be expected to accelerate as scholarship students are pushed away from universities who need the hard currency of wealthy foreign and US students, post-bailout.

Your post, with its numerous solecisms, is good for a laugh. However, your species of pompous and foolish aliteracy appeared on the Internet circa the first Iraq war when I was charged, falsely, with an error in using “aliteracy” (to mean not the inability to read or write: to mean the refusal and the preference for TV) in an antiwar post. Your aliteracy, shading to illiteracy is connected with the growth of international and interpersonal barbarism.

I can write: you cannot. I can code: but I discovered over thirty years that most people cannot: one result, predicted by Dijkstra in 1976, is the collapse of Western civilization, in large part due to the idiot coding of structured financial instruments. Your post gives evidence that the rot is thoroughgoing, and has no more significance than the ravings of a crazed sixth century monk about what he thinks is “correct” dog Latin.

Have a nice day.

My reply is out of scale, of course, and it’s the sort of reply that will get one fired if aired at the typical business meeting.

In American computer programming, the fear as expressed by the commenter is of having one’s “incompetence” exposed. There’s a passage from Adorno’s Minima Moralia:

But far from finding anything inimical in the prohibitions on thinking, the candidates – and all scientists are candidates for posts – feel relieved. Because thinking burdens them with a subjective responsibility which their objective position in the productive process does not allow them to meet, they renounce it, shiver a bit, and run to join their opponents. Dislike of thinking rapidly becomes incapacity for it: people who can effortlessly discover the most sophisticated statistical objections when it is a question of sabotaging a piece of knowledge, are unable to make ex cathedra the simplest predictions. They hit out at speculation and in it kill common sense. The more intelligent of them suspect the sickness of their intellectual powers, since it first appears not universally but in the organs whose services they sell. Many wait in fear and shame for their defect to be discovered. But they all find it publicly acclaimed as a moral achievment, and see themselves recognized for a scientific asceticism which to them is none, but the secret contour of their weakness. Their rancour is socially rationalized with the argument: thinking is unscientific. At the same time, their mental power has, in a number of dimensions, been prodigiously increased by control mechanisms. The collective stupidity of research technicians is not simply an absence or regression of intellectual faculties, but a proliferation of the thinking faculty itself, which consumes thought with its own strength.

After I fled the “consulting” firm for a much more satisfying job doing compiler development in Silicon Valley, which remained satisfying until objective circurmstances caught up with that company and changed it into a money machine for the top men of its parent, some of whom have now been jailed, I read this passage.

Since the “consultants” at the “consulting” firm were paid like the lawyers in John Grisham’s book The Firm, lavishly, they were held hostage to a competitive and nasty environment of secrets and lies. We were required to attend a “team building” lecture and dinner every week: note that when firms destroy solidarity, Job One becomes the creation of all sorts of false culture and false solidarity in the form of social events which resemble Hitler Youth festivals in some respects, accenting as they do the systematic humiliation of outlier characters, and representations of customers and clients, as a safety valve which never works.

Programming competence was never completely defined throughout the history of American programming, a history which has come to an end with the globalization of software work. In the pages of a then-popular trade journal, it was defined as the complement of the set “academic theorizing”: Fortran was good and Algol was bad because Fortran didn’t originate inside a university, whereas Algol did, and Algol’s louche, Left Bank, and flaneur connotations were confirmed by its European origin.

But to define something as the complement of a set isn’t precise. It was made deliberately impossible to know what constituted good, artisan-like, solid and workmanlike design in business programming.

I realized that the issue wasn’t technical … it was a matter of the social control of university graduates who, when unemployed and unemployable as they have often been in developing economies, start reading Marx. The pornographic company was an instance of social control that had evolved quite independently of the personality of its founder, who himself was “academic” and a “geek” but in self-hatred created a machine for the destruction of academic geeks.

Leszek Kolakowski, in Main Currents of Marxism, emphasises that the capitalist does not buy labor time: he buys labor power. Programmers believe that their “skills” allow them control over their labor time, and that they can work “just enough” to fund their “life styles”…only to regularly find that the system of distributing assignments works against this. When the programmer needs to work part time, the only jobs available will be 24/7. When he needs to work 24/7 to pay for a baby, you can be sure that only part time contract jobs will be available.

The firm needs, over and above the programmer’s time, something more inclusive. It needs his labor power considered as a unit.

Time saving techniques, unless blessed and sanctified, and reified, as “Rational” systems for “generating code”, are discouraged, because the telos of the system is the destruction of the purchased labour power. The programmer must at least act alienated and hollowed out as a form of reassurance.

This generated and continues to generate snarling about “verbosity” since any excess complexity is unmanageable, given that the human managers of complexity have been subordinated to the machine.

Follow

Get every new post delivered to your Inbox.

Join 538 other followers