Archive for the 'Programming' Category

« Previous Entries
Next Entries »

In search of stream-based desktop metaphors

Thursday, June 26th, 2008

I just ran across an excellent article comparing two competing desktop worldviews, documents and streams. The author argues that everything in our desktop environments is set up to support a document-based metaphor, when actually what is more relevant to the majority of our work these days is streams. He makes a very persuasive argument:

The prevailing UI paradigm today is built around the notion of document authoring. It expects that the main thing you do is create spreadsheets, word documents, presentations, and so on. There is a task bar to remind you of what documents you’re editing, there is cross-application cut and paste so you can put pieces of one document into another. You can place documents on your desktop surface itself, so you can organize your work. You can define which applications to use for which types of docs. You can set up a default printer to put your documents to hard copy. You can set up system-wide fonts to use in documents. You can put icons to apps and even documents onto your panel. And on and on. [...]

Really, what I mostly do today is stream management. And I suspect this is true for the vast majority of people. I don’t deal with writing documents, but with changes to documents. I put comments onto things. I slap patches onto things. I tweak the states of things. Once in a rare while I may author a completely new thingee, but even there I usually end up working with it as a stream of changes that I build up over time (and usually in collaboration with a few other people who stream changes to me).

I’m sold.

The problem is, our virtual desktops (and pretty much all OSes fail equally at this) do not support stream-centric interfaces to data. I can create discrete files just fine, even organize them into nice little directories, but what about my precious streams? I’m talking about my constantly updating server logs, the weather, stocks quotes, news, emails, instant messages, IRC messages, downloads, and more. Everything is handled separately and discordantly.

I can use an ugly hackish little program that outputs system log tails directly to my desktop. I have a Firefox plugin that tells me the current weather and a couple days’ forecasts. My investing service offers a streaming stock quote desktop application, but it only runs on Windows. Mozilla Thunderbird and Azureus pop up email notices and download completion notices, respectively. Instant messages are handled by Pidgin while incoming IRC messages are handled by X-Chat, both of which blink in my taskbar. As for the news — I can use a KDE plugin called Knewsticker that snarfs up RSS feeds. And I haven’t yet found a good way to track, say, SVN commits to the pyWikipediaBot project, so I’m stuck with getting a new email on every commit. Brilliant.

Read the rest of this entry »

Posted in Programming | 11 Comments »

Why programmers make good editors

Thursday, June 19th, 2008

A couple days ago, whilst reading a post on a well-known blog (though I no longer remember which one), I noticed an unmatched parenthesis. A long parenthetical aside, fully two paragraphs in length, was not terminated with a matching right parenthesis. This is quite an easy mistake for most to make, and I do not fault the author overmuch. The length of a parenthetical aside is inversely proportional to how likely the reader is to remember he is still in a parenthetical aside by the time he reaches its end. In this case, the person doing the reading was also the post’s author going back through to edit it, and he simply missed the missing closing parenthesis.

But I’m the kind of person who notices these errors, and I’m also the kind of person who often thoroughly analyzes situations (note that I did not say “over-analyzes”), so I got to thinking, why do I notice these kinds of errors especially well when many people tend not to? I don’t think it’s just because I personally enjoy using parentheses so much that I keep a careful watch for abused parentheses everywhere I go, like some superheroic defender of downtrodden punctuation. No, that’s not it. Then it hit me. Keeping track of matching syntax is a very important activity in my day job — computer programming. Programmers run into time-consuming compiler errors early and often if they can’t keep their parentheses, angle brackets, curly braces, and square brackets tightly wrangled.

Therefore, it’s worthwhile to keep track of syntax nesting levels in your head as you write or read code, adding a mental “+1″ for each opening character you come across and taking off a mental “-1″ for each associated closing character. By the end of the chunk of code, you should be back to the number you started off with (for my fellow computer scientists out there, the best representation of this would be a stack (i.e., push a left symbol onto it and pop a right symbol off, and the stack should be empty at the end (this is how compilers work)). I’m not saying I’m perfect at it; when I’m twenty levels of parentheses deep in a particularly ugly Lisp subroutine, I have no choice but to rely on the compiler’s auto-indentation to make matching manageable. But I definitely think I’m better than most, simply because I regularly work in an environment where it matters a lot.

So as I’m reading prose and I encounter a left parenthesis, some kind of state subconsciously switches in my mind, and I go into parenthetical aside mode. I stay in that mode until a right parenthesis is encountered. If one isn’t forthcoming, I quickly scan ahead in the text to see if there even is one, or maybe there was one already that I simply missed. More often than not, the author has simply forgotten the closing parenthesis. It is my experience that long parenthetical asides are more rare than unclosed short ones. This same mental trick even works for parenthetical asides inside of parenthetical asides and even parenthetical asides inside of parenthetical asides inside of parenthetical asides, but that’s about as far as I go. Luckily for me, you don’t tend to see levels of parenthetical nesting four or more deep, and if you do, it’s probably some Lisp programmer forgetting in what medium they’re writing. If the latter is the case, watch out for cars and cdrs as well.

But I don’t just notice mismatching syntax errors in the written word; I tend to notice all errors (so long as I’m reading with the intent of editing, anyway; when I’m speed reading, I often miss errors on account of not seeing them). I’m a good editor — and not to sound vain, I’ll balance that out by saying I’m a terrible bowler. But I can’t help but think that being good at editing comes naturally to computer scientists. Many of the skills — noticing slight deviations from the rules, especially in the form of syntax — are exactly the same. Both the English language and all programming languages have well-defined rules about how words/clauses may or may not be used together. It’s simply a matter of identifying violations of the rules.

I will add one rather large caveat to my thesis: I’ve known many programmers who cannot spell worth a damn (maybe they flee to computer science because it involves very little essay writing?). Some of them have been dyslexic. I don’t know if anyone’s established a correlation between dyslexia and going into computer science, but I definitely think there is one. So I think programmers make good editors, with the exception of the many programmers who cannot spell well. But if the spelling is good, by virtue of their profession, I bet they’ll be darn good at noticing all of the other errors one encounters in prose.

And for those of you following along closely at home, did you notice the mismatched parenthesis in this post? In the comments below, let me know if you noticed it, and whether you are a programmer. Be honest! Let’s try to get some data that, while not conclusive, will at least be one step above anecdotal.

Posted in Personal, Programming | 6 Comments »

Don’t ever be ashamed of your code

Friday, June 13th, 2008

Are you ever ashamed of your code? Don’t be! Being ashamed of your code is harmful, as artfully explained by Ben Collins-Sussman. It’s better to make your mistakes in the open where they can be quickly corrected than in private where they can fester for months, even years. Note that we aren’t necessarily talking about open source code here. Being ashamed of your code could also mean not sharing code with other people at your company.

Ben uses some anecdotes to illustrate just how badly situations can get when programmers (or small groups of programmers) sit on their code for months on end without any outside sanity checking whatsoever. But these anecdotes are more humorous than necessary, as it’s pretty much a truism in computer science that coding off on your own in secret is a bad idea. The people who are doing it know it’s bad, and the only reason they persist is because they are ashamed. Oftentimes they’ll rationalize it by saying “I’ll just clean it up before I let others see it” — which, when combined with procrastination, can mean no one else sees it for months or even years. And if poor architecture decisions have been made, as they often are, the problem is too large for a simple clean up; a partial or full rewrite is necessary. This is not the situation you want to find yourself in.

Luckily, I can’t say I’ve ever felt ashamed of my code. And that’s not for lack of writing some truly terrible programs, either. I just value the feedback I get from others more than any personal attachment I might have to my code. In other words, I don’t take it personally. And to demonstrate that, I’m going to post a truly terrible program I wrote back in high school. My only excuse is that I was young and ignorant.

The program in question is “makeSite”, a program I wrote to create my blog-like website before the word “blog” even existed and before any real blogging software had been developed. I was writing what was effectively a blog at the time (you can see an archive of it here), but I got tired of having to hand-edit the HTML to copy over a previous entry and modify it each time I wrote a new entry. So, naturally enough, I wrote a C++ program to statically compile a bunch of text “data” files containing my own custom pseudo-HTML-like syntax into a website. I won’t defend the decision to do it this way, other than to say that I didn’t know any better. What this effectively meant was that every time I updated any part of my site, even to fix a one-character typo, my entire site had to be re-compiled by re-running the program, a task that, because my program wasn’t very efficient, was taking minutes after my site grew to be rather big. I toyed with the idea of some sort of incremental site compilation, only updating the pages corresponding to the changed data files, but I never got that working.

I think it’ll help to illustrate how bad this program truly is by individually discussing some of the more egregious parts of it.

#include “apstring.h”

For those of you who aren’t familiar with the “apstring” string library, here’s a hint: ap stands for Advanced Placement. That’s right, instead of using a standard, widely used string library (like “string.h”), I used the apstring library (by the College Board), because that’s what we were taught in class. It was just like a real string library, only it didn’t have as many features. Frankly, there’s no excuse for its existence, as tests should conform to reality and not the other way around. If you ever see it in production code, you should run like hell.

headFile.open(“pages.dat”);
output.open(“pages2.dat”);
while (headFile.get(ch))
output < < ch;
output.close();
headFile.close();

Yes, you really are looking at a character-by-character copy of a file. Never mind that there's an OS function to do this in one line (and much more efficiently, I might add). But the reason I did it this way is even worse than the way I did it, if that’s possible: I wanted a second copy of the file so I could parse through the original string-by-string, and then when I hit upon a page that was a subpage of another page, I would consult this copy to find out what its parent page was. This was to get around the problem of not being able to have two file handles open to the same file. I suppose the concept of just loading the whole file into memory and parsing through that didn’t occur to me. And notice the hard-coded file names; that’s a nice touch.

Read the rest of this entry »

Posted in Net, Personal, Programming | 3 Comments »

WordPress vulnerability in the wild

Monday, June 9th, 2008

Five days ago, I noticed that the number of hits coming to this blog decreased drastically. Specifically, the WordPress.com Stats plugin showed that my incoming hits from search engines had decreased to nearly zero. And since Google is this blog’s front page, that represented a rather drastic drop in traffic. I was down to just the regulars, those faithful readers who either refresh this blog regularly or are subscribed to the RSS feed (you know who you are). I wondered if I had pissed off the Google gods, and tried to wrack my brain for any bad SEO juju I could have employed.

Two days later, my traffic still hadn’t improved. My visitor numbers were in the doldrums for three straight days. You can imagine how disheartening it is to spend a year and a half working on building up a blog, only to slip backwards by over a year’s worth in traffic numbers. I was starting to foster an irrational hatred of Google. Then I randomly ran across a new WordPress vulnerability in the wild thanks to a link on Reddit. Here’s the description from the linked blog post:

Many sites that are running WordPress blogs have been hacked by a very clever and hidden PHP Injection which is redirecting all requests from Google, MSN, Live, Altavista, Ask, Yahoo, and other search engines and redirecting it to ‘anyresults.net’ a site filled with pay-per-click ads and redirects to other landing pages. This is a very clever trick as visiting a web site either through a direct navigation type in or a bookmark does not display the problem. Only search engine visits are redirected and many site owners are delayed at discovering this problem until they notice huge dips in traffic or revenue stats.

Wow, did that ever sound familiar! And after a quick check, what do you know, my site had been hit by the 0-day WordPress vulnerability described in the linked post. The fix was very simple: remove the offending code from my wp-blog-header.php file. How in the hell it got in there is still unknown. Hopefully WordPress fixes this soon. In the mean time, I’m just keeping a very close eye on my visitor statistics, and I have a pre-exploit backup I can revert to if absolutely necessary. Very thankfully, my visitor numbers have returned to what they were from before the exploit, so it looks like I won’t face any permanent damage.

As you can imagine, I’m a bit peeved at the WordPress folks right now. I’ve complained about the security problems in their software before, and new problems are always being discovered. If you don’t keep on top of WordPress upgrades very regularly — or even if you do, as I have discovered — you can be hit with all sorts of bad stuff. It’s more proof of that ancient pearl of wisdom in computer science: truly secure programs are designed with security in mind from the ground up. You can’t possibly make a program secure by trying to play whack-a-mole with all of the security holes in insecure software. If security wasn’t front and center in your mind from the get go, you can’t fix it later on, as security drives major architecture decisions that are much harder to revisit in mature codebases than simply dashing out another stop-gap security patch.

Posted in Net, Programming | 8 Comments »

The failings of development in Windows

Tuesday, May 27th, 2008

Drinian (regular commenter here) pointed me to a great series of articles on the failings of Microsoft in recent years. Particularly, the Windows APIs are inconsistent and not pleasurable to use from a development perspective, and with Windows Vista and its flagship applications, Microsoft has released a wildly inconsistent smattering of user interfaces. I’m not going to try to sum up the articles in any further detail; they’re so full of content that you really have to read them for yourself:

  1. From Win32 to Cocoa: a Windows user’s conversion to Mac OS X – Part I
  2. From Win32 to Cocoa: a Windows user’s conversion to Mac OS X – Part II
  3. From Win32 to Cocoa: a Windows user’s conversion to Mac OS X – Part III (Updated 2008-06-01)

And yes, I know there’s a lot of “Mac OS X” in the title there, but the majority of the content really is about Microsoft and Windows. The third part in the series isn’t out yet, but when it is, I’ll try to update this blog post with a link to it.

For the record, I personally agree with pretty much everything Peter Bright says about Windows development. I did a good bit of it at my previous job and it was ugly. .NET hasn’t made significant improvements in this regard because it makes way too many concessions to long-deprecated functionality. And the wide variety of official Microsoft user interfaces in Windows Vista is incredibly off-putting. Why does every large Microsoft application function completely differently?! If I’m a third party developer writing my own application, what do I try to make it look like? The answer isn’t Microsoft Office 2007 (even though it’s my favorite new interface of the lot), because the ribbon menu implementation is specific to the Office codebase and doesn’t even have a public API! Brilliant!

Posted in Microsoft, Programming | No Comments »

Pining for the coding fjords

Wednesday, May 14th, 2008

I’m deep into the third week of a technical writing project at work, and boy do I miss coding! It turns out there was a good reason I went into computer science in college and got a job as a programmer afterwards; I really am passionate about it. That feeling just gets lost when I’m doing programming that I don’t really find enjoyable (and let’s be honest: unless you work for a game company, and often not even then, the kind of programming you’re doing isn’t fun).

So I’ve started by writing some simple algorithmic programs in C++: linked list classes, binary search trees; you know, nothing spectacular, but not nearly as trivial as a “Hello world!” either. I just wanted a refresher on C++ because I haven’t used it in awhile. And boy did I forget some things! The programs were algorithmically correct on the first try (which is absolutely not something I would be able to do when I was first learning these data structures back in high school, so at least those skills stuck with me), but the syntax was horrid. Imagine a tsunami of syntax errors impacting a rain forest of pointer errors. Uh, yeah, something like that.

Pride being one of the worst sins in a programmer, I immediately looked up some C++ tutorials on Google to find out what I was doing wrong. I had missing semicolons at the end of class declarations (I forgot all about that quirk; Java doesn’t need them), I was missing function declarations (again, not needed in Java), and I was even accidentally passing a pointer to a function that really wanted a value by reference (don’t ask). But it wasn’t too bad; after a few minutes of cleaning out the errors, the program compiled successfully and ran perfectly on the first try. It was fun using pointers again after these past couple of years of not touching them. They’re fun in a masochistic kind of way. I think it has something to do with working closely enough to the hardware to be able to allocate individual blocks of memory and daisy-chain them all together. It’s cool when you think about what you’re doing.

These programs were still pretty simple though. I don’t have a real desire to make a much more complex project in C++, because it does start getting ugly after you’ve grown accustomed to the niceties of C#, Java, and Python, so now I’m off to code up a game in Java. I have a general idea of what I want to write, which I’ve already made allusions to, but I don’t want to talk more about it in case the nifty idea in my head utterly stumbles upon execution. It’s happened before, so I’ve learned the fine art of hedging.

After some preliminary research, I’ve decided that I’ll use the Lightweight Java Game Library (LWJGL) to code up what I have in mind. I don’t want to make the same mistake that I did back in college when we wrote the game A Day in the Bay for our four-year undergraduate research project. We wrote it from scratch in pure Java AWT (not even using Swing!), so we spent a lot of time reinventing the wheel, and to boot, our wheel wasn’t particularly good because it didn’t have any hardware acceleration*. I don’t want to have to spend so time much time doing performance optimizations this time around, so I’ll just use LWJGL and hopefully get to pure coding a lot more quickly.

Now unfortunately, it can be awfully intimidating first starting off with something as complex as LWJGL. Just look at their API; it doesn’t make much immediate sense even if you are a Java developer. This is the critical juncture where many people get turned off and decide to write something on their own (as we did back in college). This is a big mistake. It’s a lot less effort to learn how to use a new library than to essentially write it from scratch. So how do I learn a new library? Examples! And luckily, LWJGL has some awesome examples of fully-fledged games to peruse. Check out Space Invaders and 3D Asteroids. Using these as a reference, and then the API for other functionality not used by these sample games, I’ll have my game up and running in no time.

So keep your eyes peeled. I’ll eventually have some progress to report back on, and then at some point the fruits of my labor will be downloadable for your enjoyment. I’ve already set up my SVN server and committed a skeleton class file, so there’s no turning back! Programming that’s fun again, here I come!

*While we’re mixing metaphors this badly, I’ll just come out and declare that a wheel with hardware acceleration is an automobile. Or, you know, a motorized unicycle.

Posted in Personal, Programming | 2 Comments »

Writing fifty games in one semester

Tuesday, May 6th, 2008

Four computer science graduate students recently created fifty playable game prototypes in one semester. Each student worked alone, putting out around twelve games at a rate of one per week. And they were responsible for not only the programming, but also the graphics and sound. That is quite the Herculean effort, and their results are impressive. I’ve seen that Swarm game before (I guess it was linked on Digg or something), but I didn’t know to examine it in the context of this rapid development game project.

The idea of creating lots of simple game prototypes in rapid succession really appeals to me. Yes, not all of them will be great, but some will be good. Little enough time is invested in each one that even if only one pays off, it’s all worth it. Compare this to the traditional game development process, which takes longer to create one game than these guys could use to pump out 100, and often yields terrible results nonetheless. Yes, that’s right, some of these fifty games are already better than what professional studios spend man-decades creating.

Unfortunately, I just don’t have the free time at the moment to devote my attention full-bore to creating lots of neat games in short periods of time (what with work and all). But I do have enough free time to create a couple, so I think I shall have to try it. Flash seems like the obvious language to do this in, but I’m not experienced in it, and I am concerned by its closed, proprietary nature. I think I’ll do what I did a lot of in high school: making prototype-sized Java applets. I guess I’ll have to read up on some Free Software Java libraries, because I don’t want to have to code something as simple as sprite rotation from scratch.

And working on creating some fun little games will also give me the opportunity to try out the ultimate form of game loop which I expressed a desire to attempt a month and a half ago. Now, I just need an idea. Hrm, stats in RPGs are fun, why not try to play around with that mechanic? I’ll see what I can do.

Posted in Gaming, Investing, Programming | 1 Comment »

How the programming magic happens

Monday, May 5th, 2008

Here’s your thought for the day: Programmers don’t really know how they program. I stared in bemusement at that simple claim for a couple of seconds, went on to read the whole thing, and then realized the truth of it.

I don’t really know how I program. I couldn’t describe it in a series of steps like I might be able to describe how I drive a vehicle, or even solve a math problem (whole classes of which generally have certain steps that, when followed in order, yield the solution like peeling layers off an onion). Come to think of it, I recall initially being thrown off in my first computer science classes in middle school when the teachers weren’t really teaching us how to program. They showed us lots of example programs and how they worked, but the real sink or swim moment was when you were asked to program something you hadn’t yet programmed before. No wonder so many students in my computer science classes were put off by it; they were being asked by their teachers to come up with solutions completely on their own, to a degree they had never yet experienced before in school.

Here is my general “process” when I’m programming something tricky:

  1. I think about the problem for awhile. If there’s a written description, I read it over until I grasp the whole of it in my mind, then I just kind of stare blankly at the screen. Sometimes I close my eyes. This can go on for a couple of minutes, so when I’m doing something at work, I’d rather not have the boss come in during these moments lest he think I’m not being productive.
  2. I start outlining how my solution will work. Sometimes I do this in comments directly in the IDE and flesh it out with code later; other times I just write it in Notepad, or even on a physical sheet of paper when I need to sketch something. If I have a good grasp of the problem, I might just keep it all in my head.
  3. I go research anything I don’t yet know that I will need to complete my solution. Generally this involves Googling. After many years of doing exactly this, I’ve become remarkably efficient at researching programming problems online. Oftentimes I have an API or some documentation in mind when I start my search.
  4. Here’s where the magic happens: I write the code. I can’t really explain it in any more detail than that. Oftentimes my planning was woefully incomplete (or just plain wrong), so I revise it as I go along on at this step.
  5. Testing, and then debugging if something isn’t working.

Kind of enigmatic, huh? I think I could do a decent job of explaining how the planning stage works (I identify the problem, then rack my memory for any prior experience I have with that sort of problem, combining various common programming patterns as necessary, until I have something that should yield the answer), but I can’t explain how the programming stage works. Even more problematic, sometimes I completely draw a blank when I’m planning out a solution, so I just go directly to the coding, writing out line after line until I’ve arrived at some code that vaguely does what I’m looking for. I couldn’t say how that works, and it’s not nearly as simple as “Here’s how to design the program, and then writing the code is a simple transformation process on the design.” No, it’s not even nearly that simple.

Read the rest of this entry »

Posted in Personal, Programming | 12 Comments »

File extensions and the pigeonhole principle

Friday, April 4th, 2008

Earlier today I downloaded some map data from the United States Geological Survey. It’s file extension was .adf. Hrmmm, that was something I wasn’t really familiar with, so I headed off to that great oracle of Internet knowledge, Google, to find my answer, and Google did not disappoint.

Apparently ADF is a file format for a bunch of different programs. It’s can be, among other things:

  1. A saved game file for A.R.S.E.N.A.L.: Extended Power, a game I’ve never heard of that has a curiously acronymed name. I can’t be bothered to look up what the acronym stands for, but even without looking, I can guarantee it’ll be utterly contrived.
  2. An “Actual Drawing File”. Your guess is as good as mine. Presumably contains actual drawing data, unlike the “Fake Drawing File” format.
  3. An “Adapter Description File”, used in some obsoleted suite of IBM software. If I had a nickel for every piece of software that IBM has abandoned by the side of the roadway, I’d be able to afford some actual IBM software.
  4. An “Admin Config File”. No other information is provided, so presumably it contains such vital information as height, weight, favorite food, preferred Linux distribution, favorite Star Trek movie (not V!), etc.
  5. An Amiga Disk File. These are still around?!
  6. An “Antenna Data File”. Seeing as how I just got into ham radio, that’s an interesting coincidence. I bet some ham somewhere out there is using ADFs to design his killer Yagi array.
  7. A “Dog Creek QC Mask File”. I’m not sure how they got “ADF” out of that name, so your guess is as good as mine.
  8. A Grand Theft Auto Vice City Radio Station file — alright, finally one I’m familiar with! I actually played that game, and on the computer to boot! So I’ve actually used these ADF files before. Unfortunately, this isn’t quite the kind of thing the USGS makes available for download, though they really should.
  9. An “I-DEAS Associated Data File”. This is some kind of executable binary code having something to do with Matlab. Yeah, I’ll stay far away from it, for both reasons.
  10. A “MicroSim PCBoard Photoplot Aperture Definition File”. Man, we’re getting incredibly esoteric here. I wonder how many ADF files of this type there are on the entire planet?
  11. Something to do with “ReliaSoft ALTA 1″. My guess would be something to do with quilting and shuffleboard (wow, obscure reference?), but apparently ALTA actually stands for something: Quantitative Accelerated Life Testing Analysis. So, uh, murder by spreadsheet it is?
  12. “Wyatt Technology ASTRA(r)Chromatography”. Finally, one that needs no explanation.
  13. An ArcView ARC/INFO Coverage Data File. You know, map stuff.

Obviously the last one is the correct one in the context of the file I downloaded, but I think I had more fun looking at the disparate things ADF has been used for in the past than in actually being able to open the file itself. All of these different uses of the same file extension brought to mind the pigeonhole principle, a mathematical principle which says, in essence, if you have more pigeons than pigeonholes, then at least one pigeonhole must have more than one pigeon in it. It sounds obvious and trivial, but when properly applied in mathematical proofs, it can actually yield some surprising and nontrivial results. We used it in a lot of my theory computer science courses at University of Maryland.

Read the rest of this entry »

Posted in Programming | 3 Comments »

WordPress finally discovers salted passwords

Saturday, March 29th, 2008

WordPress 2.5 is out today and it looks mighty impressive. I’m going to wait a few days for reports of compatibility with the plugins I’m using before I upgrade, but after that, expect to see WordPress 2.5 on this blog soon.

Looking through the changes list, I did notice one odd thing. WordPress 2.5 finally adds salt to stored password hashes. It’s nearly inconceivable to me that WordPress went so long without salted passwords — it’s an incredibly important security technique that essentially has zero implementation cost. When I was helping to design the software infrastructure that powers Veropedia, I made sure that password hash salting was in our alpha. And yet it takes the fine folks over at WordPress until version 2.5 to implement it? Did they not realize how important it is to security?

Here’s why password salting is so important. The naive algorithm for storing login passwords in a database is to store them as plaintext. User tries to login, the inputted password is matched against the password field under their username in the database, and if it matches, the login is successful. The reason this is terrible security practice is because if the database is compromised (which is surprisingly easy to accomplish even remotely using SQL injection) the entire list of passwords can be revealed, compromising the entire site and everyone who is registered to use it.

So the next step in the evolution of login security (and this happened decades ago) was to use a one-way function called a hash function to store the password in the database. I won’t go into the details of how a hash function works, but the key point to know is that it is one way: given an input, you can quickly calculate the output, but given the output, you cannot calculate what the input was. So, now password hashes are stored in the database instead of the raw password, and when a user goes to log in, their input is hashed and compared against the value in the database. This is what WordPress used up until version 2.5.

There’s just one major flaw with this seemingly secure system. There are only a few widely-used hash algorithms, and they all necessarily run quickly on small inputs, so it’s trivial to pre-compute a huge list of potential passwords and their associated hash values. This is called a rainbow table, and larger rainbow tables have trillions of entries in them, pretty much guaranteeing a successful attack against less secure passwords (short ones, ones that don’t use numbers and punctuation, etc.). So we’re pretty much back to square one: database is compromised, the hashed passwords are compared with the rainbow table nearly instantaneously, and lots of accounts can be compromised.

Read the rest of this entry »

Posted in Meta, Programming | 11 Comments »

« Previous Entries
Next Entries »