Honor among developers: self-assessing productivity
There is a discussion brewing on HN today about measuring programmer productivity. Reg Braithwaite chimes in to ask a very pertinent question:
Programmers talk about productivity all the time. They say they’re more productive with static typing or without it. With Ruby or with Java. … It seems every programmer I meet has strong opinions about when they are more productive and when they are not. So, here’s my question:
How are all these people measuring programmer productivity?
The observation is spot-on. Most developers know exactly what makes them more, or, less productive. They may not be able to assess if something they haven’t used yet will boost their productivity; but of the tools and habits they have tried in the past, they have strong feelings about their current choice.
In an intangible way, every good developer I’ve met knows when she has had a productive day. It’s mostly a feeling that comes without rational explanation. This intuition is what developers are using when they express their choice for Java or Ruby. They have used both, and they know which felt right and which did not.
But there is a catch: you can’t harness this intuition if there are incentives to assess your own productivity too positively. Say, like, an annual performance review that your job depends on. Yet, this isn’t because developers are crooks. Something so intangible like an intuition is susceptible to being skewed for very innocent psychological reasons. This is also why a good manager will allow you to freely raise the issue when you haven’t been productive. It’s the only way she can find out why and address the root cause whatever it may be.
Professional assessment attaches very real consequences to being productive. You can’t let a developer search her own soul about if she should get a raise this year. She would really like to believe she should. However, if you simply want to make your team as productive as possible, let them be honest with you about it (and, by extension with themselves).
Push notifications in the age firewalls
In the world of client-server networking, web browsers are the ultimate clients. They are always sending SYN packets, but never receiving any. They are always establishing outgoing connections, but never get to listen to any ports.
Due to the ubiquity of the web, we have built our networks around the need to browse. Most NATs will not let any incoming connection near the private subnet. Similarly, our corporate firewalls cull incoming connections with extreme prejudice. As a result, an outgoing SYN packet on port 80 is the X86 of network connectivity. It’s as portable a networking tool as it gets, but it’s getting kludgier by the minute.
Take RSS readers: imagine a website that posts 10 new items a day, if you poll this site every 5 minutes, you are wasting your time 97% of the time. Granted, you are getting 304 Not Modified responses without any HTTP response body. But these requests still consume threads, CPU and network resources on the server. At the end of the day we are also only getting 2.5 minutes of average latency, which in computer terms, is not very fast. In short, polling sucks.
The PubSub Hubbub protocol aims to solve this problem by allowing the server to push content to the readers directly. Unfortunately, as explained above, the server cannot establish a connection to a lot of clients out there. You would be able to connect to web based feed readers like Google Reader, because Google understands how to configure a network to allow incoming connections. But the average user either is not permitted to modify their network configuration (office) or don’t know how to do it themselves (home). As cool as PubSub is, it just does not solve the problem at hand.
Which brings me to the actual solution to the HTTP push problem: long polling. With long polling, instead of promptly responding with a 304, the server lets the connection go idle as long as there is no content to send. The client on the other hand, reinitiates a new request as soon as it gets a response. The server also does not need to allocate threads to each incoming request, a single thread can wait on multiple incoming requests and respond to them as new data becomes available. The client gets an almost immediate notification, the server does not have to waste time handing out 304s. Everybody is happy.
With that we have implemented yet another version reverse-tunneling to deal with NAT. Honestly, for the near future, we are going to have to fit the square peg of push notification into the round hole of browser acting as a client.
