The mysteries of software development, revealed!

OK, not really. But the problems of large-scale software development are still an obsession for me, despite it being a few years since anyone last tried to shame me into committing to deliver a fixed scope by a fixed date that I had no confidence in.

There’s now an outlet for these thoughts, while not distracting the gentle readers among you who don’t have to suffer the frustrations of working in teams. And this blog will continue its focus on the technical and business aspects of building small web tools.

If you’re at all interested, please head over to our new blog with some wisdom on Lean Software Engineering and join us as a subscriber.

Maxing out your MacBook Upgrade

Running out of space on your MacBook? Things getting slow because you’re low on memory?

Maxing out the disk and memory in your MacBook can be a relatively stress-free and simple process. The only extras you need are an external drive enclosure and software to create a bootable drive image.

Apple charges a premium for upgrades, so here you get both the benefit of less expensive suppliers and the ability to wait until you really need the extra space down the road — when components are cheaper.

What worked well for me (my MacBook is a 1st gen Core Duo) basically fits in 3 steps.

  1. Buy compatible upgrade gear — SATA drives and SODIMMs
  2. Put the new drive in the enclosure, connect it to your MacBook, and create a bootable image of your old drive using SuperDuper
  3. Open up your MacBook, swap the old/new drives and memory

Now the details on each of these …

1. Buy the upgrade gear

You can buy a complete set of compatible gear from a store like Other World Computing, or go shopping for individual pieces as I did.

SODIMM PicOn the memory front, I’ve always had luck with crucial.com in terms of identifying the type and amount of memory that can be upgraded. I bought the 2GB kit (1GBx2), 200-pin SODIMM Upgrade for a Apple MacBook 1.83GHz Intel Core Duo (13-inch White) from them (currently $131.99. I paid $203.99 in Feb)

The MacBook is my first laptop with an internal SATA connector for the hard disk (SATA 150 compatible). That meant none of my existing ATA/IDE enclosures would cut it for the upgrade. When looking for one to buy, I would have liked to find one that supported Firewire 400 for the external connection, since Firewire gets closer to the actual disk throughput on the Mac (and also on the PC, if you have a few other USB devices attached) — but only USB 2.0 was available. If anyone has tried a Firewire 400 enclosure that works, comment and I’ll update this post.

SataI bought the nice little Mini 2.5 inch SATA to USB 2.0 Aluminum Enclosure ($24.98) from satadrives.com.

For the drive, 200GB was the largest available. I bought a Toshiba 200GB 4200 RPM 8MB Cache Serial ATA150 drive ($179.99. I paid $223.99 in Feb)

2. Put the new drive in the enclosure, connect it to your MacBook, and create a bootable image of your old drive using SuperDuper

Follow the enclosure’s instructions. For more on the drive image part, SuperDuper’s defaults do exactly what you want. But if you want to see more, look at MacWorld’s disk cloning with SuperDuper article. Leave a few hours for this step, as the full drive image will be pounding that USB 2.0 connection and drive for a while. I initially did this with the free, unlicensed demo of SuperDuper and it was problem-free. But, especially with your old drive and new enclosure available as a backup destination — it’s worth buying a licensed copy to get the faster incremental backup functionality.

3. Open up your MacBook, swap the old/new drives and memory

Macworld made a great video to watch before you begin all this, giving you an idea of what’s ahead.

But the best video, one that includes key hints like the amount of force required to re-insert the memory DIMMs is from the good folks at OtherWorldComputing:

OWC MacBook Upgrade Video

Watch that one a few times, possibly keep it up on another PC while upgrading to ease any fears you have.

There you have it. How is the performance? Effect on machine temperature? Subjectively, both have been great. The memory upgrade and extra cache in the drive overwhelm any disadvantage from the slower 4200RPM rotation speed. And all that extra space means I finally can develop, listen to music, edit my videos, and still have room for a some VMs or a BootCamp partition on my little wonder laptop.

Notes:

  • Apple doesn’t mind you upgrading memory, but may charge you extra for any future service to a machine that’s had a hard disk upgrade. My local Apple store gave me this warning, but then serviced the machine for free anyway.

open_id_authentication (part 2)

In part 1, we looked at some changes to DHH’s OpenID plugin for rails. Now we look at the app’s authenticated sessions controller, which uses that library code. The goals were:

  • Support both OpenID and password authentication
  • Support all the usual login goodies, including before_filter :login_required, remember me functionality (with cookies), HTTP basic authentication, etc.
  • Get this by integrating the restful_authentication plugin/generation and open_id_authentication plugin

First, I installed both plugins and ran the restful_authentication generator

./script/generate authenticated user authenticated_sessions

I chose the name “authenticated_sessions” to distinguish from the built-in (non-authenticated) Rails sessions functionality.

routes.rb adds

  map.resource :authenticated_session, :member => { :complete => :get }
  map.resources :users

Then it’s a matter of some substantial edits to the generated app/controllers/authenticated_session_controller.rb Here’s where it would be nice to have a combined generator like Eastmedia’s. If you’re just trying to get an app and and running, you don’t want to be reading a blog posting like this. :)

The smartest design decisions in the file below (which all came from DHH’s code), are having centralized successful_login(), failed_login(), and destroy() functions, which are common across the otherwise separate openid/password authentication paths.

# This controller handles the login/logout function of the site. 
# File created with restful_authentication generator 
class AuthenticatedSessionController < ApplicationController
 
  # render new.rhtml
  def new
  end
 
  # Added after restful_authentication generator, using code from http://www.loudthinking.com/arc/000604.html
  # Complete arrives to us via a browser redirect from the OpenID provider, which happens after create/begin.
  # Obviously only called in the OpenID path through authentication.
  def complete
    complete_open_id_authentication(params[:openid_url]) do |result, identity_url, sreg|
      case result
      when :canceled
        failed_login "OpenID verification was canceled"
      when :failed
        failed_login "Sorry, the OpenID verification failed"
      when :successful
        if self.current_user = User.find_by_openid_url(identity_url) || User.create(:openid_url => identity_url, :login => sreg['nickname'], :email => sreg['email'])
          successful_login
        else
          failed_login "Sorry, no user by that identity URL exists (#{identity_url})"
        end
      else
        failed_login "Unknown error logging in #{identity_url}"
      end
    end
  end
 
  # Handle the creation of a new authenticated session, OpenID or password authentication path
  def create
    if using_open_id?
      begin_open_id_authentication(params[:openid_url], :required => "nickname, email") do |result, identity_url|
        case result
        when :missing
          failed_login "Sorry, the OpenID server couldn’t be found"
        else
          failed_login "Unknown error in openid begin for #{identity_url}"        
        end
      end
    elsif params[:login]
      password_authentication(params[:login], params[:password])
    else
      failed_login "No valid credentials passed to create authenticated session"
    end
  end
 
  def destroy
    logger.info "logging out #{self.current_user.inspect}"
    self.current_user.forget_me if logged_in?
    cookies.delete :auth_token
    @session[:user] = nil
    flash[:notice] = "You have been logged out."
    redirect_back_or_default(calls_path())
  end
 
protected
 
  def password_authentication(login, password)
    if self.current_user = User.authenticate(params[:login], params[:password])
      successful_login
    else
      failed_login("Invalid login or password")
    end
  end
 
private
 
  def successful_login
    logger.info "sucessful login for #{self.current_user.inspect}"
    if params[:remember_me] == "1"
      self.current_user.remember_me
      cookies[:auth_token] = { :value => self.current_user.remember_token , :expires => self.current_user.remember_token_expires_at }
    end
    redirect_back_or_default("/")
    flash[:notice] = "#{self.current_user.login} Logged in successfully"
  end
 
  def failed_login(message)
    logger.info 'Login failed. ' + message
    flash[:notice] = message
    redirect_to new_authenticated_session_path()
  end
 
end

Lastly, views/authenticated_session/new.rhtml is where the initial form is presented when users are redirected from an action that requires authentication. I chose to put the password and openid authentication forms on the same page, but within their own divs with IDs and a common rounding class, so the styles can be tuned in the .css file to make it look nice.

<div id='password-login' class='rounded'>
	<% form_tag authenticated_session_path do -%>
	<p><label for="login">Login</label><br/>
	<%= text_field_tag 'login' %></p>
 
	<p><label for="password">Password</label><br/>
	<%= password_field_tag 'password' %></p>
	<p><label for="remember_me">Remember Me</label><%= check_box_tag 'remember_me' %></p>
	<p><%= submit_tag 'Log in' %></p>
	<% end -%>
</div>
 
<div id='openid-login' class='rounded'>
	<% form_tag authenticated_session_path do %>
	  <p><label for="openid_url">OpenID</label><%= text_field_tag 'openid_url' %></p>
	  <p><label for="remember_me">Remember Me</label><%= check_box_tag 'remember_me' %></p>
	  <%= submit_tag 'Login' %>
	<% end %>
</div>

You’ll notice that most of my postings and a lot of the plugins have headed in the direction of making authenticated sessions a REST resource, using the newer RESTful support in Rails. Reflecting back on it all, it isn’t the best conceptual or practical match. As long as the end-user functionality is the same (support for remember_me, HTTP basic authentication, etc.) — it seems more strightforward to implement authenticated sessions as a traditional controller, not a REST resource.

open_id_authentication (Part 1)

open_id_authentication is the OpenID consumer plugin for Rails, being written by DHH and some on the Rails core team.

This plugin was getting some quick updates at first, but things have slowed down as the 37signals crew has focused on launching the app they developed it for — highrise.

So, in the absence of other updates over the last few weeks, here are some changes I made for one of my own Rails apps. In part one of two posts, we look here at what changes were made on the library side. In part two, we’ll look at the the library client (rails controller) side.

On the left is http://svn.rubyonrails.org/rails/plugins/open_id_authentication lib/open_id_authentication.rb revision 6452.

On the right (or down below, if the browser isn’t wide enough) is the modified version used for my rails app.

60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
  protected
    def normalize_url(url)
      OpenIdAuthentication.normalize_url(url)
    end
 
    # The parameter name of "openid_url" is used rather than the Rails convention "open_id_url"
    # because that's what the specification dictates in order to get browser auto-complete working across sites
    def using_open_id?(identity_url = params[:openid_url]) #:doc:
      !identity_url.blank? || params[:open_id_complete]
    end
 
    def authenticate_with_open_id(identity_url = params[:openid_url], fields = {}, &block) #:doc:
      if params[:open_id_complete].nil?
        begin_open_id_authentication(normalize_url(identity_url), fields, &block)
      else
        complete_open_id_authentication(&block)
      end
    end
60
61
62
63
64
65
66
67
68
69
  protected
    def normalize_url(url)
      OpenIdAuthentication.normalize_url(url)
    end
 
    # The parameter name of "openid_url" is used rather than the Rails convention "open_id_url"
    # because that's what the specification dictates in order to get browser auto-complete working across sites
    def using_open_id?(identity_url = params[:openid_url]) #:doc:
      !identity_url.blank? || params[:open_id_complete]
    end
In these diffs, you see authenticate_with_open_id has been removed. The original plugin code on the left routes OpenID’s begin and complete operations through this single function and keys off an HTTP parameter to know which operation we’re really doing. In the implementation on the right, the client calls directly into the appropriate (and newly un-private) begin & complete methods, and we’ll now have to make sure to normalize the url in those functions, as necessary.
80
81
82
83
84
85
86
87
88
89
90
91
  private
    def begin_open_id_authentication(identity_url, fields = {})
      open_id_response = timeout_protection_from_identity_server { open_id_consumer.begin(identity_url) }
 
      case open_id_response.status
      when OpenID::FAILURE
        yield Result[:missing], identity_url, nil
      when OpenID::SUCCESS
        add_simple_registration_fields(open_id_response, fields)
        redirect_to(open_id_redirect_url(open_id_response))
      end
    end
70
71
72
73
74
75
76
77
78
79
80
81
82
83
 
    def begin_open_id_authentication(identity_url, fields = {})
      identity_url = normalize_url(identity_url)
      logger.info "beginning openid authentication for #{identity_url}"
      open_id_response = timeout_protection_from_identity_server { open_id_consumer.begin(identity_url) }
 
      case open_id_response.status
      when OpenID::FAILURE
        yield Result[:missing], identity_url, nil
      when OpenID::SUCCESS
        add_simple_registration_fields(open_id_response, fields)
        redirect_to(open_id_redirect_url(open_id_response, identity_url))
      end
    end
Notice the “private” is gone on the right — it’s moved down below the begin and complete methods. Otherwise nothing too exciting here. Just added the url normalization and a log message to help illuminate looking back on a failed or successful login.
93
94
95
96
97
98
99
100
101
102
103
104
105
106
    def complete_open_id_authentication
      open_id_response = timeout_protection_from_identity_server { open_id_consumer.complete(params) }
      identity_url     = normalize_url(open_id_response.identity_url) if open_id_response.identity_url
 
      case open_id_response.status
      when OpenID::CANCEL
        yield Result[:canceled], identity_url, nil
      when OpenID::FAILURE
        logger.info "OpenID authentication failed: #{open_id_response.msg}"
        yield Result[:failed], identity_url, nil
      when OpenID::SUCCESS
        yield Result[:successful], identity_url, open_id_response.extension_response('sreg')
      end      
    end
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
    def complete_open_id_authentication(identity_url)
      identity_url = normalize_url(identity_url)
      logger.info "completing openid authentication for #{identity_url}"
      open_id_response = timeout_protection_from_identity_server { open_id_consumer.complete(params) }
 
      case open_id_response.status
      when OpenID::CANCEL
        yield :canceled, identity_url, nil
      when OpenID::FAILURE
        logger.info "OpenID authentication failed: #{open_id_response.msg}"
        yield :failed, identity_url, nil
      when OpenID::SUCCESS
        yield :successful, identity_url, open_id_response.extension_response('sreg')
      end
    end
The Result[] class on the left was added in one of the last few changes, and broke the plugin (at least for me and Ben). So the code on the right just reverts back to the previous working code, which returns the result code directly as a ruby symbol. As the plugin gets updates from the core team, I’m sure this’ll get a fix.
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
 
    def open_id_consumer
      OpenID::Consumer.new(session, OpenID::FilesystemStore.new(OPEN_ID_AUTHENTICATION_DIR))
    end
 
 
    def add_simple_registration_fields(open_id_response, fields)
      open_id_response.add_extension_arg('sreg', 'required', [ fields[:required] ].flatten * ',') if fields[:required]
      open_id_response.add_extension_arg('sreg', 'optional', [ fields[:optional] ].flatten * ',') if fields[:optional]
    end
 
    def open_id_redirect_url(open_id_response)
      open_id_response.redirect_url(
        request.protocol + request.host_with_port + "/",
        open_id_response.return_to("#{request.url}?open_id_complete=1")
      )     
    end
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
private
 
    def open_id_consumer
      OpenID::Consumer.new(session, OpenID::FilesystemStore.new(OPEN_ID_AUTHENTICATION_DIR))
    end
 
    def add_simple_registration_fields(open_id_response, fields)
      open_id_response.add_extension_arg('sreg', 'required', [ fields[:required] ].flatten * ',') if fields[:required]
      open_id_response.add_extension_arg('sreg', 'optional', [ fields[:optional] ].flatten * ',') if fields[:optional]
    end
 
    def open_id_redirect_url(open_id_response, identity_url)
      open_id_response.redirect_url(
        request.protocol + request.host_with_port + "/",
        open_id_response.return_to("#{request.url};complete?openid_url=#{identity_url}")
      )     
    end

The openid_redirect_url function is where the interesting change is. Both left and right code contain a convention (as Rails is wont to do) on how the HTTP request is formatted to distinguish the OpenID complete operation from begin.

The code on the right uses an explicit, separate action instead of one action and a param switch to encode the complete. This is cleaner IMHO, and allows begin and complete to have distinct Rails routing signatures. For the code on the left, the Rails routing is

map.open_id_complete 'session', :controller => "session", :action => "create", :requirements => { :method => :get }

And for the code on the right, it’s

  map.resource :authenticated_session, :member => { :complete => :get }

Note, the code on the right is bad in that it doesn’t use the recommended path helper, so now, as of a recent edge rails change, the resource ‘;complete’ action should be ‘/complete’ (so I should use that helper!)

Why pass the identity_url from the begin operation to complete? Because in the case of link rel redirection, we want to make sure to have our identity url reflect what the user gave us (e.g. his blog URI), not what it later resolves to (the provider URI) — or else, we’ll break user’s ability to switch providers transparently with redirection.

There we go. Let me know which of my decisions were good, bad, or broken. And next, we’ll see what this looks like from the client perspective.