Thinking about a first project

The Internet has enabled us to flood ourselves with data. To make sense of this all, we have to be able to prioritize and summarize what we look at. We have a lot of web tools and web services to help us, but not surprisingly, most of it is text based — and sometimes a picture is worth a thousand words.

For those pictures, we already have many vertical market sites which provide great charts (http://finance.yahoo.com/ being one common example for stocks, etc.). What we don’t have is a service — and a business model to support it — to bring this kind of data visualization to the “long tail” of other data on the internet. Charting for things like: bugs or checkins to open source projects, historical government spending, census data, or the # of daily classified job postings in your field etc, etc.

In order for that to happen, you’d need a freely available service which lets you point it at data, generate nice charts and visualizations, and have them be hosted so you can link to them from web pages, blogs, wikis, discussion boards, etc.

This is not a big business — there are limits to how much this visualization is worth to people. And for one-off stuff, you can always hand data to your buddy who is handy with Excel. But it’s a nice niche, and it’s part of a much larger category of knowledge mangement and data mining tools, which is an area of work which I love for some reason, and keep getting drawn back to.

What do you think — is this something you could ever imagine using (or seeing someone else use)?

But ideas are cheap — it’s really all about execution. For that, see coming posts.

Lean close and tell me what you’re up to

I love that technology has given individuals the tools to strike out on their own, craft solutions to problems they care about, and (well, this part is shakier) make a living at it.

It is through this kind of massive, decentralized innovation that we’ll see continued, accelerating progress of humanity.

“Lean” is what (successful) small, unfunded startups naturally are — growing both the products and the business organically and flexibly, through a rich feedback loop with customers. It’s the antithesis of the funded startup that spends 2-3 secretive years getting a big-bang product out. We need those secretive companies occasionally, but get them too often.

We’ve also seen the rise of the “portfolio” product company. The realization that — in software and web services at least — you can both tackle projects incrementally (frequent releases) and leverage synergies of technology and customer mindshare across products to make the whole more than the sum of the parts. Google, 37 signals, Fog Creek, Geoffrey Grosenbach are inspirations at various levels of scale — it’s possible to have an ongoing product portfolio down to the 1-person-company level. The book Micro-ISV is another recent inspiration for software businesses.

This is where leancode.com is coming from. But, with no customers yet to guide us — where should it go first?

Update on Vonage

It turns out my Vonage problems were unrelated to Bittorrent/Azureus.  More than one customer in our area was having problems with calls to a Vonage number resulting in ‘circuit busy’ or other messages.  The problem seems to have originated with Vonage, and resolved itself after a few days.  Nice timing for this kind of problem, being around that company’s attempted IPO and free Skype calls in the US/Canada this calendar year.
So the .torrent below is still up with at least one seed, and I’ll try to keep that going until a decision about what direction to take RORVM — continue delivering it as a finished VM, or turn it into capistrano script to configure a VM from parts.

Shoveling rails

I’m deploying a Rails application to textdrive using Geoffrey Grosenbach’s Shovel—Rails Deployment with Lighttpd. Thanks to Geoff for putting all his learnings into a script for the rest of us. Here’s some things I ran across on the way:

  • Textdrive has changed a few aspects of the server config. I updated Geoffrey’s script to cover one of the changes — users/home/[username]/sites is now users/home/[username]/domains. This is now one of the variables you should set for your case at the top of the script. I also added a variable for svn branch path, in case you use a trunk/branch/tag structure in svn as I do. This updated script is here.
  • Running the rake task, I was having this common problem with ruby require not finding the ‘openssl’ library. The solution on Ubuntu 5.10 (Breezy Badger) was to install libopenssl-ruby (sudo apt-get install libopenssl-ruby). Don’t be fooled, other packages like ‘openssl’, etc. are not what you’re looking for. A little commentary: I can’t recommend Ubuntu for Ruby/Rails developers — I’ve lost a lot of hours on various problems. The ubuntu/debian package maintainers are working at odds with Ruby’s distribution mechanisms (which works best when you can install ruby+rubygems and use gems to get everything else). Right now, Ruby on Ubuntu is broken in pieces and available in both places, without accurate dependency maps to help you get what you need. Until this is fixed, consider finding another alternative. If you have a Mac, I’ve had good experiences with DarwinPorts+gems.
  • Error messages in most of the tools here are not particularly helpful. Often it’s a Ruby exception where the real problem is only discernable way up the stack. Googling the error message (such as it is) is a good first step for debugging.
  • Textdrive has moved the ruby bin to /usr/local/bin/ruby. If a ruby script doesn’t work, check the top for this.
  • If you keep rails’ database.yml out of source control (e.g. if following subversion with rails), don’t forget to create it on the server now
  • After running the shovel script, /users/home/[username]/domains/[application]/current/config/lighttpd.conf wasn’t populated. Running script/server copied an example from the gems directory, but I needed to edit it to provide my textdrive-supplied port for lighttpd, and fix up the ruby location.

At this point, running ‘rake deploy’ and ‘rake rollback’ should work as intended if run from where I’ve checked out the production branch.
Some of the error messages that I hit before hitting on the solutions above:

  • (mod_fastcgi.c.1022) execve failed [turned out to be the /usr/local/bin/ruby problem]
  • no such file to load — openssl [sudo apt-get install libopenssl-ruby]

Any other updates or suggestions to do this better are welcome.

Azureus, Capistrano, and Vonage

A few things I learned from putting out RORVM last week:

  • When you publish a torrent, you can publish with a “torrent tracker” site, or do it “trackerless” (dht). I used Azureus to do it the trackerless way, and I think that was the wrong choice. Going trackerless reduces the ability for others to find the torrent, which then reduces the number of other seeders to help distribute it. So I’ll try a tracker for the next one, but I’m still not sure of the best tracker to use, though.
  • For a “Linux Appliance VM”, which mine is, you could imagine another means of distribution. Instead, download a standard VM image (say a straight Ubuntu Breezy 5.10 server install — or just the CD iso and install yourself) — and then re-configure the VM with a tool like Ruby’s Capistrano. So my instructions would change to 1. download ruby+capistrano to your PC. 2. download a straight Ubuntu VM. 3. download my capistrano script to re-configure the VM. Wouldn’t require a torrent on my part. But the cost of a few more steps to get it running.
  • If you do try to download my current VM image as it stands, you might find no seeds available. I was keeping Azureus running on my home box to make sure we had at least one seed, but I’ve had a problem — Vonage. I’ve been making full use of the Comcast cable connection, running a lot of stuff over it, including my home phone. Well, with the 40K of upload traffic that was consumed by seeding my torrent, we started having problems with “line is busy” or other errors in calling our Vonage number. Not 100% sure it’s cause-and-effect, and don’t know why Vonage isn’t falling over to the backup number, but I’m blaming it on the torrent for now, and may need to find another way.

Ruby on Rails Virtual Machine for VMware (RORVMv0.10)

[update: the .torrent is no longer up. I'd recommend using Ubuntu 6.06 or later, as the Ruby support is much cleaner]

I’ve been working on a project using the Ruby on Rails framework, which is ultimately deployed to a webhost running a Unix variant. At first, I was building this in an OS X environment, but then I had to switch over to a Windows laptop.

So in order to have a (at least somewhat) common development environment, I wanted to make use of some of the amazing progress that’s being made in virtualization software to do my development in a Unix VM on Windows.
So here is a very early, rough release of a VM intended to meet my needs (and perhaps yours). The basis of the VM is Ubuntu 5.10 (Breezy) Linux text-mode-only server. On top of that are Ruby and Rails packages, several fixes (for example, 5.10 comes with Ruby 1.8.3, which is incompatible with rails — so I upgraded the VM with Ruby 1.8.4 from Ubuntu 6.0), and some customization useful for running in a VM (e.g. prominantly displaying the DHCP-assigned IP address). All off-the-shelf open source pieces — thanks to the tons of people involved to make all that work, and to VMware for releasing a great, free VM “player”.
I’ll be tracking the rbuntu live CD work, as that comes out, possibly merging this work with that. I’m just making this available via Azureus torrents for now — it’s intended to be a minimal VM, but it’s still 250MB. :)
Steps to install RORVMv0.10

  1. Install VMware Player
  2. Install Azureus bittorrent client (.torrent below may only be avilable via Azureus’ decentralized tracking)
  3. Click the RORVMv0.10 .torrent file to have Azureus download it, and unzip.
  4. Click on the .vmx file to have VMware player run the VM. Follow directions before login.

Let me know if something just doesn’t work here, or if you have feedback or ideas to make this more useful.

creating this site with wordpress

I’ve used a bunch of CMS systems over the years: perforce + raw HTML, wikis, zope, sharepoint, textpattern, etc.

But with leancode, I needed to strike a balance where the site had to be great for time-oriented information (blogging, project news) while being functional enough for static pages. WordPress seemed like a good fit, but it required some customization. Here’s what was done:

  1. Followed textdrive’s instructions for installing wordpress
  2. Chose the andreas08 theme from themes.wordpress.net because it combined a navbar based on the sites ‘pages’ with a clean, functional design.
  3. Followed instructions for using a wordpress ‘page’ as the site front page including making blog appear in non-root folder, creating a ‘blog’ page, and changing the permalink definition as described
  4. Installed the static front page plugin for wordpress (again, a wget from within the target directory on the hosting site), created a ‘home’ page for it in the wordpress panel, and activated the plugin so that this home page became the new front for leancode.com
  5. Went to the theme’s directory (under wp-content/themes/andr*), and edited header.php to remove the extra list element in the navbar for the duplicate ‘home’ entry on the menu
  6. Changed the wordpress page order to reflect the home, blog, about order I wanted.

The result of all this is a wordpress install which is only slightly customized (theme with one tweak, and one plugin), but which otherwise can be managed from the wordpress control panel. And it looks and behaves enough like a website done with a more sophisticated CMS.
Some paths I went down and rejected:

Things that still bother me

  • The URL for the blog is leancode.com/blog, but the URL for the feed is leancode.com/feed. I would have liked to see clear separation of the blog feed (e.g. leancode.com/blog/feed) from the other content on the site that one might subscribe to

Any suggestions on this or other aspects of the site design are welcome …

Thanks to all the creators of wordpress, andreas for the theme, and all the others involved in the deep stack of open source software who make this stuff work.