It looks like the Mono project (which is an alternate implementation of .NET) has been ported to the iPhone.

C# and iPhone together at last.  Brilliant!

This brings up the following random thoughts and/or conerns:

1. Is Git really better?  It looks like it may be.  While it has obvious advantages for a distributed team, it would seem the support for commits while disconnected has advantages even for single-developer projects.  Why just the other day I wanted to commit some changes on one of my projects when I found my Internet down.  Git would have come in handy.
2. Although Subversion works great for me, should I change for change’s sake?  To learn a new technology and avoid becoming that guy still clinging to Subversion 4o years from now?
3. The header image on Git’s home page has an orange monster eating trees.  Subversion’s home page does not.  Game/set/match Git.  Subversion better come up with an equally cool monster doing some equally cool or I may be forced to switch.

My favorite feature?  Built in CSS/HTML W3C validation.  You would be surprised how many of your CSS bugs that you are quick to blame on browser deficiencies are actually a result of your invalid CSS :/

1. Alastair Brunton, with improvements from Harry Love, created a means of generating your Sitemap dynamically for each model in your database.
2. Phil Misiowiec at Webficient created a tool to generate a Sitemap for your Rails app’s static content.

While each is incredibly useful, I wanted a solution that combined both.  I thus took the code created by all of the above, and extended their solutions to generate a Sitemap for both your dynamic and static content all at once.  Curiously, I also ran into and fixed a problem with the dynamic Sitemap generator whereby the XML created was a single line and Google was rejecting it with a non-descript error.

To get up and running with all of this, do the following:

1. Make sure you have the “mechanize” gem installed:

sudo gem install mechanize

2. Be sure to create a “sitemaps” subfolder in your [rails_app]/public directory.

3. Copy the two files below to your [rails_app]/lib directory:

# static_crawler.rb

require 'mechanize'

class StaticCrawler

  # EXTENSIONS_IGNORED = %w[.csv .doc .docx .gif .jpg .jpeg .js .mp3 .mp4 .mpg .mpeg .pdf .png .ppt .rss .swf .txt .xls .xlsx .xml]
  # BRB - In my case, I want to index document types like doc and pdf
  EXTENSIONS_IGNORED = %w[.csv .gif .jpg .jpeg .js .mp3 .mp4 .mpg .mpeg .png .rss .swf .xml]

  PROTOCOLS_IGNORED = %w[feed ftp itms javascript mailto]

  def initialize(starting_url, credentials = nil, quiet_mode = false, sitemap = false, debug = false)
    @bad_pages = []
    @agent = WWW::Mechanize.new
    @sitemap = sitemap
    @debug = debug
    @visited_pages = []

    if credentials
      creds = credentials.split(':')
      @agent.basic_auth(creds[0], creds[1])
    end

    @quiet_mode = quiet_mode
    @starting_url = starting_url
    @starting_url_domain = starting_url[/([a-z0-9-]+)\.([a-z.]+)/i]
    puts "domain: #{@starting_url_domain}" if @debug
    extract_and_call_urls(starting_url)
    generate_sitemap if @sitemap
  end

  def extract_and_call_urls(url)
    #get page
    puts "#{@visited_pages.size+1} #{url}" unless @quiet_mode
    begin
      page = @agent.get(url)
    rescue => exception
      @bad_pages << url
      puts "error: #{url}, #{exception.message}"
      return
    end

    #for any content types we may have missed above, exit if content type is not html
    return if page.instance_of?(WWW::Mechanize::File) || page.content_type.index('text/html') == nil

    #add to array
    @visited_pages << url

    #get links found on page
    links = page.links

    #for each link, call the url if not in history
    links.each{ |link| extract_and_call_urls(link.href) unless
      ignore_url?(link.href) || @visited_pages.include?(link.href) }
  end

  private

  def ignore_url?(url)
    begin
      return ignored = true if url.nil? ||
                       (url.include? 'http' and !url.include?("webficient.com")) ||
                       @bad_pages.include?(url) ||
                       PROTOCOLS_IGNORED.find{ |prt| url =~ /#{prt}:/ } != nil ||
                       EXTENSIONS_IGNORED.find{ |ext| url =~ /#{ext}$/ } != nil
    ensure
      puts "ignored: #{url}" if ignored and @debug
    end
  end

  def generate_sitemap
  	xml_str = ""
  	xml = Builder::XmlMarkup.new(:target => xml_str, :indent => 2)

  	xml.instruct!
  	xml.urlset(:xmlns=>'http://www.sitemaps.org/schemas/sitemap/0.9') {
  		@visited_pages.each do |url|
  		  unless @starting_url == url
    	    xml.url {
      	    xml.loc(@starting_url + url)
      			xml.lastmod(Time.now.utc.strftime("%Y-%m-%dT%H:%M:%S+00:00"))
      			xml.changefreq('weekly')
   			  }
   			end
  		end
  	}

  	save_file(xml_str)

  	# BRB - don't need to call this as something similar is called at the end of ModelCrawler
  	# update_google
  end

	# Saves the xml file to disc. This could also be used to ping the webmaster tools
	def save_file(xml)
		File.open(RAILS_ROOT + '/public/sitemaps/static.xml', "w+") do |f|
			f.write(xml)
		end
	end

	# Notify google of the new sitemap
	# def update_google
	#     sitemap_uri = @starting_url + '/sitemap.xml'
	#     escaped_sitemap_uri = URI.escape(sitemap_uri)
	#     Net::HTTP.get('www.google.com',
	#                   '/webmasters/tools/ping?sitemap=' +
	#                   escaped_sitemap_uri)
	# end

end

# model_crawler.rb

require 'net/http'
require 'uri'
require 'zlib'

# A class specific to the application which generates a google sitemap from the contents of the database.
# Author: Alastair Brunton
# Modified: Harry Love 2008-06-09
class ModelCrawler

  def initialize(base_url, sources)
    @base_url = base_url
    @sources = sources
  end

  # 1. Iterate through each model's #get_paths method
  # 2. Create sitemap file for each model
  # 3. Create sitemap index file
  # 4. Ping Google
  def generate
    path_ar = []
    sitemaps = []
    @sources.each do |source|
      # initialize the class and call the get_paths method on it.
      path_ar = eval("#{source}.get_paths")
      xml = generate_sitemap(path_ar)
      save_file(source, xml)
    end
    index = generate_sitemap_index(@sources)
    save_file('index', index)
    update_google
  end

  # Create a sitemap document for a model
  def generate_sitemap(path_ar)
    xml_str = ""
    xml = Builder::XmlMarkup.new(:target => xml_str)
    xml.instruct!
    xml.urlset(:xmlns => 'http://www.sitemaps.org/schemas/sitemap/0.9') {
      path_ar.each do |path|
        xml.url {
      	  xml.loc(@base_url + path[:url])
      	  xml.lastmod(path[:last_mod])
      	  xml.changefreq('weekly')
        }
      end
    }
    xml_str
  end

  # Create a sitemap index document
  def generate_sitemap_index(sitemaps)
    xml_str = ""
    xml = Builder::XmlMarkup.new(:target => xml_str, :indent => 2)
    xml.instruct!
    xml.sitemapindex(:xmlns => 'http://www.sitemaps.org/schemas/sitemap/0.9') {
      xml.sitemap {
    	  xml.loc(@base_url + "/sitemaps/static.xml")
    	  xml.lastmod(Time.now.strftime('%Y-%m-%d'))
 	    }
      sitemaps.each do |site|
        xml.sitemap {
      	  xml.loc(@base_url + "/sitemaps/#{site}.xml.gz")
      	  xml.lastmod(Time.now.strftime('%Y-%m-%d'))
   	    }
      end
    }
    xml_str
  end

  # Save the xml file (gzipped) to disk
  def save_file(source, xml)
    File.open(RAILS_ROOT + "/public/sitemaps/#{source}.xml.gz", 'w+') do |f|
      gz = Zlib::GzipWriter.new(f)
      gz.write xml
      gz.close
    end
  end

  # Notify Google of the new sitemap index file
  def update_google
    sitemap_uri = @base_url + '/sitemaps/index.xml.gz'
    escaped_sitemap_uri = URI.escape(sitemap_uri)
    Net::HTTP.get('www.google.com', '/webmasters/tools/ping?sitemap=' + escaped_sitemap_uri)
  end
end

4. Alter deploy.rb

Now you’ll need an entry in your [rails_app]/config/deploy.rb file to copy your Sitemaps over with each new release:

namespace :sitemap do
  desc "Copy the sitemap files after deploy"
  task :copy_sitemap, :roles => :app, :on_error => :continue do
    puts "copying Rails sitemap files"
    run "cp #{previous_release}/public/sitemaps/* #{current_release}/public/sitemaps/"
  end
end

after :deploy, 'sitemap:copy_sitemap'

5. Create a rake task

Now add a rake task to actually perform the Sitemap generation by creating the [rails_app]/lib/tasks/sitemap.rake file and adding the following code:

require 'static_crawler'
require 'model_crawler'

site_url = ENV['URL'] || 'http://localhost:3000'

namespace :sitemap do

  desc 'Crawl the site and create sitemap xml files for both static and dynamic content.  Set CREDS as username:password if you are hitting a password protected site.'

  task(:generate => :environment) do
    # Generate static sitemap
    sitemap = StaticCrawler.new(site_url, (ENV['CREDS'] if ENV['CREDS']), true, true, false)

    # Generate dynamic sitemaps for each of the models listed in the array
    models = %w( Project )
    sitemap = ModelCrawler.new(site_url, models)
    sitemap.generate
  end

end

6. Setup a cron task

Finally, add an entry in your crontab to periodically run the rake task and generate Sitemaps:

30 9 * * * cd /path/to/rails/app && /path/to/rake sitemap:generate URL=http://domain.com RAILS_ENV=production

Be sure to verify the path to your rake command.  It can be different on some systems.

But now you are thinking about going live to the world, and suddenly realize there is still so much more to be done!  This is exactly where I found myself recently with the deployment of brockbouchard.net.  Below are some of the tasks that are easy to overlook and how to go about completing them.

1. Keeping Log Files Tidy

While you could have your single production.log file fill up indefinitely on your server, it is better to create a rake task that will automatically truncate and create separate files for logs older than X days.  Below is a rake task that I use called “clean_logs” that does just this.  Place this code in a ruby file in your [rails_app]/lib/task directory:

require 'ftools'

desc "Truncates and backs up log files.  Deletes backup logs older than ENV['CLEAN_LOGS_DAYS_OLD'] days, default is 30.  Set RAILS_ENV on command line to target correct log file."
task :clean_logs => :environment do

  days_ago = ENV['CLEAN_LOGS_DAYS_OLD'] || 30

  environment_name = ENV['RAILS_ENV'].downcase
  log_file = "log/#{environment_name}.log"
  log_file_backup = "log/#{environment_name}-#{Date.today.to_s}.log"

  if File.exist?(log_file)
    # move log file
    File.move log_file, log_file_backup, true
    # delete log files older than days_ago
    puts "Deleting log file backups prior to #{Date.today - days_ago}"
    Dir.foreach "log" do |filename|
      if filename =~ /#{environment_name}-\d\d\d\d-\d\d-\d\d/
        backup_date = Date.parse filename.scan(/\d\d\d\d-\d\d-\d\d/)[0]
        File.delete "log/#{filename}" if Date.today - backup_date > days_ago
      end
    end
  else
    puts "#{log_file_name} does not exist."
  end

end

Note here that X defaults to 30, you can change the default to whatever you like, or you can pass in the number of days to keep.  With this rake task in place, add an entry for it to your crontab:

0 9 * * * cd /path/to/rails/app && /path/to/rake clean_logs RAILS_ENV=production

Notice that the full path to the rake executable is specified.  In my case, it is /usr/bin/rake.  However on some systems it may be different.

2. Backing Up Your Database

Even if you run a small site with a small audience, or don’t have access to fancy backup drives or tapes, you should employ some kind of database backup.  This is indeed my situation: brockbouchard.net and its MySQL database all run on one affordable VPS hosting plan.  Nonetheless I run a database backup every night with the following shell script…

#!/bin/sh
/usr/bin/mysqldump -h localhost database_name -uusername -ppassword > /path/to/db/backup/directory/`date +%Y%m%d`.sql
/bin/gzip /path/to/db/backup/directory/`date +%Y%m%d`.sql

…that is called from cron with the following entry…

0 10 * * * /path/to/db_backup_script

Be sure to verify the exact location of your mysqldump and gzip commands.  They could be different on your system.  Indeed I was loathe at first to go through setting this up, but just the other day MySQL decided to corrupt my live database upon server restart.  Fortunately I had a backup ready and waiting!

3. Handling Errors

The first thing to do here is create custom 404, 422 and 500 error pages.  You’ll notice that your Rails app has an html file corresponding to each error number under the [rails_app]/public directory.  While the pages created by rails may suffice, it would probably be more helpful to create a page with relevant contact information on it!

The next thing to do is install the Exception Notifier plugin.  It’s a pretty straight-forward installation, with the following catch.  If you are using a recent version of Rails (I believe 2.2 or later), you’ll need to add a new ruby file to your [rails_app]/config/initializers directory where you will initialize the exception notifier plugin:

ExceptionNotifier.exception_recipients = %w(errors@yourdomain.com)
ExceptionNotifier.sender_address = %(errors@yourdomain.com)

In earlier versions of Rails you could configure ExceptionNotifier right inside of your production.rb configuration file.  But apparently something changed in a recent version of Rails such that doing so no longer works.

Finally, in order to actually get error emails, don’t forget to configure your mail server as I talk about below!

4. Configure Your Mail Server

Oh wow did I ever get burnt by this.  I spent way too much of my time wondering why emails sent by cron and my Rails app were not arriving.  While your mail server situation may be more complex and may already be setup by an IT department, in my case my app is a one man show running on an affordable VPS hosting plan.  By default my mail server (exim4) was limiting emails to local domains only.  I only discovered this after looking in my /var/log/exim4/mainlog file:

... Mailing to remote domains not supported ...

Once I turned this off everything was fine.  If you are in the same boat as me, refer to your mail server’s documentation for info on how to disable this behavior.

5. Permissions

Make sure the directory hosting your application is accessible by the user your web server (Apache, nginx, etc) runs as, and make sure new files placed there pick up those permissions.  I host brockbouchard.net on Slice Host (which I highly recommend) and they have a great article about setting up Apache permissions.  While it follows the deployment conventions of their tutorials, it is nonetheless useful as one could  easily draw parallels to their particular environment:

1. http://articles.slicehost.com/2007/9/18/apache-virtual-hosts-permissions

6. Maintaing Files in /public

You may have files in your [rails_app]/public directory that are not part of your Rails app; these could be files you upload manually or files that others upload.  However if you are using Capistrano to deploy your Rails app (and you should be), each successive deployment will leave the files in your [rails_app]/public directory that are not under source control in the directory for the previous release!  Fortunately this is easy enough to get around by putting something similar to the following in your [rails_app]/config/deploy.rb file:

namespace :deploy do
  task :move_uploaded_files, :on_error => :continue do
    run "mv #{previous_release}/public/uploaded_files #{current_release}/public"
  end
end

after "deploy:update_code", "deploy:move_uploaded_files"

7. Basic Search Engine Optimization (SEO)

While I am not an expert in this field, I knew I had to do something about this for my site!  I found a number of good tutorials for this with Rails in mind:

1. http://www.seoonrails.com/
   
2. http://www.bingocardcreator.com/articles/rails-seo-tips.htm
   
3. http://www.tonyspencer.com/2007/01/26/seo-for-ruby-on-rails/
   
4. http://noobonrails.blogspot.com/2006/09/good-seo-mojo-with-rails.html

I also set out to get my Rails app setup with a Google sitemap and Google webmaster tools.  This was a little bit more complicated and I will leave it for my next post!

http://www.slashdotdash.net/2007/12/04/iphone-on-rails-creating-an-iphone-optimised-version-of-your-rails-site-using-iui-and-rails-2/

I then said to myself that I wanted my site to behave as follows:

1. Occupy the width of the iPhone’s screen
2. When the device is rotated, expand the width of the site and have flow content adjust accordingly (and do not zoom in on content)

This how the iPhone version of ESPN’s website behaves, and is something I wanted to mimic.  However, this is where all of the “easy” came to a crashing halt.

I googled around, eventually finding a link to Apple’s official support pages for building iPhone-specific sites:

1. http://developer.apple.com/safari/
2. http://developer.apple.com/safari/library/codinghowtos/Mobile/UserExperience/index.html
3. http://developer.apple.com/safari/library/documentation/AppleApplications/Reference/SafariWebContent/Introduction/Introduction.html#//apple_ref/doc/uid/TP40002079-SW1

While generally helpful, nothing in any of those articles, or the articles immediately linked to those, helped to answer my problem.  With everything I tried, my site would always stay the same absolute width, zooming in on the content when I rotated.  I eventually stumbled upon a reference of all of the viewport settings one can apply to Mobile Safari:

1. http://devworld.apple.com/safari/library/documentation/AppleApplications/Reference/SafariHTMLRef/Articles/MetaTags.html

Sure enough the answer is to use the maximum-scale setting and set it to 1.0:

<meta name="viewport" content="maximum-scale = 1.0, user-scalable = no, width = device-width" />

Maybe it was my fault, but I feel like it was way to hard and took way too long to find that piece of information :/

One last thought: I found the iPhone’s screen to render colors in a much “darker” fashion than on my iMac.  Even with the brightness all the way up on the iPhone, and the brightness toned down on the iMac, I still found I had to use a separate set of “brighter” colors for the iPhone site to compensate for its screen.  Thus when developing an iPhone site it is important to test it on the actual device itself, and not solely on any of the various simulators out there!

What are the benefits of doing this? There are many:

1. A professional graphic designer can totally retool your site’s design in a manner limited only by his or her imagination and knowledge of CSS.  Your interaction from a programming perspective with the designer is very little if not zero!
2. With the focus of your HTML being solely content and structure, it is easier to appreciate and focus on the semantic statement that your markup is making about your content to search engines.  For example, search engines will look for titles to be in <h1>, <h2> and <h3> header tags, and menus to be in a <ul> list.  With CSS you can then go wild as to how an <h1> or <ul> tag is displayed.  Your header for example, could have its text hidden in order to show a background image representing your site’s header.  And the <ul> containing your site’s menu can be altered to display horizontally or vertically, and without list item markers.  This is indeed how brockbouchard.net was designed.
3. Your site will behave nicely if loaded by a primitive browser that does not support CSS.  Indeed this is a less of a concern with the ever growing use of smartphones with full-featured browsers.  However, there are still many ordinary phones with internet access that feature a limited browsing environment.  A pure-content HTML site will render nicely in this scenario without the need for a special mobile version of the site.

How do we go about doing this? Let’s look at the basic structure of brockbouchard.net as an example.  Below is the basic HTML layout of the site, minus some of the actual information I display:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">

<head>
<title>My Title</title>
	<link href="styles.css" media="screen" rel="stylesheet" type="text/css" />
</head>

<body>
<div id="container">
<div id="header">
<h1><a href="/" title="My Website"><span>My Website</span></a></h1>
</div>
<div id="menu">
<ul>
	<li id="Home"><a href="/home" title="Home"><span>Home</span></a></li>
	<li id="Contact"><a href="/contact" title="Contact"><span>Contact</span></a></li>
	<li id="Projects"><a href="/projects" title="Projects"><span>Projects</span></a></li>
	<li id="Blog"><a href="/blog" title="Blog"><span>Blog</span></a></li>
	<li id="Resume"><a href="/resume" title="Resume"><span>Resume</span></a></li>
</ul>
</div>
<div id="content">
<h2 class="pageTitle">Welcome</h2>
<div id="contentForLayout">
Content goes here</div>
</div>
<div id="footer">
Pick a theme:
<a href="?theme=theme1" title="Theme 1">Theme 1</a> |
<a href="?theme=theme2" title="Theme 2">Theme 2</a></div>
</div>
</body>

</html>

Some things to notice:

1. The placement of <div> and <span> tags, along with the ids and classes associated with them, are totally arbitrary and up to you as the programmer.  Just because I have a <div> tag named “footer” doesn’t mean that <div> has to appear at the bottom of the page.  You could use CSS to place your “footer” anywhere!
2. Think of the use of <div> and <span> tags as “hooks” for finer grained control inside of CSS.  While you don’t want to go crazy placing <div> and <span> tags everywhere, in some cases you will find it helpful.  In my case, I used <span> tags inside of my <a> tags as I wanted my menu links to appear as images I had designed in Illustrator.  However, I found that with some browsers the only way to make the text of the link totally disappear and give way to the image was to put the text inside of a <span> tag and hide that.  Unfortunately applying “font-size: 0″ to <a> tags in my CSS was not working on all browsers and platforms!
3. I put links at the bottom of the page illustrating how the theme could be changed.  In my case, I have my Rails Application controller check to see at the beginning of all requests if a “theme” parameter is specified.  If so, the theme name is stored in a session variable (with the session variable defaulting to the default theme name).  Then, in my view where I spit out the <link /> tag for the style sheet, I choose which style sheet to use based on the theme name.

How about the CSS? While I can’t post all of the CSS here, here are some of the highlights.  If you want to see how I did everything then feel free to go to brockbouchard.net, view the source, then follow the link to my style sheet.

#container
{
	/* this makes the page a fixed width and centers it */
	margin-left: auto;
	margin-right: auto;
	width: 980px;
}

#header
{
	margin: 0px;
	/* this corresponds to the size of the header image */
	width: 980px;
	height: 90px;
	background-image: url("/images/main/theme1/header.jpg");
	background-repeat: no-repeat;
}

#header h1
{
	padding: 0px;
	margin: 0px;
	width: 100%;
	height: 100%;
}

#header h1 a
{
	/* these rules make the link inside of the header take up the entire space of the header */
	/* thus this makes the header a clickable link */
	display: block;
	width: 100%;
	height: 100%;
	text-decoration: none;
	border-width: 0px;
	margin: 0px;
	padding: 0px;
}

#header h1 a span
{
	/* hide the text inside of the header link */
	/* doing "font-size: 0" for the <a> does not work across all browsers */
	visibility: hidden;
}

#menu
{
	margin: 0px;
	/* the menu is rendered as all one image */
	/* the size of the menu items defined below defines the clickable area for each menu item */
	background-image: url("/images/main/theme1/menu.jpg");
	background-repeat: no-repeat;
	/* this corresponds to the size of the menu image */
	width: 980px;
	height: 45px;
}

#menu ul
{
	/* this corresponds to the size of the menu image */
	width: 980px;
	height: 45px;
	margin: 0px;
	padding: 0px;
	list-style-type: none;
}

/* this is the necessary CSS to turn list items into invisible, clickable areas over the menu image */

#menu ul li
{
	position: relative;
	display: block;
	height: 40px;
	top: 0px;
}

#menu ul li a
{
	display: block;
	width: 100%;
	height: 100%;
	text-decoration: none;
	border-width: 0px;
	margin: 0px;
	padding: 0px;
}

#menu ul li a span
{
	visibility: hidden;
}

#HomeMenuItem
{
	/* something like this would be repeated for each menu item with the appropriate width filled in */
	float: left;
	width: 104px;
}

#HomeMenuItem a:hover
{
	/* change the appearance of the menu item when the user hovers over */
	/* would need to do something similar for each menu item */
	background: transparent url("/images/main/theme1/home_hover.jpg") no-repeat scroll top left;
}

The site was built using Rails, and I took the opportunity to over-engineer a few features to learn some new tricks.  I also host the site on VPS plan where I had to deploy and configure everything myself.  This was a highly painful, but more importantly highly informative, experience.  I plan on blogging about what I learned building the site over my next several posts.

UPDATE: Apple posted a patch rather promptly, and thus my iPhone continues to be the greatest thing to ever happen to humanity.

This got me wondering: is Multi Touch hardware starting to emerge in anticipation of Windows 7?  It turns out it is.  Take for example these offerings from HP and Dell.

Thus I ponder the following:

1. Will I be able to buy one of these HP or Dell All-In-Ones and start creating Multi Touch WPF applications out of the box?  Or am I going to need to interface with whacky low-level Windows APIs or vendor-specific SDKs?
2. Why is Apple and its community of followers totally quiet about the idea of a Multi Touch iMac?  Maybe Apple figures that Multi Touch only works with something you hold in your hand.  I could see my arm getting tired using a Multi Touch All-In-One all of the time.
3. Multi Touch applications for full-fledged Windows PCs could be the killer selling point and marketing appeal of Windows 7 that no one has seen coming yet.

Well I hope this all shakes out the way I want: to buy a Windows 7 All-In-One this fall and start writing Multi Touch applications in pure WPF.