Preface on why I love git

Roughly four years ago I switched to git for version control and it’s changed my development life. While SVN is sufficient to “get the job done” in terms of keeping a history of your commits with a group of people, Git is truly the next evolutionary step up in every way. I could expound a whole article on the joy of git, but many others already have, and this articles not about that. But I will kindly throw in my 0.02 on why I personally love it:

• It’s fast. Git keeps your whole repo’s history local. So no need to ping your repo server for every command. You can even checkout your codebase from 3 weeks ago or a year ago in under a second.”
• It works offline. That’s right, no internet. Sync up later.
• It has useful modern tools. Like git stash and git bisect. Both will save you a lot of time.
• It’s your third hand. Let’s face it, besides svn being slow, it’s not friendly enough to do more than throw your past commits into an empty hole you will probably never see again. With very little setup, you have beautiful colored output in your logs and diffs with git. And “git log” actually pages by default (while keeping colors) instead of flooding your console with the entire repos history! Want to query something in your history log? What about all commits from your friend Bob in the past 2 weeks?

  git log --date=relative --author=nizam --since="2 weeks ago"

  Bam. Done.

• Easy repo creation. You don’t have to be an admin to setup a git repo. Just cd into a folder and type “git init”. You just created a new repo. I do this all the time, for example in my folder of todo lists, I use it to track history of my todos. At work someone kept messing up the apache config file, all I had to do was “git init” in /etc/httpd/conf and now I can easily “git diff” new changes that were made and revert / adjust if needed.
• Many ways to use it. Git is promoted as “decentralized” version control, but it can be an easy drop in replacement for any job SVN can do. Which leads me to…

With all those advantages, many larger companies haven’t jumped on the git bandwagon. I believe there’s two reasons this is the case.

• As programmers get more experienced, they generally become more pragmatic (which is generally a good thing.) They adopt a “if it ain’t broke, don’t fix it.” mentality. After all, SVN get’s the job done.
• There is a bit of a learning curve with git. But there are a ton of good resources for learning it. I recommend getting your hands dirty with Git Immersion if you’re new or would like to learn. I don’t recommend wrapping git around svn as I’m about to show you for a person who’s learning git.

Getting down to business.

In order to use git with an svn repo, you’ll have to run some commands to import that svn repo, commit by commit into a git repo.

1. First, make sure you have a version of git installed with svn bindings. If you type “git svn” and it give you an error, I’d google one of the resources in reinstalling git with svn bindings.

2. We need to keep

# In your existing svn repo, lets grab a list of authors
svn log -q | awk -F '|' '/^r/ {sub("^ ", "", $2); sub(" $", "", $2); print $2" = "$2" <"$2">"}' | sort -u > authors.txt
# create a new folder
cd ..; mkdir NEWREPO.git;
git svn clone -A ../OLDREPO/authors.txt https://svn/repo/trunk
git svn fetch # this could take a long time as it checks the repo out one rev at a time.
 
# update your working copy:
git-svn rebase
 
# commit your changes to svn server
git-svn dcommit

kudos to:

http://flavio.castelli.name/howto_use_git_with_svn

function gdaily {
    NEXT=$(date +%F)
    echo "CHANGELOG"
    echo ----------------------
    git log --no-merges --format="%cd" --date=short | sort -u -r | while read DATE ; do
        echo
        echo [$DATE]
        GIT_PAGER=cat git log --no-merges --format=" * %s" --since=$DATE --until=$NEXT
        NEXT=$DATE
    done
}

Which looks like:

[2012-01-07]
 * Refactor list portfolio page css and js.
 * namespace.data include to load fake api data until it is ready. Portfolio list working again.
 * CurrentUser model defaults and code cleanup.

[2012-01-06]
 * Upgrade portfolio call from GET and POST. To be discussed in tomorrow's standup.

My latest approach has been modifying a Linksys NSLU2, an old low-powered network storage device, and installing Debian on it (thanks to great documentation from the NSLU/Linux community). The end product is a web server that runs between 3-5.5 watts! And it’s able to support two usb drives. Not the fastest machine but certainly useable, especially at the cost of only $35 used for the device off Craigslist (compared also to the $225 or so I spent assembling the machine above.)

While I think it’s the approach I’m going to go with, I found a link where someone installed Ubuntu on their Android device. This piqued my interest as it’s also a low wattage device (and the fact that I have a Galaxy S sitting around doing nothing.) I followed the steps, and though it worked well, the Ubuntu package manager has problems when you want to install any new packages or do an update. It just 404′s like so:

Err http://ports.ubuntu.com karmic/main Packages
  404  Not Found

After much research I ended up having to change /etc/apt/sources.list from:

deb http://ports.ubuntu.com/ubuntu-ports karmic main universe

to:

deb http://old-releases.ubuntu.com/ubuntu/ karmic main universe

I tried to post this update to the blog above but it errored out, so I’m leaving the solution here instead should anyone else have the issue.

Additional note In the case of the NSLU2: if DHCP has assigned the Slug an ip address but your wireless router doesn’t find it, use nmap to ping for port 22 on your subnet:

nmap -p 22 --open -sV 10.0.0.0/24

If you’ve experienced a Backbone error that’s not on the list, feel free to post it in the comments.

• 0.1 second is about the limit for having the user feel that the system is reacting instantaneously, meaning that no special feedback is necessary except to display the result.
• 1.0 second is about the limit for the user’s flow of thought to stay uninterrupted, even though the user will notice the delay. Normally, no special feedback is necessary during delays of more than 0.1 but less than 1.0 second, but the user does lose the feeling of operating directly on the data.
• 10 seconds is about the limit for keeping the user’s attention focused on the dialogue. For longer delays, users will want to perform other tasks while waiting for the computer to finish, so they should be given feedback indicating when the computer expects to be done. Feedback during the delay is especially important if the response time is likely to be highly variable, since users will then not know what to expect.

From the book Usability Engineering (1993). More info here.

Fast forward to now and I need to make some general web pages again for a project and I need to support IE once more. These pages have some light-to-moderate js functionality and though I could write a small library from scratch, I’d be a fool in wasting so much time, though minimally, I only need:

• Cross-browser event support
• Cross-browser ajax support

Though one does arguably write faster and more concisely (smaller file size) if one has:

• A chainable API
• A CSS selector engine

The answer? Use an older version of jQuery. jQuery 1.2.6 for example is 17.18 KB (compressed and gzipped). Almost half as small as the current version. It gives me all four requirements above. Every version of jQuery has been battle-hardened against the most finicky older browsers and the newest batch of browsers aren’t really causing too much trouble. I had been writing with Zepto in Chrome initially when I made the switch to test in IE. I made the switch to 1.2.6 without issue. IE and all browsers are happy.

While I would love to take credit for thinking this through, it actually happened by accident. I referenced jQuery version 1.2.6 from Google’s CDN instead of 1.6.2.

I’m sure someone’s going to comment how I’m missing some major optimizations in later versions, but after trying the project out, I already have split second performance using an old build. If I start doing crazy animations or need .live, I’ll move up some revisions. If you do plan to try this out for yourself, know that jQuery 1.3 (20.08 KB) is when Sizzle was introduced and in jQuery 1.6 a lot of the API was modified or had signature changes (nothing that’s affected me so far).

Here’s a list of some versions along with their file sizes. Check the blog for more details.
1.6.2 (32.18 KB) – misc small bugfixes
1.5 (30.12 KB) – ajax timeout and abort
1.4.2 (24.86 KB) – .delegate
1.4 (27.25 KB) – extended .live support
1.3 (20.08 KB) – Sizzle and Live
1.2.6 (17.18) – 100% faster events
1.2 (17.21 KB)

And don’t forget if you’re using Google’s CDN (a good idea), always fallback to your own version, too, just in case.

*2.6.1, I use Charles Debugging Proxy for measurements.

You want to optimize static content

You have an Apache install that’s hosting a bunch of sites and friend’s sites through vhosts. One of your blogs is getting a lot of hits and you want to optimize it’s static content – or even the static content of all sites. You’re not quite ready for a CDN-type deal, just to place the content outside of Apache and into something more lighter-weight so it’s not running through WordPress / Apache and unnecessarily using up threads (at roughly 2mb a thread).

You want all your web apps on port 80!

You started really getting into Node (or Ruby on Rails or Django) but every web app needs to be binded to it’s own port and port 80 is taken by your Apache which is hosting a lot. You don’t want to be giving out the url: http://mycoolnewapp.com:81.

Sure you can use Apache’s proxypass but you’re gaining overhead and, in case of Node (or Ruby’s EventMachine or Python’s Twisted), you’re losing the whole point of having an optimized, non-blocking / non-threaded, web app.

The solution

Nginx! Nginx is a web server, reverse-proxy, load balancer, and mail server all in one. Like Node, it was built with the concept of the event loop, not threads so it’s highly optimized for high concurrency. The idea is to setup Nginx to be a reverse-proxy for all your other services.

I’ll skip over how to install Nginx as it’s pretty straightforward and you can google it. I’ll go over the main steps to getting Apache to work through Nginx – it’s truly easy, I did it on my first try. The only problem that I encountered is that since the user only interfaces with Nginx – it’s Nginx that is making the requests to Apache / Node. So from Apache’s perspective, all requests are coming from 127.0.0.1. We also fix this in the steps below.

1. Once Nginx is installed, edit your /etc/httpd/conf/httpd.conf so that it listens on another port. Say 127.0.0.1:8080.
2. Switch all your vhosts to also listen on this port.
3. Edit your /etc/nginx/nginx.conf file. Make sure it’s set to listen on port 80 and add a server entry per each apache vhost. Here’s an example:

       server {
              listen       80;
              server_name  mysite.com;
   
   	   location / {
                       proxy_pass http://127.0.0.1:8080;
                       include /etc/nginx/conf.d/proxy.conf;
              }
       }
   

4. Next we’ll add that proxy.conf reference by creating /etc/nginx/conf.d/proxy.conf:

   proxy_redirect off;
   proxy_set_header Host $host;
   proxy_set_header X-Real-IP $remote_addr;
   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
   client_max_body_size 8m;
   client_body_buffer_size 256k;
   proxy_connect_timeout 60;
   proxy_send_timeout 60;
   proxy_read_timeout 60;
   proxy_buffer_size 4k;
   proxy_buffers 32 256k;
   proxy_busy_buffers_size 512k;
   proxy_temp_file_write_size 256k;
   

5. At his point you need to install mod-rpaf for Apache. This enable Apache to use the extra headers Nginx is passing in the request. If you’re using a flavor of Linux that uses apt-get you’re in luck. Just run: sudo apt-get install libapache2-mod-rpaf. If you’re using a system that uses Yum, you’ll have to compile it yourself. Just follow the steps here.

As for serving static content. Inside each “server” declaration you can add the following (modified to your taste):

       location ~* ^.+\.(jpg|jpeg|gif|png|ico|tgz|gz|pdf|rar|bz2|exe|ppt|txt|tar|mid|midi|wav|bmp|rtf) {
            root /folder/to/static/files;
            expires 90d;
       }
       location ~* ^.+\.(css|js)$ {
            root /folder/to/static/files;
            expires 30d;
      }

Ad that’s it you’re done!

Additional reading

Nginx Primer
Nginx Primer 2: From Apache to Nginx
Apache with Nginx
Really solid config samples

http://jashkenas.github.com/docco/

Creates pages like this:

http://documentcloud.github.com/backbone/docs/todos.html

A few quick things to note:

• Gerrit is made by guys at Google and is used in well known projects such as the Android platform and is used at companies like digg.com.
• I found it fairly easy to setup, just run it’s .war file and answer a few questions (few prerequisites below).
• The Gerrit platform internally, uses a java implementation of git as well as it’s own java SSH daemon. Wonder why? I did too, and posted in the Gerrit Google Group, and got some helpful answers.

Getting started

• Before hand you should have mysql running with an empty database (default: reviewdb)
• Decide on which port you want to run the Jetty webserver beforehand, and which port you want to run the SSH daemon (it must be different than your default one). I recommend you not run any SSH daemons on the default port (22) if your server is directly accessibly from the internet.
• To be secure, you should prepare a separate unix user that gerrit runs as (container process), as well as a gerrit mysql user with full access to the reviewdb.
• You should also have SMTP setup either on the server or through another source – note that you cannot just use sendmail, as I also asked here.

Quick commands:

cd /tmp
wget http://gerrit.googlecode.com/files/gerrit-2.2.1.war
 # run the install wizard
sudo java -jar gerrit-2.2.1.war init -d /srv/gerrit

During the install, when it asks you if you want to copy the war file, agree.
sudo java -jar gerrit-2.2.1.war daemon -d /srv/gerrit # to test it out

Other Helpful Info

gerrit’s home is:
/srv/gerrit

Create /etc/default/gerritcodereview containing the line:
GERRIT_SITE=/srv/gerrit
so you can use the shell script for turning off / on:

As for SMPTP you can just set up a gerrit.yourdomain gmail account. Mark SSL encryption on port 443.

exception:Invalid UTF8 character detected

Use iconv, a program that seems to be on both mac and most unix systems:

iconv -f ISO-8859-1 -t utf-8 accounts.csv > accounts-utf8.csv

Then continue on your merry way:

mongoimport --type csv --headerline --drop --port 7777  -d dbname -c collectionname --file accounts-utf8.csv