Map Yahoo! Address Book Contacts

Map Yahoo! Address Book Contacts

Plot the locations of your friends and colleagues on a map of the world with your Yahoo! Address Book and the worldkit mapping application.

The Yahoo! Address Book lets you store more than just names and email addresses. You can store complete contact information for everyone, including street address, city, state, and country. If you have more than a dozen or so contacts stored, you might not realize how geographically dispersed everyone is. This hack plots the locations of all of your Yahoo! Address Book contacts on a map, so you can visualize where your friends, family, and coworkers live.

Plotting each contact's location is possible thanks to Yahoo!'s Address Book export feature, which provides all of your address book data in commaseparated value (CSV) format. From there, a freely available web service called Geocoder (http://geocoder.us) translates each address into its longitude and latitude. And finally, worldkit (http://brainoff.com/worldkit) plots each point on a map.

Preparing Your Address Book

Right now, your address book might be in sloppy condition (mine was), including misspelled addresses, missing states and countries, and abbreviated cities. Geocoding requires some degree of accuracy to find good matches, so before getting started, tidy your address book up. It's not necessary to produce a pristine address book, but you need at least a city and country, and a state abbreviation for U.S. locations. Street addresses for U.S. contacts can be used too. You'll likely need to iterate this cleanup a couple of times, with feedback from the geocoding script. Export your contacts by loading http://address.yahoo.com, selecting Import/Export, and selecting Export Now! for the Yahoo! CSV format.

The Code

yadr2geo.pl is a Perl script that takes the name of your downloaded address book and outputs a geocoded RSS file to use with worldkit. Geocoded RSS refers to any flavor of RSS extended to include item-level latitude and longitude. More details on the format can be found at http://brainoff.com/worldkit/doc/rss.php#basic.

This script requires commonly installed modules: URI::Escape for formatting web service requests, LWP::Simple for making those requests, and XML::Simple for parsing the responses.

This script does not use a module to parse CSV, because modules such as Text::CSV assume that a newline indicates a new record, while in Yahoo! CSV (and most flavors of this unofficial spec) it's legal to include a newline within an entry if that entry is quoted. CSV is discussed in more detail at http://www.creativyst.com/Doc/Articles/CSV/CSV01.htm.

Yahoo! CSV is simple: all entries are guaranteed to be quoted, the first line gives field names, and there's no extraneous whitespace. So it's straight-forward to program a script to parse Yahoo! CSV character by character. The subroutine geTRecord() takes an open filehandle as an argument and returns an array containing the next CSV record.

Save the following code to a file called yadr2geo.pl:

	
	#!/usr/bin/perl -w

	use strict;
	use XML::Simple qw(XMLin);
	use LWP::Simple qw(get);
	use URI::Escape qw(uri_escape);

	# Map your personal country naming conventions
	# to country codes listed at http://brainoff.com/geocoder/countryselect.php
	# and change the default country if you wish
	my %countrycode = ('USA' => 'US');
	my $defaultcountry = 'US';

	print <<RSSHEADER;
	<?xml version="1.0"?>
	<rss version="2.0">
	<channel>	
		<title>Yahoo! Address Book</title>
		<link>http://address.yahoo.com/</link>
		<description>My geocoded Yahoo! Address Book</description>
	RSSHEADER

	my (%hash, @vals, $arg, $loc, $lat, $lon, $success, $country);

	# First line of Yahoo! CVS is keys
	my @keys = @{ getrecord(*STDIN) };

	while (! eof(STDIN)) {
		@vals = @{ getrecord(*STDIN) };
		@hash{ @keys } = @vals;

		$success = 0;
		undef($loc);
		$country = $countrycode{ $hash{'Home Country'} } || $defaultcountry;

		# Check for sufficient information to geocode

	if (length($hash{'Home City'}) == 0
		|| ($country eq "US" && length($hash{'Home State'}) == 0)) {
		print STDERR "Couldn't geocode: \""
			. join ("\",\"", @vals) . "\"\n";
		next;
	}

	# Try geocoding US street address
	if ($country eq 'US'
		&& length($hash{'Home Address'}) > 0) {
		$arg = $hash{'Home Address'} . "," . $hash{'Home City'}
			. "," . $hash{'Home State'};

		eval {
			# Be patient, geocoder.us free service is rate limited
			$loc = XMLin(
				get("http://geocoder.us/service/rest/?address="
				. uri_escape($arg) )
			);
		};
	
		if (!$@ && defined($loc->{"geo:Point"}->{"geo:long"}) &&
			defined($loc->{"geo:Point"}->{"geo:lat"})) {
			$success = 1;
		}
	}

	# Try geocoding world city
	if ($country ne 'US' || ! $success) {
		if ($country ne "US") {
			$arg = $hash{'Home City'} . "," . $country;
		} else {
			$arg = $hash{'Home City'} . "," . $hash{'Home State'}
			. "," . $country;
		}
		eval {
			$loc = XMLin(
				get("http://brainoff.com/geocoder/rest?city="
				. uri_escape($arg))
			);
		};

		if (!$@ && defined($loc->{"geo:Point"}->{"geo:long"}) &&
			defined($loc->{"geo:Point"}->{"geo:lat"})) {
			$success = 1;
		}
	}

	if ($success) {
		print <<ITEM;
	<item>
		<title>$hash{'First'} $hash{'Last'}</title>
		<geo:lat>$loc->{"geo:Point"}->{"geo:lat"}</geo:lat>
		<geo:long>$loc->{"geo:Point"}->{"geo:long"}</geo:long>
	</item>

	ITEM
		} else {
			print STDERR "Couldn't geocode: \""
				. join ("\",\"", @vals) . "\"\n";
		}
	}

	print "</channel></rss>\n";

	#
	# "getrecord" returns the next record as an array from an open
	# filehandle. It is a simple state machine, that expects a file
	# formatted in 'Yahoo! CVS'
	#
	sub getrecord {
		my $fh = shift;
		my $c = "";
		my $st = 0;
		my @record;
		my $entry = "";
		while (defined($c)) {
			$c = getc($fh);
			if ($st == 0) {
				if ($c eq "\n" || ! $c) {
				return \@record;
				} elsif ($c eq "\"") {
				$st = 1;
				} else {
				die "error: parsing state:$st char:$c\n";
				}
			} elsif ($st == 1) {
				if ($c eq "\"") {
				$st = 2;
				} else {
				$entry .= $c;
				}
			} elsif ($st == 2) {
				if ($c eq "\"") {
				$entry .= "\"";
				$st = 1;
				} elsif ($c eq ",") {
				push @record, $entry;
				$entry = "";
				$st = 0;
				} elsif ($c eq "\n") {
				push @record, $entry;
				return \@record;
				} else {
				die "error: parsing state:$st char:$c\n";
				}
			}
		}
		die "error: premature end of file\n";
	}

The main body of the script builds a hash from the current record, attempts to geocode the address, and outputs an RSS item if it's successful. For U.S. locations with full street address, the REST service from http://geocoder.us is employed. It expects an address, city name, and state abbreviation, and it returns a small bit of XML containing a latitude/longitude pair if it's successful. The free service is rate limited, so you'll notice pauses during requests. For non-U.S. locationsand for unsuccessful Geocoder requestsa request is made to the REST interface of the Geocoder at http://brainoff.com/geocoder, which expects a city, state abbreviation for U.S. cities, and country code.

The country codes are particular to the GNS (http://earth-info.nga.mil/gns/html) database that backs this service. To look up the codes, go to http://brainoff.com/geocoder/countryselect.php and select a country; a JavaScript alert will give you the code. You will need to map the country names used in your address book to these codes, by adding entries to %countrycode in the script.

If you use a non-English language on Yahoo!, you might have different field names from the ones expected. The script uses Home Address, Home City, Home State, and Home Country. You might need to examine your CSV export and replace these field names in the code. Similarly, if you wanted to map work addresses, you'd replace Home with Work in each of these field names. Another modification to try is adding a <description> or <link> field to each item, set, for example, to the Personal Website field.

Running the Hack

With the script and the Yahoo! CSV export file (yahoo.csv) in place, call the script like this:

	perl yadr2geo.pl < yahoo.csv > rss.xml

The file rss.xml will contain each of the entries from your Yahoo! Address Book, along with its geocoded location.

Plotting the Addresses

The final step is to download worldkit from http://brainoff.com/worldkit and install it on your server or locally. Loading the included index.html in your browser displays the default map. Replace the included rss.xml with the output of yadr2geo.pl and reload the map. You'll see the locations of your friends spread over the globe, as in the geographically dispersed map in Figure.

Yahoo! Address Book entries plotted on a worldkit map

There are many possible customizations described in the worldkit documentation, from changing the map from global to city scale, to changing the annotation colors according to the category of each contact.

Mikel Maron