Google Developer Day across the World

I was tempted to get up in the wee hours to see the first live webcasts of Google Developer Day 2007 from London. But my brain's desperate need for delta waves overrode its thirst for beta.

Fortunately Google is posting them to YouTube and I'm already seeing one from Sydney on Ajax.

I will probably try to catch a few of the live ones from California. There's twenty five sessions on five different tracks. But there's one on Python design patterns and a few on geospatial that I'm particularly interested in.

There's something about a live webcast that makes it more fun. It's like watching a live concert or television broadcast; you know your brain is processing the same information at the exact same moment as millions of others on the planet.

[UPDATE]

Notes

Watching the keynote now.

There will be over 160 sessions around the world! All free with free food. Hopefully they will all be posted to YouTube. [Hopefully they will all be posted to YouTube.]

Google Mashup Editor lets you build mashups via an online tool in as little as 3 lines of code and serve your mashup via other Google services like Google Desktop.

Google has now become the largest network. Over .5 billion visitors (per day, month?).

Google gadgets are very popular and are generating huge amounts of traffic for their authors (in the millions of pages views per month for some). Example is Expedia Fare Calendar.

Can drop them in Google Pages, Desktop, and hook them into your site and it's served by Google. Pundits talking about "the Google Gadget economy."

"Maplets bring Gadgets to Maps."
Orbitz has a gadget that shows hotel room availability on Google maps.

The power comes from building "mashups of mashups" from disparate services to combine useful data like mashing in weather data, restaurants nearby, etc.

Tools to build the world's next generation of web applications.

Google Web Toolkit lets you code in Java and compiles it to JavaScript and HTML for deployment. [There needs to be a Ruby and Python version. Hmm...]
"Vibrant community around it."

Limits of Ajax: For one you have to be connected to the net.

Gears lets you take your web apps offline [similar to Joyent's Slingshot or Dojo's offline features?]

Works on Firefox, IE, Safari on Mac, Windows, and Linux.

Adobe using Gears with Flex apps and Apollo to install to your desktop. Works on Windows, Mac, and Linux. Drag and drop data from native application from Apollo to desktop. Uses Sqlite for local datasource.

Everything will be linked to from code.google.com.

Sergei Brin speaking now.
AI hasn't worked. "You don't have to eliminate the person. You can create the person."

[Uh oh, here comes the evil part. Skynet is coming!]

First offspring from online dating is about 12 years old and they are creating maplets right now. So the internet is improving itself already.

Google Gears session

Are offline apps necessary? Aren't we always connected now?

But... it's not always reliable even when you can be connected so it's important to have your data available offline. And you have to pay for net access in a lot of cases. Can take 5 minutes or more to hook up.

You are offline more than you think you are, especially if you are a laptop user.

Performance of offline is better. No network roundtrip latency. Takes more time than data from the desktop.

Early release of Google Gears.

In a perfect world...

• One application, one URL
• Seamless transitions between online and offline
• Ability to use local data, even when online
• Available to all users on all platforms

The user shouldn't have to do syncing. The application should make this transparent.

[Seems like we're just talking about offline as a cache. You could use local data caching to improve performance of online apps too. For example, first 2000 blog posts show up in Google Reader are cached on your desktop. You want more, it connects to the web.]

Ajax has been missing offline capability.

An incremental approach

• Do for offline what XMLHttpRequest did for web apps
• Browser plugin
• IE, Firefox (Wind, Mac, Linux), and Safari (almost!)
• Open source: http://code.google.com/p/google-gears (New BSD)

What do Gears apps look like?

• Read and write using local store
• Changes are queued for later synchronization
• Server communication is completely ?...

Tools to build but not to guide you in synchronization and conflict resolution yet.

With Gears most operations stay local and synchronization happens in background.

APIs:

• Localserver, to access the app offline
• Database, to store large amounts of structure data
• WorkerPool, to perform long operations

ManagedResourceStore: Capture entire applications

• List application resources in a separate manifest
• Gears captures and updates the application resources atomically.
• Gears automatically updates application resources on each view (within reason).
• Always run locally.
• Supports multiple users per application.

Uses Sqlite for local store. Parameterized SQL for protection against SQL injection attacks.

Open source. Looks like most of the code is C.

Full text searching in Sqlite using Google search algorithms. Joint project with Sqlite.

Google and the Geoweb
Bent Hagemark, Lior Ron, Michael Jones

Michael

The Google Geo group wants a Star Trek tricoder so you can feel like a local when you're not.

Geographically organize the world's info and make it universally accessible and useful.

Odyssey and Iliad mapped on Google Earth.

Some info belongs on a map, some on a page, some either way.

People are building maplets while they're viewing the Google Developer Day session.

Connecting with users. Local data with buildings in 3d. In Zurick can see subway and train lines. Real time traffic. U.S. Senator made a geoblog.

Sketchup for 3d geometry.

Be the agent of change. Surf the wave and win.

Alan Kay. "What's the easiest way to invent the future? Invent it!"

Bent Hagermark with KML overview.

[Uh oh. XML...]

Placemark give you:

3d/2d view
list view
balloon

Paste KML into an app and it'll actually do something.

Placemark:LookAt
lat, long, range, tilt, icons, rollover styles, balloons, points, lines, shapes, 3d shapes, styling, 3d textured models, ground overlay, screen overlay, folder for grouping elements and hierarchies, time with timestamps and timespans, NetworkLink,

NetworkLink
Divide your docs into many docs. Like and include in C.

view-based refresh. After motion is stopped can call another script/resource, can do time-based refreshes, refresh after expiration.

Region-based allows to trigger a fetch of a link when a user moves into a particular region.

http://code.google.com/apis/kml

KML is:
geographic display language
hosted XML files
a standard

[Boy, there needs to be a YAML version, KAML?]

Amazing developer examples demo with Google Earth flying you around from point to point in 3d. It's like you're flying from point to point like a Superman. All controlled via KML definitions.

Design interactive and motion-based presentations.

All Google Earth stuff also works on maps.

Images from devastation of Kansas town destroyed by tornado.

Can host the KML files anywhere and have the same functionality.

3 levels of imagery can be viewed.

User can fly around and view your data with timespan control for things like viewing a time window of a person walking with a GPS device.

[Can't wait til some network games take advantage of Google Earth. This has to be the next step toward the holodeck. :)]

Time instants for time span data. If your data is imagery, you use a timespan and sweep with timestamps. [Did I get that right?]

Can pop up balloons that are very detailed like a web page. You can keep the balloon up while flying from one location to another. User can navigate dataset without using the side panel, zooming from one point to another.

[This is amazing to watch!!!]

New beta spec for photo overlays. You can fly away from the photo and see that they are in space.

Leor on Geosearch:

Angelina Jolie has tattooed the long/lat of her adopted children on her arm.

Maps and balloons for content that has lat/long.

Can see user created content from geosearch.

Can link to other results and resources from the geosearch results.

Now searching for Osama bin Laden. [Too bad Bush doesn't know about this. :)]

Now searching for all windsurfing locations with aggregated results from 5 different web sites.

Now searching for dog parks in the Bay area and near his house.

Users are helping to map the world.

Use for trip planning. Now searching for tango places in Buenos Aires.

Can also do business transactions, e.g. can book a place from a map balloon directly.

Three steps:

• Use KML or GeoRSS instead of JavaScript so your data can be indexed by Google.
• Put it on a public server.
• Inform them in a sitemap so it can be indexed properly.

[This is frikkin' awesome stuff!! Time to write a KML/GeoRSS plugin for Rails. ;-)]

Mashups with the Google Mashup Editor
Rich Burdon, Paul McDonald

Rich:

'Hello, world' mashup showing the world in a map.

Mashups is about feeds.
Pasting url of feed into the editor.

Feed browser hooks into the Reader framework. Pulls in a feed URL that is a GeoRSS feed.
Creates a template (autocompletion!). Table row with repeat attribute will repeat elements from feed in table rows.

[Can't really see the code. Someone tell him to increase his browser view!!!]

Images pulled in now.
Extended map to use the feed and geo points. Setup events so the map can respond to those events from the list.

Now map has images on the right that you can click to navigate the map.

List tag with variable substitution syntax to reference app's own custom data store. (Setting up a dataset.) Goes back to add authentication for a login to secure the data.

Uses atom standard and read-write data store.

Now he shows a read/write datagrid for changing the data online.

[Finally some one asked him to increase the size of the browser view!]

Simple relations supported in the data store.

Can use different templates and controls. Schema and query work is done for you.

Google Base integration, pulling in GData feed. Geoencoded info and images pulled in.
Search tag with autocomplete with parameters, binds data from Google Base.
List tag with variable substitution hooked to id of search control.
Builds a list from the data and binds map to results of the query.
Hits Test tab to load the mashup in the sandbox.

[Wow, it occurs to me. How does one write test for all this stuff?]

Now he's deploying the application.

Can now deploy mashup as a Google Gadget to add to his Google Desktop.

Paul:

• Simple online interfaces
• Pluggable extensible services
• Standards based approach

Still experimental and need some help.
A lot of people are signing up [myself included] but right now it's a limited beta.

[Wow, the room looked kind of empty as they closed out so I wonder if another session was more popular.]

Python Design Patterns
Alex Martelli

Forces controted with not under your control.

Facade interposes a simpler object/class exposing a controlled subset of functiionality.

Patterns not created but rather discovered from "Known Uses." (KU)

Some Facade KUs
* dbhash
* os.path: split + indexing; isdir; os.stat

A structural DP like Adapter.
Sometimes they merge.

No good books on Python design patterns.

Creational
Structural
Behavioral
Also can have object or class-level.

Program to an interface, not an implementation.
Python's duck typing.

Favor object composition over class inheritance.
In Python, hold, or wrap
Inherit only with it's really convenient.

Hold: strong coupling on the wrong axis. Revealing to world as it's built.

Wrap: explicit, automatic, gets coupling right (Law of Demeter: have only one . in your expression.)

Creational pattern not common in Python because factories are built into the language.

Only one instance to exist. Use a module. (works 90% of time).
Can't subclass and not special methods (can't override method)
Make 1 instance (no enforcement) and trust that we're all adults.

Singleton ("Highlander")
There can be only one.
Subclassing is a problem.

Monostate ("Borg") Guido dislikes it.
We don't care how many instances as long as they share state.

Subclassing is not a problem.

Python lets you override data not just methods.

Dependency injection. Don't create anything at all in the class. Creation is done from outside and passed into the object/class. Doesn't work write if you need to do mulitple creations. Factory fixes that.

[Here I gave up trying to keep up with Alex. But I found his slides and some other presentations on design patterns online.]

"ReCAPTCHA" your forms and email while helping the world to read

The computer scientists from the School of Computer Science at Carnegie Mellon University, with support from Intel, Novell and The MacArthur Foundation, have put together a really nifty service that kills two birds with one click of the submit button.

reCAPTCHA is a free web service that anyone can use to prevent spammers and bots from posting to your web form or sending you email by scraping your mailto links. They do a really nice job of explaining their service and the whole subject of CAPTCHA in general but I'll try to sum it up.

To submit a form or reveal an email link on a web page, you must enter two words presented to you as images that have been slightly obfuscated so that only a human brain is likely to make them out. One of the words is known by the system and if you enter it correctly, the submission is considered to be by made by a human and not some spambot. The other word is not known by the system but if you entered the the known word correctly, the system will make the reasonable assumption that the other word is also likely correct - that is, was entered by someone with a brain - and it will compare your entry with the other potentially correct entries for that word to determine what that unknown word really is.

OK, well, so what? Well, those words are very important. They come from texts that have been scanned in with OCR (optical character recognition) software but were not fully recognized and matched to their text equivalents. So by using the reCAPTCHA service, you are helping to digitize books from the Internet Archive. With each accurate challenge response input by a human - or very lucky monkey - a hard-to-decipher piece of text from a book gets that much closer to being translated.

Simple but clever solution, eh? Turing would be proud.

It's sort of like SETI@home except instead of sharing processors to hunt for alien messages from space, they're farming human neurons to decode messages - in the form of books - from our own earthbound species. It's a real-time human neural net, a collectively conscious form of web-based wetware!

Some important things to note about how this service is different than many others:

• The widget offers a reload button so if you can't identify the two words you can try another two.
• The widget provides audio CAPTCHA so it's accessible to the visually impaired. A series of numbers are read out with a slight fuzzing noise in the background (a aural equivalent to the visual obfuscation in the image version) and you enter them instead of the words.
  
• The same widget GUI is available in a bonus feature, reCAPTCHA Mailhide, that lets you hide your mailto links via a popup window linked to the obfuscated email address.
  

Most of the software on the backend is FOSS, heavy on the Python side. But there are plugins and libraries for wiring up PHP, Python, Ruby, Rails, Perl and some of the more popular bulletin boards, blogging, and publishing tools out there like Wordpress, phpBB, and MoveableType to name just a few. I'm sure the list will grow.

In fact, I wanted support in both Rails and Django for controlling the CSS themes and tabindex in the widget per the "Look and Feel" section of the API, so I just contributed a little code of my own to Jason L. Perry's most excellent Rails plugin and Ben Maurer's handy Python library. (I know the functionality is now in the Rails plugin and it will probably be available shortly in the Python version.)

There is a Google reCAPTCHA group for questions and support and you can download some of the plugin and library code with subversion from Google Code.

Kudos to the folks at CMU for such a nice implementation and all the developer's who've been contributing plugins for it. I'm going to recommend this solution to my clients as it's the best I've seen to date.

Please help spread the word about this great new service and contribute code to help support their effort.

Death to spam!!!

[UPDATE]
I've just checked in the same widget styling functionality to McClain Looney's recaptcha Ruby gem. The nice thing about McClain's solution is that it can also be used outside of Rails. It also supports Mailhide nicely.

(In the process McClain introduced me to Mercurial, a light-weight, distributed scm built in Python. There's even a TextMate bundle for Mercurial.)

p.s. Here are the Wired article and Ben Maurer's blog post that first turned me on this.

p.p.s. For you Django developers, here's a view code example of a contact form (with newforms) using Ben's library with my changes for CSS themes and tabindex.

settings.py:
...
# you API keys here...
RECAPTCHA_PUBLIC_KEY = 'xxxxxxxxxxxxxxxxxxxxxxxxx'
RECAPTCHA_PRIVATE_KEY = 'yyyyyyyyyyyyyyyyyyyyyyyyyyy'
...

forms.py:

from django import newforms as forms

class ContactForm(forms.Form):
subject = forms.CharField(max_length=100)
message = forms.CharField()
sender = forms.EmailField()
cc_myself = forms.BooleanField(required=False)

views.py:

from recaptcha.client import captcha
from citizencoder.blog.forms import ContactForm
from citizencoder import settings

def contact(request):
captcha_error = ''
captcha_html = captcha.displayhtml(settings.RECAPTCHA_PUBLIC_KEY,
                                 True, theme='white', tabindex=5)
if request.method == 'POST':
  form = ContactForm(request.POST, auto_id=True)
  captcha_response = captcha.submit(request.POST['recaptcha_challenge_field'],
                                    request.POST['recaptcha_response_field'],
                                    settings.RECAPTCHA_PRIVATE_KEY,
                                    request.META['REMOTE_ADDR'])
  if captcha_response.is_valid:
      if form.is_valid():
          # Form processing here...
          return HttpResponseRedirect('/blog/thanks/')
      else:
          captcha_error = """
Captcha was correct but you will need to reenter it
because other form fields weren't correct.
"""
  else:
      #captcha_error = captcha_response.error_code
      # TODO: map captcha error_codes to different messages...
      captcha_error = "Invalid captcha entry. Please try again."
else:
  form = ContactForm(auto_id=True)

return render_to_response('blog/contact.html',
                        {'form' : form,
                         'captcha_html' : captcha_html,
                         'captcha_error' : captcha_error })

contact.html:

{% extends "base.html" %}
{% block title %} Contact Me {% endblock %}
{% block content %}
...
<form method="post">
<table>
{{ form }}
</table>
{% if captcha_error %}
<p>{{ captcha_error }}</p>
{% endif %}
<p>{{ captcha_html }}</p>
<input type="submit" value="submit" />
</form>
{% endblock %}

Eureka 2nd season coming in July

Got nothing to watch now that Lost and Heroes are over.

Waiting for season two of my brother Andrew's TV series Eureka which now comes up as #2 on Google right after the vacuum cleaner company (no sucks jokes, pleeease).

Hoping they punch it up with a bit more edginess this round.

Crime does pay! - but in a good way

Adrian Holavaty, one of the creators of Django ¹ and the mind behind ChicagoCrime.org and many wonderful creations at Lawrence.com and the Washington Post, was just awarded a grant from the Knight News Challenge to build some really cool, participatory, civic-minded sites - something this country really needs. Adrian's new startup, EveryBlock, will be something to keep an eye on in the coming months.

I've been following Adrian's work ever since I heard a phenomenal podcast by his former boss Rob Curley about their work at Lawrence.com and World Online. It was like the Xerox PARC of online journalism. These guys really did some wonderful stuff and innovated every day.

As a freelance Web developer ² and as someone who majored in politics ³ and came fickled-finger-of-fate-close to going back to school for a masters in journalism, I've been toying with some ideas of my own about how to use the Web to give this country and the neighborhoods back to the citizens who should truly own them and bring participatory democracy back to this nation.

That's why another grant awarded by the Knight News Challenge to Northwestern University for nine scholarships to developers interested in journalism is so very tempting to me - despite the chill factor of Chicago ⁴. There's still that nagging twinge of "the road not taken" that pinches the back of my noggin' every so often. And with the many recent failures of our media to challenge the powers-that-be and hold them accountable to the truth, the siren call to take the news back from the sophists and profiteers is getting that much louder.

Delusions of grandeur? Maybe. But check in with Adrian in a year or two and you might be surprised. Oh, and the dude is only only 26.

1 - As someone who's currently working in Ruby on Rails, I may be labeled a heretic by saying this but I think Django is really excellent project with a lot to teach Rails. In fact, these two projects could learn a lot from each other if there was less Ruby vs. Python machismo in both camps (yeah shut up already, you know who you are).
2 - Yes, I've left GE Healthcare for those who haven't heard. Been so busy I haven't even gotten my own shingle up.
3 - Irvine (go Anteaters!) and Indiana (go Hoosiers!)
4 - I spent a couple of years of my youth living in the burbs outside Chicago and I have the frostbite scars to prove it.