During the talks we invited the ~100 people in the room to beat up the apps, and here’s what we found.

First – both services withstood the inbound load without issue.

Node.js and Socket.io

Demo site: http://node.dtsagile.com

This was running on an AWS Micro-Instance, and there were no issues handling the load. Over the 4 hours from when we opened it up, 999 points were added to the system.

In the first 10 minutes, 780 points were added, and looking at the timestamps we had several points where there were 10, 12 15 points added per-second. Not bad for a wee little instance! Here’s a screen cap from Brian Noyle’s phone after the session was over. Down to 3 users, but you can see all the points stacked up.

What we did find is that once there were a lot of points, the system hit a snag querying them back out and sending them to the clients. Although we could have had the clients query the ArcGIS Server FeatureService directly, part of the experiment was to do that work in Node, and then push the features to the client. This all worked smoothly except the FeatureService got slower as the number of points went up (we are still looking at why this is), and the timeout in the Node xhr request was set to be 1 second. Thus, the request timed out, and newly connecting clients did not get the “current” crop of points. Newly added points worked just fine.

At this point things are working smoothly again – and that’s because the query only pulls points added in the last 24 hours, and there are very few points added on any given day. I have not had any time to replicate the issue, but I expect that extending the timeout would fix things. A better idea would be to cache the data in something like Redis, and then lazy-write it into a geodatabase or feature service – maybe there would be a second or so lag between the two, but I think the user experience would be better.

code: https://github.com/dtsagile/dmg-express

ASP.NET MVC and SignalR

Demo site: http://signalr.dtsagile.com

This was running on a server in our Fort Collins office, and although overloaded, it’s a pretty powerful box, so there were no issues with the load. Over the first 4 hours, 536 points were added to the system. In the first 10 minutes, 426 were entered, with a peak of 5 per second. We can’t compare this directly to the numbers for node as there is now way to tell if more or less people were entering data. However, we can safely say that the server easily handled this load.

This version of the app simply stored the points in a SQL table with x,y values, and interestingly there was no issue with scaling this up – mainly because there was no xhr in the mix to time out, and because direct sql access is much faster than going through a feature service, which then hits the database.

code: https://github.com/dtsagile.com/dmg-signalr

Videos of the Talks

We recorded screencasts of the talks as we gave them, and they are now up at Vimeo.

Node & Socket.IO:  http://vimeo.com/dtsgis/realtime-node

SignalR & ASP.NET MVC: http://vimeo.com/dtsgis/realtime-signalr

After years of being told we should use plug-in technologies such as Flex or Silverlight for our web applications, the seemingly sudden switch to HTML5 has left developers and organizations with many questions…

• What is HTML5?
• How does Javascript fit into this?
• When should we stick with plug-ins?
• When should we use HTML5?
• What is CSS3 and how can we use that to create amazing interactions?
• How do we serve different versions of an app to different clients?

On April 11th, 2012, Brian Noyle and I are doing a URISA webinar where we will cover how DTSAgile builds geospatial HTML5 apps that work on desktops, tablets and smart phones. We’ll be covering a lot of ground in this fast-paced talk that is aimed at both developers and managers.

Registration: http://www.urisa.org/URISAConnectGISinMobileEra

Talking Tech: Presentation Zen for Everyone

1pm Wednesday, Mojave Learning Center (Dave Bouwman and Brian Noyle)

Let’s face it, most technical presentations leave a bit to be desired. Myself and Brian Noyle will talk about our journey from power-point hell to something better. We’ve got ~100 slides and 30 minutes, so join us for a fun session that will make your next presentation rock.

Have a Little BackboneJS

4:30pm, Wednesday, Mojave Learning Center (Brian Noyle and Mike Juniper)

This fast paced talk will introduce the GIS community to the Backbone.js framework in the context of existing ESRI web technologies.  We’ll start with some MVC and HTML 5 love and explain the Backbone.js code structure and implementation pattern with a focus on maintainability and unit testing.  We’ll take a few ESRI examples from the JavaScript API and show developers how to implement them using Backbone.js. We’ll also look at a large and complex development initiative and illustrate how Backbone.js has made our lives easier.

Getting Real-Time Part -1: Node.js and Socket.io

10:15am Thursday, Mesquite B (Dave Bouwman)

Some scenarios (i.e. zombie flash mobs) require the ability to keep all users view of the data in sync, in real-time, without requiring an update or page refresh. In this talk we will look at a simple demo application the DTSAgile team created to investigate real-time technologies. This Damage Assessment demo app uses real-time communication to allow users (desktop or mobile) to share real-time “state” as different types of assessments are completed. The idea being that when an “injury” is reported, appropriate medical teams can be dispatched to the location.

In this part, we will show a simple node.js app that uses socket.io to send notifications between clients and the server, as well as sending the point locations to an ArcGIS Server feature service. Bring your notebook/iPhone/iPad/Android devices and let’s see how many sessions we can run on an AWS micro instance before we crash the wireless network, or overload the server!

Code: http://github.com/dtsagile/dmg-express

Getting Real-Time Part -2: ASP.NET MVC and SignalR

11:00am Thursday, Mesquite B (Mike Juniper)

In this session, Mike Juniper will show how to implement the same Damage Assessment demo app, but this time, working on the Microsoft platform, using ASP.NET MVC and the fast evolving SignalR library.  Finish out the Dev Summit with a bang and go back home armed with ideas on how to easily add real-time capabilities to your existing apps.

Code: https://github.com/dtsagile/dmg-signalr (coming soon!)

Before I get into this – I’m not trying to point fingers and call anyone out on this, I’m simply trying to show a technique that can really help make your maps fast.

Ok – the site is “Climate Change and African Political Stability” and it’s at http://ccaps.aiddata.org/ . I got to it from a blog post titled “Crisis Mapping Climate Change, Conflict and Aid in Africa”, and when I first saw the screen cap in the blog post, I thought it looked like something by DevelopmentSeed or Vizzuality. So, yeah, the site looks good. Here’s the screen cap on first load…

Cool! So it’s using the Esri Javasctipt API, and some fancy client-side clustering. What I could not get in the screen cap is that the little clusters have fly-out “octopus” arms depicting how many items are in the cluster. Very cool.

The intent of the app is to have the users manipulate the slider at the bottom to look at changes over time. So, when the app first loads, we are looking at data for 2010.

Whenever I see clustered data like this, I’m curious to see what’s on the wire… so a quick peek into Firebug tells another part of the development story…

The line I high-lighted is a little hard to see, but the key things are POST, 343k and 3.6 seconds. First off the POST is kinda hokey, to say nothing of turing the stomach of every RESTafarian around. Why? Because POST is meant to CREATE or UPDATE data… queries should be done with GET my friend. And, no one else can POST to that url, to get the data. Maybe that’s by design – I’m not sure. The next two numbers are really performance indicators, and really not too bad, but let’s look at what’s actually coming down.

The actual object does not see to have a much extraneous info, which is good, and 343k is not far off the size of a map tile, so it’s all good right?

So, here’s where I’m gonna start making suggestions… using long property names is wasteful. Being a nerd, I grabbed the response data, stuffed it into an editor, and started hacking. Changing the properties to be much more concise (lat,lng,did,et,ed,a1,a2,c,l corresponding to the items in the order shown above) drops almost 1/3 of the file size – down to 260K.

But really, there is another way to shrink that down that’s much much more effective. You see, there are 1418 data points coming down in that package. To show then effectively on the map, they have to be clustered, or the map would be a total schear of points, and be impossible to make sense of. Looking at the map, we are seeing *maybe* 30 points total? So that means that we have an extra 1388 data points on the wire. We could make exactly the same map, with roughly 0.03% of the data on the wire. And you know what that means? The app will be monstrously faster.

“But Dave, a few seconds when the page loads – who cares?” Ah, yes, but let’s get back to the point of the app – to look at this data over time. And not just at different points in time, but aggregated over time. What say we slide that date back to 2003 shall we? Go ahead, try this… I’ll wait…

Impatient? Ok, here’s what you’ll see if it finishes loading…

Kinda similar eh? I moved the map a bit for this screen cap, but really we see pretty much the same thing – approximately 30 points. Except for that heinous lag while it loaded, the map is pretty much the same. And what does Firebug tell us? Yeah – that huge lag was what happens when you jam 5MB of JSON down the wire. That little hack using short property names to save 30% doesn’t look so stupid now does it

And the load time on that? 1 minute, 18 seconds. Firebug did not even JSONify the payload, but swaging is based on file-size, this is somewhere around  25,000 points. And in the end the map shows 30 points? Right.

A Better Way…

Server-side clustering is the solution here. Based on the Url’s we are seeing in FireBug, I’m guessing that this data is stored in a simple tabular database, which makes things even easier. But first a demo…

Conveniently, we launched a site last week which shows server-side clustering in action. Check it out at http://maps.saferoutesinfo.org. Basically it’s a map of the projects which have been funded by Safe Routes to School – if there are no projects in your area, contact your school board and congress person.  Anyhow, the data covers 6 years, and has roughly of 13,000 data points.

What’s great about this clustering solution is that the payload on the wire is small, and stays small. We load the map with the “worst case scenario” – all types of awards, for all years, for the entire country. Payload is close to the other app, at 211Kb, 1.4 seconds and we are showing 450 points. If you go poking into the JSON, you’ll see that we were under a time-crunch to get this live, so it’s not optimized – we could likely save another 20% by dropping extra / unused attributes. Speaking of Url, this is a simple REST service so you can run this same query using this link.

The idea here is to store the data in a normal SQL table, send in the bounding box and use that in a simple SQL query to get all the points currently in the map. We then cluster the points, JSON that up and shoot it back. The only downside here is that we need to re-fetch the points every time we pan or zoom. But given the fact that the points are often rendered before the map tiles come down, this does not seem to be a performance issue.

The clustering is based on a 22 pixel square at the current zoom resolution, so we never get back any more than 1 point for every 20×20 pixels of map space.

What’s more, we can do some interesting stuff – in this app, we have 3 types of funding “awards” – School Awards, District Awards and State-wide Awards. If a cluster contains only schools, we use the green cluster icon. If it has 1 or more District Awards, it gets the blue cluster icon, and if there is a State-wide award in there, we show the gold icon. I’m mentioning this because the gist I posted has this logic in there – if you just want to show points, you’ll need to rip out that extra logic.

Since  I’m running on the el-cheapo WordPress.com hosting, I can’t embed the gist of the code, but you can get it here: https://gist.github.com/2188210

Summary

I hope this shows why server side clustering is a killer way to build really fast web apps. In the end, you can have great design, great data, a great story, but if the app is slow, all that work is lost.

Also – if you are the developer/project manager/benevolent overlord/in-any-way-associated with the Climate Change and African Political Stability App, I encourage you to get in touch – although it looks like you are not using ASP.NET, I am more than willing to help you work through converting the C# over to whatever you are using. The site tells an important story, and it deserves to be told well.

Given that I’d likely have an aneurysm if I paid any attention to the electoral insanity of the next 11 months, I’m opting to spend my evenings and weekends getting real with Rails. And I’ll be  blogging about my adventures along the way.

What about .NET and Esri? Are you going all FOSS?

In a word no. DTS is and will remain a .NET shop, and we’ll still be cooking ASP.NET MVC apps backed with  Esri technology, and I’ll still be blogging and speaking about that stack (some cool stuff is coming at 10.1 but can’t talk about it yet!). My interest in Rails is mainly to get another perspective on web development, and to see how well it works. I’ve got a few non-spatial ideas kicking around, and I want to use them as a platform for learning Rails. There may come a time when we start doing project work on Rails, but that will be a ways out.

That said, if I could avoid the hell that is the Microsoft Volume Licensing site, I’d be more than a little happy

“Getting Started With Rails” Resources

There are a ton of resources out there to get you going – here are some that I found useful:

Glenn Goodrich (@ruprictgeek) has an on-going series at RubySource.com – I literally got my Mac setup with Rails using this post on RVM. His series continues and is integrating Backbone.js and some geospatial flair using leaflet, but I had to take a side journey into Ruby 101 and Rails.

The day after I got RVM setup, Rob Conery tweeted about $100 off an annual subscription to Tekpub.com. I love love love the This Developers Life podcast which Rob edits, and I knew that Tekpub had a Rails 3 series ($28 if you just want that series), so it was an easy sale. Over the break I managed to watch 9 of the 11 videos in that series, and I have to say they are really good in terms of giving you a mix of theory and practice. Rob shows how to use scaffolders, and then how to do it “better”. There is quite a bit on testing in here, using the built in test tools as well as rspec. Other goodies covered include generators, haml, factory_girl, cheat, devise and autotest. Rob also mixes it up doing some episodes using VIM, others in Textmate, and others in RubyMine. Some on Linux and some on Mac. Quality work sir!

I really enjoyed a NetTuts video called “The Intro to Rails Screencast I Wish I Had” and is presented by Jeffery Way. Once you have a general understanding of Rails 3, this is a great 45 minute overview of how it all goes together, doing TDD and not using any scaffolding. It also goes into Capybara, which is wicked slick if you’ve ever tried to do any front-end testing. I’m definitely going to check out more NetTuts Rails stuff.

I also got the Kindle version of “The Ruby Programming Language” book for the whopping sum of $5.98. I’ve been reading this in chunks mainly as a means to get my mind into reading ruby code, and thus “knowing” how to express my intent in the language.

While I strive to be “all digital”, having cheat-sheets lying around is always helpful, so I’m compiling a list of useful ones that I’ll post sometime soon.

So that’s it for now. I’ve got a lot of ideas on things I want to built, and at this point I merely have an idea of how much I don’t know (linux, nginx, postgres), but I think it’s gonna be a fun journey!

Here’s my EF Model. Lots of lovely navigation properties allowing me to get from any entity to any other one. Handy-dandy in .NET land…

But no-bueno when serializing using the out of the box Json serializer…

[HttpGet]
public JsonResult Get(int id)
{
    return Json(itemcategoryRepository.Find(id), 
        JsonRequestBehavior.AllowGet);
}

Now, this sucks because how awesomely elegant is that code? What you get is an error like this:

A circular reference was detected while serializing an object of type ‘System.Data.Entity.DynamicProxies.ItemCategory_A79…’.

Json Serialization Options

There are a few ways to go here – you could use ViewModels everywhere and sloppy-copy properties across as needed. Even with Automapper to take the drugery out of this, it rubs me the wrong way – what’s the point of using EF if I still have to have a separate set of Poco ViewModels?

Another option is to add a .ToJson method to the EF classes, and emit strings. Lots of control, but… meh.

Using Json.net comes up a lot, but it’s also labor intensive.

What I ended up doing is using Linq to project the EF object graph into an anonymous object graph. Here’s an example that serializes two levels of objects (a list of ItemCategories, each of which can contain a list of Items – see the object model above)

[HttpGet]
public JsonResult List()
{           
    var data = itemcategoryRepository
        .AllIncluding(itemcategory => itemcategory.Items);
    //Project into anonymous objects because Serializers
    //can't handle circular refs in the EF magic
    var collection = data.Select(x => new
    {
        id = x.Id,
        name = x.Name,
        items = x.Items.Select(item => new
        {
            id=item.Id,
            name = item.Name
        })
    });
    return Json(collection, 
        JsonRequestBehavior.AllowGet);
}

I like this solution because I have control of the Json in the controller, rather than forcing an object to always serialize the same way in all cases (sometimes I want the Item to serialize it’s Entries, but only when I’m getting a single Item).

The situation:

I have ASP.NET MVC3 project, and I created an Area for my json API.  In my area I create a simple controller called CategoriesController, and I registered some routes in my AreaRegistration…

//handles get for a list of all categories
context.MapRoute(
    "ListCategories",
    "API/Categories",
    new { controller = "Categories", action = "List" }
);

I used the very awesome MvcRouteUnitTester (on Nuget) to write unit tests to make sure this route was working. All systems go!

Before I started writing tests against the actual controller, I used a browser to hit http://localhost/myapp/API/Categories – which should match this route, and spit back some “hello world” json. I get a 404.

Even with the unit tests, I decided to pull in the RouteDebugger NuGet package, just to make sure that the routes were working, and that showed that all was good

So, I’m stumped. No Exceptions, everything seems to line up, but still a 404.

After about 15 minutes of useless Googling, I remember that I’ve got logging setup in this project, and that’s in my Global.asax.cs and tied into Application_Error…

protected void Application_Error()
{
    Exception lastException = Server.GetLastError();
    var logger = DependencyResolver.Current.GetService<ILogger>();
    logger.Fatal(lastException);
}

SO I look in the log… and find this error

The controller for path ‘/statsme/API/Categories/List’ was not found or does not implement IController.

(full trace because this will help Google index this when others go lookin)…

Message :The controller for path '/statsme/API/Categories/List' was not found or does not implement IController.
Source :System.Web.Mvc
Stack Trace :   at System.Web.Mvc.DefaultControllerFactory.GetControllerInstance(RequestContext requestContext, Type controllerType)
   at System.Web.Mvc.DefaultControllerFactory.CreateController(RequestContext requestContext, String controllerName)
   at System.Web.Mvc.MvcHandler.ProcessRequestInit(HttpContextBase httpContext, IController& controller, IControllerFactory& factory)
   at System.Web.Mvc.MvcHandler.<>c__DisplayClass6.<BeginProcessRequest>b__2()
   at System.Web.Mvc.SecurityUtil.<>c__DisplayClassb`1.<ProcessInApplicationTrust>b__a()
   at System.Web.Mvc.SecurityUtil.ProcessInApplicationTrust[TResult](Func`1 func)
   at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
TargetSite :System.Web.Mvc.IController GetControllerInstance(System.Web.Routing.RequestContext, System.Type)

Ok, WTF? CategoriesController inherits from a BaseController which inherits from Controller, so we’re clear on the IController bit. I know the class is there… what is going on?

Google… Google… more Google. Random hacking… cursing… Ah HA!

The issue was the namespace on my controller. Turns out that MVC has a convention that the area name is expected to be in the namespace of the controller.

So – I had StatsMe.API.CategoriesController, but MVC was looking for StatsMe.Areas.API.CategoriesController.

What finally tipped me off was some stuff in the RouteDebugger output…

Annoying because the error message really could have helped out by saying “Hey – I’m looking for StatsMe.Areas.API.CategoriesController and I can’t find it”, rather than the very generic message it spit out.

Configuration Overriding Convention

As we all know, in MVC where there is a convention, there is a way to override this, and that’s in the route definition… like so…

context.MapRoute(
    "ListCategories",
    "API/Categories",
    new { controller = "Categories", action = "List" },
    new[] { "StatsMe.API.Controllers" } 
);

Viola! Hope this helps someone else avoid a few hours of frustration.

As noted in the previous post, things got started back in September with the FOSS4G conference. That post sums things up pretty well, so I’ll just move the story forward from there… up up and away!

Orlando

The following week I was in Orlando doing some work with a client. While DTS is headquartered in Orlando, our offices are in Avalon Park, which is well outside the city itself. But, this client is downtown, and this gave me a chance to check out another area of the city. I do enough business travel that staying in yet another cookie cutter hotel has long lost it’s appeal, so when I can, I step out and stay at non-chain places – like the OE Inn. Great little place, right on Lake Eola Park (photo below), half a block to the client’s office, a Panera Bread on the main floor, and a great sushi place around the block. Since this was right after FOSS4G, and I had my evenings free, node hacking was in order (more on this in the next post!).

Next Stop… Kentucky

From time to time we have internal meetings – usually in Orlando, and usually out at Avalon Park. This was getting kinda boring, so Dave Buckley (our VP of GIS and Wildfire dude) suggested we do something different… like tie the meetings in with some caving in Kentucky. And so it was that 8 of the DTS core team met up at Mammoth Cave National Park for a mix of meetings, burbon and caving.

This was the easy walking-on-the-path Crystal Cave. While the iPhone 4 can take good photos, low-light conditions are not it’s forte.

This is from the “Wild Cave Tour” which had a few “tight spots” and a fair amount of crawling around. A great way to spend 4 hours, and get photos of the DTS core team looking like convicts…

Colorado River Trip

From Kentucky it was back home to Fort Collins, and then off to Fruita to float the Ruby Horse Thief section of the Colorado river. For the last 3 years we’ve been doing this family river trip with a group of friends. This year the dates got moved around a bunch due to my travel schedule so it was a smaller group, but still a ton of fun for all involved.

I’m not much of a “water” person, and this is perfect – no real whitewater, but still a lot of fun camping and being out in the desert.

Off to Europe

I was at home for a few weeks and then headed out to Europe for a 10 day whirlwind tour of sorts. The basic plan was to go do a few presentations at the Esri European User Conference, which was held in Madrid. Our Spanish partner in the Wildfire business, Joaquin Ramirez / Technosylva, has come over to the US for the last two Esri User Conferences in San Diego, so it only seemed fair that we come over there since it was in Spain. And since we were going over to Europe, why not tack on some meetings in Zurich and Geneva?

Zurich

We got micro rooms at non-micro rates right near the lake and downtown. I managed to get out and walk about (with a side-trip to the Apple store to get a USB-Cat5 dongle/adapter). Stunning place.

Off to Madrid

When dealing with airlines, nothing is straight forward, so while it would make sense to fly from Geneva (we drove down from Zurich) directly to Madrid, that particular flight was gonna cost most than the rest of the trip combined. So, we hop from Geneva, back to Frankfurt and then down to Madrid… much farther, but through the miracle of airline math, less than 10% of the price of the direct flight. Go figure. Of note – Spanair really crams the seats onto their planes. If you are 6’ or taller, you’re gonna have a leg-room issue. I think there were 8 additional rows of seats on this plane as compared to the Lufthansa flight… in the exact same model of aircraft. Ok, back to it.

Ah Jack. Whenever possible, I always make his plenary sessions. Sure I’ve heard the content before, but as someone who does a lot of speaking, there is a lot I can learn from Jack’s style and how he presents his vision, and structures the talk in general. Some people call this kool-aid, I call it a great speaker. Moving on… the conference was fun, and our talks went really well – packed rooms for all of them (special thanks to Ismael Chivite who I believe corralled a mess of people into my talk about deploying apps to the cloud!). But the real fun is after hours! We went to the Mercado San Miguel and liked it so much we want back the next night. This place is how WholeFoods would do a tapas / wine bar / open space. Awesome fun.

And apparently you can’t eat in Spain without consuming Jamon. Nom Nom Nom!

Seville

So the conference wrapped up and we jumped on a train with Joaquin, and headed to Seville for a few days of “strategic planning” for DTSWildfire. And by strategic we mean…

and…

and…

Somewhere along the line we did talk about wildfire products and markets and marketing and general strategy, but in a very civilized manner.

Back Home

I have a 5 year old, which means that Halloween is big deal. Thanks to the magic of jet travel I was able to get from Madrid to Fort Collins (via Frankfurt… of course) in a single (very long) day, arriving just in time to go trick or treating. In keeping with the general frenzy of the fall, the next day my wife went to New York to visit her sister and our new nephew, so I was a solo parent for about a week. This was actually really good, because it was perfectly acceptable for me to go to sleep at 7:30 when my son goes to bed. And it’s always nice to get outside and have a picnic in Lory State Park…

That was supposed to be it for travel. Note the “supposed” part. But with our acquisition of VueWorks earlier this year, we also acquired their annual user conference. And since I’m helping define some major changes in the product, I had to pop over to Orlando for a few days of strategic meetings (see how that comes up so often?) and then do a talk on the vision for the product line.

(Note: the vision does not involve “pivoting to monetize crowd sourced photos of giant dump trucks” – I’ll leave that up to you.)

In contrast to my previous trip to Orland, this time I was staying at the Disney Coronado Springs – the very epitome of corporate lodging. Don’t get me wrong, it was all fine, but just a little to homogenized for my liking.

And that about winds up the last few months. Thanks for slogging through this – hope it was a fun diversion from the usual tech-centric posts. With all that travel out of the way, I’ll be working on some posts about real-time collaboration, discussing two research apps we built – one using node and socket.io and the other using ASP.NET MVC and SignalR.

One thing that struck me, and apparently also Stu Lester - the maturity of the FOSS GIS software was really on display. That said, I’m not about to write off a certain large vendor by any means – but make no mistake – FOSS4G software is certainly usable beyond the “gcc” crowd. I will note that the only Windows machine I saw was someone who’d just grabbed the Win8 bits, so the crowd was certainly it’s roots.

Some particularly cool things that stood out for me…

PostGIS Cloud Services are clearly hot. SpacialDB and CartoDB are leading the charge here. Both will be offered as services, and both have their own take on how this should work. Javier de le Torre (@jatorre) and Simon Tokumine (@tokumin) of Vizzuality were pretty direct in saying that CartoDB was modeled on Google Fusion Tables, and the demo shown was really slick. Here’s a video of CartoDB accessed from an iPad.

On the SpacialDB side of things, Kashif Rasul (@krasul) and Shoaib Burq (@sabman) did a tutorial/talk on Async apps with NodeJs and focused more on how to use NodeJS for real-time collaboration, and just glazed over SpacialDB. One thing that stood out to me was that when you created a “table” you got back 4 keys for the 4 HTTP verbs. Kinda slick, and I’ve since used their example code to hack up some other stuff (read: blog later). Notably they were also the only people talking about LeafletJS. Not sure what the deal is with that, as Leaflet is pretty frikin’ sweet, but whatever.

Back to PostGIS – the upcoming version 2.0 will seriously raise the bar with the addition of has more built-in features – from “clean” operations, to 3D support to raster algebra / focal operations (as in Spatial Analyst ‘fu in the database). Quite simply this is gonna be a huge release – I think I heard that the beta is slated for end of the year. More info here.

Vizzuality also released CartoSet (on github), a Rails framework for simple “Home/Explore/Details” type apps. As someone who cares about the design of web apps, I think this is a great idea. Standardize what can be standardized, but leave the UI completely open ended – a sure way to avoid cookie cutter sites. Vizzuality++

NodeJs - the new new shiny thing, javascript on the server, async, non-blocking goodies that few people are sure about how to leverage effectively (case in point: all demos are “chat”). Sure you can use Express and create server side MVC apps, but since that type of app is well serviced by a host of other frameworks, I’m not seeing that as the real use-case. Realtime steaming seems to be the best fit at this time, and it certainly is fun to play with. If you are on Windows and want to play with Node, check out this post on using Node with IIS and WebMatrix.

A nice thing about a smaller geo conference is that you are much more likely to actually talk with vendors. So, talked with the folks at SkyGone, and got the skinny – although a “cloud” provider, they are NOT like EC2. The difference starts with a “system builder” tool which allows you to layout an entire multi-server system, using pre-configured machine images (I’m sure they have some other name). What really struck me was that a running instance can be scaled by adding resources on the fly (think scaling from an EC2 micro-instance to an x-large instance on the fly without launching new instances). This is especially attractive for emergency response applications which need to run with minimal resources until an event occurs and then needs to scale up quickly.

Design Matters…  attendees packed the room to hear Sergio Alvarez (@saleiva) of Vizzuality talk about design. Since virtually every Vizzuality project I’ve seen is a work of art, you know the room was packed. Sergio walked through a series of design principles, giving examples of most from their work or via interesting videos. I’ll credit Apple and the app store with making the value of design obvious to a much wider audience. Simply put – ugly apps will fail.

Ok, I did not want this to be atotal Vizzuality love-fest, but when you show up and every presentation your team gives is top notch, you deserve credit. So, in that vein, check out this javascript web preso on HTML5 hosted at github by Javier Alvarez (@xavijam) and while you’re in the mood for cool HTML5 stuff, check out his profile page. Question: Where do I find people like this?

Overall, there was a ton of great content, but realistically there is only so much you can absorb in a few short days (ok, the late nights in LoDo may have had some impact!). Beyond reconnecting with old friends and meeting tweeple for real –  for me, the long term value from an event like this is getting the lay of the land, and knowing what tools and technologies are at a point where I feel we can use them in solutions. From here, the hard work of getting up to speed on those technologies starts – watch for blog posts!

I expect FOSS4G to be similar – lots of great, creative ideas, lots of code and “how-to-make-this-rock-now” type sessions. And beer.

So, why am I going to a “dirty-hippy-give-it-all-away-GPL-forevah-yo” event? After all, building on the Esri platform is (and will continue to be) my bread and butter. The simple fact is that spatial has not been “special” for a long time now. Integration and scalability are key factors when designing systems and products – and this applies across all software markets. Esri has a great software stack, and even they are embracing the use of open source technologies. Python is an integral part of the ArcGIS ecosystem, and postgresql is the database in their cloud solution. At 10.1 there is PostGIS native access via Spatial Data Service. And then there is the whole GeoServices REST Specification (pdf).  So, from my place in the world, I want to get the skinny on this stuff, and FOSS4G is the place to be!

Sure, some of it’s pretty bleeding edge (I’m not likely to propose nodejs + couchdb to a state agency!) but that’s not the point. As a developer, in order to keep my edge, I need to continually challenge myself. I do this by learning new languages, tools and techniques. So maybe I don’t *implement* solutions on nodejs for my clients, but upping my javascript skills by using backbonejs on the server will certainly help with other solutions.

And who wouldn’t want to hear Schuyler Erle pontificate on ”Pivoting to Monetize Mobile Hyperlocal Social Gamification by Going Viral”

Hope to see you there!

Of note: the Thursday night party is $99 on top of the $649 registration fee, but people tell me it’s gonna be totally awesome and @mattpriour will be there and that alone is worth the price of admission