I’ve been diggin in deep this week – at 4:30pm on Friday this was my desktop…

(click to view full res -500k)

I’ve got the ArcDeveloper REST project up, a testing site where I’m furiously mixing the Dojo Framework with MS Ajax, and a DotNetNuke module where I’m bringing it all together…

This is just the start, but I’ve got Virtual Earth in a Dojo Split Container, (there will be a Dojo Tree in the lovely light blue area) with a Dojo Toolbar (icons to be changed). Not shown but also working are some Dojo dialog forms. The orange blob is a feature brought into VE via the ArcDeveloper.net REST service. I still have to wire in some point clustering, and the tree view, but given it’s been a productive week.

Back in November I was asked to write an article on Agile Practices for “GIS Development” magazine. For those who are not familiar this publication, it’s based in India, and is quite similar to GeoWorld. I quickly recruited Chris Spagnuolo and we whipped up a 5 page article covering the high-points of agile development and some of the tools used to enable the process.

A week or so ago we got hard copies (always cool if you don’t publish a lot of stuff!), and I was just notified that the article is now online.

Article: Agile Project Management for GIS

I’ve been working with Virtual Earth and SQL 2008 – mainly pulling points out of the database, clustering them, and throwing them on the map. All well and good, but I was getting some weirdness when zoomed out to the first of second zoom levels.

Specifically I was getting this error:

The specified input does not represent a valid geography instance because it exceeds a single hemisphere. Each geography instance must fit inside a single hemisphere. A common reason for this error is that a polygon has the wrong ring orientation.

The issue was that the bounding box of the map view port was more than a “hemisphere” in size, and so I was going to have to split up these queries into SQL Server. Now that sounds simple enough, but took some experimenting to work out exactly how to do this. Steve Kass (I believe of the SQL Server team @ Microsoft) has a post on the hemisphere requirement that talks about ring-direction. While we’re on that sub-topic, ring-direction matters a lot here. The “left side of the line” is the “inside”, and if you get it wrong you will also get this same exception.

So – my observations – it seems that your geometry has to be less than a hemisphere. The “exceeds” in the error message is a red herring.

For example the “western hemisphere” as defined below will throw an exception

declare @WKT varchar(max);
set @WKT = 'POLYGON((90 -180, -90 -180, -90 0, 90 0, 90 -180))'
SELECT count([PointLocationID]) FROM [dbo].[DemoPoints]
WHERE [Location].STIntersects(geography::STPolyFromText(@WKT, 4326)) = 1

However, if we tuck these values in a little bit…

declare @WKT varchar(max);
set @WKT = 'POLYGON((89.9 -179.9, -89.9 -179.9, -89.9 0, 89.9 0, 89.9 -179.9))'
SELECT count([PointLocationID]) FROM [dbo].[DemoPoints]
WHERE [Location].STIntersects(geography::STPolyFromText(@WKT, 4326)) = 1

All is well again.

Further experiments seem to indicate that which way you cut the globe matters as well. Despite having 1000’s of points in the souther hemisphere, this query does not return any points

declare @WKT varchar(max);
set @WKT = 'POLYGON((0 -179, -89 -179, -89 179, -1 179 , 0 -179))'
SELECT count([PointLocationID]) FROM [dbo].[DemoPoints]
WHERE [Location].STIntersects(geography::STPolyFromText(@WKT, 4326)) = 1

If I then split the southern hemisphere into two parts – East and West, I get the points again…

declare @WKT varchar(max);
set @WKT ='POLYGON((0 0, -89 0, -89 179, -1 179 , 0 0))' --eastern half of southern hemisphere
SELECT count([PointLocationID]) FROM [dbo].[DemoPoints]
WHERE [Location].STIntersects(geography::STPolyFromText(@WKT, 4326)) = 1

and

declare @WKT varchar(max);
set @WKT ='POLYGON((0 -179, -89 -179, -89 0, -1 0 , 0 -179))'--western half of southern hemisphere
SELECT count([PointLocationID]) FROM [dbo].[DemoPoints]
WHERE [Location].STIntersects(geography::STPolyFromText(@WKT, 4326)) = 1

So – it would seem that the “hemisphere” they are referring to is really an EAST – WEST range. As long as your query geometry is less than 180 degrees wide, things are pretty good. Take this query, which is a patch that is ~180 wide and 90 high centered over 0,0.

declare @WKT varchar(max);
set @WKT = 'POLYGON((45 -89.9, -45 -89.9, -45 89.9, 45 89.9, 45 -89.9))'
SELECT count([PointLocationID]) FROM [dbo].[DemoPoints]
WHERE [Location].STIntersects(geography::STPolyFromText(@WKT, 4326)) = 1

This works just fine, so clearly we are not restricted by the actual Lat/Lon values just the range of values.

So as far as I can tell the rules for a valid geometry are:
1) Latitude (Y) Range must be less than or equal to 90 degrees

2) Longitude (X) Range must be less than 180 degrees

Hope this helps someone else avoid a few hours of head scratching and manual SQL querying.

I spent today doing some research on using SQL 2008 as a back end for a Virtual Earth application. Here’s a rough breakdown of the steps I took to get some stuff up and running.

Get All the Parts and Pieces

First, download the November CTP of SQL Server 2008

I originally installed the Express version, but that did not include the Management Studio. Install the full version by downloading the DVD Image. You can mount the image using MagicISO Mounter

During the installation, select everything to ensure that you get the Management Studio and SQL Server Integration Services (aka “SQL Server Business Intelligence Development Studio”).

Connecting from Visual Studio 2005

Download the Visual Studio 2005 patch allows it to connect to SQL 2008

This will allow you to connect to the database engine, and while the connection works, Visual Studio tries to parse the queries and complains on spatial data types. You get a message like this, and then the query proceeds.

Getting Started…

I started with John O’Brien’s post and code titled “Virtual Earth and SQL 2008 Spatial – a first impression”. Once I got this running, I started to experiment with my data.

Loading X,Y Data into SQL 2008 Geometry

My test data set has 250,000 records, with a Lat and Long stored as fields in the table. My first hurdle was to get the table from a SQL 2005 instance to my 2008 instance. Enter SQL Server Integration Services – Microsoft’s ETL platform. This is essentially an add-in that puts more functionality into Visual Studio.

I created a new Integration Services Project in Visual Studio…

and added an ADO.NET Data Flow Source and an ADO.NET Data Flow Destination to the design canvas.

When setting up the destination, I was prompted to create the table which was handy. It’s important to note that these tools do not know about the spatial data types so I could not add the geography column until the data was transferred.

Add Geometry Column and Load

I added the “Location” geography column did this right in SQL Management Studio 2008 – very simple. Then I used some SQL to convert the Latitude / Longitude columns into points in the Location column:

update  TestPoints
set location =
geography::STPointFromText(’Point(’ + CAST(Latitude as varchar(50)) + ‘ ‘ + cast(Longitude as varchar(50)) + ‘)’, 4326)
from TestPoints
where Latitude is not null and longitude is not null and Latitude <= 90 and latitude >=-90

The data I was working with had a few instances where the Lat and Long were reversed, hence the where clause.

Viewing in Virtual Earth

As I mentioned earlier, I used John O’Brien’s sample code to get things up and running quickly. I just had to change the connection string in his code, create some new stored procs that used my table instead of his, and change the call in his Web Service. And Voila…

Overall Impression

So this is pretty simple stuff really – show some points on the map. But most of my effort was in getting SQL 2008 installed with all the bells and whistles, and getting Visual Studio to talk to it. Beyond that things went really smoothly.

No doubt that there are many issues left – not the least of which is actually dealing with the 250,000 points. Luckily John has a post on implementing point clustering!

I will also be looking at the Vector Tiling stuff as I’ve got some polygons that need to be put into the map as well – but that will be another post!

When I got in this morning I had an email from ESRI asking me to take the 2008 ESRI Developer Survey. I asked the rest of the team and they had not gotten it (yet?) so I thought I’d shoot the link out to everyone. The survey is a bunch of questions related to ESRI developer resources – basically what are you using, how often to you use it, what languages are involved. I’m not sure who all is “officially” invited to take this, but it seems that anyone can access the survey. So – if you want to be heard, this is a chance to let ESRI know what you think and what you want. Get over there and take the survey!

I’ve been doing some work with SharpMap v0.9  (from CodePlex) for a windows forms based mapping tool, and while there are quite a few samples which show how to do basic mapping stuff, it was very difficult to find out how to locate features using an attribute query.

While SharpMap has a very obvious method for doing spatial queries –

IProvider.ExecuteIntersectionQuery(Geometry, targetDataSet)

There is no similar method for attribute queries.

After scouring the SharpMap discussions over at CodePlex, I initially resorted to a bit of a hack – I’d get the FeatureDataTable from the SharpMap.Data.FeatureDataSet, cast it to a System.Data.DataTable, and create a DataView with a filter. This worked for showing the attributes in a grid, but I had no way to get the Features because they are not stored in the DataTable.

I decided to put this on hold for a while and work on some other aspects of the application, and while debugging I was stepping through the ExecuteIntersectionQuery code in the ShapeFile provider and saw this:

What was this FilterDelegate? A little searching in the source and found a long comment section by Morten that describes how to use delegates to filter the results returned in a DataSource. You can view the highlighted source here (by the way Koders.com rocks for their indexing of open source code!)

Since this was buried, I thought I’d show a little code on how to do it. But first – if you are not familiar with Delegates, they are essentially pointers to functions.  Here are some resources: Wikipedia description, how they relate to events, and MSDN article on them.

Here’s how the filter works:

As the ExecuteIntersectionQuery is looping over the dataset, it will pass the current row into the delegate and add the row to the results if the delegate returned true. Once you dig into the source code, it’s actually pretty clear what’s going on, but it would have been nice for this to be exposed in a more obvious manner.

Implementing A Filter 

The straight-forward way to do this is create a static method that do the evaluation and assign that method to the SharpMap.Data.Providers.ShapeFile.FilterMethod . The code snippet below shows the IsOwnerNameSet filter being used. 

This works pretty well if your filter criteria are static – like “is the ownername set?” But what if your criteria are changing? Since you can’t pass additional arguments to the delegate, the other option is to create static member variables that the delegate will use to determine that match. This has a bit of a code-smell to me, so I ended up using Anonymous methods. In the code snippet below I am creating the anonymous method, and at the same time defining the value it is checking against. In this case I’m searching for a parcel by owner.

This allows you to specify the value (owner) without resorting to static members.

Hope this helps people just jumping into SharpMap

While it’s main purpose is to provide an Inversion of Control container, and allow you to develop applications that are highly cohesive yet loosely coupled, a secondary benefit of using the Castle Windsor container is how it simplifies dealing with configuration files.

Typically if you need to have some external configuration, you’d put this into your own custom configuration section and write a class which either implemented IConfigurationSectionHandler or inherited from ConfigurationSection. If you needed to change the configuration, you’d need to change these handler classes. While this is unspeakably better than parsing .ini files, it’s still pretty brittle – especially if you are being agile and refactoring your code to any great extent.

What is Castle Windsor?

It’s a container that figures out how to create classes, along with all their dependencies, on the fly, from a configuration file. Somewhat like Configuration Section Handlers, but it’s an awful lot more. For example…

<component id=”VirtualEarthTileServiceManager”
                 type=”ArcDeveloper.TileServer.Core.VirtualEarthTileServiceManager, ArcDeveloper.TileServer.Core”>
        <parameters>
           <services>
            <dictionary>
              <item key=”colorado”>${ColoradoVETileService}</item>
              <item key=”tahoe”>${TahoeVETileService}</item>
            </dictionary>
          </services>
        </parameters>
      </component>

This little snippet of XML defines how the VirtualEarthTileServiceManager should be created. Basically the type is instantiated, and an IDictionary containing the keys and items it depends upon is passed into the constructor. The ${somename} notation refers to other classes in the same configuration – and they use similar syntax to have their dependencies injected at instantiation.

To actually get a VirtualEarthTileServiceManager, you just use this code…

IWindsorContainer container = new WindsorContainer(new XmlInterpreter());           
VirtualEarthTileServiceManager veTileManager = container.Resolve<VirtualEarthTileServiceManager>();

The container.Resolve call looks up the name (VirtualEarthTileServiceManager in this case) and creates the class, with all of it’s dependencies.

Learning More

This post has just scratched the surface, so I recommend checking out some more information on Castle Windsor. There are quite a few articles on how Windsor works, but the best I found are a series of 4 written by Simone Busoli, and you can read them here: Part 1, Part 2, Part 3, and Part 4.

Since I’m also a fan of Scott Hanselman, give a listen to this podcast on Mock Objects, and this one on MonoRail and the Windsor container.

For more on the Virtual Earth Tile Service, see this previous post, or check out the ArcDeveloper project at Assembla.

If you are a .NET developer I recommend checking out DocProject. It’s an open source front end for the SandCastle documentation engine. Sandcastle picks up where NDoc left off, and is a Microsoft open source project that provides services for creating MSDN style HTML and compiled help. Unfortunately it’s a bit of a bear to work with (lots of Xml configuration files, batch files etc).

This is where DocProject comes in. The basic process is to add a “DocProject” or “DocSite” to your solution, and then add references to all the other sites/assemblies in the solution into the DocProject. Then during a build, DocProject actually runs Sandcastle to create the doc.

I’ve only just played with it a little, and it’s pretty easy to get setup and running, but the actual compilation of the help was pretty slow. It was running for >15  minutes on my Core 2 Duo notebook! After digging around a little, it seemed the issue was that I have some ArcGIS Server WebService proxies in one of the assemblies, and it was cranking out all the help doc for that – which is substantially larger than the actual code I was trying to document. Once done, the output is pretty good.

Combine this with GhostDoc, and there is no excuse for not having good developer API documentation for whatever you write.

Check it out at www.codeplex.com/docproject

Ok, it’s a little grainy, but if you look closely you can see the shiny new ArcGIS Server REST API behind all that fur.

Kudos to ESRI for actually showing some beta software. Typically this sort of stuff is locked down until the final release. Does this mean that people in the beta program will be able to stage up applications for the public to view? Will they be able to talk / blog about it? And most of all – does this mean that we’ll see a release before the User Conference?

In my moving of the ArcDeveloper TileServer code between Subversion instances, I somehow managed to drop ITileProvider. Thanks to akhor who posted this issue over in the comments @ SpatiallyAdjusted.com.

The ArcScripts version has been updated (seemed that I had to delete the original one), and the file is now in the ArcDeveloper Subversion repository as well.