We’re looking for a motivated person to join our cutting edge geospatial development team. Located in Fort Collins, this tight knit team is focused on building cutting edge GIS application using both ESRI and OGC technologies. This position is focused on web development using ASP.NET and AJAX, to build dynamic geospatial portals. We have a number of on-going large scale projects which require these skills, and we’d love to have you join our team.

Must Have Skills:

• ASP.NET (C# or VB.NET)
• Cross-browser Javascript +Ajax essential

Other skills:

• Experience with Ajax.NET
• Background/Experience/Interest in GIS, and/or spatial technologies
• Good design skills (Photoshop + CSS)
• Experience with DotNetNuke
• Experience with ArcIMS / OGC WMS
• Database skills (MS SQL, Stored procs etc)

The next step in the ongoing upgrade saga was to load an existing feature class into a subtype of one of the new featureclasses. Of course, the new schema has fields which are not in the original featureclass, or which are now a different type.

The approach I took was to add some additional fields to the old featureclass, then create a multi-version view for that feature class, and use SQL to populate the fields in question, and then load the features into the new class using the Object Loader.

Adding the Fields
To get this done quickly, I simply added the new fields via ArcCatalog. Only hitch here is un-versioning the data. Luckily for us, this application applies all edits to the Default version, so there was no issue dropping the existing versions.

Creating the View
Once the fields were created, I fired up our ViewMaker tool. This tool simplifies the creation of multi-version views by generating the sdetable commands for you. This is very handy when you need to create views for an entire geodatabase. It also has a companion tool which will generate lookup tables from all the domains in a geodatabase – very handy when you want to generate reports using SQL.

Updating the fields
Now, why do this when I could simply “calc” the fields in ArcMap? Well, multi-version views allow you to update attribute data via SQL. Thus, I could write one SQL statement, and run it in 3 seconds rather than fiddling with the field calculator. And I like multi-version views.

[SQL in QueryAnalyzer]

Loading the Data
I now had the data in a format that I could load using the Object Loader. For those not familliar with this tool, it’s in both ArcMap and ArcCatalog, and it allows you to load data from one feature class into another, while defining field mappings.

This worked really nicely, the only beef I have is that the tool does not have any messaging upon completion. I then refreshed the view of the layer, and saw no records, and assumed something went wrong. So I ran it again. Again, there was no “X Records loaded” or “Processing Complete” or “Done!” message. This time, I disconnected from SDE and re-connected, where I saw that I had two copies of each record loaded. After dropping all the records (via edit session in ArcMap), I re-loaded the data, and all was well. Now I just need to get the data for the other 2 sub-types and load it

From here, the main tasks are systemic in nature. The schema changes that we applied concern a critical component of the data hiearchy – actual land ownership boundaries. It is at this level that we apply our spatial permissioning model – thus changes to this layer will have major ripple effects across the application. At this point, I’m mostly turning project over to other staff members, but I’ll be posting on any hilights or problems we run across. Maybe I’ll be able to convince them to start blogging too!

I’ve done a little more tweaking with my dasBlog theme. The idea was to avoid Jakob Nielsen’s Weblog Usability: Top 10 Design Mistakes.

A quick review of the lists says I’m still missing an #1 – an Author Biography, #5 – my “classic hits” are still buried, and #6 – my navigation to older posts is abyssmal (I don’t even have a calendar!)

Anyhow – for anyone running dasBlog and interested in making these same mistakes, my theme is downloadable from the navigation bar on the left.

Problem:
We have a geodatabase which was originally modeled in UML, but the UML is out of synch with the geodatabase, and we don’t have the time to update the model to match the geodatabase. But, we need to add some new featureclasses to the geodatabase, and we want to design and document these using UML. How do we get the new items into the old geodatabase?

Option 1: Import XMI into the geodatabase
I’d hoped that this would work, but ArcCatalog simply crashed. After some head scratching (mostly because my other options seemed overly difficult), I seemed to have a vague recollection that you need to import models through a connection that attaches to the database as the SDE user. It seems that you can import a model as any user who has correct permissions, BUT, if you import a model which references a domain which already exists, then you need to be connected as SDE. If not, you’ll get another informative error:

Unable to aquire schema lock for {domain name}

I really wish ESRI put some better messaging into the CASE Tools – how hard would it be to pop a message reminding you to connect as SDE when starting the model import tool? And while I’m in this topic, the first time I tried to import my XMI file, I got this error:

Unable to load model from XML file {path}
An error occured while parsing the XML document.
Error number: 516

How useless is this? You can google this error, and get to this page at ESRI that tells you the problem – you need the uml.dtd file in the same folder as your xmi file. Now, since the code clearly knows what’s wrong (error 516) why not just tell me!

Anyhow, turns out that you can apply a partial model into an existing geodatabase, if you connect as SDE. Of course this causes the newly created featureclass to be owned by SDE, and not the user you want to own it. So – back to the drawing board and…

Option 2: Import XMI into a new geodatabase, Export the featureclasses to XML schema, and import that into target geodatabase

This time, I created a new database in my SQL instance, set it up for SDE, and imported the model into it as SDE. Then exported an XML workspace document. And then attached to my target geodatabase as the ‘owner’, and imported the XML Workspace document. And it worked. In order to make sure I did not mess anything up, I had been working on a copy of the real geodatabase, so the real test was to try the same thing on the real one. It worked. Really. So, it appears that the XML schema stuff is the way to go. Hopefully ESRI will create a geodatabase designer which spits out this XML because it would be a whole lot better than messing with XMI/Visio/UML kludge.

Now this this is sorted out, I now have to figure out how to massage the existing data which is going to populate one of the 3 subytpes in one of the new feature classes. But that can wait for a few days.

Next item in the upgrade is addressing some schema changes.

When we started development, we designed a large geodatabase using Visio & UML. Nice. Over the following years, we wrote the ArcMap tools and other applications which use this geodatabase. Still nice. As is the case with most development, some changes must be made to the database model. In most cases, we were good, and updated the Visio UML model, but it’s been a few years, and we’re pretty sure they are now out of sync.

The question is: How can we find these changes so we can easily update the model so it’s back in sync with the geodatabase?

Option 1: Manually check the UML against the contents of the geodatabase
Ok, this is not an option because I need to do this in hours, not weeks. And I’m pretty sure nobody on staff would actually do this!

Option 2: Reverse engineer the geodatabase back into UML.
Wishful thinking – this is still not possible.

Option 3: Compare XML Schemas
This is what I went with, and it should be straight-forward right? Just dump the workspace schema from the existing geodatabase, and from a geodatabase generated from the current UML model, and run them through a diff program, and see what’s different? Oh were it so simple.

The Coded Value Domain Problem:
As part of the workspace schema export, coded value domains are written out, and this caused me some problems. Things may have been easier if we followed ALL geodatabase UML design guidelines. And we do for most things – just not Coded Value Domains. It’s not that we don’t create the domains in the model, we do. We just don’t populate all of them. In our experience we’ve found that the domain values you get from a client at the beginning of a project are different from those needed at the end. Instead of editing & re-applying the UML model a zillion times (which is a pain, and requires that everything be un-versioned), we simply load the domains out of SQL tables that hold the values.
Anyhow – the problem is that the “real” geodatabase has all the domain values which have been loaded from the tables, and in the “new” geodatabase, the domains are empty, which means I can’t compare the XML files directly.

Transforming the XML Export:
Since the XML schema has more information than I wanted, I figured I should be able to just extract out the information I was interested in. ( I should say here that I’m not an XML guru by any means).
After a little quality time with XMLSpy, I cooked up an xml transform (XSLT) that parses out the information I want – Featureclass name, the fields, their name, type, length, and domain if applicable.

After this, I thought I was good to go. Nope. Apparently the ESRI XML schema exporter writes out the xml in some order other than alphebetical by featureclass name. Thus, the order of the feature classes in the output files are not the same, so a simple diff will not work. XML.com had a simple example of sorting in xsl, and after sorting the featureclasses by name, and the fields by name I had something to work with.

So, here’s what it outputs:

At this point I started looking for a simple diff tool so I could locate the differences between the files. Before I got too far on that, I noticed that the XML from the “live” geodatabase was 89k, and the XML from the geodatabase generated from the model was 59k. Not good.

It turns out that we had a whole bunch of new tables registered with the geodatabase. Part of the system involves the collection of very detailed forest information – measurements of indivudal trees. Once this data is loaded into the system, a bunch of processing is applied to the raw tree values to calculate timber volumes. All the tables used to hold the outputs from these processes were not modeled in the UML. Uggh.

Since we don’t have time to spend days entering all these tables into the model, and updating the model is not a priority for our client, it was time to figure out how to “nicely” merge two UML models into one operational Geodatabase…

Over the last 3 years we’ve been building a state-wide enterprise forest managment system for Michigan. During the main development and stabilization phases, we locked the application at ArcGIS 8.3. However, with the upcoming release of ArcGIS 9.2, version 8.3 will become deprecated. What this means is that ESRI will no longer support a direct upgrade between ArcSDE 8.3 and ArcSDE 9.2. So, it’s time to bring the entire app up to the latest version – 9.1. Since I need to document what I’m doing, I thought that I’d share a generalized version of the various steps, how they go, the problems we encounter, and the solutions we use. Hopefully it will be interesting/useful reading…

Code Upgrade: ArcGIS 8.3 to 9.1:
The thing to address is the code – since there was a breaking change in ArcObjects between 8.3 and 9.0, all the code must be re-built, using new references the the new libraries (bye-bye esricore!). Now, it’s worth noting that this is a big system. We’ve got 53 VB6 COM dlls, containing 100’s of commands, GUIs, class extensions, user managment, permission managment etc etc, so we were somewhat concerned about how much work the code upgrade would be.

Happily, we found that the ESRI VB Code Converter did a pretty good job of it. I talked with the developer who handled this upgrade, and besides having to work out the build order of the projects because of dependancies (my bad – I should have told him where to find that!), there were very few errors, and it took less than 8 hours to complete. In a few cases, class names need to be fully qualified when dimming them (i.e. IPoint must be esriGeometry.IPoint)

Not a whole lot to report – the tools work, and it’s not that big a deal.

Testing the Changes:

From this point, we pushed the build out onto a “beta” system in our Citrix farm, for our client to test. I mention this because having this Citrix farm has been really helpful in getting rapid feedback from our clients. However, from time to time we have some strange behavior on these systems. For example, we posted the code, our client logged into the box, and something did not work. We checked it on the development boxes, and it ran just fine. We logged onto the “beta” box via Citrix, and try it – it works. We tell our client. They log back into the same box, with us shadowing them, and it fails.

So you’re following along: on one box (beta) with one install of ArcGIS, and one set of application dlls, the application works for one user and not another. Joy. It gets more interesting – when we have our client log into Citrix using one of our logins, it fails. When we login, it works. Additionally, the “failure” is not really a failure in the typical sense. Basically a part of the interface freezes, and the cursor changes to a cross hair. There is no code in the form in question which changes the cursor.

This is still an on-going problem, so if anyone has ideas, I’d love to hear about them.

Up Next: Upgrading ArcSDE to 9.1, and making some schema changes…

Over the last 3 years we’ve been building a state-wide enterprise forest managment system for Michigan. During the main development and stabilization phases, we locked the application at ArcGIS 8.3. However, with the upcoming release of ArcGIS 9.2, version 8.3 will become deprecated. What this means is that ESRI will no longer support a direct upgrade between ArcSDE 8.3 and ArcSDE 9.2. So, it’s time to bring the entire app up to the latest version – 9.1. I thought that I’d try to write about the various steps, how they go, the problems we encounter, and the solutions we use. Hopefully it will be interesting reading…

The first thing to address is the code – since there was a breaking change in ArcObjects between 8.3 and 9.0, all the code must be re-built, using new references the the new libraries (bye-bye esricore!). Now, it’s worth noting that this is a big system. We’ve got 53 VB6 COM dlls, containing 100’s of commands, GUIs, class extensions, user managment, permission managment etc etc, so we were somewhat concerned about the code upgrade.

Happily, we found that the ESRI VB Code Converter did a pretty good job of it. I talked with the developer who did this upgrade, and besides having to work out the build order of the projects because of dependancies, there were very few errors, and it took less than 8 hours to complete. In a few cases, class names need to be fully qualified when dimming them (i.e. IPoint must be esriGeometry.IPoint)

At this point we’ve got the updated 9.1 code on a the beta system in our Citrix farm, and our client is running through the app to check for any bugs we may have missed.

Next: Upgrade ArcSDE, and plan for some schema changes

While I did not really beat up on it, I must say that Brian Flood’s Arc2Earth tool is amazing. It was super simple to use, and really fast. I think that I was lucky in that the data I had, happened to be in an area that had hi-res imagery, and the data alignment was really good. I really should post a screen cap, but I’m whupped, and I’ve got to pack because I’m escaping for a while. We tried to go to Tulum Mexico, but United is not flying into Cancun until Christmas time, so we’re off to Hawaii for a week. Woot!

No big secret that I’m a bit of a geek. While putting together my presentation on Google Maps/Earth, I found a network link from Geobloggers.com that loads in the last 50 Flickr geotagged photos within your view area. While this is very cool, my immediate idea was that I wanted to be able to see my photos, not just random shots from some random person (this way I can send the link to my family, and they can see where we’re at while travelling). In case I’ve never mentioned it, my other big interest besides GIS & Software development is travelling. Anyhow…

Being a developer geek, I created my own. I’m sure that someone else has cooked this up (maybe even Geobloggers.com!), but I saw this as an opportunity to jump into the mashup pool. Admittedly, I’m pretty late to the party, but better late than never.

The Project:
Allow a user to create a network link of geotagged photos for a particular Flickr user. Pretty simple really. Just enter a Flickr user (use “Dave Bouwman” for example), and if they have any geotagged photos, it will create a network link for you. The network link itself calls a “REST”ful aspx page that I also cooked up. All very fun. And done very quickly. Very quickly. That said, I’ve got a few things left to address before I post up the source code for this little thing.

Tools:
For interfacing with Flickr, I used the Flickr.NET API. While the API is pretty good (I only had to make one change in the source), it’s somewhat difficult to figure out who’s in charge of the project. I was in a hurry, so maybe it’s buried at one of the sites. (oh – and if you’re reading this, your wdevs.com site is down a lot – Check with your provider)

Once I grabbed the photo info from Flickr it was a cinch to whip that into some KML. Ok, it was a cinch after finding TJ’s KML Documentation. Hightly recommend this if you’re going to jump into KML.

Without further ado, I present (my) Flickr Network Link

My “labs” have opened!

My plan in setting up this site was to have a place to fiddle with various things not work related. After a year, I’ve finally gotten around to it. Woot! I hope to add more projects to this area (I’ve been playing with ArcWeb Services in a winforms app…), and over time release most if not all the source code.