UPDATE: Apparently Royce has been slammed by the flu and will be re-scheduling – we’ll still talk Flex, and other random stuff though

Tomorrow, Oct 9th, we will be chatting with Royce Simpson, the developer who built the award winning City of Greeley “ORIGIN” mapping applications.

(Royce is on the right, Brian is on the left, and Jack is givin’ out the hugs)

We’ll talk about his background as a developer, what got him interested in Flex development, we’ll have him take us through the app and discuss various tools and how they were implemented.

Check out the apps, and join the chat to ask questions.

When: Oct 9, 2009 4:00pm MT

Where: http://geogeek.tv/live.aspx 

Follow Royce: @roycesimpson

Assuming the wireless at University of Denver is stable and fast, I’ll be streaming various sessions at WhereCamp5280 (8/14 and 8/15). Since this will be an “un-conference” it’s hard to say who will be talking on what topics, but between Peter Batty, James Fee, Steve Coast and a host of others, there will certainly be a wide array of topics covered. We’ll also try to do some ad-hoc discussions with people, as those are a lot of fun.

As usual I’ll tweet about what we are about to broadcast so you’ll know when to tune in (follow me @dbouwman on twitter).

The url as usual is: http://www.ustream.tv/channel/GeoGeekTV

Until then you can check out some of the recordings from GeoWeb:

Architecture Panel Discussion Part 1

The REST vs SOAP vs Event Driven throw down panel discussion. Actually the SOAP guy’s intro is really good, and all three intro’s are worth watching.

James Fee’s Talk at GeoWeb 

James talks about data sharing on the GeoWeb. James is slinging some zen slides, so you can more or less see what he’s talking about

Andrew Turner: GeoWeb Standards – Where are We?

This one has really low volume, but it was very interesting. I recorded this while sitting next to Ron Lake (aka “father of GML”) and at a few points, Andrew quite deftly slams GML (right around 9 minutes in). We streamed Sean Gorman’s talk, but the recording was borked (as in I failed to press “record”)

Random Chat in the Hallway at GeoWeb

This is likely the funniest of the videos recorded at GeoWeb – James Fee, Steve Citron-Pousty, Brian Noyle and I riff on random stuff

Just a quick note that Brian Noyle and I will give a one day Agile Project Management training workshop on Friday, June 26, 2009 in Denver, CO.  This course will cover both project management practices and development/engineering practices. While we *talk* about developer tools & practices, this is not just for developers – the whole day is equally pertinent to technical and management staff.

We’ll begin with an introduction to agile practices and rapidly progress to specific methodology examples (Scrum), cover roles and responsibilities, project controls, and how to scale the agile process in your organization. 

In addition, we’ll introduce specific software development processes that mesh well with the agile process including automation for code documentation and unit testing, design patterns, refactoring tools, and automated builds and continuous integration.  Throughout the course we’ll give you specific examples, the good, the bad, and the ugly, from our own experiences using the methodology in our shop. We’ll also have hands-on exercises so you can try out some of the ideas we are discussing.

Course Details:

Course Material: Introduction to Agile: Project Management and Development

Date: Friday, June 26, 2009

Location: USGS Mtg Room, Fed Center, Lakewood CO (suburb of Denver for those not from Colorado)

More Information: www.rm-urisa.org/news.html

Please use the “more information” link above to read the full course abstract and get additional details including how to register, maps of how to get there, etc. 

Hope to see some of you in June!

Apparently there was a configuration issue with FeedBurner, and this was not aggregated – thus the repost.

The meeting will be at the Boulder County Administration Building, 1325 Pearl Street, Boulder, CO 80302. The meeting will start at 8:00 am with coffee, juice, and breakfast-y fare. Please be prompt to get some food and hear about GISCO goings-on and a GIS “state-of-the-state” from Colorado State GIS Coordinator Jon Gottsegen. Again, you do NOT have to be a GISCO member to attend  and there is NO COST for the meeting. Here’s the latest agenda that I’ve received – I’ll be talking at 1:00pm – hope to see you there!

8:00-8:30am

Registration – Coffee, juice, donuts, etc.

8:30-8:45am

Welcome, greeting, business announcements – Frank Orr - GISCO

8:45-9:15am

Update on State Activities – Jon Gottsegen - Jon.Gottsegen@state.co.us

9:15-9:45am

Using a GIS to manage Agricultural Water Resources at Boulder County Parks and Open Space

Kristina VanDenBosch,– kvandenbosch@bouldercounty.org

Summary: The Boulder County Parks and Open Space program began in 1975 as a single property. Since that first acquisition in 1975 the Parks and Open Space program has grown to more than 97,000 acres, including directly owned properties and conservation easements held over private property. In addition to the land, BCPOS has a large water portfolio. These water resources, used predominantly for agricultural production, have a value in excess of an additional $60 million. Making decisions about the management of these water resources for agricultural production requires a thorough understanding of complex information, including climatic conditions, natural and artificial water systems, agricultural practices, and legal conditions. Learn how Boulder County Parks and Open Space has designed and implemented an integrated GIS, based on the existing technology structure, to effectively manage these water resources.

9:45-10:15am

Automatically Assigning User-Defined Unique IDs in ArcSDE

Dave Murry – DMurray@CityofWestminster.us

Summary: The automatic assignment of unique IDs in a GIS environment is described in this presentation. The process uses a versioned ArcSDE database environment and Microsoft SQL Server script to assign a unique ID to each new record. The key to the successful automated assignment of unique IDs in a versioned environment is making sure the database is compressed and that no edits are outstanding. This requires the automation of the SDE version process and the editors adhering to these versions. The business process allows for the ID assignment to be one day out of sequence. Some organizations require immediate propagation of IDs and this method would not work for them. Also, environments that do not version their datasets would not necessarily need to use this process but they could take advantage of the automated steps described here. The presentation will review various alternatives explored when developing this method. Future enhancements as well as possibilities for different approaches will be discussed.

10:15-10:30am

Break

10:30-11:00am

On-Demand Spatial Solutions

Frank Orr - Frank.Orr@CH2M.com

Summary: CH2M HILL’s ODSS Small Government Edition is a managed enterprise GIS for government organizations without the time or specialized resources to build and maintain a comprehensive, scalable GIS. CH2M HILL can provide the specialty servers, software, and support so that your staff spend more time focused on your core mission and less time figuring out how to access and support the GIS. ODSS provides a number of advantages compared to other enterprise GIS implementation approaches:

· Lower up-front capital and overall operating costs

· Predictable, subscription-based pricing

· Rapid deployment of industry leading GIS technology and data sources

· GIS accessible by all staff using only a browser—no need for specialty software!

· Guaranteed service levels and scalability

11:00-11:30am

DRCOG Data Catalog

Jonathan Harahush - JHarahush@drcog.org

Sara Eberhardt - SEberhardt@drcog.org

Summary: The Denver Regional Council of Governments (DRCOG) had a business need to develop and maintain a regional data catalog comprised of up-to-date regional data and its associated metadata. The DRCOG Geospatial Team did not have significant budget for additional commercial software or vendor support to build and deploy this Web-based application so we leveraged our collective knowledge and experience and built a framework consisting of Free and Open Source Software (FOSS) in our existing IT environment. The basis for developing this application was to create an open data framework to share regional land use and transportation planning data with DRCOG member governments, internal staff, partner agencies and the public through a searchable web interface. We’ll highlight the project focusing on the open source solutions involved such as PostgreSQL with PostGIS, Geoserver and OpenLayers to name a few. We will also discuss the project’s success and challenges and what the future holds for future FOSS projects at DRCO.

11:30-12:00 noon

Best Practices for Mobile GIS Technology Integration

Glenn Vlass - glenn@cartopac.com

Summary: Today, many local and state governments are deploying mobile solutions, many of which have workforces in the field with more than a hundred users. Determining requirements, defining data work flow and understanding how data is to be moved between the office and the field are all essential steps in order to be successful. This presentation will help you understand key issues and determine how to plan for a mobile implementation and integrate it into an existing GIS. Already have a mobile application in the field? Learn best practices, improve your efficiency and increase your ROI. Learning objectives include:

· Discuss the essential requirements to successfully deploy mobile applications within your workforce.

· Learn the value of defining a data work flow for your mobile application before deployment of your mobile workforce.

· Understand the challenges faced by many Municipalities when deploying mobile applications.

1:00-2:00pm

Building Great Apps for Your Maps

Dave Bouwman - dbouwman@dtsagile.com

Brian Noyle – bnoyle@dtsagile.com

Summary: The GIS community is faced with a number of challenges and opportunities today when developing applications for the GeoWeb. In this session, speaker Dave Bouwman (http://blog.davebouwman.com) will discuss what to consider when developing GeoWeb applications. Real-world examples will be used to illustrate and explain development concepts relating to using an agile development process, selecting map canvases, interface design, back-end service design, and unit testing. Some of the technologies covered will include ASP.NET model view controller (MVC), MbUnit, esri.map, and OpenLayers.

2:00-2:30pm

Getting The Lay of the Land: Free and (mostly) Easy Mapping of Land Resources With The Help Of Government Web Services

Brian Timoney - brian@thetimoneygroup.com

Summary: Be a hero to your clients and co-workers by quickly whipping up comprehensive snapshots of land use in the west by tapping the WMS services offered by the USGS and BLM. More than simply topos and land grid, with these services you can stream federal land ownership, oil & gas leases, coal & geothermal, etc. Imagine: no shapefile downloads, no stacks of DVDs on your desk. Even better, your users won’t need fancy software to view your handiwork: we’ll show you how stream this info into Google Earth as well as create your own browser-based map using Open Layers. By showing the utility and flexibility of streaming open-standards web services, we’ll highlight the difference between public entities making their data accessible and merely making it available (e.g. shapefile download via FTP–hello 1999!).

2:30 – 3:00

The GIS in Higher Education Summit

Jamie Fuller - j.d.full@gmail.com

Summary: On Friday, April 24, 2009, professors and instructors from across the state gathered at Colorado State University in Fort Collins to discuss the role of higher education in geospatial technologies. The GIS in Higher Education Summit attracted representatives from over 15 colleges and universities and facilitated discussion on topics including: strategies for coordinating geospatial programs and certificates offered across the state; articulation between community colleges and universities; possibilities for sharing resources across campuses—curricula, data, teaching methods; efforts to establish standards for certificates, minors, majors, and other degrees; and what is expected from GIS students once they are out in the workforce. The Summit marked a starting point for collaboration among academic institutions teaching GIS and related subjects. A centralized website and a listserv are initial items to establish; coordination with both GIS Colorado and the state GIS office on these efforts seems imperative! The “GIS in Higher Education” website (currently under construction) will be demo’d and comments will be welcomed.

3:00 – 3:15

Break

3:15 – ??

Breakout Sessions:

• Commercial and Public Data Updates – Dave Murray
• GIS Education and Mentoring – Esther Worker
• National Hydrography Dataset Stewardship Committee – Chris Brown
• SWUG 2009 Planning Committee

I’ve been happily coding along on an ASP.NET MVC project using Test Driven Development (TDD). When you write the test first, you usually end up with very high code coverage numbers…

(Obviously the LoginController was not written TDD – creating tests for it is next on my list!)

Anyhow – I’m currently working on the SecurityHelper – which does some rather ugly User.Roles to Workflow-Status comparisons to determine if a user can View/Edit fields on the Project domain object. Since this is really the core of this system’s security model, I really wanted this to have high coverage. With TDD I should have 100% right? What’s with that 99%

Digging into NCoverExplorer, we can see the coverage by method…

So – my problem is the GetEditAccess method. When we dig in further, we can see the actual code that’s not executed during the test run… (how awesome is NCoverExplorer!)

This is the default clause in a switch/case block that is switching on an enumeration. I even noted that this code would only be hit if someone added a value to the enum, re-compiled the app and passed that into the method. And, should this occur, I want to ensure that security is not compromised.

So -  the question becomes – if I can’t* write a test to exercise this code, should it be there? Rigorous TDD would say no, but having guard clauses is a good idea – especially a default in a switch/case.

Personally, I’m leaving the guard clause in there and I’ll just have to live with 99% coverage for this class, but it’s something to think about – what amount of coverage is enough? What is too much?

For this project, we have used code-generation to create some “domain object” starting points (we can’t do pure Domain Driven design here because we have inherited a populated database). At this point in the project, we’ve got about 30 of these domain classes, which are dominantly just properties. Since we “trust” the .NET CLR to do it’s job and correctly handle the getters and setters, we are not spending time with testing that code. When we add logic to these classes, we add tests, and we expect high-coverage on them.

What’s  your take on Code Coverage? Worth it or waste of time?

*[Ok, you could likely do something with Reflection, but that would waste a few hours figuring out how to do it - does not see like very efficient way to spend time.]

Just a quick note to any ESRI Canada readers heading to the Tech Trek next week – Thursday afternoon I’ll be giving a workshop about Tools and Techniques for the Agile Developer – I’ll be talking about a wide range of things – from source control to  unit testing to refactoring to design patterns – basically a whole wack of good (free/open-source) stuff that you can use to make your software development projects more sane. This is a zero-to-sixty talk – it assumes very little coming in, and you’ll walk out with a whole lot of stuff you can use to get fully up to speed!

Hope to see you there!

The main complexity was the database that contains the scientific observations. I should note that this is a database used by scientists to enter all sorts of avian influenza data – simple count surveys as well as detailed blood sampling. As a result, the database has over 2000 tables, and 10000 stored procedures (mostly code-generated). There are more than 35,000 location records in the system, and many more samples or observations. Sure the locations (Lat / Lon) were in one table, but recall that we needed to allow users to filter the data shown on the map by multiple criteria simultaneously. Other than Country, all the other criteria required at least one join, and most required multiple joins. Oh, and it has to be fast. Ideally we want the points drawn on the VE Control at or near the same speed that the tiles load.

This effectively took ArcSDE and ArcGIS Server out of the picture since this stack does not effectively handle this sort of situation. We did some testing just pulling the points out of ArcSDE via ArcGIS Server, converting them into our custom point type, which then runs through a clustering algorithm, and is finally converted to GeoJSON and shipped to the client. While this worked, it was slow. And we could not see any way to handle the query criteria without doing the critera filtering in code vs in the database. Since we were already looking at several seconds to just get the points, I stepped back and too a different approach.

Clearly if I was going to handle the criteria filtering, and hit the performance goals, I needed to do all the filtering (spatial & attribute) in the database itself. This sounded like a good opportunity to play with SQL 2008, thus the initial posting on the R&D back in February. That post goes into much of the details, but what I found (and you can see on this demo page) that it was in the ballpark in terms of performance. On our LAN, zoomed in so there were not 1000’s of points returned, I was getting the points rendering in Virtual Earth about the same time the tiles would arrive. Good. But when we zoomed out (level 1 or 2) there were 10,000+ points being returned, and we still had 1-2 second lag times. As noted in this post, I also had some issues with the single hemisphere thing.

While I was working on that, more or less on a lark, I thought I would try working directly with the Lat / Long fields in SQL 2005. I cooked up a quick Stored proc that took 2 Lat/Long pairs defining a box, and returned points that inside. Simple, and for the purposes of this project accurate enough. Oh, and really really fast. Even at zoom level 1 (whole earth), I was still getting the points back rendered in Virtual Earth before the tiles. Checking performance in SQL Server, this was running in less than 100ms.

So that pretty much decided how I would handle the locations. From there I took a set of Stored Procedures written by WCS, and added in the Lat/Long bounds, and I had the criteria working. The main stored procedure than handles all the point and criteria queries is run every time you move or zoom the map.

It’s looking really dark outside and the trees are whippin around, so I brought up the weather channel’s radar map to see this…

Time to hunker down!

Just wrapping up a project that’s built on DotNetNuke, and was using the built in module packager to create a nice zip installation. Great. Only problem is that it packs up all my Subversion “_svn” folders as well as the actual files I want. Doh! Initially I just exported from my Subversion working copy to a new location, added that as a virtual directory in IIS, and used that instance to create the package. But this is a real pain when you need to release small patches and bug fixes, so this being open source and all, I dove into the code.

And there is a simple fix… assuming you have the entire source tree laying around ( you can download it from www.dotnetnuke.com if you don’t have it)

The module packager is the PaWriter class, which is in the DotNetNuke.Library project, under the Components/Modules path.

All you need to do is add two snippets of code to the two ParseFolder private methods:

Change:

to:

Once you’ve made this change, re-build the DotNetNuke.Library, and copy the DLL into the bin folder of your DNN instance (if you are working directly in the full source tree, just build the website).

Given that Subversion is so very widely used, it would be nice if the DNN team would add this (and .svn) to the “core” code for the packager.

One issue I seem to run into a lot is the gap between client expectations of ArcGIS Server, and the reality. Although I did bring this up at the closing session of the Developer Summit, and we were told this would be addressed, I think it’s worth pointing out some specifics.

On the one hand we read “Fulfilling the Promise of Complete Enterprise GIS” in the Fall 2007 ArcNews issue.  Some quotes…

… to support fast and efficient visualization and analytics applications, regardless of the amount of data held within an organization.

Any model or tool authored in ArcGIS Desktop can be shared to a broad audience via ArcGIS Server…

It’s  typical of much of the marketing. Pushing the simple author-publish-consume model, in which Joe GIS can just take his map, and geoprocessing models and fire it up onto the web in a few short clicks. To be sure this makes a great marketing story… “Wow – I can do that! Lets buy ArcGIS Server!”

But what happens when reality steps in? When Joe GIS takes his map, and geoprocessing service and publishes it?

Typically the site is slow, the geoprocessing tasks randomly fail and take ages when they succeed, and the Joe GIS gets (understandably) negative feedback from his users and superiors. So he packs up his bags and heads out to the developer summit, with one agenda: make his site faster. He drops into the “Architecting ArcGIS Server Solutions for Performance and Scalability Session“. On slide 5th slide, the first golden nugget of truth is revealed:

The item in question is somewhat small on this slide, and it’s not spelled out directly, so let me help out – “Design your maps specifically for server deployment”.

Somehow this never makes it into the ArcNews articles or the demos. Nor do the items on the following few slides…

• only use the ESRI Optimized symbology,
• use annotation instead of labels,
• skip the fancy cartography – including highway symbols
• cache everything you can,
• and only load a few layers.

So Joe GIS now realizes he’s going to have to re-work all his maps. But there’s got to be some good news about the geoprocessing right?

Enter golden nugget of truth two – and nope. Seems Joe is going to have to do a bunch of pre-processing, and stage up his data differently. So much for just publishing the data as it is. What does this “Understand Performance Expectations” mean? Oh yeah – basically if it’s slow in desktop, it will be slow in server. And this last bit about only one instance can update data at a time? That does not sound too good. And Joe’s ESRI reps have been telling him to use geoprocessing to do everything, but if he uses Geoprocessing his site can’t handle all 5 of his users at once! Maybe it’s time to take a look at .NET and that ADF thing…

I could go on through the entire presentation, and point out the various inconsistencies between what we see in the ArcGIS Server marketing and the reality (aside: check out slide 28 where they tell you to disable seamless panning and drop the overview map – how many demos have you seen where seamless panning is disabled?!)

What’s really interesting is this is ESRI telling you both things!  Right hand meet left hand. Marketing meet the technical staff. Take each other out for lunch. Talk. Listen. Learn. ArcGIS Server is a great product – it can do lots of great things, but it’s not a point and shoot camera. It’s a serious SLR, that needs professional skill to operate effectively and efficiently.

Until the large scale marketing shift happens, here are a few suggestions for the ESRI technical marketing people out talking to clients:

• Be up front about what ArcGIS server can do, and what it can’t.
• Be up front about having to specifically author maps for server, and the limitations on cartography, number of layers and caching.
• Be up front about having to pre-process data used in geoprocessing tasks, and concurrency limitations.
• Be up-front about the fact that to do much beyond pan and zoom, there will be some coding needed, and the water gets deep fast.
• Be up front about the server loads and the licensing costs to scale out.

Then show some totally kick-ass demos. Show something truly amazing, and be rock stars about it, but just don’t lip synch. Be up front about the reality of running ArcGIS Server, because every single one of your users are going to experience this reality. Set customer expectations realistically, and everyone wins.

Link to the presentation listed above