Thursday, June 21, 2007

On Competition

Sorry for the long delay in blog posts - once I got home from a week in Orlando for TechEd, the real world jumped on me with both feet and hasn't let up since. I haven't been to the SSIS Forums, I haven't been posting here - I've barely had time to think and breathe. [1]

But while I've been running here and there weathering the changes going on around me, I've had a background thread running and doing a lot of thinking about the nature of competition. And the soundtrack for this thread has been a Manowar[2] song titled "Number 1." Here's an excerpt of the lyrics:

We are sending you a challenge, it’s very clear
We came to win, that is why we are here
Demanding to be tested, tested by the best
Not to be forgotten like all the rest
Let the contest begin, play hard, fight to win
Immortality, victory and pain[3]

The emphasis is mine, but I think it's vital to the meaning of the verse and of the song. Winning is not the most important thing. Winning when pitted against the best, the strongest, the meanest, the fiercest challenge that the world can throw at you - that is when winning matters, when winning really means something.[4]

I've been very fortunate this year. Not only did I win the Iron Architect[5] competition at TechEd in Orlando, I also won several other competitions, including one that earned me a place on stage with Manowar at their Magic Circle Festival this July in Germany. All in all it's been a very good year.

But the thing I remember about the competitions in which I've participated is not the moment of victory. It's not even the brief "moment of fame" that accompanies the victory.[6] The memories that stick around are those of the uncertainty, the dread, the drive, the energy - the whole mish-mash of emotions that swirl through your head when you know that this is the time when you need to do your absolute best or else you're going to be "forgotten like all the rest."

When do you ever feel more alive than when you're faced with the absolute certainty that the only thing that stands between you and glorious victory (or inglorious defeat) is yourself? And when you emerge victorious at the end of the day, will you look back and say "I won because I was better than I have ever been before" or will you say "I won because there was no competition?"

And which one means something, and which one does not?

[1] I wrote this intro on June 21st, so hopefully this should give you an idea about how busy things have been of late. Where did July go, anyway?
[2] The Kings of Metal, the loudest band in the world, my favorite band - despite the fact that they have such a horrible web site...
[3] This line actually goes "Immortality, victory and fame" but I'd always heard "pain" when listening to the song, and I think that this fits better anyway. Victory without pain is usually empty; the pain lets you know that you did something real.
[4] Back when I was in college I used to fence competitively. In all honesty, I was never great, but I was pretty good, I was generally the best fencer in whatever group I joined. (Again here there is a theme of never being better than your competition, eh?) I won hundreds of bouts over the years, but I have difficulty remembering any of them. The ones that have stuck with me through the years are the ones where I was pushed to my limits and beyond by a superlative opponent. One bout in particular is still vivid in my memory: I was having my best day ever, and was beating my opponents readily, progressing through the tournament. But I was eliminated before the final round by a fencer much better than myself. Normally I would not have given him much challenge, but that day I was on, and probably fenced better than I ever had before or since. In the end he defeated me 15-14, and went on to win the tournament. After the day was done he sought me out and told me "Our bout was the best bout I've ever fenced - by the end I was afraid to attack!" Give me competition that means something over an easy victory any day...
[5] Although I have yet to begin work on my Microsoft Certified Architect track - ugh.
[6] But being on stage with Manowar in Germany is about as cool as cool can get, no matter how you look at it. ;-)

Saturday, June 9, 2007

TechEd 2007 and Space Shuttle Atlantis

Well, TechEd 2007 in Orlando is officially over, and NASA decided to see it off in grand style. [1]

Last night I drove to Cocoa, FL with about 30 other Microsoft Certified Trainers [2] to watch the launch of the space shuttle Atlantis. The traffic was horrible, but my group made it to the observation point with almost eight minutes to spare, and that was what was really important. I took a few hundred pictures - here are the ones I've found so far that I like the most. Enjoy!

Wow. The photos do not really capture the majesty of the event, but hopefully a little bit of it shines through. Maybe we can do it again next year too...

[1] I don't know if this was quite as cool as the fact that Manowar was playing in town the day before the conference started the last time TechEd was in Orlando back in 2005, but it was pretty darned close.
[2] What a great community!

Friday, June 8, 2007


Well, the Iron Architect competition at TechEd 2007 is over - the final showdown (with tasting and judgment) took place Thursday evening, and in the end I emerged victorious.

There were four contestants in the final showdown, each one of whom came up with an architectural proposal for the same problem posted on the MSDN Iron Architect Blog. Each contestant had a very different approach. My architecture was based on using SQL Server Analysis Services Data Mining as the core engine to segment and categorize attendees and to define relationships between new attendees during the registration process and the existing registered attendee base. This core engine would then be implemented as a discrete service that would then interact with external systems (such as the conference scheduling systems and the hotel registration systems) via BizTalk Server.

The competition was pretty fierce. I went second, and although I thought I did a better job than the first and third contestants, but the fourth guy started out really strong, focusing on the build vs. buy decision, which I had not considered at all. Still, in the end the judges (who were delightfully critical - I love getting critical feedback!) thought that my solution and my presentation were strongest over all, and awarded me the grand prize.

Of course, although this content was stressful, the hard work is now just beginning. The prize I've won is basically a scholarship to get accepted into the Microsoft Certified Architect program. Now I have to do all of the work for the MCA, and from what I hear, that's far from trivial. Time will tell, and I'll be sure to post about it here...

Thursday, June 7, 2007

Iron Architect, Here I Come!

I received an email last night informing me that the judges for the Iron Architect contest liked my submission enough to declare me a finalist, and invite me to present and defend my proposal tonight here at TechEd.

So... if you're in Orlando this week, come down to the ARC track interactive theater (blue 9) this afternoon at 4:30 and watch me sweat!

Tuesday, June 5, 2007

Are Donald Farmer and Dr. Ivan Brady Related?

Another great quote:
"You have to understand the nature of your exceptions"

You need to understand the underlying business context of the data and problems with the data before you can solve the problems. And you need to remember that it may be a business solution, not a technical solution.

Sound familiar?

Words of Wisdom from Donald Farmer

My favorite SSIS quote of the day:
"Rolling your own ETL is king of like rolling your own cigarette - if you
roll it too tightly, you can't get any throughput"

-- Donald Farmer

ETL Lifecycle - An Interesting Thought

During this ETL Framework session, Larry Barnes (the person with whom Donald Farmer is co-presenting) raised an interesting point:
ETL processes tend to stick around forever, because people are horrified to
touch them.

He used the image of a Perl script that runs on an Oracle server pulling data from a Sybase database and loads it into Informix (can't you just picture that?) but I have no trouble visualizing the same thing happening with SSIS as well.

This may not be technical or of interest to anyone but me, but it strikes me as being profound in some way. I tend to be very aggressive in my commenting, annotation and documentation, but this really drives home how important this is for SSIS. In addition, this really demonstrates the importance of having a well-documented and understood process for making changes, moving from dev to test to prod just like with traditional application code, because only through this process will people truly be comfortable making changes to such business-critical code.

Donald Farmer Moving On

This may only be news to me, but Donald Farmer is no longer on the SSIS team. He's now a Principal Program Manager on the SQL Server BI team, and is working with the data mining team. This came as something of a shock when he said this, since his name is the first one that leaps to my mind when I think of SSIS.

With that said, he's co-presenting on an ETL Framework session as I type, so his love of SSIS still remains.

Hopefully he'll be able to do for data mining what he did for ETL. Good luck, Mr. Farmer!

Katmai for the BI Guy

I attended a session yesterday here at TechEd 2007 in Orlando - DAT 201, SQL Server 2008 "Katmai" Overview with SQL Server General Manager David Campbel presenting. Although there was very little detail information, it was quite a good session, as it presented a nice survey-level overview of the features coming in SQL Server 2008.

The development process for Katmai was structured differently from previous SQL Server releases. Instead of each team building their own feature in isolation, development was structured around "themes"[1] and within each theme there are scenarios, where each scenario encompasses a group of features. Also, a single feature can help complete multiple scenarios.

The theme that is most relevant to business intelligence developers is the "Pervasive insight" theme. This theme has three scenarios, with quite a few features included. Here's the deal:

Scenario: Integrate all data into enterprise data warehouse
  • Enterprise partitioning parallelism
  • Enhanced DW query optimizations
  • Data compression
  • Resource governor
  • Persistent lookups
  • Change data capture - find those changed records since last point
  • MERGE (UPSERT!!) SQL Statement (how will this affect SCDs?)
  • Data profiling

Scenario: Reach all users with a scalable BI platform

  • Scalable report engine
  • Scale out analysis
  • Subspace computations
  • Streamlined analysis tools
  • Scalable backup tools
  • New cube design tools
  • Best practice design alerts
  • IIS agnostic report deployment - Host SSRS w/o IIS

Scenario: Empower every user with actionable insight

  • New Word rendering
  • Improved Excel rendering
  • Report builder enhancements
  • Rich formatted data
  • Enhanced data visualization
  • MOPLAP enhanced writeback
  • And there was one more feature bullet here, but they moved on too quickly before I could write it down.
Sadly, I do not yet have the information to drill down on many of these feature bullets, but hopefully there is enough here to whet your appetite as it has whetted mine...

[1] I've heard other product groups use the term "pillar" to describe similar high-level meta-feature groupings.

Monday, June 4, 2007

Loud Music Not Conducive to Technical Discussions

What were they thinking?

Although a loud band can sometimes be a good thing [1], technical conferences are not included in those times. I think that whoever decided to put a band in the middle of the conference floor at lunch today at TechEd did not understand the conference's demographic. Sure, it's cool that we're celebrating 15 years and all, but when people are trying to meet and eat and have technical discussions, it's damned annoying when you have to shout to hear yourself… Next time around, let's please have the noise at the Worldwide Partner Conference, not at TechEd.


[1] Or a great thing, if it's the world's loudest band - Manowar, the Kings of Metal!

SQL Server 2008... Katmai No More...

So here I am in Orlando, feeling old. For years we've been calling it "Katmai," but now...


...Mike Woods did a demo talking about service enablement for applications. A big part of the demo was BizTalk 2006 R2 orchestrations (yes, they do make a good demo for people who have never worked with BizTalk) but it also included some new SQL Server Reporting Services capabilities for SQL Server 2008.


That's right - no more Katmai, just like there's no more Longhorn. Sad, really. I always feel like I've lost a friend when a code name is retired.


But apparently Microsoft has purchased some additional SSRS report items from Dundas. Mike showed off the Map report item, and would be surprised if this was the only one. Data visualization is vital to BI, and it's one place where SSRS today doesn't really shine...

Friday, June 1, 2007

From the mouths of babes...

"They used to be pigs, but now they're bacon!"

-- Moira, age 4

Happy birthday, Moira!