I Love My Job!

One of my biggest complaints about being an independent consultant for so many years was that it was a significant effort to find the type of cooperative and competitive peer environment in which my skills could be tested, my guidance could be questioned and I could be pushed to continually better myself. When you're a "lone wolf" you have to rely on yourself to push yourself, and that can be quite a challenge sometimes...

But I've just finished up a meeting with the architectural board (which is actually a preparatory meeting for another higher-pressure meeting tomorrow morning ;-) where I was surrounded by bright, assertive and vocal people all presenting ideas, arguing them, shooting them down, backing them up and basically all pushing each other.

And I love it!

It's still weird not being my own boss (and my own everything) but I'm so happy that I took my new job.

And that's a great feeling.

A Little More On Risk

When you read the title of this post, please be certain to focus on the phonetics and not the spelling of the title. With me? Ok, let's proceed.


Risk management is pretty simple when you think about it. Microsoft has defined a sick management process that is part of both MOF and MSF, and it looks like this:



I love this picture - it really shows what you need to do and when. But you can boil down the essentials of risk management even further:

1. Figure out things that could go wrong - these are your risks
2. For each risk, figure out how likely it is to happen[1]
3. For each risk, figure out how bad life will be if it happens[2]
4. Multiply these two together[3] to find the highest priority risks
5. For the highest priority risks, figure out what actions you're going to take to keep them from happening
6. For the highest priority risks, come up with a plan to minimize the damage done if the actions in step 5 don't work as planned

This makes sense, right? It's amazing how often you can take small steps early on that prevent big project-threatening problems later on. Many people (and many software development organizations) still don't see this, however - they think that people who focus on risk early are "being negative" when in fact that they're simply being proactive.

Why do I mention this today? Well, I've run into a problem where someone (no names will be named) has lost around four (although some might say seven) days' productivity when asking for a little information could have reduced this to an hour or less of lost time. The good news is that I now have a little .NET utility that generates a DTEXEC.EXE /VALIDATE batch file for all of the packages in a project, a solution or a file system folder and its sub-folders. It's simple, handy and proactive, and it has that certain someone back on track with (cross your fingers!) no more lost time. Now back to work...

[1] This is generally measured in percentage.

[2] This is generally measured in pounds of poop that will hit the fan.

[3] The resulting figure is measured in poopercentage, which is a unit rarely encountered outside risk management, but you can simply think of it as a number if you'd like.

More MOF Quotes

In the Microsoft Operations Framework class I’m taking this week, we were talking about the MOF Risk Management Discipline[1] and there was another quote that made me smile:

“If everyone owns it, no one owns it”

And there was another related quote that came up yesterday while talking about the MOF Process Model, which fits in very nicely:

“Every process needs an owner: One throat to choke!”

These two quotes really drive home the fact that on any project, on any team and in any process there is a real need for accountability and ownership. Not only does this help foster an environment of responsibility, it also helps define channels of communication between groups and members of groups.

It’s a big thing. When was the last time you were working on a project and asked “who’s responsible for this server[2]?” and no one could get you the answer you needed? I’ll bet it hasn’t been too long.

Ownership is a core component to doing things right, because if no one owns something, no one will take care of it. If it’s not worth assigning an owner, it’s probably not worth doing, so you should question why you’re doing it in the first place.

Take a look at your teams and projects and processes. What do you own? Do other people on your team agree that you own it? Does management? Only you know the answers – or do you?

[1] Risk management discipline is not unique to MOF; it’s also a core part of MSF (the Microsoft Solutions Framework) and any well-run software, database and BI project. If you don’t use it, you really should.
[2] Or software, or resource or process or whatever.

Random Thoughts on Usability

I’ve been working a bit with Windows Presentation Framework lately. Don’t worry – it’s not as bad as it seems. No one has asked me to design a new cutting-edge user interface for their software. I plan on remaining on the back end as close to the data as I can get without burning myself, and closer if necessary.[1] But the opportunities that exist for rich data visualization with WPF and WPF/E are too exciting to ignore, so even if I never write a line of production XAML, I still want to know enough about it to be dangerous.

Besides, it’s new and cool, right?

Right?

Yes, it is. And that’s what’s been scaring me. I’ve been thinking about usability, and how the temptations presented[2] by WPF may overwhelm those developers who are tasked with designing that next great UI, causing them to forget their users and instead focusing on the cool new features of the cool new platform.

And we know that’s not good, right?

Right?

Boy, I sure hope we do by now.

Anyway, I’ve had a few random thoughts running around my head that my tagging service[3] has categorized as “usability,” and once this tag bubbled to the top it was immediately correlated with WPF. So in no particular order, here they are: Random usability thoughts.

1) The Price Tower. The Price Tower is a 19-story building in Bartlesville, Oklahoma that was designed by the renowned architect Frank Lloyd Wright and has since been converted into a hotel without modifying his design. He’s a famous guy, right? He’s a master architect and artist who really knows his stuff, right?

This may be the case, but this building is a usability nightmare. Before I get into any details on that front, let me start off by saying the tower is beautiful – it has delightful angles and motifs that really embody the Frank Lloyd Wright look and feel, and it’s aesthetically very pleasing. But as a usable space it falls flat. Flat onto rusty metal spikes. The rooms themselves are split onto two levels, with a narrow[4], steep and angled stairway connecting them. The stairs are dangerous. The rooms themselves are lovely, but bizarre. There is not a right angle to be found. The ceiling in places is high and lofty; in others it is so low that I need to duck to avoid scalping myself. The bathrooms are tight and confined – if you can picture the last bathroom you used on an airplane and squeeze a shower into that space you’re not far off. And you can’t get to the toilet without closing the bathroom door, so there’s no way to escape the claustrophobia when nature calls.

There are many more usability horrors in the Price Tower (such as the hexagonal elevators that only hold one person at a time) but I won’t attempt to list them all here.[5] But it sure is pretty!

2) Thrifty Rent-A-Car. This usability horror story is a little less blatant, but no less frustrating. I recently rented a car from Thrifty – I reserved it online using their web site to be picked up Sunday night when my flight arrived, and returned on Friday when my return flight departed. As part of the reservation process I provided profile information (name, address, phone, etc.), trip information (airline and flight number) and was in return given a reservation number, a daily rate and a total estimated price. Standard fair so far.

But my flight on Sunday was delayed due to weather, and I arrived at 11:30 instead of 9:30 as planned. No problem right? The reason you provide your airline and flight number when you reserve a car is so that if the flight is delayed, the rental company employees can wait for the delayed customers and not cancel the reservation or leave, right?

Right?

Apparently not. When I got to the rental counter, there was no one there. The very helpful young woman at the Alamo rental counter next door let me know that they’d all left. Gone. No note, no notification, no communication, no phone call, no nothing. And certainly no car.
I won’t go into the many negative emotions I felt at the moment. I was able to rent a Nissan Murano SUV from Alamo for one day, and got to my hotel before I fell asleep, which is really what matters.

But I was seriously let down by Thrifty. There is an implied contract between the rental company and renter in the exchange of flight information, and that contract was broken. When I called Thrifty the next day to ensure that I still had a car waiting for me and told the “customer service” representative about my disappointment, the only thing that he could say[6] was “we go home at eleven.” I won’t go into more frustrating details, but I certainly will never rent a car from Thrifty at this location if my flight is scheduled to arrive later than mid-afternoon. It’s just not an acceptable risk, because a rental provider that doesn’t reliably provide rental cars isn’t usable.[7]

3) Nissan Murano. (http://www.nissanusa.com/murano/index.html) As I mentioned above, I rented this SUV from Alamo when there was no service at the Thrifty counter. I wanted a compact car (I don’t like driving large SUVs) but this was the best vehicle available. I drove it for the day on Monday, and took it back to the airport Monday evening so I could pick up my reserved car at Thrifty. I didn’t pre-purchase fuel, so I stopped off at a gas station near the airport to fill up the tank before returning the car.

This should be a simple operation, right? There is a standard way to do this, right?

Right?

Wrong!

First, I reached down to the floor to the left of the driver’s seat to pull up the little lever that is there on 99% of the cars I’ve driven in the last five years. Nope – nothing there. Next, I looked over the dashboard to see if there was a “fuel door release” button. Nope. Then I looked inside the glove compartment to see if there is a release button in there, as I vaguely remembered some older cars having. No go. So I got out and looked at the fuel door itself to see if it was designed to open manually from the outside. And this was not the case. WTF? What other options could possibly exist for this simple task? I could think of none, so I repeated the steps above several times, looking more closely and more slowly each time. I probably spent the better part of ten minutes doing this, to no effect. There simply was not a button or lever or anything to open the fuel door.

Then I found that thing you almost never find in a rental car: the user manual. I flipped through the table of contents and found the diagrams of the control panels. Four or five pages with diagrams; dozens of labels for details on the diagrams. No mention of anything related to fuel other than the fuel gauge. WTF? I then flipped to the index, found the entry for the fuel tank door and flipped to the center of the book where the topic was buried. There I found the instructions for opening the fuel tank door. Ready for them? This is what you need to do:

* Exit the car
* Close the driver’s door
* Insert the key into the lock on the driver’s door
* Turn the key twice clockwise in the lock on the driver’s door
* Remove the key from the lock
* Push on the right side of the fuel tank door

Am I allowed to say “WTF” one more time in a single blog entry? Who thought of this? Who approved it? Who failed in their review to allow this incredibly horrible and completely unusable design to make its way from the drawing board through manufacturing? And perhaps even more importantly, where are their bodies buried?

To be fair, I’m sure that this usability disaster would be greatly lessened with effective training, but to be blunt, training should not be required to complete a simple task for which a standard solution is ubiquitous.[8] Nissan’s decision to reinvent the wheel, to implement a proprietary solution to a well understood function for which a standard solution already exists resulted in an incredibly negative user experience.

So what does all of this have to do with WPF?[9] My concern is that developers will give in to the siren call of new features to build UIs that they could never build before just for the sake of doing it, without remembering all of the usability practices and guidelines that “applied to the old tools and technology.” So I’m still very excited about the new capabilities that WPF provides, but that excitement is tempered with more than a little trepidation on how real developers in the real world use these capabilities to build applications that are not only beautiful and cool, but also usable…

[1] Or should that be “and closer if possible?”
[2] Pun intended
[3] I don’t know exactly where they’ve come from or who installed them, but I have a serious number of background services running all the time. Even though they’re supposed to all run at lower priority so that their threads don’t impact the performance of mission-critical threads, sometimes it seems that this isn’t always the case these days. Perhaps I need a hardware upgrade…
[4] The word “narrow” does not do these stairs any justice. Although I did not measure them, I would be surprised if they were more than 15” wide, and amazed if they were more than 18” wide.
[5] Instead, ask my colleague Ted Malone ;-)
[6] Although to his credit, he did say it over and over again, so at least he was consistent…
[7] Yeah, the usability link is a little weak for this story, but it’s a necessary lead-in to the next one.
[8] I did not have this in mind when I started writing, but this sure sounds a lot like the Office 2007 “Ribbon” UI redesign, doesn’t it? I personally love the ribbon and love Office 2007 in general, but because of this rant against Nissan I finally better understand where people are coming from when they rail against the removal of menus and toolbars in Office 2007.
[9] Oh yeah – that’s what we were talking about!

Clear Eye for the KPI Guy

I'm attending a Microsoft Operations Framework class this week[1] and the instructor[2] had a great quote today:

If your KPIs are always green, or if your KPIs are always red, then you're measuring the wrong thing.

Now he didn't actually say KPI - we were talking about the metrics you might use when defining and auditing service level agreements, and not about BI, but the concepts are still the same. In order for a KPI to have the most value there needs to be circumstances when it is not green, and if a KPI is always red, then either you've set the goal unrealistically hard, or else you're not willing or able to use the KPI the way it was intended.


[1] This is one way I'm trying to practice what I preach - MOF and ITIL are essentially the "business side" of the BI project I'm working on, so the more I understand its nuances, the better I'll be able to help bring business value to my users.
[2] Monte Whitbeck from Microsoft - he's doing a killer job. I wish I got to attend more training from people like Monte who really understand the material they present and have worked with it extensively in the real world.

ACM Professional Development Seminar

Hey – before I forget (yet again) I have one more quick announcement to make. I’m going to be delivering a one-day SSIS seminar as part of the ACM professional development series at the New Horizons Computer Learning Center of Syracuse[1]. Odds are that if you’re interested enough in SSIS and BI to be reading this blog you’re not in the target audience for the seminar[2], but perhaps you know someone who is. Just in case, here’s the session description I gave to the organizers:

This one-day seminar will provide attendees with an introduction to SQL Server Integration Services, the enterprise-scale replacement for Data Transformation Services in SQL Server 2005. Through a combination of lecture, demonstrations and hands-on exercises, attendees will learn:

• Overview of the SSIS platform


• Developing SSIS packages using the Business Intelligence Development Studio


• Building package logic using SSIS Control Flow


• Building high-performance data pipelines using SSIS Data Flow


• SSIS Deployment tools and techniques

So if you know anyone in the right area and in the right demographic, why not let them know? We're not going to drill deep covering these topics in one day, but we'll still learn a lot and have a lot of fun.

[1] I’ve worked with these guys occasionally for the last six years or so, although my schedule is busy enough it’s taken me this long to find a slot to present one of these ACM seminars. If you’re looking for a good training center in Upstate New York, this is one of the best training centers I’ve worked with in all my years as an MCT.
[2] People with little or no ETL experience, or some past DTS experience.

A Good Deed

Have you ever noticed that the unique circumstances of our lives sometimes make it easy for us to make a big difference in someone else’s lives? Each one of us has unique skills that can benefit others; sometimes we can spend ten minutes to save someone else ten hours, or even more.

It’s easy to look at Bill Gates or other insanely rich people and think “how easy it must be for him to make a difference.” But how often do we look at ourselves and think the same thing?

When was the last time that you did a good deed, and gave a little bit of yourself to make a big difference to someone else?

Blogger Weirdness

Apparently Blogger keeps track of the date when you start working on a blog post and uses that date - and not when you actually publish the post - to track the post in your blog.

Now where do we post blogger bugs?

Anyway, if you're interested in a new post titled "On Competition" that I published a few minutes ago (I'm taking advantage of airport time today, you see) you should look at June 21st.

Really.

The Company You Keep

I went to a birthday party yesterday - one of my daughter's schoolmates had just turned four, so the children (and parents) of the class all gathered at My Gym for the rituals of gift exchange and cake consumption.

During the party I was talking to the father of the birthday boy, and was surprised to learn that he and his family were going to be moving to Atlanta in the next few days, and not returning to our wonderful school when the school year starts up next week.[1] We spent some time discussing the move and its motivations, and although I was sad to see them leave, I certainly understood the reasons behind the move.

Apparently the father is a forensic psychologist, who works at a large maximum security prison in the area. For 50 hours each week[2] he talks to, councils, and analyzes the worst of society's worst, and that was starting to take its toll, not only on him, but on his family as well. Although they did not really want to move, it was the least of the available evils from which they could choose.

This reminded me strongly of an article I read somewhere (I think it was the New York Times, but it probably doesn't matter at this point) about a married couple who were in the process of going through a divorce. The husband and wife were both highly educated and were educators themselves, but they had chosen different specializations. The husband taught graduate students at a large university, while the wife chose a career working with young children and specialized in early growth and learning. They were divorcing because - despite their shared background and training - they could no longer relate to each other. He spent his days around PhD candidates; she spent hers around kindergartners. And each one had as a matter of necessity refined his methods of communication to match the audience and the context[3] in which he spent his time.

By this point you're probably asking "what on earth does this have to do with business intelligence?"

As a wise man[4] once said, "the answer is within the question."

Business intelligence is about business. It's not about technology. At some level we all know this, or at least we've heard it or read it somewhere before. But because the application of technology is crucial to business intelligence - and because it is often because of that technology that we are involved in business intelligence - it is very easy to forget this fact that forms the foundation of what we do. Although the technology is what makes business intelligence possible, if we ever lose sight of the business drivers that necessitate the projects on which we work and the business problems which our projects are designed to solve, then we've already placed on foot firmly on the path to failure.

As I mentioned already, we all know this already, right? But then why are so many BI projects run by technology professionals, and not business professionals? Why is there not a stronger involvement by business decision makers and process/domain experts on BI projects? Why does it still seem to be about technology and not about business?

I think the questions are probably more interesting than any answers that I could provide, so I'm just going to leave them dangling there as food for thought. But the next time that you find yourself talking about memory buffer optimization or index tuning or aggregation strategies or proactive caching, make sure step back - take two steps back just to be sure! - and remind yourself of the business problem you're solving through the current task. And if you can't remember, then it's about time to take a break and see how you got to where you are, and ask yourself if you aren't headed for a nasty divorce yourself...

[1] If you are a parent of school age children you probably understand the glee with which I typed the words "last week." ;-)
[2] Which is the majority of one's waking hours, when you think about it.
[3] Which, as we all know, is practically everything when it comes to determining meaning.
[4] Not really - it was really me.

SSIS == Fresh Peaches

Here in Upstate New York, local peaches are at the height of their season. I bought a bushel of them last night at a little family-run produce stand, and today their delightful aroma is filling my house. For breakfast this morning I made waffle cakes[1] for the kids, and for my serving I chopped up a few peaches and threw them onto the batter on the griddle, so I ended up with peach waffle cakes. Instead of topping them with maple syrup, I chopped up a few more and poured them over the top of the stack. The peaches inside the waffle cakes were hot and slightly caramelized, and the peaches on top were cool and fresh - it was a delicious combination. And while I was eating, I was flipping through one of my favorite cookbooks[2] finding more delicious ways to use these incredibly flavorful fresh peaches.

Life is good.

And, of course[3], these peaches and the vast range of amazing things that you can do with them got me to thinking of SQL Server Integration Services.

How so? Well first let me introduce you to my brother. For this discussion we'll call him Jason. Jason doesn't cook. The height of his culinary expertise is tater tots. He pours some from the bag onto a cookie sheet and puts them in the over following the instructions on the bag. When he's feeling particularly adventurous, he might put a little ketchup on the plate once they're done.

No, I'm not making this up.

For Jason, the perfect peach is the peach that comes in a can, swimming bisected in syrup with a few of its closest friends. Open can, pour into bowl, eat. What could be better, Jason thinks? All of the effort that has to go into fresh peaches, including peeling, pitting, cutting and (in the worst possible scenario) cooking as part of some larger dish, is not even worth considering. Why bother? Even if the end product is something that you could not get with canned peaches, why even consider it if you can have a bowl of canned peaches for almost zero effort?

Now you see, don't you?

In the world of ETL tools and platforms, Data Transformation Services is canned peaches. It's really incredibly easy to get started and to do very simple things. But if you want to do something truly amazing[4] you're pretty much doomed from the start. For some circumstances you simply "can't get there from here" and even if you can, the end product is never quite what you hoped it would be.

SQL Server Integration Services, on the other hand, is fresh peaches picked at the peak of freshness. Practically anything that you want to do, you can do and do well. It might take a little extra effort to do things right, but if you take your time you can make the most delicious[5] recipes imaginable.

But it's not always as simple as if you were using DTS[6] and because of that, people like my brother sometimes throw up their hands in disgust. I think this is one area where the SSIS product group needs to spend a little effort before they think about deprecating DTS entirely. As I've said many times before, I love SSIS. I really do. And I'll do just about anything[7] to avoid using DTS for anything but the most trivial tasks. But before it can be the right ETL tool for my brother, SSIS needs to be ready to get poured into a bowl, ready to eat.[8]

I'm somewhat ashamed to admit this, but I have yet to look at the Import and Export Wizard in SQL Server 2008. Nine times out of ten I'd rather build a package by hand than use a wizard to get me started, so I haven't made it a priority. But these delicious peaches which are calling my name made me think of my brother and of how the SSIS tools need to mature before we can get rid of those nasty canned peaches once and for all.

And now I have a bushel of peaches to peel and prepare, so please forgive me...

[1] Basically this is waffle batter, made with whipped egg whites to be extra fluffy, but cooked on the griddle in a pancake-like form factor, instead of being cooked in a waffle iron. If you want the recipe, just ask.
[2] The Pie and Pastry Bible by Rose Levy Beranbaum, if you must know.
[3] What? You didn't see this coming?
[4] You should see the Peaches and Cream Tart recipe I have planned!
[5] And high performing, maintainable, scalable, etc. Everything a great dessert needs to be.
[6] Canned peaches, remember?
[7] Yes, Lynn...
[8] So to speak.

But What About Excel 2007?

That last post on loading multiple Excel files is probably going to be less useful if you have files in the new Excel 2007 XSLX file format. The built-in Excel source component in SSIS only works with the "classic"[1] XLS file format. Fortunately, Microsoft has made available for download a set of 2007 Office System Driver: Data Connectivity Components that allow you to connect to XSLX files via OLE DB or ODBC.

There's also a good article on this on the ssis.wik.is web site, with a few more details than you'll find on the download page.

Enjoy!

[1] I think that "classic" has a much nicer ring to it than "legacy"[2]
[2] Which was, of course, the new "old"[3]
[2] Which was, of course, the old "old"

Loading Multiple Excel Files with SSIS

Now that I have the time to look up and see what's going on in the world around me, I'm trying to spend more time on the SSIS Forums on MSDN. This is a great community-driven resource where lots of really bright and dedicated people (and sometimes me, too) help the growing body of SSIS developers solve their problems. And looking around the forums I've found several posts (including this one) asking about how to load data from multiple Excel files using a single SSIS package. This is an exceptionally easy task for SSIS, but I could not find[1] any online resource that walked you through step by step.

So here you go: Step by step instructions on how to load data from multiple Excel files with a single SSIS package.

1. Create an SSIS project in Visual Studio


2. Create multiple Excel files in the project folder – make sure that they have an identical format but different data

3. Add an Excel connection manager to the package – point it at of one Excel file

4. Add a Data Flow task to package

5. Add an Excel source component to the Data Flow

6. Configure the Excel source to use Excel connection manager and the correct worksheet (Sheet1 in this example)

7. Click the Preview button to be certain that it works

8. Build the rest of data flow – in this case we’re simply counting the rows, but you might want to put the data into a database or something ;-)

9. Make sure the Excel file is not open – this will cause errors every time if Excel has the file open when you try to load it
10. Execute the package

11. At this point we know that we have everything built correctly to load a single Excel file – now we need to modify the package to work with an arbitrary set of files
12. Copy and paste into notepad at the ConnectionString property of the connection manager – it should look something like this: Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\Projects2005\SSIS_Excel_Loop\SSIS_Excel_Loop\email_book_01.xls;Extended Properties="Excel 8.0;HDR=YES";
13. Create a package variable called FileName and give it the value of the file name from the ConnectionString

14. Update the connection manager’s ConnectionString property to be based on an expression that uses the FileName variable to duplicate its tested value - "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + @[User::FileName] + ";Extended Properties=\"Excel 8.0;HDR=YES\";"
(Note how we're using the backslash "\" character to escape the double-quotes that are part of the connection string without breaking the expression!)

15. Run the package again to make sure it still works – By testing after each change you will catch errors early, and not need to try to figure out where you went wrong if you get to the last step and things don’t work

16. Add a Foreach Loop Container to the control flow and drag the Data Flow task to be contained within it

17. Edit the Foreach Loop Container so that it loops through all xls files in the folder and retrieves the fully qualified file name and maps that value to the FileName variable

18. Run the package again – Voila!! We've now loaded multiple Excel files with a single SSIS package

19. If you're feeling particularly skeptical (after all, how can you be certain that really looped through the files?) you can easily add a Data Viewer to the data flow and look at the data being loaded from each Excel file

And there you have it. It's not exactly a complicated scenario, but it is very common, very "real world" and it shows off some of SSIS's most powerful features, namely variables and property expressions. We could also easily update this example to use a configuration to set the path of the folder through which the Foreach Loop Container enumerates, but I'll leave hat as an exercise for the reader.

[1] If you do find an existing online resource that goes through this step-by-step, please don't tell me. Loading images using Blogger is painful, and I'd hate to think I went through that torture for naught. ;-)

MSDN Beginner Developer Learning Center

I also considered these alternate titles:

• Enough free beginner training to choke a rhino
• Holy video training, Batman!
• I've done up and polluted my sample, dag nub it all

My friend Kris is starting to go through the SQL Server video training I shared with him (and with you yesterday) and is more excited than ever. That's the good news. The bad news is that I let him know that I was using his journey of self-discovery[1] as blog fodder. So he's been sharing with me other goodies he's found related to beginning .NET development. That's the bad news. Once your experimental test subjects know that they're participating in an experiment at all, you've totally skewed[2] your tests results. Bah.

Not that this is an experiment, of course. [3]

Anyway, with help from Kris, I've found what is probably the best starting point for beginning developers who want to learn the Microsoft development platform: The MSDN Beginner Developer Learning Center. Here's what you get:

1. The launch page, with a bunch of links: http://msdn.microsoft.com/vstudio/express/beginner/
2. A "learning path" page that lists out all of the lessons, in order, for Windows and Web development paths: http://msdn.microsoft.com/vstudio/express/beginner/learningpath/
3. Individual lesson overview pages, such as Introduction to CSS and Visual Web Developer Express Feature Tour and Handling Exceptions.
4. Some of these lesson pages have PDF lessons to download. Some of them have videos to download or stream. Some have full lesson packages, including videos, transcripts, starter project files plus study guides and exercises.

Wow! Not only is the content available and free, it looks like some real thought has gone into its organization and planning. I've put together basic developer training paths in the past, but have never before seen a pre-built repository of learning resources like this available freely online.

Now let's see what Kris will do with it, while I try to think up another experiment...

[1] Before you can know code, you must know yourself, grasshopper. Or something like that.

[2] Or something that rhymes with skewed, anyway.

[3] Why doesn't HTML have a diabolicallaughter tag?

Starting Anew

I've been doing this software stuff for a long time. I started programming when I was eleven, which was longer ago than I care to admit, and have been building software on the Microsoft platform full time for well over ten years. One of the periodic revelations[1] that I've been having during this time is that "the more you learn, the more you realize you don't know."

This fact has been driven home lately in a new and different way. I have a friend[2] who is making a career change from IT help desk support to software development. He's got a good head on his shoulders, but this is a pretty big change, and he's got a long road ahead, so I've volunteered[3] to help him get up to speed on the big bad world of Visual Studio, SQL Server and .NET. [4]

So I've had two main thoughts going through my head for the last few days as this friend (let's call him Kris[5]) has been asking questions via IM. These thoughts are:

1) WTF was I thinking?
2) Wow! There are a lot of cool resources out there that I never looked at before!

I'm trying to focus on the second one. ;-)

Anyway, after you've been building software professionally for long enough[6], you pretty much stop looking at the 100- and 200-level introductory resources, and I have not been tracking all of the amazing things that Microsoft has been doing in this arena. I've started Kris looking at SQL Server (it is me after all - what did you expect?), and have found lots of great little gems that I'd never bothered to look at so far. The ones that have me smiling today are two series of free videos and webcasts that Microsoft has put together to help people get started:

1) Video Series: SQL Server 2005 Express Edition for Beginners: This is over seven hours worth of video training that starts with "what is a database?" and works its way through stored procedures and full text search, touching on web development, BI and security along the way.

2) Coding Basics: Use Visual Studio and SQL Server Express Editions: This is an insane number of 100- and 200-level webcasts covering web development, database development, game development and more.

Needless to say, Kris is both delighted and overwhelmed. And I am too - if there had been resources like these online back when I was learning programming... Well, since there wasn't really much of an internet back then and I was using either an NCR mainframe or an Atari 400, I guess it would not have done me much good . But you get the idea.

So... As Kris travels down this path of discovery (and I swear every time I see that blinking on the taskbar) I'll use this space to post new goodies that I find for beginners.

[1] No, don't look for Matthew's periodic table of revelations - these are just things that I keep realizing over and over again. It sucks to get old...
[2] Sort of the husband of a third cousin, once removed - something like that.
[3] Yes, you may start laughing at me now.
[4] Yes, please keep laughing. No need to stop.
[5] We'll call him this mainly because it is his name
[6] And after you stop teaching intro courses when the training market doesn't support half your regular daily consulting rate

Webcast Available on Channel 9

I just got am email from Lynn Langit, one of the hosts[1] of Microsoft's geekSpeak show, that my show is now online on Channel 9, here: http://channel9.msdn.com/Shows/geekSpeak. Enjoy!

[1] I have a feeling that Lynn is responsible for the delightful synopsis of the show at the URL above. If I knew for a fact, I would identify her as "the evil host" but since I'm not certain, I'll withhold judgment. For now.

Salary Man, or "I Remember Paychecks"

This isn't truly news any more, but it is one of the main reasons why I've been so dark lately. After nearly five years of working as an independent consultant and trainer, I've recently accepted an offer of full time employment from Configuresoft, Inc.

I'm working with Configuresoft as a Data Architect, focusing on building a business intelligence product offering to complement their flagship Enterprise Configuration Management product as part of their "Configuration Intelligence" initiative. They have a great product and a great team, and I'm really looking forward to the opportunities that this change will bring.

Now that I'm pretty much settled in to my new role, hopefully I'll be able to start posting here and in the MSDN SSIS Forums again. Let's just hope that life stops interrupting quite so much...

SSISIS or "Reverse Viking Invasion"

That's "SQL Server Integration Services In Stockholm" of course - Sweden, here I come!

I'm in the process of putting the finishing touches on planning for a two-day "SSIS advanced topics" seminar in Stockholm, Sweden on October 11th and 12th for RBAM AB. I've never been to Sweden - although many of my favorite bands are from Sweden - so I'm awfully excited, and understandably so. The seminar will cover best practices, scripting and component development and other "advanced topics." I'll post more information here when I have it to share...

...but the strangest thing is that the thing that has me most excited is not the seminar, and not the opportunity to visit Sweden, but is instead the fact that I'll be flying back to New York on the top deck of a 747. I've never flown in a 747 before at all, and for some reason the thought of flying in the upper deck in the front of the plane has me as giddy as a a schoolboy...

Putting Things in Perspective

A close friend of mine was recently visiting (I think this was late June, if that puts into perspective how insane my schedule has been of late) and I was playing some new music for him. He lives out on the west coast, so it's rare that we get a chance to catch up in person.

Since the last time we've sat down together, he's had surgery for Acoustic Neuroma, which is a potentially life-threatening (although non-cancerous) tumor on the acoustic nerve. His (also potentially life-threatening) surgery was successful; the tumor was removed, but he lost all of his hearing in his right ear.

While we were listening to the music, I asked him what he thought. As part of the conversation that followed, he said:

It's in mono. The whole world is in mono. But it's better than being dead.

Kind of puts things in perspective, doesn't it?

I'll try to keep this in mind while I wrestle with SSIS deployment stuff today. ;-)

Webcast Available on Demand

Back in July I did a live webcast on SQL Server Integration Services as part of Microsoft's geekSpeak webcast series. I had a great time and the feedback we received was very positive[1], and now the webcast is available on-demand here: http://msevents.microsoft.com/CUI/WebCastEventDetails.aspx?culture=en-US&EventID=1032344832&CountryCode=US. Check it out, and submit your feedback on the geekSpeak blog. There has already been some talk of having a follow-up SSIS session with a more targeted set of topics and questions, so if there is something specific you'd like covered, let Glenn and Lynn (the hosts of geekSpeak) know and maybe I can get back on their speaker schedule with a more focused agenda and set of demos.


[1] Although the one hour time limit proved to be much tighter than I'd anticipated, and I ran very short toward the end of the show.

Swearing a Lot Today

I've been thinking today of making up a t-shirt that expresses my opinions on Windows environment variables, but I wouldn't be able to wear it anywhere. It's been one of those days, where 6.02x10^23 things have gone wrong. None of them was noteworthy on its own, but when viewed in the aggregate, they made for a very frustrating day.
[1]

As you can see, most of my mental anguish today was caused by Windows environment variables. Specifically, I've been fighting with the SETX utility, which I use to define environment variables for indirect XML configurations for my SSIS packages. I've been using this setup for many months, but have only recently discovered the SETX utility, which is new to Windows Server 2003 and Vista; before now I've been using Registry scripts to define the environment variables, an approach which has drawbacks and gotchas of its own.


In any event, I define a package variable in my master packages which stores the root deployment folder - this (when combined with configurations and property expressions) allows me to deploy my packages to arbitrary file system folders as my clients' environments and corporate policies dictate. It works quite nicely once the environment variables are created. But I set this variable with an environment variable configuration, and based on the logic in my packages (which is now quite well established, and would be a major pain to revise) the path stored in the environment variable needs to end with a trailing backslash, like this:


C:\DeploymentRootFolder\


It works. It's been working for months, because I have a batch file that contains a line that looks a lot like this:


SETX /M SSIS_Root_Folder C:\DeploymentRootFolder\


That's right - I have an environment variable that contains my path in the format my packages require, and I can verify it by running the SET command and testing:


SSIS_Root_Folder=C:\DeploymentRootFolder\


Yay! It works as desired and as expected!


But today I deployed my packages to a folder that contained a space in the path. (How this scenario could have remained untested for so long is only my own fault, I know.) I updated the batch file to contain a line that looked a lot like this:


SETX /M SSIS_Root_Folder C:\Deployment Root Folder\


When I ran it, I got back output that looked quite a lot like this:


ERROR: Invalid syntax. Default option is not allowed more than '2' time(s).Type "SETX /?" for usage.


WTF, you ask? WTF indeed! Ahhh... that must be it - spaces in the path. Silly me. I'll just wrap that path in quotes and all will be good once more. I next updated the batch file to contain a line that looked a lot like this:

SETX /M SSIS_Root_Folder "C:\Deployment Root Folder\"


It worked - hooray! Of course, how could it not work? This is how Windows command line utilities always work, right?


But, of course, when I attempted to run my packages, it did not work. In fact, it did the opposite of work. When I got around to running the SET utility again to check the values actually stored in the environment variable, I found this:


SSIS_Root_Folder=C:\Deployment Root Folder"


WTF? Oh! I see, it's treating the backslash as an escape character and then treating the ending quote as part of the variable value. Not quite what I'd expect, but I can see the logic. Easy enough to work around, right?


SETX /M SSIS_Root_Folder "C:\Deployment Root Folder\\"


Finally!


SSIS_Root_Folder=C:\Deployment Root Folder\


But this afternoon either I was doing something subtly different (the more likely explanation, I know) or else this wasn't actually working. I ended up with two lines in my batch file that looked like this:

SETX /M SSIS_Trailing_Backslash \
SETX /M SSIS_Root_Folder "C:\Deployment Root Folder"%CIA_Trailing_Backslash%


The key, of course, was the backslash as the escape character and the quotes to support spaces in the file names, but it was a long, aggravating process to make it work correctly. Hopefully tomorrow will be a better day...


[1] This is a none-too subtle homage to xkcg, for those of you who paid attention to my last post ;-)

xkcd

Oh.

My.

God. [1]

I've found the online comic strip that I would have written if I were funny and had a (very) small degree of artistic talent: xkcd. I stumbled across the gem a few evenings ago and my sides still hurt from laughing so hard as I flipped through the comic archive. This is the one I found first:

And it only gets better from there.

From the volume of xkcd resources I've found online already, I'm behind the curve here, but I would be remiss if I did not do my little bit to spread the word. Check it out.

[1] Everyone whose brains continued on at this point with the line "Becky, look at her butt" need to stop reading and seek professional help.

Snag Better Bugs

I've been using TechSmith's SnagIt application for the last year or so [1], and it's been a great help when writing articles and documentation for my BI projects. If you've never seen it (and are too lazy to click on that link before continuing) you should think of it as PrintScrn++ or even PrintScrn#, since it's so darned slick. It has enough features and options that I'm not going to try to list them here (just visit the product page for that) but I just discovered one that truly opened my eyes: SnagIt can now natively output to Team Foundation Server.

That's right - capture your image, edit and tweak as necessary, and publish it to TFS, either as a new work item or as an update to an existing work item. They even have a video showing it in action.

How cool is that? It's never been easier to include screen shots with bug reports, feature requests and other work items. Being a Microsoft developer just keeps getting better...

[1] Thanks to my friend Chris Randall who pointed me to it - it's proven to be just about the best $39.95 I've ever spent.

Katmai July CTP Available for Download

No, I'm not dead, but people keep throwing poop at the fan (and projects and other offers at my schedule) and I've pretty much been overcome since TechEd finished last month.

But this news is too big to ignore! Microsoft has made available a new CTP release for SQL Server 2008. You can download it from the Microsoft Connect site here: https://connect.microsoft.com/SQLServer/content/content.aspx?ContentID=5395

Hopefully I'll have a chance to catch up on things (including things like blogging) now that life seems to be calming down a little...