Tuesday, June 5, 2007

ETL Lifecycle - An Interesting Thought

During this ETL Framework session, Larry Barnes (the person with whom Donald Farmer is co-presenting) raised an interesting point:
ETL processes tend to stick around forever, because people are horrified to
touch them.

He used the image of a Perl script that runs on an Oracle server pulling data from a Sybase database and loads it into Informix (can't you just picture that?) but I have no trouble visualizing the same thing happening with SSIS as well.

This may not be technical or of interest to anyone but me, but it strikes me as being profound in some way. I tend to be very aggressive in my commenting, annotation and documentation, but this really drives home how important this is for SSIS. In addition, this really demonstrates the importance of having a well-documented and understood process for making changes, moving from dev to test to prod just like with traditional application code, because only through this process will people truly be comfortable making changes to such business-critical code.

No comments: