A Developer Rants for a Bit

The ERP consulting world is consistently undermined by many of the very people who participate in the profession. They undermine those of us who take pride in delivering value for our clients by failing to charge a fair amount for their services. They undermine those of us who perform well with the consistently poor quality of their work. They undermine those of us who attempt to show our client that we are as humble as we are talented, and just want to be members of the team by treating client employees as stupid and acting like rock stars.

As a consultant searching for their next project, it angers me that I consistently have to pay for the sins of consultants past. It pisses me off that many potential clients, because of past experience with bad consultants, would rather pay for five, low-dollar, cheap consultants than a mid-priced consultant who will easily perform the work of the five low-dollar losers. (Low dollar consultants are low dollar for a reason.) It frustrates me that those potential clients who have money to spend would rather overspend on a gaggle of under-qualified big four consultants who show little ability outside of running up giant dinner bills every evening, flying first class without an upgrade, and walking around with their noses stuck up in the air. (Many potential clients believe that hiring a large firm will alleviate the problems of hiring independents ... it doesn't work that way.)

So, here's my helpful guide for ERP customers to use when evaluating what type of help they need for staffing their ERP installations and upgrades:

1. Hire a good PM from a boutique firm or an independent with impeccable references. Avoid Big Four PM's like the plague and do NOT choose this position from in house. A good PM will occasionally need to piss everyone off and you don't want that baggage hanging around for too long after you go live. Make sure that you give your PM absolute control over everyone working on the project. Yes, that means consultants and employees.
2. There's nothing more important than staffing your projects with quality business analysts from day one. The configuration decisions made at the start of an ERP project are the most important decisions to be made. If you used a boutique firm to find your PM, you should use the same firm to begin staffing your project. By using consultants from these small firms, you are far more likely to find quality people who have worked together in the past, and work together well. Otherwise, rely on your PM for recommendations of people or firms they have worked with in the past. Do NOT, under any circumstances, use a recruiter who has no real knowledge of the product for which you need help.
3. The same goes for your technical consultants. Get them in house early and integrate them immediately with your technical staff. A good consultant will want to be a part of the team and will want to learn about your current processes and will want to begin teaching right away. Again, favor consultants from your PM or BA's firm or network.
4. If you are using a small boutique firm, and you find you need a few more consultants, rely on the best of your onsite consultants for recommendations of talented independents. Once you have people on site, I would strongly advise against advertising to fill consultant spots. You want people who are known quantities ... not best guesses. 
5. Do not, under any circumstances, use one of the big four consulting firms. Yes, the big four firms have numerous talented consultants, but, they have a multitude of consultants and there's no way to know for sure the quality of the people you are getting. Boutique firms are almost totally dependent upon word of mouth for their survival ... big firms rely on branding and branding doesn't get the software live.
6. Do not, under any circumstances, staff your project with consultants from the same vendor who sold you the software. Again, many of their consultants are talented and want to deliver for their clients, but ultimately, they have a conflict of interest. Does their loyalty belong to the client or to their employer, the software vendor?
7. Do not, under any circumstances, staff your project with low dollar consultants. People who are willing to work for the smallest amounts generally have good reason to do so. It's the only thing they have to sell about themselves. A quality mid-price consultant will generally demand two to three times the rate of a low dollar guy, but will deliver at least FIVE times the value. Count on it.
8. If you run into problems with the project, STRONGLY resist the urge to bring lots more consultants in house. In fact, consider getting rid of a few consultants who aren't performing up to expectations and further empower those who are doing the best work for you. You can watch them rise to the occasion.

I could go on forever with his, but I'm tired, and I'll be coming back to this topic over and over.

Is there anything your fellow consultants do to piss you off?

A Developer Extols the Virtues of Groovy #1

Perhaps the most common task for any Lawson technical consultant is that of the simple, outbound interface. These interfaces are generally written in the Lawson 4GL and will iterate over the Lawson database, perhaps perform a calculation or two, and then generate an output file, usually in the CSV format. The file is then sent to a vendor for processing. Seems easy enough, right?

Now, writing these interfaces in Lawson 4GL is not difficult by any means, and there are some compelling reasons to write these interfaces in the 4GL. All programs written in the 4GL will appear the same to a user as any other Lawson program. Programs written in the 4GL can be run using Lawson's job scheduler. The Lawson CSV file processing API makes generating CSV files a snap in COBOL. Still, although the process is not difficult, there are a number of steps that have to be taken that make the process more cumbersome and time consuming than need be. The put it simply: the 4GL dampens productivity.

When I build a Lawson interface, here are the steps I typically take ...

1. I create the program metadata in Lawson using the Lawson pgmdef utility.
2. I paint a "report" with most of the fields needed on the resultant output file with Lawson's CASE tool.
3. I paint a form with any parameters needed using the CASE tool.
4. I define the format of the output CSV file using Lawson's workdef utility.
5. I generate the COBOL and 4GL source files.
6. I clean up formatting issues in the .scr and .rpt 4GL source files.
7. I clean up the revolting COBOL code generated in the PD and WS files. (If you are going to generate COBOL code that needs to be modified, why would you abandon best practice? Yes, COBOL is overly verbose, but why abandon readability in COBOL code when it's one of the few things the language has going for it?)
8. I modify the COBOL logic to process any database calls that were not generated correctly, and I insert the logic to create the CSV file.
9. The program is tested.

When completed, the COBOL code is awfully long for what is a very simple process ... on the order of several hundred lines. The code is bloated not just for your typical COBOL reasons, but because of Lawson's use of a data access API rather than SQL for accessing the database. Yes, the Lawson data access API has a few benefits (such as filtering all access according to user-defined Lawson security rules). Still, these benefits are dwarfed by the loss in productivity that results from being unable to join SQL tables. (Why exactly are they using an RDBMS?) Additionally, when writing a 4GL batch program, Lawson forces you to generate a report as well. This isn't always a bad thing with some data formats, but in a CSV file, the file is the report more often than not.

I wish I could place the PROCEDURE DIVISION from a typical Lawson 4GL generated COBOL interface in this post, but it would make this already long post ... well ... longer. Not to mention, I'd need the permission of one of my clients. I'm not sure that people who came to programming after the age of COBOL really understand just how verbose and unwieldy COBOL can be. Not to mention, the ultimate irony of the 4GL is that it removes so much verbosity from COBOL code thanks to the CSV file API, only to add it all back, along with a complication of the logic, thanks to the ridiculous data access API. OK. Enough about that. Hopefully I've conveyed just how long, verbose, and inefficient the typical Lawson 4GL solution is for creating a typical Lawson 4GL outbound interface.

Now, we look at a solution for the same type of program in Groovy. I'm not going to list all the reasons why Groovy is better than COBOL. I would assume, excepting the opinions of a few stray dinosaurs, there isn't really any debate here. Still, the conciseness of the Groovy solution is astounding when compared to the COBOL. On top of that, the Groovy solution is so easily digestible, that it is far more "readable" than the COBOL.

Let's look at the Groovy solution then I'll explain a bit why I love it to the old COBOL dudes. Here's the code:

import groovy.sql.Sql

def sql = Sql.newInstance("jdbc:oracle:thin:@the.ip.address:1521:PROD", "username", "password", "oracle.jdbc.OracleDriver")

def output = new File("regular_pay.csv")

def query = '''
SELECT a.company, a.employee, rtrim(a.last_name), rtrim(a.first_name), 
       a.date_hired, b.birthdate,
       nvl(sum(d.hours),0) rg1_hours, nvl(sum(d.wage_amount),0) rg1_wages,
       nvl(sum(e.hours),0) rg2_hours, nvl(sum(e.wage_amount),0) rg2_wages,
       nvl(sum(f.hours),0) rg3_hours, nvl(sum(f.wage_amount),0) rg3_wages,
       nvl(sum(g.hours),0) rg4_hours, nvl(sum(g.wage_amount),0) rg4_wages
  FROM lawson.employee a, lawson.paemployee b, lawson.paymastr c,
       lawson.prtime d, lawson.prtime e, lawson.prtime f, lawson.prtime g
 WHERE a.company = 100
   AND b.company = a.company AND b.employee = a.employee
   AND c.company = a.company AND c.employee = a.employee AND c.check_date >= (SYSDATE - 365)
   AND d.company (+) = c.company AND d.employee (+) = c.employee
   AND d.check_id (+) = c.check_id AND d.pay_sum_grp (+) = 'RG1'
   AND e.company (+) = c.company AND e.employee (+) = c.employee
   AND e.check_id (+) = c.check_id AND e.pay_sum_grp (+) = 'RG2'
   AND f.company (+) = c.company AND f.employee (+) = c.employee
   AND f.check_id (+) = c.check_id AND f.pay_sum_grp (+) = 'RG3'
   AND g.company (+) = c.company AND g.employee (+) = c.employee
   AND g.check_id (+) = c.check_id AND g.pay_sum_grp (+) = 'RG4'
 GROUP BY a.company, a.employee, a.last_name, a.first_name, a.date_hired, b.birthdate
 ORDER BY a.company, a.employee
'''

sql.eachRow (query) { row ->
    output << "${row.toRowResult().values().join(',')}\n"
}

Now, how short, sweet and to the point is this code? Why do I love it so? Well, I love it because ...

1. I love it because all I have to do is build my SQL query using a tool like TOAD, and that's all I essentially need to build my CSV file. I just select the fields in the order I want them to appear on the file. A good experienced Lawson consultant can build an outbound interface in the time it takes to write an ad hoc query.
2. HEREDOCS!!! Yes, with Groovy, I can create an easy multi-line string using the here doc format. This means that all I have to do is paste my query into the code. I don't have to put the entire query on a single line that scrolls of the screen. I don't have to put the query into a number of different string literals and concatenate them together. I can format SQL query in the most readable format possible.
3. I love the obviousness of the code. When I call the eachRow() method with my query, it's obvious that I'm processing each row through the closure.

The ship sailed on the productivity of COBOL a long, long time ago. Thanks to modern languages like Groovy, Lawson outbound interfaces can be created in the time it takes to build a SQL query. The only problem is convincing your Lawson clients to abandon the 4GL. More on that later.

A Developer's Many Hats

As a working consultant, you will have good clients and you will have bad clients. (I should add that if you work for a consulting company rather than on your own, your boss will make this differentiation based solely on how quickly they pay their invoices.) You will have work that is fun and you will have work that is a drudge. You will be excited almost beyond words at times, and at other times, you will be bored to tears. One thing I've learned is that when the client asks you to wear multiple hats, even if you aren't flexing all of your muscles under any of the hats, it's a lot harder to be bored with your work.

At one of my recent clients, I've had to wear the following hats:

Oracle DBA - I got my foot in the door by migrating a database from Oracle 9i to 10g. I've also worked in an admin capacity supporting their Lawson Oracle installation, including the development of database replication and backup processes.

Shell Script Developer - Since this is a UNIX environment, I developed a number of shell scripts to automate some of the common DBA tasks they'll be using. Later during my time at the client, I would use shell scripting in conjunction with some of the other technologies.

Perl Developer - Anyone who's worked for long enough in UNIX environments will find themselves using Perl. It's inevitable. Not only did we use Perl for system admin scripting, but we used Perl to develop the type of interfaces normally written in the 4GL (thanks to the wonderfully simply DBI package). For one project, we even used Perl to perform some post-processing on a report generated by a 4GL program. Personally, I love Perl, but not everybody agrees!

Lawson 4GL Developer - As I have on every Lawson technical project, I've written the usual assortment of reports, interfaces and bolt-on systems using the Lawson 4GL. You want hear me say a lot of positive things about the 4GL, but sometimes, it's the option that just makes the most sense in a Lawson shop.

Groovy Developer - For one interface, the vendor required that the data be placed in the format of a Microsoft Excel spreadsheet. The interface included multiple worksheets within the Excel workbook. The meat of the interface was written in Perl to retrieve the data and generate a CSV file for each worksheet in the spreadsheet. Then, the Perl script called a short Groovy script that built the spreadsheet. The Groovy script was written to be reusable. It takes a list of CSV files and an Excel template file. It then copies each CSV file in order to each worksheet of the spreadsheet, in order, using the JExcelAPI open source project. (Yes, I know about the Perl::WriteSpreadsheet module, but that was not an option.) I also wrote a Groovy process that took an Excel spreadsheet (again using JExcelAPI) and applied security and user changes to the LAUA database in the GEN product line. They wanted me to make those changes, hundreds of them, by hand. Yeah, right. Like Larry Wall said, laziness is one of the great virtues of a programmer.

PHP / MySQL Developer - The client upgraded from Lawson's old Time Accrual system to the Leave Management system. They had an employee facing system written in PHP and MySQL that combined data from their Kronos time clocking system with Time Accrual data from Lawson. I made changes to the MySQL database, the PHP scripting, as well as the Lawson data export process.

Microsoft Access Developer - As a result of new government regulations and an external supply chain system, we needed a small database with querying and reporting attached to it. I was instructed to develop it in Microsoft Access, although I rewrote it later as a bolt-on in the Lawson 4GL.

I'm sure this all reads a little dry, but the work days go a little quicker when you spend your time working with  different technologies, even when you are already overly familiar with some of them. If you're given the chance to wear a new hat, take the opportunity.

A Developer's Blog

A Developer's Blog is a place to discuss the problems of Old IT Think and the limitations of Old IT technologies. We can't convince everyone to move forward with new technologies unless we can make a convincing case that the old ones need to be thrown out. (And for those New IT folks with no experience in Old IT, perhaps we can show them the good reasons these technologies were used in the first place.)

A Developer's Blog is a place to discuss the difficulties of moving from an Old IT job to a New IT job ... especially without a major drop in your standard of living. What is the best way to shed those golden handcuffs?

A Developer's Blog is a place to discuss New IT technologies that Old IT folks looking to advance should be learning. We will focus on the problems they solve and specifically, how they overcome problems with Old IT.

A Developer's Blog is a place where I will be detailing my learning of New IT technologies, along with the occasional tutorial. I will compare and contrast these technologies with Old IT technologies and otherwise, I'll just generally evangelize on the things I find cool about them.

A Developer's Blog will be a resource for reviews of the best programming books and web sites. After all, these are the resources most of us use when we start to learn something new.

A Developer's Blog will be a resource on the process of learning itself. We will learn the best ways to learn the new technologies.

A Developer's Blog will attempt to motivate Old IT folk into learning new technologies. We don't want to see anyone stuck in Old IT.

Join us! Comment! Contribute! Help! Please!

A Developer's Career

College had become a struggle for me for several reasons. (The work was not among them, I loved then, as I love now going into a classroom to learn.) To get money to pay for school, I took a full time job as a microfilm clerk. Not only did I only make enough money to be a part time student, but my shift was constantly changing from 1st to 3rd to 2nd to 1st which left me with two quarters where I had to withdraw from all of my classes with little to no warning.

Our top local IT shop made a splash when they announced they were hiring a hundred programmers at once. Although I didn't agree with all of their decisions (for instance, they were not necessarily looking for people with any experience in programming, just people with an aptitude for it ... they were going to offer six months of training run by the same local college where I had been taking classes), I relished the opportunity to finally become a professional programmer. Tens of thousands of people applied for the job, and I was selected. Training was run by the very same professors that had been teaching me the past several years and I exceled through the classes and was ready to hit the ground running.

My first employer was a large credit card processor and I was employed to support a system running on the usual array of technologies in a large, "Big Iron" shop. (MVS, JCL, COBOL, CICS, DB2, IMS, etc.) It would be an understatement to say I performed very well on the job. I was the "rock star" of my department and would quickly become the go to guy for virtually every problem, even by the more experienced members of our team. Although I loved working with my teammates and especially my boss, I grew bored with the work. I grew bored with the 3am phone calls. I grew bored when my dinner was interrupted by a page. I grew bored with the mainframe. Although I created some interesting work for myself, which included creating a web page for our department listing our nightly job schedule and on call list and writing a few short VB programs, I longed to do more.

On a whim, I applied for a job with a large consulting company's Chicago office and was hired. I was nervous moving to Chicago, but excited about the prospect of new work. My agreement with the recruiter was that they would move me to Chicago and I would spend my first nine months working a mainframe contract using my existing skill set, and afterwards I would be sent to Boston for extensive C++ training, which is where most of my schooling was and it was a language that I had kept up with on my own. Roughly seven months later, my boss told me that their plans for me had changed. They were going to go "whole hog" after Y2K business and would need me to continue making use of my mainframe skills for the foreseeable future. I was crushed and I began looking for a new job immediately.

A friend of mine was talking with a small consulting shop in Austin that did Lawson ERP consulting and put me in touch with their recruiter. I was hired and we began work on the same day. It is now 12 years later and I'm still spend most of my days working on the Lawson ERP project. My client's have ranged from small 3,000 employee hospitals, to impossibly large hospital chains. The work has been very profitable for me, yet, something is missing ... one of the reasons for this blog.

In my early days of Lawson consulting, most of my work was confined to the Lawson 4GL, a strange combination of a CASE tool, proprietary file formats, and MicroFocus COBOL. I didn't mind sticking with COBOL so much because I was now working on UNIX and Windows platforms. Almost immediately, I had to develop skills in shell scripting and I would tackle Perl soon as well. One thing was for certain, if I wanted to advance, if I didn't want to be pigeonholed, I had to look for every opportunity to learn something new, every opportunity to do something different, and go after it with gusto. I learned as much as I could about the business side of the Lawson software from the business analysts. I found the "why" as important as the "how". This allowed me to position myself as the top guy for most of my business analysts. I have succeeded as a Lawson consultant because I found a way to set myself apart from most of my fellow technical consultants.

I am fortunate to be working for a company I love surrounded by consultants who are the best in our chosen field, but something is still missing. I take every opportunity to convince my clients to tackle new technologies, that the increase in productivity is more than worth the risk. More often than not, I get frustrated, but sometimes, I find myself pleasantly surprised. The main focus of this blog will be on my attempts to move beyond the technologies of Old IT. More to come ...

A Developer's High School Education

During my Sophomore year in High School I fully embraced my inner geek and took the first two of four high school programming classes, An Introduction to BASIC and Advanced BASIC. It was, to say the least, a major disappointment. It would not be an exaggeration to say that I already knew more than my teachers. Really, it wasn't even close. My mind wandered and I found myself getting into trouble. First, I discovered Mario Brothers for the Apple IIe's and would play the game as often as I could get away with it, which was often because the teachers were a bit clueless. Then, I turned my interests towards other activities, like pranks. For instance, we had a group of old TRS-80's in addition to the Apples and I wrote a short program that mimicked the interface to Apple BASIC. The key difference ... after you hit a key ... any key ... the computer responded with "Up Yours". Should I mention that I got in trouble?

My Junior year saved my high school programming education and put me back on my career path. The first half of the year I took a class called "Data Processing" where I was taught RPG and the second half of the year I took "Computer Programming" where I learned COBOL. Both classes were taught by Mrs. Waller on IBM System/34 mini-computers. Yeah, I knew that I did NOT want to ever work in RPG as soon as I got my first set of input sheets and output sheets, but I my complicated relationship with COBOL would begin, and to this day, the language that has been both good and bad to me. (More on that in a later post.) Ultimately, for the first time, I was writing an approximation of business applications using a serious programming language. The classes were both fun and thrilling to me.

There were no more classes to take my Senior Year, but Mrs. Waller was given a batch of PCs and would begin teaching her classes using MS-DOS versions of COBOL and RPG. She had no real experience with PCs, so she asked me to help her set them up and get the software working. To make things easier on her and the students, I wrote a simple menu system in C so the students wouldn't have to mess with the command line. The theory at the time was that they were in the class to learn a programming language, not MS-DOS.

That same school year, an individual who worked as a dBase III consultant and had his own small business contacted the school looking for some help with some of his work. I was recommended and I began my first paying job as a programmer, for the sum of five dollars an hour, which, when compared to the minimum wage I was earning at Godfather's Pizza, didn't seem so bad at all.

All in all, my high school programming experience ended on a high note, and prepared me for college where life took a few funny turns.

A Developer's Beginning

I discovered computing and programming by accident. It was Christmas of 1982 and our local J.C. Penny's announced that they were marking down a stash of TI 99/4A's. My parents, along with many others I might add, arrived their the next morning when they opened. A melee ensued. Dad would almost get into a fight. More importantly, he got me my first computer. A 12 year old was set on a journey that continues to this day.

Now, I was a kid, so my time on the computer was first spent playing games like Parsec, but it would be just a matter of months when I would learn the joys of BASIC. The first program I ever wrote listed each of my 12 favorite baseball teams. When you selected a team, I displayed details about the team such as the manager, stadium, star players and minor league affiliates. I was hooked. My mother would later take me to get the Advanced BASIC cartridge and I learned I could do even more.

Perhaps my favorite past time was typing in programs that were published in various magazines and books, and then, I would edit them to change their functionality. I can remember this one game where you would fly through space going after aliens and you would use the joystick and once you got an alien in the crosshair, the alien would explode. So of course, I changed the program to require you to press the button. Then I added a sprite to simulate a laser blast. I changed it to display the score at the top of the screen as you played. I added multiple levels and multiple aliens at the same time. I even changed it to save the high scores to a cassette tape! I may not have been a hacker, but I was a programmer.

Later, after all my friends had their Commodore 64s, I would beg my parents to replace my old box. Eventually, they would buy me a Commodore 128 for a birthday and I would begin to pick up more programming skills. The 128 was marketed as three computers in one: a Commodore 128, a Commodore 64 and more importantly, a CP/M machine. I used the Commodore 128 mode to add to my BASIC programming skills. The Commodore 64 mode was used to play games, and later to hack them using tools like a sector editor. I would also use the Commodore 64 mode to start logging onto BBS's over a modem. It was there that I discovered a wealth of free software available for the CP/M operating system.

The first thing I learned about CP/M software is that I would need to learn to build the software myself since it was usually distributed as source code. So, I learned to use the included C compiler and make utility to build and install software. More importantly, I discovered the C programming language, and with the help of a few excellent books, I would teach myself to program C, my first real programming language.

It was then that my high school education would advance me further.