Sep 6, 2008

Industrial Revolution for White-Collar workers: Data Mining and BI meet Orwell’s Big Brother (based on a BusinessWeek Article)

In BusinessWeek's 2006 and 2008 cover stories, "Math Will Rock Your World," and a new book called “The Numerati”, by Stephen Baker, a new age of numbers was announced. With the rise of new tools (called Business Intelligence and Data Mining), all of us are channeling the details of our lives into databases. Every credit-card purchase, every cell-phone call, every click on the computer mouse enter this data warehouses.

Companies that hire people with the tools and skills to make sense of them are beggining to decipher our movements, desires, diseases, and shopping habits—and predict our behavior. This promises to transform business and society.

One of the most promising laboratories for the Numerati is the workplace, where every keystroke, click, and e-mail can be studied. In the book, work at IBM (where mathematicians are building predictive models of employees) is shown. This work promises to be the most important until now.

Dr Samer Takriti, Syrian-born mathematician, heads up a team (40 PhDs, from data miners and statisticians to anthropologists) at IBM, that's piecing together mathematical models of 50,000 of IBM's tech consultants. The idea is to pile up inventories of all of their skills and then to calculate, mathematically, how best to deploy them. If this works, they plan to apply these models to other companies and to automate management. Their assignment is to translate the complexity of highly intelligent knowledge workers into the same types of equations and algorithms that are used to fine-tune shipping or predict the life span and production of a mainframe computer. With time, IBM hopes to build detailed models for each worker, each one complete with a person's quirks, daily commute, and allies, perhaps even enemies. These models might one day include whether the workers eat beef or pork, how seriously they take the Sabbath, whether a bee sting or a peanut sauce could lay them low. No doubt, some of them thrive even in the filthy air in Beijing or Mexico City, while others wheeze. If so, the models would eventually include this detail, among countless others. The idea is to build richly textured models that behave in their symbolic realm just like their flesh-and-blood counterparts. Then planners can manipulate them, looking for the most efficient combinations.

IBM has long been a leader in converting all kinds of complex systems into numbers. Right after World War II, Operations Research was used to construct a mathematical model of the company's industrial supply chain. It included its costs and capabilities, as well as limitations, or constraints. Once the supply chain existed as numbers, engineers could experiment with it–optimizing it–and later incorporate the improvements in the real-life version. This drove efficiency and lowered costs. It was wonderful for manufacturing.

But now, as IBM has shifted from products to services, the corporate supply chain is made up less of machine parts than of people. Today, IBM wants to optimize their use of 300,000 workers.

Personnel files, which include annual evaluations, are off-limits at IBM. But practically every other bit of data is fair game. Sifting through résumés and project records, the team can assemble a profile of each worker's skills and experience. Online calendars show how employees use their time and who they meet with. By tracking the use of cell phones and handheld computers, IBM's researchers may be able to map the workers' movements. Call records and e-mails define the social networks of each consultant. Whom do they copy on their e-mails? Do they send blind copies to certain people? These hidden messages could point to the growth of informal networks within the company. They may show that a midlevel manager is quietly leading an important group of colleagues and that his boss is out of the loop. Eventually, say experts, e-mail analysis may single out workers whose behavior places them outside the known networks. Are these outliers depressed, about to jump ship, consorting with the competition? In companies around the world, the Numerati will be hunting for statistical clues.

Even without reading all the e-mails, managers can automatically spot the most common words that circulate within each group of workers. This permits them to establish the nature of each relationship. They can also see how communications shift with time.

This is management in a world run by Numerati.

As IBM sees it, the company has little choice. The workforce is too big, the world too vast and complicated for managers to get a grip on their workers the old-fashioned way– by talking to people who know people who know people. Word of mouth is too foggy and slow for the global economy. Personal connections are too constricted. Managers need the zip of automation to unearth a consultant in New Delhi, just the way a generation ago they located a shipment of condensers in Chicago. For this to work, the consultant–just like the condensers–must be represented as a series of numbers.

Eventually, companies could take this knowledge much further, using the numbers, in a sense, to clone us. Imagine, that the company has a superior worker named Joe Smith. Management could really benefit from two or three others just like him, or even a dozen. Once the company has built rich mathematical profiles of Smith and his fellow workers, it might be possible to identify at least a few of the experiences or routines that make Joe Smith so good. There are a lot of job roles that are really commodities and if people turn out to be poorly designed for these jobs, they'll be reconfigured, first mathematically and then in life.

IBM does not expect to do this with superstars. Since they make lots of money for the company during short bursts of activity, they get plenty of time on the bench. But grunt workers in this hierarchy get far less consideration. They're calculated as commodities. Their skills are "fungible." This means these workers are virtually indistinguishable from others, whether they're in India or Uruguay. They contribute little to profits. They have varying skills and potential to grow. But looking at it mathematically, IBM should keep its commodity workers laboring as close as possible to 100% of the time. Not much kickback time on the bench for them.

Consider IBM's superstar consultant. He's roused off the bench, whether he's on a ski lift at St. Moritz or leading a seminar at Armonk, N.Y. He reaches into his pocket and sees a message asking for 10 minutes of his precious time. He might know just the right algorithm, or perhaps a contact or a customer he assumes his place in what is a virtual assembly line.

No comments: