Welcome to the June 2007 issue of the IT newsletter.

Articles

1. Value from change
2. Website Measurements
3. Project Governance
4. Trends 2007
5. Further reading and references

1. VALUE FROM CHANGE

Despite many years of well-documented experience in implementing new IT projects, recent studies have shown that two-thirds of change programs fail to live up to expectations. So why the large failure rate of change projects? Tracey Windsor describes how Cubic Consulting has undertaken research and developed a model that can assist in measuring ROI on change.

Introduction

Business leaders who make a significant investment in managing the deployment of change programs expect - and deserve - to realise the financial return from their investment. Unfortunately, this is often not the case. Recent studies have shown that two-thirds of change programs fail to live up to expectations. One reason for Return on Investment (ROI) failures is that a framework for measuring the success of these initiatives was not adequately determined, built, deployed, monitored or reported.

Gartner found that 65 percent of senior business executives planned to adopt a "balanced set of metrics" to guide business-related IT decisions, claiming that future IT operations would need to be governed by value to the business instead of by raw costs. ¹ However, only 45 percent of the business managers surveyed said they had effectively steered their companies' IT performance; and just over 40 percent had an IT strategy that was closely aligned with their strategic business objectives. As a result, Gartner discovered, fewer than 25 percent of major application deployment projects and similar IT engagements undertaken through 2003-2004 had delivered monetary benefits in excess of their implementation costs.

Usually the mandate of change programs, ROI deployment has become a hot topic. Across many industries implementing change initiatives, consistent thematic issues and challenges are evident demanding a review of the approach for achieving ROI. Our survey conducted over a four month period from April to July 2006, sought to identify and discuss these consistent challenges experienced within organisations.

We present these challenges and issues in regard to ROI and a solution through the application of conceptual frameworks that can be provided by a simple practical ROI Model. By connecting user behaviour to business value, we believe these concepts are helpful in focusing on the metrics and financial analytics of an implementation. The true value of any change initiative lies in the measurable outcomes achieved. A practical ROI model (Traceability Model )² that enables measurable outcomes through the consistent development and deployment of these strategies is critical to successful IT implementations.

The Survey

One hundred clients across eight industry sectors (Table 1) were targeted and invited to participate in our survey and 82 responses were received.

Download table 1 PDF (419kb)

Of the targeted industry groups, the largest number of responses received was from the Information/ Communication & Technology sector that have the greatest volume of change programs associated with system implementations.

This was followed by Finance and Banking, Retail/Wholesale with a 15% return and Manufacturing & Distribution with a 10% return. By role, the greatest number of respondents (Table 2) were - Project Managers; Vice Presidents, IT/IS Managers and Training Managers again reflecting the large volume of change activity being associated with system implementations.

Download table 2 PDF (479kb)

The greatest number of respondents were Project Managers who are usually directly associated for a period of the project lifecycle with 'the plan' to attain ROI. Additionally, Vice Presidents though less directly involved on a day-to-day basis have ultimate 'ownership and responsibility' for delivering the expected ROI of the business case... Participants were then asked to provide an overall assessment of their experience of change within their organisation.

The following Table 3 reflects their summaries regarding change initiatives.

Download table 3 PDF (223kb)

While considered important, change is however, often seen as less than successful, with the highest number of responses recorded for 'Done Poorly' and 'Considered flavour of the month'. Most commented that change was usually not successful and certainly not sustainable. Respondents were also asked to then identify change management practices that they believed were critical to the successful attainment of ROI as in Table 4 - Best Practice Elements.

Download table 4 PDF (242kb)

Planning is Essential

If a 'solid business case' was seen as fundamental why is ROI so hard to attain? Challenges still remain with deploying and sustaining the 'plan' associated long enough to measure anything. User involvement, Executive sponsorship, defined baseline & metrics and effective communication channels were all deemed critical, but given that ROI is still not readily attainable it points to difficulties with deployment.

In order to attain ROI, significant planning must be undertaken, inclusive of the identification of specific metrics and measures with which to determine the success from a change initiative. Once planned, the deployment involves specific communications, designated ownership and the development of a sustainable support model.

Our results demonstrate the distinction between theory and practice, (what we know and what we do). Many change practitioners 'know' what is required to attain ROI successfully, though still find it challenging to deploy, highlighting the importance of an easily understood and useable ROI model.

Respondents were asked to then identify the most challenging elements of deploying the plan for ROI as in Table 5. 'Breakdown in communications' was identified as the most common challenge. Having a business case is one thing, creating a sustainable plan to measure ROI quite another. If no one knows what is being measured or why, it will be impossible to later gather reports to reflect expected business case elements. A difficulty with 'defining metrics and baselines' remains one of the biggest challenges.

Download table 5 PDF (245kb)

This suggests that the process of decomposition of Business Case elements into behavioural, measurable metrics tends not to be fulfilled as a practice. Without these fundamental first steps, the other elements of best practice e.g. sponsorship, communication, ownership and data collection become irrelevant.

The "Right People"

There was also acknowledgement that the 'right' people were not always involved, with comments such as 'those who had to actually do the work and measure the changes; usage; savings - were not included or involved early enough'. When individuals who will be deemed 'responsible' are not included in the planning or deployment of the Business Case, the uptake of that initiative is poor, resulting in 'failure to measure ROI' and by association then 'a failed implementation' and a 'failed change program'.

While many elements of best practice are known as critical and some elements are evident within organisations, it appears that the ability to include all elements and plan for successful deployment and sustainability is difficult as seen in Table 6.

Download table 6 PDF (194kb)

The most common challenge for deployment seems to be the 'lack of definition around metrics; poor baseline reporting and limited links to the Business Case'. If metric definition is poor, associated documentation will be almost impossible to establish and maintain. Key metrics, once identified then, should become a baseline for future reference and integrated into roles for the long term through Key Performance Indicators (KPIs) or Short Term Incentives (STIs) as appropriate.

The Traceability Model

The Traceability Model defines the integral steps required to connect the business case to the system or change initiative and then to the user. The Model provides the practical link between the Business Case and the day to day usage of a new system and the implementation of new processes and behaviours. Without this model and the transitional support of change agents, it will be impossible to fully quantify the business case.

Our years of experience have assisted us in defining the 'best practice' critical elements required in order to achieve accurate and sustainable ROI for each and every change initiative and to incorporate these into a simple Traceability Model. This report clearly emphasizes that the 'Business Case' as a mandatory starting point is 'critical' and all 80 respondents strongly agree with an aggregate, score of 4.2 on the Lickert Scale.

In direct contrast, only half (40) of the respondents claim that metrics and processes arising out of the Business Case are easily identified and less than half of that number agree that metrics are then 'easily deployed'. This is validated by a score of 0.8 (Strongly disagree).

Traceability Model 'working sessions' target the essential first steps of defining metrics, baselines and targets. Functional team members along with the change team members can participate in driving metric identification and tracking mechanisms for their identified processes.

Reports identified as requisite to metrics tracking become known and the strategies for ensuring 'best practice' data input and output can be utilized. Business rules and policies that support the identified, agreed metrics can be identified and reinforced.

Identifying and then communicating ownership of metrics remains an issue, demonstrated by 82 participants returning an aggregate score of 2.3 (disagree) for this element. While risk management ranked well as often practiced, comments such as 'improved since we undertook a review of previous projects' and 'as most critical elements are linked around ROI; if one is failing, the others start to dwindle...the sustainability is the big challenge'.

Metrics Rule

Providing the capacity to sustain change is also an important part of the change process. Along with the necessary support mechanisms for the transition, further reinforcement and support must be provided through the alignment of human resource processes, such as role review, performance review, the induction process for new hires or the ongoing training and development processes for staff. At the end of the day it is about developing solid, quantitative measurements that are traceable over time through sustainable change practices.

Tracey Windsor is the Singapore Branch Manager for Cubic Consulting, a company specialising in Change Management. For more information about the research or the Traceability Model send an email to Tracey at TWindsor@cubicconsulting.com.au or visit the Cubic Consulting website at www.cubicconsulting.com.au

Top

2. WEBSITE MEASUREMENTS

How many times have you heard someone say - "yes we have a website but we don't know if anyone looks at it"? At the same time, we have all visited websites where we know our visit is being logged. So what happens to all the information collected by websites? What can it tell us? Carolyn Wilson provides some steps to help us better understand the role of web analytics in the world of the web.

In a business sense, we have moved way beyond thinking that the internet is an intangible, stab-in-the-dark marketing tool. With the use of various data collected from the website and its visitors, the information can be gathered in ways that are suitable for your particular business, dealing with your particular market.

Web analytics is an extremely rich source of information that helps your business understand how well your web investment is doing, and how you might be able to do better. It's a learning experience. And over time, you can dramatically improve your results, and you can prove it.

Improving your business using web analytics entails a continual improvement process. We will go through the steps of this web analytics process to better understand what is involved:

Web Analytics: Continual Improvement Process

Define Strategy

This comes from your business-wide goals and objectives on how the website will support the business. In other words, the reasons you have a website:

• Increase orders
• Improve retention
• Cost savings
• Generate leads
• Manage public relations
• Improve customer service

To understand what to measure, the business must first know what it wants to achieve, not the other way around.

Define Objectives

The objectives are how you plan to achieve that strategy. This requires a deep understanding of your business and your customers:

• Increase e-mail clickthroughs (links clicked in emails)
• Increase repeat purchases
• Increase new web accounts
• Increase use of an online tool. e.g. an online directory, job request form
• Reduce cost of new customers

Measure

Measurement allows you to get the information that will tell you if your work is succeeding, failing, or perhaps worse, doesn't make a difference. It allows you to:

• Adhere to goals
• Monitor performance
• Compare actual outcomes with expected outcomes
• What you measure comes from the defined objectives. Measurement tells you what happened, but not why.

Data is gathered from:

• Server log files
• Browser activity feedback (via built-in click tags on web site pages)
• Search log files
• Customer data (feedback, relationship information, surveys)
• Usability testing and focus groups

This data is then used to create Key Performance Indicators (KPIs), such as

• orders per site visit,
• cost per conversion, and
• order conversion rate per campaign.

The particular KPIs used are agreed upon within your organisation in order to achieve the desired business goals and results. 3-8 KPIs would be included in a report for a specific recipient, and there may be a number of recipients within one organisation (eg CEO, marketing manager, customer service manager).

Analyse

Analysis looks at differences over various measures of time. Analysis mostly reveals the match between your product and your audience. You can infer customer behaviours and, if correct, predict outcomes.

Analysis involves several tasks:

• Data selection - select the correct data and KPIs.
• Data review - look over the figures.
• Causal hypothesis - what happened and why.
• Lessons learned - draw conclusions.
• Opportunity assessment - what options can take advantage of the information.
• Recommendations - Specific actions to take.

Report

Reporting is taking the analysis deliverables and writing up the results. All too often people fail to take action on web data. Written recommendations, though, tend to become to-do lists.

For reporting to be effective, it must be provided meaningfully for the recipient. For example:

• Most CEOs don't need the number of daily visitors every day. Weekly or monthly is often enough.
• However, the marketer in charge of a campaign may need those conversion numbers daily (or even hourly), and monthly is just not useful.

Action

Taking action on what you have learned is often harder than it should be. The reports, and the habit of actioning them and continuing to generate them, need to be integrated into your business to achieve any benefit out of them.

Distribute your report to the decision makers. Then meet to focus on and discuss the findings, deciding what changes need to be made to the website and/or campaigns.

Create an action plan so that required tasks are allocated to appropriate people/teams, and deadlines set.

Then do it!

Measure Again

You need to repeat the process to see if the changes worked. If the changes did not improve your results, back it out. You learned something. Try something else. Over longer periods of time, this process will change your objectives and your strategy.

The best solutions survive, you learn, and the business gets better and more profitable. That's the point of an analytics program.

Measure it: Manage it!

Carolyn Wilson is a director of Mango Design Group, offering Web Site Development and Online Marketing Services. For further information on Web Analytics and online marketing processes, send an email to carolyn@mangodesign.com.au or visit the website at www.mangodesign.com.au

Top

3. PROJECT GOVERNANCE

Stories of project disasters abound with repercussions for corporate operations and financial performance. So if the impacts of project failure affect organisations what lessons have we learnt? What responsibility should our Boards have for these projects? Dr Raymond Young, from Macquarie University looks at these questions and the emerging demand for business project audits.

Why now?

For some, governance has become a dirty word. A colleague reported a senior board member exploding in a fit of rage and frustration "I've had it up to here with that word".

The governance pendulum has swung too far toward the management of downside risk. Boards are left with precious little time to focus on the real job of guiding their organisations towards above average returns.

This change in corporate mood may be a trigger for more business project auditing. Business projects are generally undertaken to implement strategy and improve profitability and the major risk is a failure to realise the expected benefits. Project auditing is starting to focus ³ on whether benefits are likely to be realised rather than compliance with standards.

Whether these trends will translate into an increased demand for business project auditing is difficult to say. Boards appear to be genuinely interested in improving their performance (1, 2) and almost a third of UK firms now have an executive with e-business responsibilities on the board. PRONED, a leading recruiter of non-executive directors, believe this will also be a long-term trend within Australia. The only issue in the short term is how long the threat of litigation following the Australian Wheat Board (AWB) and James Hardie, will divert board attention from their main business.

Project Standards

In the meantime, consultants, auditors and other boardroom advisors need to get their house into order. Their advice is conflicting and there are an excessive number of 'best practice' standards (PMBOK, PRINCE2, ITIL, ISO17799, and COBIT). None of the advice has yet been tested against objective criteria to prove compliance improves business performance. The majority of these standards focus on success in project management terms (on-time, on-budget, to specification) and underemphasise the real reason why projects are undertaken - to realise business benefits. There is a naive assumption, that if a project is technically successful and users are happy then the organisational benefits will be realised.

The relationship between project management success and project success is weak (3, 4). We experts simply need to acknowledge that real success, the realisation of business benefits, is less directly influenced by our actions than we previously believed. A series of case studies published by Standards Australia (5) re-examined the practice within leading Australian organisations and confirmed a long overlooked truth: top management support is essential for success.

Their evidence suggests that top managers have more influence on the overall success of a project than any of the traditional approaches such as high level planning, project methodologies, user involvement or high quality project staff. This is not to say these other factors are unimportant, only that they are not as important as top management⁴.

The problem with this finding is that most advisors have focused their efforts of the past forty years on everything except how top management influence projects to succeed. It may be difficult for some to reconcile their past advice with this insight but the accounting profession is particularly well positioned to advance the discussion. The UK Institute of Internal Auditors has long recognised project auditing as a sub-discipline, auditors have board access and the accounting profession does not rely on conventional project management or IT technical expertise for its credibility.

Selling to the Board

At a recent seminar for IT auditors, Dr Raymond Young demonstrated how easy it was to present the case for better project governance in the language of the board and its auditors. He showed that the current ROI for IT projects was around 30% but an increased focus on the business benefits was likely to increase the ROI to 135% and eventually to around 240% (Figure 1). In dollar terms he showed an organisation that spent $160M pa on IT and 15% of this on IT projects could realise another $20M pa if its board would enforce some basic project governance disciplines. At a national level there was the potential for project governance to lead to an additional $10B-$21B pa and increase GDP by up to 3.1%.

Figure 1: National Implication

At least two organisations have accepted this message at face value and committed funding to explore whether such benefits could be realised in practice. Their commitment demonstrates that potential for more business project auditing definitely exists.

Introducing Project Audits

So what should a business project audit consist of? According to the case studies reported in HB280 (5), the essence of project governance is to know what questions to ask at various stages during a project and knowing how to interpret the answers. The handbook frames six key questions in the context of the ASX corporate governance framework (Table 1).

Table 1: Corporate Governance Framework

Each of the key project governance questions can be cascaded into a series of sub-questions to deal with the different answers that might be received and the subtleties of interpretation (5). What should be noted is that none of the key questions requires any particular technical knowledge and that the main requirement is for sound business knowledge and an understanding of the subtleties of human behaviour.

Next Steps

The strength of what has been presented is that it sounds doable. It is too soon to know whether it will amount to anything significant. This article has presented some indicators that suggest there is a demand for business project audits and some evidence to suggest the intellectual frameworks are developing to support the service. Are there any stronger indicators?

Perhaps the appointment of more board members with IT experience will trigger the demand. Boards trust the advice of the world's leading consulting firms and a supply side trigger would be the marketing of project audit services by a McKinsey, BCG or some other major. A meaningful sign will be when the word 'business projects' replaces the word 'IT projects' in the business jargon. There is seldom such a thing as a pure IT project anymore, but it requires business managers to step up to the plate and recognise their responsibility to realise the benefits. You should also keep an eye on BRW; the 'Michael Jordan' effect of one or two influential companies achieving large benefits from project governance which could set off a wave of demand. Or maybe we will be pushed into it with a James Hardie scale disaster relating to an IT failure. It would be better if boards proactively embraced business project governance for its benefits, but one way or another; the signs suggest it is not so far away.

It is only a matter of time before the demands of Sarbanes-Oxley, ASX ... and other corporate governance reforms will force boards to be more accountable for the inability to implement strategy efficiently and the massive waste that occurs ( ... 40% of which is approved at board level) (5)

References

1. Leblanc, R. and J. Gillies, Inside the Boardroom: the coming revolution in corporate governance. 2005, Toronto: John Wiley and Sons.
   
2. Young, R.C. and E. Jordan. Lifting the Game: Board views on e-commerce risk. Refer IFIP TG8.6 the adoption and diffusion of IT in an environment of critical change. 2002. Sydney: Pearson Publishing Service.
   
3. Markus, M.L., et al., Learning from adopters' experience with ERP: problems encountered and success achieved. Journal of Information Technology, 2000. 15: p. 245-265.
   
4. Cooke-Davies, T., The "real" success factors on projects. International Journal of Project Management, 2002. 20: p. 185-190.
   
5. HB280, How Boards and Senior Management Have Governed ICT Projects to Succeed (or Fail), ed. R. Young. 2006, Sydney: Standards Australia.

Raymond Young (PhD, MBA, BBSc) is a former CIO and currently lectures in the Department of Accounting and Finance at Macquarie University. He is a founding member of Standards Australia's working group on IT Governance and a director of the NSW Chapter of ISACA, the professional organisation for IT auditors. Raymond can be contacted at: ryoung@efs.mq.edu.au

Top

4. TRENDS 2007

Judith Merryweather visited CEBIT Australia (CEBIT.AU) 2007 and shares some observations about the latest trends in technology evident from the trade fair and its presentations.

From its small beginnings 6 years ago, CEBIT.AU has continued to grow and expand both in the exhibition space and visitors. This year saw more records broken as CEBIT.AU was hailed as the most successful in Australia. But what happens at CEBIT.AU? What trends are evident? The CEBIT.AU Business Awards, rewarding exciting innovations being developed in Australia, www.eway.com.au provides one view of trends but in the opinion of this author, the essence of CEBIT Australia 2007 can be captured in the following four words.

OPEN-ness

CEBIT.AU demonstrated that Open software is becoming more mainstream. Not only in the amount of exhibition space devoted to it but also with the introduction of a series of seminars on Open Software. But there were warnings about ensuring you still get value from Open source software raised by Professor Gernot Heiser www.zdnet.com.au and suggestions that "Free" should be rephrased to "without payment" from Simon Phipps, Chief Open Source Officer of Sun Microsystems. The Open-ness was tempered by a large presence for Microsoft's Vista and Office 2007.

Digital

If you are interested in video, voice, music, photos or glossy brochures, CEBIT.AU offered ways to develop it quicker, display it clearer, show it bigger or say it louder. Solutions featuring digital media featured on many of the stands. Large plasma screens www.pcauthority.com.au ,digital signboards, and a range of software and hardware to bring business and entertainment one step closer were on display.

Networks

Showcasing all the latest equipment for networking, large sections of the exhibition halls were given over to networks. Paul Budde gave a series of talks on communications and new media highlighting the trends and influences these will have. www.budde.com.au Please select the 'Special free report - Global - International Market Analysis April 2007'. And it isn't just about hardware for enhanced wireless networks or other connectivity. VOIP and mobile telecommunications also featured heavily among the exhibitors and presenters.

Collaboration

Collaboration and connectedness present many opportunities for businesses. Software to facilitate and improve social networks featured with over 250 Bloggers invited to operate "BloggerZone" www.itwire.com.au during the show. And then there were other Web 2.0 technologies showing everyone ways of creating communities among their members, customers and partners.

One thing is certain - technology is continuing to develop at a greater pace throwing down more challenges for business every day. The main question for businesses is how to strategically use technology to remain competitive in an ever-expanding market place?

Judith Merryweather is a Business Systems Consultant with Alexer Pty Ltd.

Top

FURTHER READING AND REFERENCES

In this issue we have identified a number of issues and trends and provided some specific website references for our contributors. Below is a miscellaneous selection of websites and references that has been compiled to help interested readers undertake further research on these topics.

Change Management

Today's business world is fraught with change and not only on a local scale. Increased global competition has created new challenges for our directors and executive management. 10 Principles of Change Management www.strategy-business.com provides a list of guiding principles to assist executives to understand what to expect, how to manage their own personal change, and how to engage the entire organization in the process.

The Australian Development Gateway provides information on Organisational Learning www.developmentgateway.com.au and Change Management. Offering a range of information including a competency checklist and a Change Management Tool Box is a site offering a broad range of change management tools, methods and strategies that can be applied to individual, team and organisational development situations.

Technology Change Management by Sandeep Mehta www.projectperfect.com.au is an article that explores specific change management approaches for technology. Topics covered include developing strategies for investing in technology, evaluating changes to technology and implementing changes in technology.

Project Management

There are many aspects that are important in Project Management. Apart from a sound project methodology, communication is critical to good project governance. "The Project Manager's Perspective and the Importance of Communication", www.qld.ipaa.org.au provides some views on critical success factors in project management.

"Project Learning Through Project Histories" www.construction-innovation.info describes some research into how important project documentation is to organisations. The paper shows how this project history and audits of projects can be used as the basis for learning and therefore improving organisational performance.

For a good selection of links to information freely available on Project Management, visit the Sydney University site www.library.usyd.edu.au. It provides a comprehensive Internet Guide to Project Management including links to databases, websites, papers, Business portals, publications and standards. Worth a look!

Web Analytics

There is no easy answer to finding a web analytics tool. Of course it all depends on what it is you want to measure. For some interesting hints on how to go about finding the right web analytics tool visit www.kaushik.net .The article includes a list of suggested questions for vendors as well as questions for the company itself.

For an interesting read on the difference between analytics and reporting, Hurol Inan has written this article 'Analytics versus Reporting' www.hurolinan.com and there is also an article by Jim Sterne on the difference between Web Analytics and Web Metrics www.marketingprofs.com which requires free registration for access).

To support its belief in the importance of web analytics in measuring the success of websites, the Victorian Government has developed a Web Analytics Toolkit www.egov.vic.gov.au .The Toolkit can be downloaded and includes information on setting objectives, developing key performance indicators (KPIs), and measurements.

Top

Disclaimer

This is an initiative of, and has been prepared by the Business and Practice Support Team of the Institute of Chartered Accountants in Australia. While every effort has been made to ensure the accuracy of the information contained therein, neither the Institute nor its employees shall be liable on any grounds whatsoever in respect of decisions or actions taken as a result of using this publication. The information provided is a general guide only and should not be used, relied on or treated as a substitute for specific professional advice or referral to the relevant specific standard. Opinions of authors are their own and do not necessarily reflect policies of the Institute or the IT Special Interest Group.

The information is this email is confidential to the named addressee and subject to copyright. No one else may read, print, store, copy, forward or act in reliance on all or any of this email or its attachments. If you are not the intended recipient, any use, reliance upon, disclosure or copying of this email is prohibited and unlawful. If you have received this email in error please notify the sender. The Institute of Chartered Accountants in Australia (ABN 50 084 642 571) does not warrant that this email and any attachments are error or virus free and recommends that all attachments be checked for computer viruses.

IT Special Interest Group Newsletter is available by subscription for $66.00 annually-financial year (inclusive of GST).

For further information please contact:

Helen Ryan
Business and Practice Support Coordinator
Ph: 02 9290 5526
Email: business_practice@icaa.org.au