Evolution forces companies to search for new means of competition. Information technology, management science, and process knowledge are no longer enough to differentiate your business and stay competitive. Business executives often suffer from poor forecasts, insufficient data, and misleading advice that directs them towards imprudent decisions. Eventually, they find their businesses unprepared for yet another crisis and they ultimately fail. Business analytics – a new competitive advantage for companies that will help your business stay ahead of its rivals, foresee future crises, survive them, and move on after them. Your data, your software, and your people define your company business analytics foundation.
The worst thing you can do is to store your data in Excel spreadsheets. It’s okay to store some data there, but don’t routinely store all your sales, process, and other kinds of data in tables that are scattered across drives, with misprints of variable names and no tracking tags. There are fairly large barriers to installing a data collection system, so business owners often hesitate to do it. As a result, they increase the costs of run installing a business analytics system in the future. This situation is analogous to the installation of enterprise resource planning (ERP) systems like SAP in 1990s. The SAP installations were extremely expensive. For a mid-sized plant, the transfer of documents to an electronic format could cost several million dollars and last from one to two years. This was not because SAP was expensive – it was because the documentation flow and interactions between the departments was a mess. Cleaning it up and bringing the process up to standard was the most time- and money-consuming part.
Sooner or later, the law of competitive evolution will force you to store and analyze your data to stay in the game – just as SAP and similar systems did in the last decade of the twentieth century. When this day comes, your pain will be so strong that even millions of dollars will not help. If you weren’t already collecting data in the proper way, then you will be at the end of the line by definition. Your competitors, who were accurately recording and properly storing their process data, will be far ahead of you in their predictive capabilities.
If you are still using Excel spreadsheets, then at least follow these three rules:
- Always store separate variables in separate columns. Always!
- Each row is a record. Your table must be nice and clean.
- You first variable is date. Your second variable is the origin of the data. Then comes the rest.
Start with these three rules. Talk to your team, and introduce a culture of data storage. Force them to participate. They will be reluctant, but they’ll get used to it – and it will save you a lot of money in the future.
This is just the start. You cannot continue using Excel for much longer and here’s a simple example to illustrate why. Say your sales representative records his sales according to the three rules, and one variable is the customer’s name. If you want to learn more about that customer, then you have to look up a card with his profile, presumably in a separate table stored somewhere. These two tables are related – but there is no good way to create relational tables in Excel. If you have thousands of customers, you simply must use relational databases. The upshot is that you will inevitably use databases, and you will have to train your personnel about the concept of relational databases beforehand. They should not afraid of databases, and they should understand the meaning of the word ‘database’. I have seen a number of cases where people have called Excel tabs a ‘database’. In the modern world, those who are afraid of databases and don’t know how to deal with them look like people who afraid of printers in early ‘80s and used typewriters.
Databases accept data through forms. There are several ways to submit data to a database, but you can think of the database interface as a web form. By the way, all websites use databases to store data. Play with Microsoft Access to get an idea of what databases look like. You don’t have to become an Access expert – but at least launch it!
When you accept the fact that you cannot avoid using databases in your business, read the ‘Database Normalization’ article on Wikipedia. You’ll learn about database structure, reference keys, and SQL abbreviation. SQL stands for Structured Query Language. You don’t have to know it, but you have to know it exists. SQL is the primary language for dealing with databases and storing and retrieving data. All this is common knowledge you need in the new digital era. You may drive your car without knowing much about its parts, but at least you have an idea of what the transmission does. At a minimum, you know that it’s a part of your car. The same goes for SQL and databases – you have to know about SQL’s existence, and you have to have the technology in your business. There are different types of database designs and vendors, and SQL is not the only technology – but leave that stuff to the technical experts.
Be serious about your software from the beginning. I am making comparisons with cars all the time, so, think of your software like your car (or a boat as on the article cover). Each time you take your place in front of the steering wheel, you entrust your life to the machine. To a great extent, you entrust the existence of your business to software. There are myriads of software companies, start-ups, university spin-offs, and open-source codes that provide the functionality needed for business analytics. Ignore it. Work only with the giants. Some open-source software might be perfect, most start-up software will be cheaper, some will provide you with dedicated support. None of this matters. The essential thing here is the company’s reliability. Do you want your business to last for 20 years? Then chose a vendor that already proved to be on the market for 20 years. When you will filter the list, only three companies will remain: IBM, Microsoft, and SAS.
At this point, you’re ready to be introduced to the terms ‘structured’ and ‘unstructured’ data. Simply put, tables contain structured data. If you have only tables, and images of similar format (such as photos of your customers, pictures of your products, or scans of related documents) then use SQL-type databases. Microsoft is good for these. On the other hand, if you have a disk drive full of miscellaneous files, then your data is unstructured. The Internet is an enormous reservoir of unstructured data. Your ‘My Documents’ folder stores unstructured data. There are few operations you can do with unstructured data. Basically, you can only search it. If your volumes are huge, and you are going to search for texts and images, then use Hadoop- type storage. Formally, Hadoop is not a database, but a file system that works well – and quickly – on large data volumes. Have you heard the term ‘Big Data’? This term mostly describes unstructured data like audio, video, images, and unsorted texts. IBM is good for sorting through that. There are SQL solutions for unstructured data by Teradata, so contact them if you operate with both structured and unstructured data. (Do you?)
Most of the time, your business will work with an SQL database. In that case, I recommend that you use Microsoft to build your predictive analytics system. They offer everything. Microsoft SQL Server can store and retrieve data, analyze it, and generate reports. The Analysis Services of MS SQL Server is so powerful that it even has neural networks and Bayesian networks under the Microsoft brand. You can plug this data server into Microsoft’s native reporting system, SharePoint, which is document management software, but also an excellent and powerful reporting tool with a web interface. (Microsoft uses the word ‘power’ everywhere, as in PowerView, PowerPivot, and PowerQuery, for example.)
IBM has all of the tools. Actually, even more tools than Microsoft, since they have IBM SPSS, a world-standard tool for business statistics. SPSS is one of the few commercial multi-purpose packages with a dedicated function for conjoint analysis, although it is not well documented. A big plus is that IBM’s tools have graphical interfaces. If in Microsoft SQL Server you have to write a program to set up your predictive analytics, in IBM you can do it through a graphical user interface (but, of course, you can also program it). For example, IBM SPSS Modeler is a user-friendly tool that’s as good for express analysis as it is for a production system. However, once you create a production system, you won’t care how was it created (i.e., through programing or via mouse manipulations).
Speaking of statistics, I cannot ignore SAS. SAS started with statistics and gained credibility with its flagship product, ‘SAS’ (the same as the company name). The pharmaceutical industry often uses SAS to process data from clinical trials. Lately, the company decided to expand into the broader field of business analytics, which requires interfaces with data storage and reporting capabilities. To me, the only reason to use SAS is legacy data and compatibility. If you already have something written in SAS, then you can consider this option. If not, then check out their ‘JMP’ software. This is one of the best applications for regression analysis, basic data mining, and data visualization. And it’s cheap.
There are number of other software packages for predictive analytics. If you want to save money use Rapid Miner (it looks like IBM’s SPSS Modeler) or use ‘R’ software, or any free SQL database with reasonable documentation. For example, MySQL’s database is a reasonably safe place to land. It is originally from Sweden, now owned by Oracle, and free.
For the other business analytics methods, there is no gold standard. For all kinds of business process modeling, try AnyLogic, which is modern and well-supported software. AnyLogic is flexible and includes both system dynamics and agent-based modeling. However, for system dynamics alone you will be fine with VenSim, an application developed at MIT. It is very handy. For agent-based modeling, you will probably need an expert. You can play with NetLogo, and read about Repast, but if you are open-minded and decide to use agents – find an expert. Most likely, you will need to write a new system from scratch.
For the rest, you will find everything you need in IBM, Microsoft, and SAS software suites.
Last but not the least, are your people. You heard it right – your people come last in this sequence, right after data collection and software. However, at the head of them all is you, the visionary, the person who oversees the whole business and understands the roles of your various personnel. Understanding roles is an important part. As you learned so far, business analytics is not just about business processes, or sales forecasting, or data management and reporting. Business analytics is just the umbrella concept. This is the new age of competition. To me, job postings that say ‘We need a business analyst’ make no sense. People often confuse business analytics with business process analysis. To them, a business analyst is just another name for a business process analyst. They also think that ‘business analysts’ are system integrators, database administrators, project managers, financial analysts, marketing analysis, sales analysts, and other specialized experts in different fields.
If you’re looking for a new hire to help you with business analytics, don’t use ‘business analyst’ in the job title. You need an expert in your particular field who possesses business analytics skills. If you’re in the banking industry, you need a financial analyst with business analytics skills. If you’re in consumer goods, you may need a marketer with business analytics skills. If you want to improve your business process, you need a process engineer with business analytics skills or an expert in business process optimization. ‘Business analytics skills’ implies knowledge of system dynamics, databases and data mining. If you appeal for help to a company that provides business analytics services, you have to delegate an expert within your business domain to work together with the company’s people.