Internal and External Data
At last, we've arrived at where we actually find the data. The short answer is, obviously, everywhere.
It may be more appropriate to talk about what kind of access and ownership you have to various data, and how you can go about making them available and utilising them. To start with the latter, we must first get acquainted with the term “data silos”.
Data silos
The good news is that all businesses have loads of data. The bad news is that they are most likely scattered all over the place.
Perhaps you’ll find something you need in an email attachment, while some other data is located in various cloud solutions or a warehouse full of paper receipts that no one has had time to digitise yet. Or it is distributed across various applications, on local servers, in the cloud, in email attachments, on your desktop and with suppliers.
When data is separated in this way, we say it is found in different “silos”. As long as the data remains in silos, it's very difficult to utilise it or compare it across the board. We therefore need to lift it out of the silos to derive meaning and value.
Example
Bike rentals
A business that rents out electric bikes has data about who rents bikes, at what times, for how long, and at what prices.
In addition, they may be interested in third-party, external data about weather, traffic, public transport, working hours, sports and cultural events, where and how people move, and so on.
By collecting several of these data, they can discover correlations and insights that help them work more efficiently and earn more money.
First, they must gather and make all the relevant data available in one place—in this case, perhaps a cloud-based software—in a format and a structure that allows the data to be used together. Where and how the data can be structured and stored, you will learn more about later.
Once the data has been lifted out of their silos and gathered in one location, the company now has much better conditions to work purposefully with, for example, where and when they distribute bikes—given that they can follow usage patterns and demand in the data. And perhaps they will also discover opportunities for new business opportunities and strategic collaborations?
Insight
What does it mean to make data available?
Making data available can in some cases be as simple as mapping where it exists, and ensuring that those who need it have the correct login details, and so on. But the process of lifting the data out of the silos is also important. It could simply mean downloading data from various sources and manually collecting them in one place, such as in an Excel spreadsheet. This is completely fine if you are looking for a specific insight and are making a report, for example.
However, when we talk about data-driven businesses, such manual work will not be adequate. Instead, you will set up some form of automation, for example, by using APIs, to collect all relevant data in one place. What kind of “place” this can be—like databases and data platforms—you will learn more about in the next chapter. There you will also see that the data often needs to be processed and cleaned up before it is usable.
What access do we have to the data?
For a business or organisation, relevant data will exist both inside and outside the business—i.e. internal and external data. This distinction is important because it concerns the ownership of the data, which affects both access and areas of use.
A useful way to sort our data sources is by our level of access. If you are conscious of this, it is also easier to know what you can do with the data and how it can be processed in later stages.
Internal data may, for example, be collected by or shared with customers, partners or suppliers and not be yours alone, or its use may be subject to laws and regulations, for example relating to privacy.
With this in mind, let's now look at where you can get data and what access you have to it.
Internal data
We can distinguish between two types of internal data:
- Internal, unshared: Data that the business itself has collected and/or controls, for example, about accounting, customers, logistics, operations, equipment, etc.
- Shared data: Data you completely or partially control and have access to, but which others also have ownership of or access to. You cannot necessarily change or use the data freely.
All businesses have internal data, but the volume, quality, and availability may vary enormously. Here are some of the categories (but far from all):
- Customers: Names, addresses, emails, demographics, locations, business registration numbers, customer service interactions, history, potential customers
- Website: User behaviour, referrals, dropouts before purchase, search engine optimisation
- Social media: Target audiences, adverts, results, engagement, spending
- Supply chains: Supplier data, prices, inventory, logistics, transport, orders, potential suppliers
- Market / industry: Competitors, prices, trends, search trends, surveys, related products and services, business models
- Operations: Sensors, control systems, equipment data, maintenance data
- Accounting and transactions: Purchases, sales, fixed expenses, fixed income, other expenses and income
This is by no means an exhaustive list, but there may be examples here that you recognise?
The tinsmith’s workshop
Let's look at an example of what kind of data a business might have internally—in this case, the fictitious workshop “The Tin Can”.
But even if we become really good at using internal data for innovation and improvement, there may be as much or more to gain by looking outward—with external data.
External data
Again, we can make a distinction between two types of external data:
- Open/public data: Data from external sources that is openly available. By “public”, we do not necessarily mean that the data comes from the public sector, but rather that it is available to the public.
- Proprietary/private data: Data from external sources that not anyone has access to or permission to use. Such data may come from suppliers and others in the value chain of a business, or it may come from third-parties who collect and sell data.
External data is data we do not own and control ourselves, but which we either find openly available or may gain access to via partners and suppliers.
This can be data about things like traffic and weather—or data about production, logistics, maintenance and process at other businesses with which we exchange data.
Just imagine how many specialised businesses are involved when, for example, a ship is being built or how many entities are connected to the export of goods and resources.
In these scenarios, many businesses are working towards the same goals and can benefit from cooperation and data exchange—whether it's to reduce emissions, deliver better services or to build smarter, more sustainable cities and communities.
External data can often be integrated directly into a business' own systems and services through APIs, gaining even greater value.
Typical examples of public data are data from the Brønnøysund Register Centre (Norwegian public registres) or the Norwegian Mapping Authority. Here there’s tons of data that anyone can access. Organisations like the World Health Organisation (WHO) and UNICEF, too, share large amounts of data that can be freely used.
There are also some public datasets that are not immediately openly available, however, such as certain information from the Population Register or the Tax Administration—though one can apply for access.
Example
Fiskher
The possibilities with open, external data is best illustrated through a real example. Namely, the popular app Fiskher (stylised form of “fish here” in Norwegian), which shows recreational fishermen all over Scandinavia the way to over 70,000 fishing areas.
The app would have been difficult to create without open data from the Norwegian Mapping Authority and Geonorge, which is freely available for everyone to use.
By combining satellite images, mapping data, weather data, and a number of other datasets with machine learning, it was possible to pinpoint thousands of habitats for fish.
Fishermen can also register their catch, which means that Fiskher produces its own data that again can become useful for others. For example, it gives institutions like the Norwegian Institute of Marine Research and the University of Tromsø a better overview of recreational fishing in the country.
Today data is a significant part of the world economy, and several suppliers of proprietary data are multinational companies with thousands of employees.
Here are some examples of proprietary data:
- Copyright-protected data: Intellectual works such as music, film, photo, novels, textbooks—as well as patented brands and technologies.
- Market research: Professional market research agencies can provide data and insights about specific topics or issues on demand.
- Data enabling digital advertising platforms: Social media platforms like Facebook, Snapchat, and TikTok—but also online newspapers like VG and Dagbladet—own data generated on their platforms. Strict privacy rules prevent the outlets from sharing or selling detailed personal information. However, compiled user and behaviour data can still be used to create tailored ads for specific target groups.
If you are a leader or employee in a business—or have your own project for that matter—it might be wise to stop and think about what external data you could benefit from, and what data you have that others could benefit from.
But remember: Even if you own data yourself or have purchased access, it doesn’t necessarily mean that the data can be used freely. You still have to comply with laws and regulations like the GDPR. More on that shortly.
But first, we have to touch on something we've skipped over so far: Namely, that collecting data means we need to collect it somewhere. The data must be stored. We'll quickly look at this in the next topic.