Industry Trends 2021: Data & Analytics
Updated: Jan 10, 2022
Why should you care?
Nothing has powered the global platform economy as much as the abundance of data on the world-wide web. MAANG (Meta, Apple, Amazon, Netflix and Google) and countless other tech companies built products that attracted millions (or even billions) of users and then went on to leverage the data they collected for new product development and optimisation. The ability to track user behaviour into the last detail has pushed innovation in a multitude of fields, from high-performing databases towards real-time collection and consumption of information. It has also caused headaches for regulators and citizens around the globe, given that privacy eroded, and that the so-called surveillance capitalism has produced tech multinationals that sometimes appear to be overwhelmingly powerful. What remains undeniable is that data is on its way to become the most important asset for economies around the world, far more important than oil has ever been. The following trends, some technical and some not, will have a groundbreaking influence on how companies and citizens will use data in the years ahead. The amount of data in the digital world is now doubling faster than every 2 years, a trend that is only accelerating. More than 90% of that data (there are various sources around that topic) is unstructured and is likely never touched or even analysed. It is generally assumed that valuable insights can be extracted with the creation of new algorithms and additional processing power. Here is what I believe will shape the space in the upcoming years.
The current key influences
It is now obvious that COVID-19 has a lasting influence on how we live and work together. Other trends, such as work-from-home or remote diagnostics, have been accelerated or amplified during the global pandemic and the resulting lockdowns. With regards to data and analytics the 4 key influences are the following:
1. Artificial Intelligence & Machine Learning are becoming the new normal and more use cases are being transformed from mere theories into actual products
2. Data and Software Engineering are becoming more specialised and tools are more fragmented than ever
3. Privacy is a growing concern for regulators and consumers alike
4. Storage, processing and analytics are as available as never before
What exactly is going on?
AI & ML
Market Size: $138B
Annual Growth: 20+%
Automated decision making to support employees, power products and AI-driven monitoring systems are emerging and allowing business managersto make faster and more informed decisions. It also enables the creation of innovative new products, such as autonomous cars. According to Pitchbook companies will spend a total of $138B on AI & ML software in 2021. It is also expected to grow at more than 20%. Big players in the market are of course the well- know Big Tech companies, such as Alphabet and Meta, but also established other players in the space such as Databricks or Waymo, the autonomous driving division of Alphabet.
Promising companies
Mimica (London, United Kingdom)
Mimica's algorithm automates the process of finding automation potential in your processes with an AI that looks for patterns. They just received a $6M Series A, led by Khosla Ventures.
Growing complexity and fragmentation in software engineering
Market Size: $88B
Annual Growth: 9%
An abundance of highly specialised tools and methodologies are creating expert roles, for example Apache Kafka Engineers or NLP Data Scientists. To keep time-to-deployment as low as possible, new products that manage the overall infrastructure and facilitate DevOps processes are becoming increasingly important. In 2021 the market was at about $88B, most of it being spend on operational and infrastructural topics. Annual growth is expected to be around 9%. A multitude of verticals have been emerging next to established players such as Gitlab and Docker, for example Netlify (Jamstack hosting) or Postman (API testing).
Promising companies
Bloop (London, United Kingdom)
They offer an IDE (integrated development environment) integration (currently only for VS Code) that adds an AI-driven search for Javascript code, based on both examples from the official documentation and commonly used methods from open source projects.
Snaplet (Berlin, Germany)
Still in beta and with a very recent (~Nov 2021) seed funding round, Snaplet intends to improve developer’s lives. Snaplet offers a tool which, quickly and without putting sensitive information at risk, creates snapshots of production databases for local testing and development.
Privacy and data protection
Market Size: $11B
Annual Growth: 9%
The growing complexity of data ecosystems, even within small organisations, requires new approaches to follow regulations such as the European GDPR. Data governance and data lineage tools have been on an upwards trajectory and have helped create transparency around the origination and the usage of data. While cloud providers have invested in physical data centres in certain geographical locations to avoid breaching regional regulations, smaller providers have focused on creating more transparency into how data is moved from A to B. Other firms, such as Darktrace, have been working on cyber security solutions to defend against hacks and other external threats. Due to the blurry limitations, the market size is difficult to grasp. Pitchbook, however, estimates the market for encryption and protection to be at around $11B. It could very well grow to be much bigger though, depending on the definition.
Promising companies
Alvin (Tallinn, Estonia)
Alvin provides an end-to-end data lineage platform, which tracks the flow of data within a database and among a variety of tools, such as Looker and Tableau. I had the opportunity to see Alvin in action (they are still in private beta) and it offers a ton of benefits to people working in Data and Business Intelligence.
Widespread availability of data processing tools
Market Size: $224B
Annual Growth: 13%
Storage and processing capabilities in the cloud have been getting more competition and getting cheaper in the last decade. At the same time a great number of analytics companies with specific focusses have emerged. As a result today's market is quite crowded and offers a lot of different products.
Big players such as Google have started developing or acquiring tools such as Looker to enhanced their cloud services. At the same time specialised analytics platforms, such as Heap for product analytics, have become key players in their niche and new market opportunities, for example ELT tools like Stitch is offering have opened up. In 2021 the market for big data had an approximate size of $224B and is expected to grow at double digits for the foreseeable future.
Promising companies
Y42 (Berlin, Germany)
The out-of-the-box data platform of Y42 profits from the fact that many data platforms still consist of a patchwork of tools that are unified under an orchestration layer. While most of the product is not revolutionary (there are plenty of tools for ELT, API connectors, and visualisation etc out there) the simplicity of having it all under one roof gives Y42 sort of a first-mover advantage in the space. In October 2021 they closed a $31M Series A, led by Atomico and Insight.
Jina (Berlin, Germany)
Also surfing the AI/ML trends, Jina offers a framework to quickly train a neural network that is supposed to power search engines for unstructured data (that is, not just text, but also images, audio and anything else). Trusting in open source, Jina is also offering a marketplace where developers can share what they have developed on the platform. With the $30M Series A raised in November 2021, the company is going to launch in the US soon.
Is that it?
Of course, this is a just a very high-level overview on what is driving innovation in the data & analytics space right now. As you can tell from the market sizes, the industry can be broken down into a multitude of smaller trends and markets that are being discovered or disrupted. The emergence of low-code/no-code tools in combination with demographic changes affecting the workforce and the digitalisation of non-tech industries has the potential to unlock a trillion-dollar market for example. As someone who worked in the industry for several years these are my 2 cents on what is going to happen in 2022 and this decade!
As always, any feedback is welcome. Please reach out if you know about (or founded) a company that is doing groundbreaking work in the space.