In this age of digital transformation, web-scale applications and the Internet of Things (IoT) support use cases such as 24-hour shopping, mobile banking, and the enhanced utilisation of aeroplanes, buildings and fleet vehicles. As a result, IT teams are focusing on achieving acceptable application performance levels as the amount of application data soars. 

Addressing performance, scalability, availability and cost have become paramount for companies facing the challenges of fast data (big data analysed in real-time to drive business decision making).

CTOs and application architects who understand today’s technical challenges, and the new strategies available for dealing with them, are improving performance and scalability while reducing down time. All at the same time that they simplify their technology stack and save time and money. With that in mind, here are three top tech trends for 2018, along with suggestions for future-proofing applications.

1. Turn Big Data Into Fast Data

Digital transformation initiatives generate new levels of big data. These initiatives create massive data sets from multiple sources ranging from transactions to sensor data to data feeds. However, driving action based on this data can be expensive and slow, requiring significant time if the data must first undergo an ETL process prior to performing analytics. For digital transformation projects to scale, IT must turn big data into “fast data” – that is, developing the ability to obtain insight from the operational data in real time using hybrid transactional/analytical processing (HTAP).

2. Improve User Experience In The Era Of Big Data

Delivering web-scale applications is no longer limited to a handful of the largest social media and e-commerce websites. Enterprises across a range of industries must be agile enough to deliver a real-time application experience as their user base is ever-expanding. Many components of a web-scale architecture are well defined but ensuring a consistently great end user experience still isn’t easy. Enterprises must be able to scale ahead of demand while managing costs. Furthermore, many applications – such as offering additional related products during an online shopping experience – require current and historical user data to be analysed in real-time, which creates a tremendous performance challenge for applications with large numbers of users.

3. Make The Internet Of Things Work

No technology holds more promise and more challenges than the internet of things (IoT). Gartner predicts 26 billion IoT devices will be creating data by 2020 – not including computers and smartphones – and much of that data will be streamed across the Internet for immediate analysis. Consider just one of thousands of use cases: traffic management. Using data from sensors on vehicles, traffic cameras, wearables and other smart devices, city traffic managers will make smart, real-time decisions about the flow of traffic. To accomplish this, however, the city’s IT infrastructure must be able to ingest and analyse hundreds of thousands (or even millions) of bits of information every second in real-time.

In-Memory Computing

A key solution to all three of these challenges is in-memory computing designed around a simple concept: data should reside in RAM, with disk-based databases used mainly for backup. Since the early days of computing, the high cost of RAM has limited in-memory computing to only the highest value use cases. Most systems today are still tiered across RAM, fast SSD disk storage, slower hard disks, and even tape drives. But memory costs have dropped significantly, which has greatly increased the range of use cases for which in-memory computing is economical.

When in-memory computing is combined with distributed processing across a cluster of commodity servers, extreme processing speeds and simple, low-cost scaling are possible. Such platforms also enable Hybrid Transactional/Analytical Processing (HTAP), allowing transactions and analytics to be performed on the same dataset, enabling enterprises to dramatically simplify their infrastructure and achieve enormous cost savings.

The sooner enterprises begin exploring their computing options, the better the position they will be in to ensure a great customer experience, cost-effectively achieve real-time insights at scale, and launch new digital transformation initiatives.