Why Traditional Database Technology Fails to Scale

Are you aware of the fundamental flaws in legacy architecture and traditional SQL databases? Did you know that SQL databases are not designed to scale reads and writes? Wondering if your traditional SQL database can create issues for online analytics processing? Unfortunately, the answer is a definitive yes. Despite labor-intensive interventions from your DBAs to scale databases beyond existing enterprise needs, the tremendous volume and velocity of business data make it very difficult to adapt to dynamic demands while avoiding downtime and delays. These challenges do not mean that scaling your SQL database is impossible. It just means that the process is fraught with challenges on every front. Let’s learn why. (For more on SQL, see How Can SQL on Hadoop Help with Big Data Analysis?)

The Shortcomings of Monolithic Database Management Systems

Conceived in a relatively centralized era when software was deployed in static environments, legacy database architectures fail to support an increasingly mobile world where applications are accessed anytime, anywhere. Today software users want consistent improvements in usability and expect SaaS vendors to deliver new features and functionalities needed to achieve their business objectives.

However, legacy database technologies fall short in serving the needs of today’s distributed and cloud environments for the following reasons:

• Inadequate failover capabilities
• Latency issues
• Insufficient provisions during peak demands
• Lack of high availability at all times
• Increasing operational costs
• Inability to meet the demands of global markets

For all of these reasons, traditional databases are unable to deliver results in a rapidly growing environment where the workload is geographically distributed across heterogeneous data centers. Upgrading to a more distributed data model is costly and complicated. But your DBAs can’t just sit back and give up on this situation. Following are three commonly used workarounds for solving scalability issues along with their business benefits and limitations.

The Pros & Cons of Common Workarounds

• Sharding
• Vertical Scaling
• Horizontal Scaling

There are two different types of sharding – horizontal and vertical. While horizontal sharding requires division of data along multiple instances, vertical sharding moves all your tables to other instances to minimize response times for queries. Sharding facilitates storage of data across multiple servers, but both horizontal and vertical sharding are complex and time-consuming. Both the processes require changes at the app level apart from redundant data to avoid cross queries.

Sharding brings you the ability to store a huge amount of data, share it across locations, access information easily on any device, eliminate duplications and reduce storage space but, it is not without limitations. Unfortunately, sharding is error-prone, requires manual failover, and it is limited in terms of capacity partitioning. Joins are highly unreliable and inefficient, and taking consistent backups can become exceedingly difficult. Sharded databases cannot survive a service outage and do not support foreign keys. Lack of referential integrity can invite inconsistencies in data which can eventually become a significant development cost to the company.

In reality, very few organizations actually need sharding. Entities like Facebook, with petabytes of data added daily, are rare. For most companies, offloading the primary write server by scaling read capacity would solve the overall scalability challenge.

Scaling up means moving to a bigger and better server that has more memory, bandwidth and I/O. When the existing solution starts slowing down, scaling up makes it easy to handle heavy loads. So what’s wrong with the solution?

Costs rise with the more expensive hardware. Database licensing gets more expensive and maintenance costs also increase. No matter how big you go, eventually all your servers are sure to run out of steam. At some point, additional scaling up is no longer possible.

Scaling out means adding more capacity by adding more individual servers that work together to serve traffic. Such scale out can also increase uptime because any single server failure does not compromise availability of the system as a whole. The challenge is that your software doesn’t know how to talk to multiple database servers. To achieve database scale out efficiently and cost-effectively, you need database load balancing software that avoids the application rework needed to talk to replicated secondaries. (To learn more about database trends, see The Software-Defined Data Center: What's Real and What's Not.)

How Database Load Balancing Efficiently Addresses Scalability Shortcomings

Database load balancing software protects your infrastructure from downtime and delays by efficiently scaling the reads of your database, offloading the primary server and therefore scaling writes. This software enables your applications to take advantage of a scaled-out database without requiring code changes. It ensures high availability and high performance by:

• Automatically sending writes to the primary server and reads to the secondaries
• Avoiding application errors during database failover by holding writes in a queue until the secondary has been made the new primary
• Tracking replication time to ensure that reads are not sent to a secondary server that is lagging behind in replication
• Distributing reads amongst all available secondary servers, and directing the workload to the best performing server
• Offering end-to-end visibility into the root cause of application performance bottlenecks

Database load balancing is without a doubt the easiest way to ensure uptime and high performance while reducing service costs and improving customer experience.