Object Storage

What Does Object Storage Mean?

Object storage is an approach to organizing and managing discrete units of storage called objects in a distributed cloud computing architecture. In this context, an object contains three main components:

Advertisements
  1. Data: The object’s content payload can be a document, an image, a video, an audio file, or any other form of unstructured or structured data.
  2. Extended metadata: Metadata is data that describes data. In this context, metadata provides context for the data that is stored within an object. Extended metadata is typically stored as a key-value pair (KVP), and the key is often used as the object’s unique identifier.
  3. Unique Identifier: Each object is uniquely identified by an object ID. The identifier is used to locate and retrieve a specific object within a distributed storage infrastructure.

Valet Parking for the Cloud

Leading cloud providers heavily rely on object storage to power their storage services, and many cloud-based applications and services use this type of architecture to store and access data.

That’s why object storage is often called valet parking for the cloud. When someone uses valet parking, they exchange their car’s key for a numbered ticket. The person doesn’t need to know where their car gets parked because, in most scenarios, that’s irrelevant. They do need to submit the numbered ticket they were given when they want to retrieve their car, however, so the correct automobile gets returned.

In keeping with this analogy, a storage object’s unique identifier is the same as a numbered ticket for valet parking. When a user or application wants to access data they’ve stored in the cloud, they need to submit the object ID they were given when they “parked their data” to have the correct data returned.

How Object Storage Works

When a cloud storage customer uploads data to the cloud, the cloud provider issues a response that includes the unique identifier for the uploaded content. This identifier is called the “object key,” “object ID,” or “object URL.”

When a user or application wants to access a storage object, they need to include the object ID in their request. This requires the requester to know the correct object ID. If they don’t know the object ID because they didn’t submit the data for storage — or they didn’t keep track of the object IDs they received — they will need to acquire the unique ID through a metadata search or by browsing through container elements.

Upon receiving the request, the storage provider uses internal algorithms to determine which node (or nodes) hold the requested object. They then retrieve the object from the appropriate node and deliver it to the requester.

Object Storage vs. Traditional Storage

Traditional file systems have a hierarchical structure that uses file paths to document where data is stored. While this architecture has been widely used for decades, it is not well-suited for storing big data and the ever-increasing volume of structured and unstructured data that are being generated by social media, the Internet of Things (IoT), and generative AI programming.

When cloud storage providers like Amazon began to remove the constraints of traditional file paths by storing objects in a flat data lake, it enabled the dynamic allocation and distribution of data across a cluster of nodes or servers, which in turn, facilitates seamless scalability.

Traditional Storage Object Storage
Data Organization Organizes data in a hierarchical file system Organizes data in a flat structure with unique identifiers
Scalability Typically restricted by hardware and file system limitations Highly scalable
Access Method Block or file-based access Accessed through RESTful APIs
Metadata Limited metadata Extended metadata
Data Retrieval Requires full file path Requires unique object ID (key)
Data Redundancy Replication-based redundancy Erasure coding-based redundancy
Use Cases Local storage, SANs, and NAS Cloud storage
Cost Efficiency High cost per GB as storage grows More cost-effective at scale
Advertisements

Related Terms

Latest Storage Terms

Related Reading

Margaret Rouse

Margaret Rouse is an award-winning technical writer and teacher known for her ability to explain complex technical subjects to a non-technical, business audience. Over the past twenty years her explanations have appeared on TechTarget websites and she's been cited as an authority in articles by the New York Times, Time Magazine, USA Today, ZDNet, PC Magazine and Discovery Magazine.Margaret's idea of a fun day is helping IT and business professionals learn to speak each other’s highly specialized languages. If you have a suggestion for a new definition or how to improve a technical explanation, please email Margaret or contact her…