Question

How do firms collect data "at the edge" of a network?

Answer
By Justin Stoltzfus | Last updated: January 9, 2017

How do firms collect data "at the edge" of a network?

Businesses may use “edge data collection” methods to capture different kinds of information for business intelligence or other operational use. Collecting data at the edge of a network involves getting data in a decentralized point in the network architecture, for example, from small individual nodes without proximity to a central data warehouse.

In the early days of big data, a consistent philosophy emerged: that best practices, in most cases, involved routing data to a central data warehouse, where it would be stored, retrieved, analyzed and sculpted. This has remained a dominant model until recently, when “edge” data collection started to arise as a practical alternative.

To collect data near the edge of a network, businesses look far afield from the data warehouse and consider how to gather and analyze data “near the end user.” An excellent example is in internet of things (IoT) systems, where it may not be practical to funnel a lot of device or sensor data into the data warehouse. When it makes more sense to collect the data and observe it directly from the mass of disparate IoT devices, businesses collect data at the edge of the network, rather than in the center. In general, companies do this using specific middleware to get the data from individual small network nodes.

By collecting data at the edge, companies can do more analysis without the cost and effort of maintaining all of the data in archives. In other cases, the data collected at the edge does travel to the center: database professionals may take advantage of the journey to normalize and otherwise refine the data in question. One big reason for data collection at the edge is that the data may not be “clean” in its initial capture – doing more at the edge means the companies can determine exactly which data goes into the center, and how it gets internalized. The company can use protocols based on the condition of the data, taking in only clean data, or again, working on the data as it progresses from the edge to the center.

The pursuit of “edge analytics” is gaining ground in IoT architectures and other types of enterprise systems. Because companies can “thin” data or otherwise cull data results, edge data collection and analytics can help with issues such as network congestion and latency.

Share this Q&A

  • Facebook
  • LinkedIn
  • Twitter

Tags

Data Watercooler

Written by Justin Stoltzfus | Contributor, Reviewer

Profile Picture of Justin Stoltzfus

Justin Stoltzfus is a freelance writer for various Web and print publications. His work has appeared in online magazines including Preservation Online, a project of the National Historic Trust, and many other venues.

More Q&As from our experts

Related Terms

Related Articles

Term of the Day

Sandbox

A sandbox is a style of game in which minimal character limitations are placed on the gamer, allowing the gamer to roam and...
Read Full Term

Tech moves fast! Stay ahead of the curve with Techopedia!

Join nearly 200,000 subscribers who receive actionable tech insights from Techopedia.

Resources
Go back to top