Jumping the S-Curve is the Quiet Professionals’ video series on continuing to adopt and push innovation and take massive disruptive action to affect change within both the commercial and government sectors, with a focus on geospatial location-based intelligence, open-source intelligence, and bringing artificial intelligence to the tactical edge.
In this episode, Malachi talks about how it’s important not to approach big data starting from a solutions and architecture, but on a conceptual basis. Big data behaves in exponential ways, that is, ways that behave differently than the human mind is used to.
About big data, Malachi says, “when traditional means are failing you, you’re in big data space.” He adds that when you can’t open a spreadsheet or image file because they’re too big, that’s big data. Big data doesn’t have a specific number attached to it but happens at different times in different situations. When you must shift strategies to use your data, it becomes big data.
This means that big data problems require a different mindset that starts with the immense amount of data and how it should be handled, i.e., the data strategy. There are three main areas to consider: volume, velocity, and variety.
Data velocity is the speed at which data is produced, which can be constant, as in Internet of Things (IoT) or social media feeds. Data variety is both the different types or categories of data that is produced, and the irregularity of the data that is produced—for example, geospatial data that doesn’t fit in a rectangular “box” has to be handled differently than other types of data due to its irregular shapes. Finally, data volume is the sheer volume of data that has be to stored and managed, often in combination with a constant, high-velocity, real-time data.
Malachi recommends approaching data strategy not by planning to bring the data to the program that will process it (the usual method), but in bringing the computational analytics and programs to the data. Doing so will inform architecture, implementation, and what problems can be solved by using the data.
Another key element of a data strategy is not to get locked into specific solutions for processing data. With technology improving exponentially, keeping the focus on the data allows you to use “best of breed” solutions. With big data issues, changing how the data is processed and analyzed is far easier than changing or moving the data itself.
Malachi asks that viewers contact him to suggest guests or topics at marketingandmedia@quietprofessionalsllc.com.
Follow us for more big-picture thinking on Jumping the S-Curve!
