Unstructured data – peeling the veil off social behavior

One of the biggest assumptions behind the use of unstructured data to understand individual behaviors is that the   behavior in a social context is always a true expression of the person’s actual thinking and belief.

This is contradictory to the globally accepted norm that people normally tend to behave in ways that are acceptable to society so that they always fit in and that is why people who are actually themselves in the social context are termed as outliers and do tend to get all the attention. In some cases people actually behave in ways that help them get attention, this is specific to people in professions like politics, entertainment etc.

A lot of unstructured data that is normally used to identify social behaviour is normally sourced from social networks where opinions are normally contextual to what the person making them wants to be perceived as within the network. A lot of unstructured data is also being used to gauge sentiment of people or cumulative sentiment of a market or social segment. The main problem of displaying sentiment is that if it is done in a particular forum that is used for such purposes, it is expected that sentiments will be a tad stronger or exaggerated.

There have been instances that unstructured data along with other sources of data have been used to predict and reduce the rate of crime but then again this is only specific to the context of people who already have a criminal background; so far there has been no evidence of first time offenders  being caught before they commit a  crime. There is however a lot of study happening around trying to find out what factors can lead to people turning into criminals and this is surely the right way to go.

This discussion does not intend to say  that unstructured data cannot help in identifying social behavior or should not be used, but it definitely poses its own challenges to scientists to be able to get to the bottom of an actual thought after peeling the veil.