Social Media Data Mining – How it Works and Who's Using it

There are more than three billion social media users today, with 11 people creating an account every second. What does this mean? 

Well, by the time you finish this sentence, hundreds of thousands of comments, tweets, pictures, and videos have been posted across social media.

It’d be an understatement to say social media is one of the top contributors to big data, but what can you do with all of this information? How can businesses, researchers, and even government agencies begin to make sense of social media?

One of the best ways of finding hidden information in social media is through social media data mining.

What is social media data mining?

We know from our guide on data mining that this commonly refers to “knowledge discovery within databases.” This may be the case for regular data mining, but social media data mining is done on a much larger scale.

What is social media data mining used for?

Social media data mining is used to uncover hidden patterns and trends from social media platforms like Twitter, LinkedIn, Facebook, and others. This is typically done through machine learning, mathematics, and statistical techniques.

 While data mining occurs within a company’s internal databases and systems, social media data mining is far less limited as to what and where it explores.

After social data is mined, results are passed on to social media analytics software to explain and visualize the insights.

How does social media data mining work?

Social data first needs to be collected and processed. This is data that is publicly available, which may include age, sex, race, geographic location, job profession, schools you’ve attended, languages you speak, friends and connections, networks you belong to, and more.

Then there’s the unstructured content of what you post on social media – like tweets, comments, and status updates – which is mainly what businesses, firms, and agencies are looking to mine. So, if your profiles are completely public, just understand this is generally fair game for social media data mining.

Then a variety of data mining techniques are applied. Some techniques may utilize machine learning, some may not. This is all dependent on how deep the “miners” are looking to explore.

Finally, all of this insight needs to be visualized in a way so that it can be interpreted. While there are a variety of data visualization tools to use, social media analytics often provides its own visualization options.

That’s how social media data mining works in a nutshell, so what are some of its use cases?

What are some uses of social media data mining?

Why would a business, research firm, or government agency look to mine social data? Well, there are a number of reasons. Here are a few of the more prominent ones:

Trend analysis

Trend analysis can be a very important metric for businesses that utilize social listening. For example, businesses may analyze which topics, mentions, and keywords on social media are currently trending and apply mining techniques to understand why.

This insight can be extremely telling; let me provide you with one of the most prominent examples of what I mean.

A recent analysis by SimplyMeasured concluded that mining sentiment on social media platforms like Twitter and Facebook leading up to the 2016 U.S. Presidential Election was actually more accurate at predicting the election’s results. Many traditional polls that year forecasted Hillary Clinton to be the winner.

As you can see, then-candidate Donald Trump had a more positive sentiment on social media than his opponent. Negative sentiment was just about neck-and-neck.

Trend analysis lets us see a different picture and understand hidden truths.

Event detection (social heat mapping)

Event detection – sometimes referred to as social heat mapping – can be an important metric for researchers and agencies who utilize social media monitoring. The example below shows why.

In early 2016, scientists at ORNL mined social data from Twitter to examine power outages across the U.S. By looking at textual and image data, paired with information on where this data was coming from (geospatial), they could see in real-time where major outages were occurring.

Just think of the many possibilities and use cases from a model like this. One I can think of is during natural disasters.

Social spam detection

Even the social media platforms we use daily are benefiting from the use of data mining. One example of this is social spam detection.

You may see it on platforms where spammers and bots are very prominent – I’m looking at you, Twitter, and Instagram.

Bots are always finding loopholes on these platforms to spam users with annoying, repetitive, and useless content. Because of how powerful automation has become, detecting these bots and squashing them can take some time. With social media data mining, platforms are steadily getting better at spam detection.

So, what could trigger spam detection? This could be things like an excessive amount of followers over an extremely short period of time. Excessive tweeting/commenting, tagging, and post updates could be triggers as well.

To be more proactive about social spamming, Twitter recently pushed an update to limit the number of accounts a user can follow in one day from 1,000 to 400.

Discovering the unknown

Whether it’s social media data mining or data mining in general, the whole purpose is to dive in and discover what isn’t visible at the surface level.

With advances in technologies such as machine learning and artificial neural networks, social media data mining will only continue to get more creative and in-depth. In the meantime, be sure to visualize your results in a way larger audiences can understand.

Social media analytics often provide their own visualizations, but for more advanced users, it may be worth seeing which data visualization options are out there.