How to monitor social networks on any topic?

Question

PHP

Hello!
There are a number of services that monitor social. networks and media on certain topics. For example: _wobot.ru (not advertising).
Interesting, the very method of monitoring. For example, vk or twitter i.e.
1. How is the social media data API used?
2. How is such a large amount of data processed and stored in this case?

Answer 1

and what is difficult? there, in most cases, there is a search for certain words or hashtags. in VK, for example, through https://vk.com/dev/wall.get you parse posts and check if there are keywords that you need.

Answer 2

Drawing an owl is easy. We draw an oval and finish the rest of the owl.
Here - the same thing, we take posts from social networks, and filter them, that's all.
Since not a single social network allows you to upload GENERALLY all your content, you need to look for new posts through the API and take them for yourself. Search by geo-tag, by hashtag, by keywords, by trending, subscribe to new posts by news makers, self-learn. Use different servers, different applications, different accounts. Why? Well, for example, because the trends in Russia and the United States are different, for example.
You also need to understand that storing posts per day may require not even one, but several servers, and searching for terabytes of information is a rather non-trivial matter. Especially - full-text search.

Answer 3

Of course they have dedicated servers. If you take VK, then there is a stupid search.