How to properly organize a microservice architecture using the Golang language?

Question

go

I have plans to implement a project, the essence of which will be in two services. One is the telegram bot interface, it implements methods for sending a message, checking for updates, etc. The second is a service for parsing data from the site.

This service should also keep track of news, for example: a new review appeared on a site publishing games, the parser read information about it, checked for the presence of the ID and, if there is no such review, saved data about it, for example, in redis (I was advised it for these purposes).

Here's the catch: part of the advice was words related to writing a server. As I suppose http server. But is there a need for it? After all, nothing prevents you from simply writing a parser that will be launched and check for the presence of a record in an endless loop and otherwise add data.

Also, I have no idea how to unify all this, how to avoid unnecessary checking for news in the bot interface when the parser has already checked everything. Can somehow track the appearing entry in the radish?

Answer 1

No need to mix all the questions together. I'm answering the question in the title.
Which one is right depends on the scale of the project. If it is small, then you do not need to spread out over many services.
The architecture describes the tasks of services.
Option A (simple).
The daemon process of a site-scanner-parser that writes to a DBMS (relational or document-oriented). Refreshes new pages. You can use queues (the same Redis) to process parsing.
cron job to clean up irrelevant entries.
Telegram bot that reads prepared information from the DBMS.
Option B (harder).
Site crawler daemon process. It scans pages and puts a message in the page content queue. Potentially holds many connections to crawled sites, handles page fetch errors, and tries again.
The page parser daemon process. Processes messages from the page content queue, extracts the required content and puts it in the DBMS (upsert).
cron job to clean up irrelevant entries.
REST API for processing requests from a Telegram bot that reads prepared information from a DBMS. Potentially more may be required.
The Telegram bot accesses the REST API to get information and other actions.
In this case, you can scale each service separately, depending on the load. Of course, any suitable XYZ language can be used instead of Go.
I don't claim to be correct. It's more of a thought on how to do it.

Answer 2

https://github.com/micro/go-micro
https://github.com/go-kit/kit