Architecture / stack for telegram - bot, where is it wrong? Aws. +Metrics. Logs?

Question

Amazon Web Services

I want to make a bot for telegram as part of a hobby project, I would like to be poked with chopsticks into the proposed architecture.

Typical flow: The
user writes to the bot -> gets a response. The answer can be either simple or the start of other "sub-teams", there are "heavy" operations (working with loaded graphics through a bot).

Priorities:
- inexpensive (up to ~$100 per month) with n users (let's say 500 per month, each with 100 messages per month)
- simple scaling in case of a sharp increase in load (the possibility of a sharp viral effect is implied, but this is not accurate :) )
- response time to the user is not critical (up to 10s)

As I see the architecture:
1) Amazon api gateway receives a request from the cart webhook.
2) Throws it to amazon Lambda
3) It does minimal logging / metrics and throws it further into the Amazon SQS queue
4) Then the queue is parsed by n backends (to start with the same Lambda), if this is a "quick" answer - answers the user in a cart if "complex" - puts in SQS and already this message is picked up by another special separate EC2 instance.
db: Amazon DynamoDB for cheapness and speed.

In case of cost overruns, ways to reduce the cost:
Points 1 and 2 are transferred to EC2 or a digital ocean instance, for example, as well as backends processing the queue. What I would like to keep on aws is the queue as a key node in terms of system fault tolerance, the main thing is to be able to clean it up in case of a blockage, everything else is secondary (probably except for a stable api gateway in front of the queue).

Unresolved issues :
1) I have little experience with prometheus and grafana, I would like to use them here as well. What is the easiest way to integrate them these days? Do they need their own database? Will it be easier and cheaper to host all this on DO, or will it be tolerable on AWS?
2) What can be used to collect logs today from the category of cheap and cheerful?

Answer 1

As I see the architecture:
1) Amazon api gateway receives a request from a cart webhook.
2) Throws it to amazon Lambda
3) It does minimal logging / metrics and throws it further into the Amazon SQS queue
4) Then the queue is parsed by n backends (for a start, the same Lambda), if this is a "quick" answer - answers the user in a cart if "complex" - puts in SQS and already this message is picked up by another special separate EC2 instance.
db: Amazon DynamoDB for cheapness and speed.

Yes, absolutely normal approach

In case of overspending of the way to reduce the cost:
Items 1 and 2 are transferred to EC2 or a digital ocean instance,

Or AWS ECS + Fargate with scale to 0. As a bonus, we will get work with containers. And still, you still have to do throttling in terms of sending speed. How many are there today - all the same 30 messages per second?

1) I have little experience with prometheus and grafana, I would like to use them here as well. What is the easiest way to integrate them these days? Do they need their own database? Will it be easier and cheaper to host all this on DO, or will it be tolerable on AWS?

expensive, time consuming to set up, and almost useless on AWS

What can be used to collect logs today from the category of cheap and cheerful?

In the ideology of AWS - nothing but CloudWatch. Outside of AWS, use what you know - deploy it yourself anyway.

Answer 2

It is not clear why the author here grafana and prometheus. Looks like an unnecessary overlay. In addition, it requires a full-fledged and expensive EC2 against the background of everything else that is almost serverless.

Answer 3

Plusan to Ivan Shumov , with a few additions.
If the pet project is a telegram bot with the architecture you described, then you should not fence another pet project "prometheus monitoring" nearby. Cloudwatch will provide you with a sufficient level of logging / monitoring for the bot, and in the future it can also be expanded with AWS services.
Of course, you can spin prometheus nearby, but, as Ivan mentioned, this will increase the cost of the project: you will need to collect function execution metrics from CloudWatch (which is quite expensive) and/or send metrics/logs from lambdas directly to your prometheus, which will increase the cost of lambda execution
I would entrust the storage of metrics and logs to cloudwatch, and solve monitoring-logging problems when they arise (if you really can’t do something with cloudwatch)