What is the best way to organize microservices based on Docker written in node.js?

Question

Amazon Web Services

There is a program or, say, a Cloud service that polls devices using different protocols and receives data from them. Further, it either displays, or archives, analyzes, visualizes, etc. The application runs on AWS services in a compute cluster. All added devices, graphics, etc. organized within the same project. That is, the user logged in, created a project, added a device to the project, and so on.

Each user-created project will run a docker container with a node on it that will poll devices. For example, a user created devices and set the polling rate to 2000 milliseconds. We will get something like

setInterval(function(){
    // опрасить устройство
    // сохранить данные в базе
}, 2000);

But one project can have multiple devices. Some can run, others can't. So when the device starts, we add a task to the database to start or stop the poll or other micro tasks. And our node container will poll the task table every 20 seconds, for example, and execute it.

Thus, there will be no direct connection between the API server and the project container, and the project container will be completely isolated and all entry ports will be closed.

By the way, the database is an AWS RDS service, so access to it will always be inside the VPC.

Questions

1. Am I on the right track?
2. What are the tools or let's say npm packages to start small cyclic tasks, and stop as well as monitor their status. I just don't know what keywords to use here to start the search.
3. The project can contain 100 devices and each device from 1 to 100 polling channels. This means up to (or maybe more) 1000 small polling processes like I gave as an example. Is it correct to run a container per project or is it better to run a container per device? I see the benefit of a per-device container in that if there is a problem polling a device, it won't affect other devices. And it will be easy to turn it off, just start or stop the container. But I see the disadvantage that there will be a lot of containers in the cluster and I worry that the containers themselves will eat up a lot of resources.

Answer 1

a container per one task can be expensive even with Fargate. I would stop doing garbage and take AWS StepFunctions with Wait :

Create a StepFunctions job with parameters and invoke Lambda from it in a loop. Everything, there will be no problems. It is only necessary to take into account that the solution has an execution limit - 1 year, sort of. Next - recreate.
You have to calculate how much it will cost, of course. On Step Functions, one such Job per month can cost about $20 ($10 for SF and $10 for Lambda), without the need for additional management. You can save wherever you want - at least in CloudWatch, at least where you need it.
For comparison, at a minimum on ECS + Fargate it will cost $10 (if there are enough resources), but you need to control somewhere in addition that everything works, the infrastructure and in general. I would pay more for the first option