How to develop an easily scalable payment system?

Question

PHP

Task: it is necessary to design a payment system similar to QIWI, select the necessary technology stack, correctly break the system into components. Critical stability and resistance to stress. Also in the plans is the implementation of its api for clients.
As I understand it, it is necessary to work with money transactions as follows: the client leaves a request for a monetary transaction (internal transfer from balance to balance, or replenishment from outside), then the request is sent to the request processor that performs this operation (makes a transfer, or issues an invoice , checks the payment status and then increases the balance).
From here, you can break the system into the following modules:
- web interface (on the part of the user, a personal account, various supporting information, etc. on the part of operators, tracking errors, viewing transaction logs, etc.);
- a queue of applications (roughly speaking, a list of ordered transactions. From where to where and the current status: completed, waiting, ready):
- management module with applications (take a list of not completed, issue an invoice to a certain payment system, check it for payment, switch status, delete application);
- a module for working with third-party payment systems (get an instruction to collect json or xml => parse the received return the result);
Initially, the idea is to implement the system on Symfony, i.e. completely on php, use postgres as a database, but several questions pop up from here.
At low loads (100 - 1000 requests for processing), a php script is hung on cron and everything will work quite quickly, but how to optimize if there are, say, 500,000 requests. You can build a java handler and run it in multi-threaded mode. But then it turns out that you need to duplicate the management module with applications and work with the module for working with third-party payment systems.
Since for the web interface it is also necessary to create an application to see the result (balance).
As an option, there was an idea to implement the backend as a RESTfull api (in java) and the web interface as a client.
I understand the topic is quite voluminous. Therefore, I will just answer both in general terms and links to articles or books that will help to better understand this issue.

Answer 1

1. A cron script is definitely a bad idea. A more correct story is a constantly running script that receives the next task from the queue. A great option for queuing rabbitmq
2. Loose component coupling is good. In your case, definitely api (I don’t really understand throwing from php to java, but it’s up to you. Write on what you know better) + extensible api for external integrations + interfaces clinging to api.
Concerning synchronism/asynchrony - it is quite possible to find room in one api. Which methods to make synchronous, which not - depends entirely on your business logic.
In my opinion, obtaining a balance, for example, can be done synchronously.
3. From the point of view of speed - you will quickly hit the base. Accordingly, one should immediately think about data sharding, how the system will behave in case of failure of one of the nodes involved in the transaction, etc.
4. Data consistency is important to you. Just think about iron. All servers are on fire. Branded servers burn in the same way as non-branded ones. Taking into account p3 - I would make nodes completely independent of each other, with physical duplication inside the node, and tie each account to one node.
5 [philosophical] Understand the important point - without a VERY serious investment in marketing, projects do not take off. If these investments were - you would not write here (no offense). Accordingly, the probability that a huge flow of transactions will suddenly come to you tends to zero. By the time you get promoted, you will have time to change the architecture of the project 5 times. You cannot have the same architecture for a startup built by one person, and for a project with high HL / HA.
Write in whatever you like, you will have plenty of time to rewrite bottlenecks for example in C.
6 [legal] Do you know that you need a bunch of licenses to operate as an electronic money and money transfer operator without opening a bank account? :)
PS I want to believe that your question is a task for self-development. Otherwise, I can’t imagine what kind of payment system it is, which is made by one person asking for advice on how to do such things (again, no offense to you) :)

Answer 2

1. Find funding, from $10M
2. Hire a team of professionals.
Yes, but you still have to finish school first.

Answer 3

when there are a lot of applications, a bunch of workers are made and a distributed queue where applications are added and where the workers take them from to execute
queue implementations a bunch

Answer 4

JVM solutions like Scala + Akka are better for you - that's how PayPal works.
I have experience in developing a financial system with different currencies, etc.