How to create a testing department?

Question

Software testing

Good day!
There is a small company in which there are two types of employees - developers and implementers. Developers make a constructor, implementers set it up for the customer, who sees only the web interface.
Developers review each other's code, try to write unit tests for new functionality and bug fixes. After the accumulation of a set of changes, it is submitted for testing to the implementers. They try and test it, and if there are no more visible errors, then a release of the constructor is released, which can be transferred to the customer after configuration.
At a certain stage, I wanted to automate the process so that a more stable constructor would get to the implementers and exclude the possibility of delivering a product to customers in which they viewed, for example, some broken editor that was added in version X, where it worked, and in version X + 1 it has broken.
The most common places for errors to occur are at the subsystem junction and the web interface.
We believe that in order for everything to become good, the system must be covered with integration tests. We think in the direction of selenium (I would be grateful if you advise something else in the comments), so that it would simply do the same thing that the implementers are doing now - pierce the interface and if the reaction is expected, we consider that everything is fine (of course, we understand that tests do not guarantee the absence of errors).
We see the following problems when organizing a new division:
1) A huge variety of options for building products from constructor blocks - what works for one customer may not work for another for various reasons. On what configuration of the constructor to conduct tests?
2) There is no clear specification for the constructor (and there won’t be at the moment), only a list of tasks in the bug tracker, which means that testers do not understand what to test. You can try to write tests for new tasks and gradually they will cover everything, slowly add tests for functionality that does not fall into new tasks. Will it work?
3) Exceeding the unit's ability to keep tests up to date. Rather, it is a far-fetched problem, but nevertheless, can it turn out that there will be too many tests and when a new version of the product is released, the testers will not have time to update the tests for the changed platform?
Thank you!

Answer 1

Take an experienced QA and he will tell you everything how to do it.
>1) A huge variety of options for building products from constructor blocks - what works for one customer may not work for another for various reasons. On what configuration of the constructor to conduct tests?
On the most frequently occurring or bringing the main income.
>2) There is no clear specification for the constructor (and there won't be at the moment), only a list of tasks in the bug tracker, which means that testers do not understand what to test. You can try to write tests for new tasks and gradually they will cover everything, slowly add tests for functionality that does not fall into new tasks. Will it work?
It will work, but the developers will have a big drawdown in time - since the testers will have to ask them a lot about the system.
>3) Exceeding the unit's ability to keep tests up to date. Rather, it is a far-fetched problem, but nevertheless, can it turn out that there will be too many tests and when a new version of the product is released, the testers will not have time to update the tests for the changed platform?
These are already problems of the testing department and how they will solve the problem, but in general there is such a problem, but there is nothing to say without specifics.
I know systems where 10k tests are kept by one person, and systems where 100 tests are maintained by a department of 6-7 people.

Answer 2

"1) Огромное разнообразие вариантов построения продуктов из блоков конструктора - что работает у одного заказчика, может не сработать у другого по разным причинам. На какой конфигурации конструктора проводить тесты?"
If you are going to create a normal team of testers who will do tests, and not test (that is, write automation), then what difference does it make how many options - test everything. The main thing is to have enough resources for the required number of virtual machines.
First of all, you need to set up a testing process in the sense that the application would be deployed, some kind of sanity test would pass, and a notification about this would go further along CI. When the system is working, it will be possible to add tests, update tests. But such a system should be designed and set up by an experienced QA - a leader who will first understand the product and be able to make a fairly flexible system into which all tests and all configurations can later be pushed.
There should be tests that depend little on the version. It should be possible to quickly disable irrelevant tests. Tests are not an end in themselves, they are an additional metric of product quality and development simplification through automation. After all, you can not test something, but at the same time it will work, because it was tested by the developer in unit testing.
Well, it’s clear that with experience comes an understanding of how much and what needs to be tested in order to be successful. There will be a lot - discard the non-critical part.

Answer 3

Видим следующие проблемы при организации нового подразделения:
1) Огромное разнообразие вариантов построения продуктов из блоков конструктора - что работает у одного заказчика, может не сработать у другого по разным причинам. На какой конфигурации конструктора проводить тесты?

Есть main product - набор всех функций.
Есть customer product * n (n - количество кастомеров)
Тестируется как главный продукт, так и для каждого кастомера в отдельности. Тк различная комбинация функций - может вызывать различное поведение и соответственно разные баги.
Это, конечно, если вы хотите избежать лишних разговоров с заказчиками.