How do you test websites with CI?

Question

Software testing

There is a site in PHP, Ruby or любая веб технология.
The development environment for the site is developed in ansible (this is not so important), and deployed on the development machine in one click (using vagrant), there are also instructions for deploying production.
I want to make Continious Integration and the question arose of how to test the code.
To do this, you need to run phpunit, rake or similar.
Do I need to reconfigure via ansible runtime before running?
And how to set up the system so that it is possible to check several commits at the same time.
Let's complicate the task: selenium is used. Therefore, you need to use your own virtual machine for each instance, inside which there is a GUI. Do they dynamically create according to the image, or immediately expand the number of virtual machines according to the number of agents?
Now we use SaaS which uses the option without selenium.
With the use of Selenium, I wanted to jump off the SaaS needle.
Choice of CI: Jenkins or TeamCity.

Answer 1

A very curious question. I'm interested in how people do things.
Whether it is necessary or not depends on whether the environment has changed. I usually just run vagrant up --provision. Provisioning takes 20-30 seconds (if nothing has changed, apt-get update is to blame, but with ansible 2, these 10 seconds spent on updating the package index will disappear, although it seems that I will switch to docker completely earlier).
I only checkout the latest commit for a branch. That is, in fact, this solves the problem with several commits (we only check what has changed since the last CI run).
not GUI but X-s, it's a little different. Again, you can pervert and put the Silenium itself on the host, and manage it from a vagrant ... but this is difficult, as for me.
As for instances, I only use one instance per job.
Well... they are dynamic and will be created if your job is constantly jumping from agent to agent. Usually they don't jump that often. vagrant up --provision solves all the problems in terms of virtual machine deployment. In this regard, I really like the prospect of migrating to Docker.
A much more interesting question is managing preconditioning for UI tests, that is, either loading fixtures for a suite, or calling console commands, or something else ... With Behat, let's say I just use the service layer of the application to set preconditioning. And if I suddenly want to test the UI of mobile phones, then in this case I wrote a simple runner for behat that allows you to run separate steps, so I can run tests for Behat in tests for calabash, for example (to install precondition tests).
The difference is not really that big. There are more plugins and cool stuff under Jenkins, but this is a dumb thing. I myself am currently sitting on Jenkins.