Factory method creates an extra abstraction?

Question

OOP

A factory method is a way of abstracting the creation of a particular instance. We have a common interface of factories, then a specific factory is built on this interface, creating an object. All clear. But I have a question - why is a separate layer in the form of a factory used, if you can combine products under a common interface and give them from the client, introducing a dependency. After all, in the end we still initialize the factory, so why not get rid of the extra layer?
upd:
Do I understand correctly that the point is to rid the client of "dependence" on the object?