Why does ssh keep asking for password in Docker?

Question

Composer

Hello.
The project has a container with openssh-server and composer. The project uses several private repositories.
When you try to install using composer, you are constantly asked for a password from ssh keys (twice for each repository). The password fits and the repositories are downloaded.
The container is logged in with docker exec.
What could be the problem ?
Example Dockerfile

FROM php:5.6-fpm

RUN apt-get update && apt-get install -y \
    openssh-server                        \
    git                                   \
    unzip                                 \
    && curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer

RUN mkdir /var/run/sshd
RUN sed -i 's/PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config

ENV NOTVISIBLE "in users profile"
RUN echo "export VISIBLE=now" >> /etc/profile

EXPOSE 22
CMD ["/usr/sbin/sshd", "-D"]