vncdotool not working, what's the problem?

Question

Python

Started x11vnc with the following parameters:
x11vnc -display :99 -localhost -rfbport 5566
I have a graphical application running on the 99th display.
I'm trying to connect:

from vncdotool import api
client = api.connect('127.0.0.1:5566', password=None)
client.keyPress('enter')

I am getting the following error:

Traceback (most recent call last):
  File "vnctest.py", line 3, in <module>
    client.keyPress('enter')
  File "/usr/local/lib/python3.6/dist-packages/vncdotool/api.py", line 108, in proxy_call
    raise VNCDoException(result)
vncdotool.api.VNCDoException: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionRefusedError'>: Connection was refused by other side: 111: Connection refused.
]

Why and how to fight?
TightVNC easily connects (I forward the port via ssh)