How to make a neural network that will receive data from opencv?

Question

Python

How to make a neural network that will receive data from opencv in real time and could use the "keyboard" function and library (which simulates key presses) to control what is happening in the same real time. How to "tell" a neuron that it can control a computer with the help of a function, and what library would you recommend for creating the network itself (keras, tensorflow, theano, etc.)?

Answer 1

Here is an example of tweaking TensorFlow + OpenCV for playing GTA5:
https://youtu.be/ks4MPfMq8aQ

Answer 2

For those who are interested, here is what I wrote:

import numpy as np
from PIL import ImageGrab
import cv2
import time
import keyboard

for i in list(range(4))[::-1]:
  print(i+1)
  time.sleep(1)
def jump():
  time.sleep(0.5)
  print("click")
  keyboard.press_and_release('space')

def process_img(image):
  original_image = image
  processed_img = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
  return processed_img
while(True):
  screen = np.array(ImageGrab.grab(bbox=(0, 35, 640, 515)))
  new_screen = process_img(screen)
  cv2.imshow('window', new_screen)
  #jump()
  if cv2.waitKey(1) & 0xFF == ord('q'):
    cv2.destroyAllWindows()
    break

There was also a black and white option. Just put this in line 17: processed_img = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)