How to fix java.io.IOException: Cannot run program "python3": CreateProcess error=2 error on Windows 10?

Question

Python

Hello! I'm trying to install everything needed to work with PySpark in Pycharm. I barely found some tutorial on how to install it all, but when I try to run a simple code in Pycharm, a bunch of errors fall out, led by these:

21/03/18 00:00:18 ERROR Executor: Exception in task 1.0 in stage 0.0 (TID 1)
java.io.IOException: Cannot run program "python3": CreateProcess error=2, �� ������� ����� ��������� ����
at java.lang.ProcessBuilder.start(Unknown Source)
  at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:165)
  at org.apache.spark.api.python.PythonWorkerFactory.create(PythonWorkerFactory.scala:107)
  at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:119)

The paths are all registered, everything seems to be installed, tell me, please, in which direction to dig?
Thank you very much in advance!

Answer 1

Long story short, I did it.
I found a note on the Internet about the PYSPARK_PYTHON variable. Added it to the environment variables with the value of the path to the file, i.e. right up to the python executable. And everything worked. It's just that this variable is rarely mentioned in any tutorials.

Answer 2

error=2 means that the executable file was not found.
There may be several reasons.
I suspect that java is trying to start python with just the 'python3' command, without an absolute path. Then you need the python binary directory to be added to %PATH%. If it is exactly spelled out, then you can try the Resource Monitor (in Windows it seems to be out of the box) to see exactly which files java is trying to open.