KMP, or can you please explain how the parameters of Selenium webdriver are arranged?

Question

Python

I climb into the official documentation , I look at how to initialize the webdriver, I do:

from selenium import webdriver

driver = webdriver.Firefox(firefox_profile='D:\\Develop\\redirect_watotg\\profile.default',
                           executable_path='D:\\Develop\\redirect_watotg\\Lib\\geckodriver.exe')
print(driver.firefox_profile.path)
driver.get('https://web.whatsapp.com/')

I start - it works. Wonders! I look at the output:

>>> C:\Users\kshnk\AppData\Local\Temp\tmp_mkfpnfj\webdriver-py-profilecopy

I look in the folder that I indicated - empty. I pass authorization on the site, close the browser, start again - there is no authorization. I didn't expect anything else, but still.
I go to Google, I find the answer in the selenium repository :

def __init__(self, profile_directory=None):
        """
        Initialises a new instance of a Firefox Profile
        :args:
         - profile_directory: Directory of profile that you want to use. If a
           directory is passed in it will be cloned and the cloned directory
           will be used by the driver when instantiated.
           This defaults to None and will create a new
           directory when object is created.
        """

Understood, this is ok. That is, what I indicated in the firefox_profile attribute indicates which directory should be used as the basis, but it will not be used, respectively, if it is empty initially, then it will remain so, Selenium will not return data from the temporary folder back in the folder I specified (or maybe there is a solution, but I did not find it). I'm looking for how to do it. Found on the mozilla site . Tried:

from selenium import webdriver
from selenium.webdriver.firefox.options import Options

driver_options = Options()
driver_options.add_argument('-profile')
driver_options.add_argument('D:\\Develop\\redirect_watotg\\profile.default')

driver = webdriver.Firefox(options=driver_options,
                           executable_path='D:\\Develop\\redirect_watotg\\Lib\\geckodriver.exe',
                           service_args=["--marionette-port", "2828"])
print(driver.firefox_profile.path)

The print(driver.firefox_profile.path) line is no longer with me:

Traceback (most recent call last):
File "D:/Develop/redirect_watotg/main.py", line 22, in
print(driver.firefox_profile.path)
AttributeError: 'NoneType' object has no attribute 'path'

Well, okay, I delete it, run it again - it started, I look in the specified folder - the browser put all my working files there, log in, close the browser, start it again - authorization is in place. Super!
The next step - I need a headless mode. I find it on the same (almost) mozilla site , add just one line:
driver_options.add_argument('-headless')
I check it works. Still great. Next - you need to make sure that css is not loaded (ideally) and it is possible to download video files, because. at the moment, when you click on the download, a window pops up with the option to save, or open. Finding a solution to disable css and to enable loading. Also, looking at the code, I remember print(driver.firefox_profile.path) and return it to its place, remove headless to look at the result. Final version of the code:

from selenium import webdriver
from selenium.webdriver.firefox.options import Options, FirefoxProfile


driver_options, driver_profile = Options(), FirefoxProfile()
# driver_options.add_argument('-headless')
driver_options.add_argument('-profile')
driver_options.add_argument('D:\\Develop\\redirect_watotg\\profile.default')
driver_profile.set_preference('permissions.default.stylesheet', 2)
driver_profile.set_preference('browser.download.folderList', 2)
driver_profile.set_preference('browser.download.manager.showWhenStarting', False)
driver_profile.set_preference('browser.download.dir', 'D:\\Develop\\redirect_watotg\\Downloads')
driver_profile.set_preference("browser.download.manager.closeWhenDone", False)
driver_profile.set_preference("browser.download.manager.focusWhenStarting", False)
driver_profile.set_preference('browser.helperApps.neverAsk.saveToDisk', "video/mp4")

driver = webdriver.Firefox(options=driver_options, firefox_profile=driver_profile,
                           executable_path='D:\\Develop\\redirect_watotg\\Lib\\geckodriver.exe',
                           service_args=["--marionette-port", "2828"])

print(driver.firefox_profile.path)
driver.get('https://web.whatsapp.com/')

I launch - all css is in place, when I try to load, a window pops up with a suggestion to open / save MP4 Video File (VLC). Until I stop the script, I go to the folder C:\Users\kshnk\AppData\Local\Temp\tmp_4shdr7q , which print() kindly told me about, I notice that there is only one file, although in theory everything should have been copied there files from the directory I specified, while the file that is there (user.js) is not in my directory. I open it, I see a lot of lines with different parameters, and among them there are those I indicated:

user_pref("permissions.default.stylesheet", 2);
user_pref("browser.download.folderList", 2);
user_pref("browser.download.manager.showWhenStarting", false);
user_pref("browser.download.dir", "D:\\Develop\\redirect_watotg\\Downloads");
user_pref("browser.download.manager.closeWhenDone", false);
user_pref("browser.download.manager.focusWhenStarting", false);
user_pref("browser.helperApps.neverAsk.saveToDisk", "video/mp4");

I look at the link - it looks strange, I change it to relative 'Downloads', I restart the script - the result is the same.
At this point, I did not find anything else.
Please tell me at what point (besides when I decided to try programming - it is obvious) from all of the above I made a mistake and misinterpreted the data I found. As a result, I have two questions:
1. Why are the files not copied to the temporary folder (although I don’t think it’s critical, but it should be so)?
2. Why don't set_preference() work?
3. Purely a bonus, if suddenly a solution of the past is found - is it possible to check the status of the download, or its name with the built-in features of selenium, without gag?
Thank you all for reading this and many thanks to those who decide to help.
UPD to
line

driver_profile.set_preference('browser.helperApps.neverAsk.saveToDisk', "video/mp4")

Added all existing mime types - did not help.
UPD 2
I studied the code inspector in Firefox, the network tab - I saw that when you click the "Download" button, nothing appears, but it is when loading a frame with a video that there is some kind of application/octet-stream , nothing else, including video/mp4 . I decided to add it - nothing, I decided to leave only it - also nothing. I noticed that when the open / load window appears, the blob is indicated in the source "from": Maybe this is the case? Went on to google.
UPD 3 Well
, well, well, by typing I came to the conclusion that it breaks the whole picture of all the parameters driver_options.add_argument() , removed options=driver_options from the initialization and voila - it workeddriver_profile.set_preference() , however, the idea of ​​disabling CSS had to be abandoned - the site completely breaks down and the script can no longer work with it, but we managed to set up the download.
But! Now back to my original problem - how to specify the profile directory so that you can save the session, because initially this directory was added just the same by the driver_options.add_argument() function .
UPD 4
So, it seems that it is crap with driver_options.add_argument('-profile') - it works by itself, but because of it, not a single set_preference() works , while driver_options.add_argument('-headless') great with everyone. But there was still trouble withset_preference('browser.download.dir', 'sdsadasd') - it doesn't work. As you can see, the parameter is an extremely strange address, but the function does not care what you write there - it downloads all files to the default download folder.
As a result, the problem with creating the initial profile and the non-working parameter of specifying the download folder are still relevant. In principle, we solve the issue - the same profile can be stupidly copied to the folder I need after executing the authorization function, and then loaded with the FirefoxProfile('D:/Develop/redirect_watotg/profile.default') parameter- this option works. The same can be done with download files, you just need to find out under what name it is loaded and when exactly it was loaded. But still, I would like to figure out what selenium gives.
PS Abnormally huge activity on the toaster today. And yes, it's more like a blog now, but I'm still hoping someone can help me.
UPD 5
Well, my plan failed in the bud - I wrote two cherished lines that should have solved my problem with saving authorization:

shutil.rmtree('D:/Develop/redirect_watotg/Profile')
shutil.copytree(profile_path, 'D:/Develop/redirect_watotg/Profile/')

Until I saw that after authorization I have only one file in the "Profile" folder - user.js and remembered the beginning of the story: specifying the profile folder in FirefoxProfile () will copy files from the profile folder, and specifying the folder and then creating profile files in it dealt with driver_options.add_argument() , which doesn't work in my case.

Answer 1

In total, I found temporary measures:
1. To save the session, I copy the directory, but since some files are occupied by the process had to wrap try-except blocks. It's good that it is the files responsible for saving the session that have time to be copied, although in general the solution is so-so:

path_file = open('D:/Develop/redirect_watotg/geckodriver.log')
matches = []
for line in path_file:
    if line.count('rust_mozprofile'):
        matches.append(line)
profile_path = 'C:/Users/kshnk/AppData/Local/Temp/' + matches[-1].split('\"')[-2].split('\\')[-1]
print(profile_path)
shutil.rmtree('D:/Develop/redirect_watotg/Profile')
try:
    shutil.copytree(profile_path, 'D:/Develop/redirect_watotg/Profile/')
except shutil.Error as e:
    print(e)

2. With the name of the downloaded file, it's a little simpler:
Before calling the file download function: And after the function has completed its work:

after = os.listdir('D:/Downloads')
difference = set(after) - set(before)
if len(difference) == 1:
    file_name = difference.pop()
    print(file_name)
    return file_name
else:
    return False

I hope in time I will find more correct solutions, if relevant.