.htaccess(ERR_TOO_MANY_REDIRECTS): how to do redirect to index.php + redirect to https:// at the same time?

Question

htaccess

Initially my .htaccess looks like this:

Options +FollowSymLinks
IndexIgnore */*

RewriteEngine on

# if a directory or a file exists, use it directly
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d

# otherwise forward it to index.php
RewriteRule . index.php

I want to redirect all requests to https, I tried to do this:

Options +FollowSymLinks
IndexIgnore */*

RewriteEngine on

RewriteCond %{HTTPS} off
RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [L,NE,R=301]

# if a directory or a file exists, use it directly
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d

# otherwise forward it to index.php
RewriteRule . index.php

But Chrome gives an error "ERR_TOO_MANY_REDIRECTS". How can these options be combined without causing multiple redirects?