How to get_terms sort by meta field?

Question

WordPress

$taxonomy =...;
$termargs = array(     
      'orderby' = > 'meta_value',
);
get_terms($taxonomy, $termargs);

and where to specify by the content of which field to sort?
If you specify
'meta_key'= 'my_field;
that, in addition to sorting, it will also select only those fields that have this field!
And how to select all fields, but sort by meta field. If not, then for these terms are placed either at the end or vice versa at the beginning. (doesn't matter)