What algorithm should be used to determine if a point is outside the triangle of space?

Question

Mathematics

Let's say there are 3 points: p1(x1,z1,y1), p2(x2,z2,y2), p3(x3,z3,y3), which form triangles in space (polygon). And there is a point p (x, y, z), how to determine whether this point is outside the boundaries of the triangle?
There was an algorithm:

Find 4 areas: 1 area is responsible for the triangle directly built on the vertices p1, p2, p3, and the remaining 3 areas of the triangle built on the vertices of the form: p1, p, p2; p2,p,p3, etc., and if the sum of such 3 areas equals the area of ​​the triangle p1p2p3, then the point lies inside the triangle

, here in this algorithm, Heron's formula was used to find the area, well, such an algorithm turned out to be inefficient, and so: what algorithm is best used to determine whether a point is outside the triangle of space?

Answer 1

If the point is not on the plane of the triangle, then you need to reformulate the problem. It's generally unclear what it means inside / outside.
If it is on a plane, then enter a coordinate system there (for example, ortho-normalize the vectors p2-p1 and p3-p1). Get the coordinates of all points (p1 can be set as the origin).
Then you have to apply the formula for the plane. You can use vector products. The product of pairs of vectors {p-p1, p2-p1}, {p-p2, p3-p2}, {p-p3, p1-p3} must all have the same sign (either all <=0 or all >=0. Equality 0 something will mean that the point is on the border).
Or you can calculate the area, again through the vector product. |(p2-p1)(p3-p1)| = |(p1-p)(p2-p)+(p2-p)(p3-p)+(p3-p)(p1-p)|

Answer 2

The algorithm was normal (because to find trilinear or barycentric (as in your case)) coordinates of a point and compare with a unit (or with an area) - this is the mathematically correct verification algorithm.
Only instead of a collective farm of school formulas (which should be left at school), matrices and determinants should be used. The whole search for coordinates comes down to inverting a 4x4 matrix, which is done quickly and easily.