Am I training the model correctly?

Question

Machine learning

Hello. There was a need to make a prototype model. The model should predict the probability of fraud (fraud) when fulfilling customer orders. As data for analysis - numbers and boolean parameters (the number of reassignments, whether it is due, etc.), which, in my opinion, can affect the likelihood of a fraud. Uploaded a sample of 140 confirmed fraud cases to azure. After training the model and testing on the same sample (70/30 ratio), the evaluation of the model shows that all values ​​are true positive. When testing the model on a sample of 30,000 already unknown claims, the evaluation of the model reports that all rows are false positive. I read a lot of documentation, tried different combinations of algorithms and selectable data, but I can’t get a sane result. I understand, that there is still very little data for training and you need at least 20 times more. But am I doing the right thing and how to understand the results correctly? With such a small sample for training, I won’t achieve anything at all, or am I setting up the model incorrectly in azure ml studio, and still something sane can be obtained at least for a raw prototype?
The algorithm that was last used is two classes boosted decision tree.
I would be grateful at least for comments from people who also tried to do something like this. Thank you!

Answer 1

Did you only send examples of fraud for training or something, without examples of "non-fraud"? It is necessary to send both positive and negative examples for two-class classification. The amount of data is desirable more, of course, but even with a small amount of correct data, there should not be results like yours.

Answer 2

There should be more data. 140 is nothing.
Alternatively, you can generate data.
We take each line, and change one parameter. That. from one line you can generate 10-20 new ones.
But this is so - a method on the forehead.