Machine Learning Model Evaluation: Metrics and Performance Guide

Help me understand and implement proper machine learning model evaluation for my project. I need: 1. CLASSIFICATION METRICS: - Accuracy: when it's misleading (imbalanced classes) - Precision vs Recall tradeoff and F1 Score - ROC-AUC: interpretation and when to use - Confusion matrix: how to read and act on it - PR-AUC: better than ROC-AUC for imbalanced datasets - Multi-class: macro vs micro vs weighted averaging 2. REGRESSION METRICS: - MAE vs RMSE: which to use and why - MAPE: advantages and pitfalls (division by zero) - R² and Adjusted R²: interpretation 3. EVALUATION BEST PRACTICES: - Train/Validation/Test split strategy and ratios - Cross-validation: k-fold, stratified k-fold, time series CV - Data leakage: how it happens and how to prevent it - Overfitting diagnosis using learning curves 4. CHOOSING THE RIGHT METRIC: - Business-aligned metric selection framework - Optimizing models for business cost (e.g., false negatives cost more) - Creating custom business metrics 5. PYTHON CODE: - sklearn metrics implementation examples - Plotting ROC curve and confusion matrix My project: - Problem type: [classification/regression/ranking] - Class balance: [balanced/imbalanced, ratio] - Business context: [what does a false positive/negative cost?]