Supervised learning, unsupervised learning, model evaluation and selection, data preprocessing, and pipelines and composition. Per-topic detail is in the other reference files in this directory.
Comprehensive algorithms for classification and regression tasks.
Key algorithms: - Linear models: Logistic Regression, Linear Regression, Ridge, Lasso, ElasticNet - Tree-based: Decision Trees, Random Forest, Gradient Boosting - Support Vector Machines: SVC, SVR with various kernels - Ensemble methods: AdaBoost, Voting, Stacking - Neural Networks: MLPClassifier, MLPRegressor - Others: Naive Bayes, K-Nearest Neighbors
When to use: - Classification: Predicting discrete categories (spam detection, image classification, fraud detection) - Regression: Predicting continuous values (price prediction, demand forecasting)
See: references/supervised_learning.md for detailed algorithm documentation, parameters, and usage examples.
Discover patterns in unlabeled data through clustering and dimensionality reduction.
Clustering algorithms: - Partition-based: K-Means, MiniBatchKMeans - Density-based: DBSCAN, HDBSCAN, OPTICS - Hierarchical: AgglomerativeClustering - Probabilistic: Gaussian Mixture Models - Others: MeanShift, SpectralClustering, BIRCH
Dimensionality reduction:
- Linear: PCA, TruncatedSVD, NMF
- Manifold learning: t-SNE, Isomap, LLE, MDS, ClassicalMDS (1.8+)
- External (install separately): UMAP (umap-learn)
- Feature extraction: FastICA, LatentDirichletAllocation
When to use: - Customer segmentation, anomaly detection, data visualization - Reducing feature dimensions, exploratory data analysis - Topic modeling, image compression
See: references/unsupervised_learning.md for detailed documentation.
Tools for robust model evaluation, cross-validation, and hyperparameter tuning.
Cross-validation strategies: - KFold, StratifiedKFold (classification) - TimeSeriesSplit (temporal data) - GroupKFold (grouped samples)
Hyperparameter tuning: - GridSearchCV (exhaustive search) - RandomizedSearchCV (random sampling) - HalvingGridSearchCV (successive halving)
Metrics: - Classification: accuracy, precision, recall, F1-score, ROC AUC, confusion matrix - Regression: MSE, RMSE, MAE, R², MAPE - Clustering: silhouette score, Calinski-Harabasz, Davies-Bouldin
When to use: - Comparing model performance objectively - Finding optimal hyperparameters - Preventing overfitting through cross-validation - Understanding model behavior with learning curves
See: references/model_evaluation.md for comprehensive metrics and tuning strategies.
Transform raw data into formats suitable for machine learning.
Scaling and normalization: - StandardScaler (zero mean, unit variance) - MinMaxScaler (bounded range) - RobustScaler (robust to outliers) - Normalizer (sample-wise normalization)
Encoding categorical variables: - OneHotEncoder (nominal categories) - OrdinalEncoder (ordered categories) - LabelEncoder (target encoding)
Handling missing values: - SimpleImputer (mean, median, most frequent) - KNNImputer (k-nearest neighbors) - IterativeImputer (multivariate imputation)
Feature engineering: - PolynomialFeatures (interaction terms) - KBinsDiscretizer (binning) - Feature selection (RFE, SelectKBest, SelectFromModel)
When to use: - Before training any algorithm that requires scaled features (SVM, KNN, Neural Networks) - Converting categorical variables to numeric format - Handling missing data systematically - Creating non-linear features for linear models
See: references/preprocessing.md for detailed preprocessing techniques.
Build reproducible, production-ready ML workflows.
Key components: - Pipeline: Chain transformers and estimators sequentially - ColumnTransformer: Apply different preprocessing to different columns - FeatureUnion: Combine multiple transformers in parallel - TransformedTargetRegressor: Transform target variable
Benefits: - Prevents data leakage in cross-validation - Simplifies code and improves maintainability - Enables joint hyperparameter tuning - Ensures consistency between training and prediction
When to use: - Always use Pipelines for production workflows - When mixing numerical and categorical features (use ColumnTransformer) - When performing cross-validation with preprocessing steps - When hyperparameter tuning includes preprocessing parameters
See: references/pipelines_and_composition.md for comprehensive pipeline patterns.