Meta Learning (AutoML)
AutoML is the attempt to automate model choice, hyperparameters, architecture search, and the other steps a practitioner usually does by hand. The page puts the personal caveats first, because automation does not replace looking at the data, then follows the systems curated by automl.org: full ML systems, hyperparameter optimization, architecture search, and auto feature engineering, and ends with PyCaret.
The same notes are in Model Families.
What is? Automated Machine Learning provides methods and processes to make Machine Learning available for non-Machine Learning experts, to improve efficiency of Machine Learning and to accelerate research on Machine Learning.
Personal note: automl algorithms in this field will bridge the gap and automate several key processes, but it will not allow a practitioner to do serious research or solve business or product problems easily. The importance of this field is to advance each subfield, whether HPO, NAS, etc. these selective novelties can help us solve specific issues, i.e, lets take HPO, we can use it to save time and money on redundant parameter searches, especially when it comes to resource heavy algorithms such as Deep learning (think GPU costs).
Personal thoughts on optimizations: be advised that optimizing problems will not guarantee a good result, you may over fit your problem in ways you are not aware of, beyond traditional overfitting and better accuracy doesn't guarantee a better result (for example if your dataset is unbalanced, needs imputing, cleaning, etc.).
Always examine the data and results in order to see if they are correct.
With those caveats stated, the projects themselves are backed up in one place: Automl.org’s github — it has a backup for the following projects., the AutoML-Freiburg-Hannover organization.
Automl.org is a joint effort between two universitie, freiburg and hannover, their website curates information regarding:
HPO — hyper parameter optimization
NAS — neural architecture search
Meta Learning — learning across datasets, warmstarting of HPO and NAS etc.
Automl aims to automate these processes:
Preprocess and clean the data.
Select and construct appropriate features.
Select an appropriate model family.
Optimize model hyperparameters.
Postprocess machine learning models.
Critically analyze the results obtained.
Historically, AFAIK AutoML’s birth started with several methods to optimize each one of the previous processes in ml. IINM, weka’s paper (2012) was the first step in aggregating these ideas into a first public working solution.
The following is referenced from AutoML.org:
ML Systems
The first group automates the whole pipeline, starting from that WEKA line of work.
The same notes are in Genetic Algorithms & Genetic Programming.
AutoWEKA is an approach for the simultaneous selection of a machine learning algorithm and its hyperparameters; combined with the WEKA package it automatically yields good models for a wide variety of data sets. Auto-sklearn is an extension of AutoWEKA using the Python library scikit-learn which is a drop-in replacement for regular scikit-learn classifiers and regressors. TPOT is a data-science assistant which optimizes machine learning pipelines using genetic programming.
The platform vendors ship their own. (google) H2O AutoML is the H2O AutoML documentation; it provides automated model selection and ensembling for the H2O machine learning and data analytics platform. Google Brain AutoML is on ( git. TransmogrifAI is an AutoML library running on top of Spark. MLBoX is an AutoML library with three components: preprocessing, optimisation and prediction
For tabular data with reports attached, MLJar (git) medium, 2 — Automated Machine Learning for tabular data mljar builds a complete Machine Learning Pipeline. Perform exploratory analysis, search for a signal in the data, and discover relationships between features in your data with AutoML. Train top ML models with advanced feature engineering, many algorithms, hyper-parameters tuning, Ensembling, and Stacking. Stay ahead of competitors and predict the future with advanced ML. Deploy your models in the cloud or use them locally. The git repo is the mljar-supervised Python package, and the medium post is about its main advantage, the markdown reports produced from model training. On top of the pipeline it adds advanced feature engineering, algorithms selection and tuning, automatic documentation, and ML explanations, as the figure below shows.
Hyper param optimization
A full system hides the search. When only the hyperparameters need automating, which is where the personal note says the money is saved, these are the tools.
The same notes are in HYPER PARAM GRID SEARCHES and Hyper Parameter Optimization.
Hyperopt is Hyperopt by jaberg, including the TPE algorithm. Sequential Model-based Algorithm Configuration (SMAC) is the SMAC page. Spearmint is a package to perform Bayesian optimization according to the algorithms outlined in the paper Practical Bayesian Optimization of Machine Learning Algorithms by Jasper Snoek, Hugo Larochelle and Ryan P. Adams. BOHB: Bayesian Optimization combined with HyperBand, and RoBO – Robust Bayesian Optimization framework, are kept at the end of the page. SMAC3 – a python re-implementation of the SMAC algorithm.
Architecture Search
Hyperparameters tune a fixed network; architecture search lets the search choose the network itself.
Auto-PyTorch is automatic architecture search and hyperparameter optimization for PyTorch. AutoKeras is the same idea for Keras. DEvol is an early POC of genetic neural architecture search. HyperAS is Keras + Hyperopt, a very simple wrapper for convenient hyperparameter optimization, and talos runs hyperparameter experiments with TensorFlow and Keras.
Auto Feature Engineering
The step before any model is the features, and that can be automated too.
The same notes are in FEATURE ENGINEERING.
The reading here is automated feature engineering on medium by will koehrsen: https://towardsdatascience.com/automated-feature-engineering-in-python-99baf11cc219.
PYCARET
After the systems and the separate search tools, PyCaret wraps the whole path in a few calls. 1. What is? by vidhaya is the Analytics Vidhya post on using PyCaret to build a model in seconds when running low on time. PyCaret is an open-source, machine learning library in Python that helps you from data preparation to model deployment. It is easy to use and you can do almost every data science project task with just one line of code.
Deprecated links
These links and images no longer work. The original wording is kept here. A same-resource copy, when one was checked, is used above.
Auto-sklearn. This address no longer opens: http://automl.github.io/auto-sklearn/stable/
H2O AutoML. This address no longer opens: http://docs.h2o.ai/h2o/latest-stable/h2o-docs/automl.html
H2O machine learning and data analytics platform. This address no longer opens: http://docs.h2o.ai/h2o/latest-stable/h2o-docs/welcome.html
2 (MLJar TDS). This address no longer opens: https://towardsdatascience.com/automating-eda-machine-learning-6ddb76c1eb4d
BOHB: Bayesian Optimization combined with HyperBand. This address no longer opens: https://www.automl.org/automl/bohb/
RoBO – Robust Bayesian Optimization framework. This address no longer opens: http://www.automl.org/automl/robo/
Last updated
Was this helpful?