ddbourgin /numpy-ml
Machine learning, in numpy
★ 16,322⑂ 3,730👁 445⑃ 72● 53▤ 0
+0
日增长
16,322
stars
3,730
forks
445
watchers
72
prs
53
issues
0
releases
README
numpy-ml
Ever wish you had an inefficient but somewhat legible collection of machine
learning algorithms implemented exclusively in NumPy? No?
Installation
For rapid experimentation
To use this code as a starting point for ML prototyping / experimentation, just clone the repository, create a new virtualenv, and start hacking:
sh
$ git clone https://github.com/ddbourgin/numpy-ml.git
$ cd numpy-ml && virtualenv npml && source npml/bin/activate
$ pip3 install -r requirements-dev.txt
As a package
If you don't plan to modify the source, you can also install numpy-ml as a
Python package: pip3 install -u numpy_ml.
The reinforcement learning agents train on environments defined in the OpenAI
gym. To install these alongside numpy-ml, you
can use pip3 install -u 'numpy_ml[rl]'.
Documentation
For more details on the available models, see the project documentation.
Available models
Click to expand!
1. Gaussian mixture model
- EM training
2. Hidden Markov model
- Viterbi decoding
- Likelihood computation
- MLE parameter estimation via Baum-Welch/forward-backward algorithm
3. Latent Dirichlet allocation (topic model)
- Standard model with MLE parameter estimation via variational EM
- Smoothed model with MAP parameter estimation via MCMC
4. Neural networks
Layers / Layer-wise ops
- Add
- Flatten
- Multiply
- Softmax
- Fully-connected/Dense
- Sparse evolutionary connections
- LSTM
- Elman-style RNN
- Max + average pooling
- Dot-product attention
- Embedding layer
- Restricted Boltzmann machine (w. CD-n training)
- 2D deconvolution (w. padding and stride)
- 2D convolution (w. padding, dilation, and stride)
- 1D convolution (w. padding, dilation, stride, and causality)
Modules
- Bidirectional LSTM
- ResNet-style residual blocks (identity and convolution)
- WaveNet-style residual blocks with dilated causal convolutions
- Transformer-style multi-headed scaled dot product attention
Regularizers
- Dropout
Normalization
- Batch normalization (spatial and temporal)
- Layer normalization (spatial and temporal)
Optimizers
- SGD w/ momentum
- AdaGrad
- RMSProp
- Adam
Learning Rate Schedulers
- Constant
- Exponential
- Noam/Transformer
- Dlib scheduler
Weight Initializers
- Glorot/Xavier uniform and normal
- He/Kaiming uniform and normal
- Standard and truncated normal
Losses
- Cross entropy
- Squared error
- Bernoulli VAE loss
- Wasserstein loss with gradient penalty
- Noise contrastive estimation loss
Activations
- ReLU
- Tanh
- Affine
- Sigmoid
- Leaky ReLU
- ELU
- SELU
- GELU
- Exponential
- Hard Sigmoid
Top issues
#34How to use for word2vec training?9 comments · 2019-07-25#79Import of collections.Hashable fails in Python 3.104 comments · 2022-10-19#14Usage to build CNN Network4 comments · 2019-07-09#46implementation of Lasso regression3 comments · 2020-02-27#69Feature Request: Clustering Kmeans (hard and soft version)2 comments · 2021-05-25#29Feature: Cosine Proximity Loss Function2 comments · 2019-07-15#93Example of MLP architecture1 comments · 2023-12-15#77`numpy_ml.linear_model.LinearRegression.predict()` generates `ValueError` when used with copy-pasted code, but pip installed version works as expected!!1 comments · 2022-03-20