Want to learn how NLP/CV Transformers, GPT, BERT, PaLM, T5, etc is made in 150 lines from scratch?

TensorFlow Show and Tell
1

Hello everyone!

If you are interested in learning how stuff like ChatGPT, NLP Transformers, NLP/NPU, etc in Machine Learning does works, I’ve a great Project to offer you all then!

I’ve written a modular and functional transformer module, completely on Python, implementing everything from scratch in well structured and user-friendly format under as few as less than 150 lines of code. Nevertheless to say, Transformer Architecture in NLP is one of the biggest advancement in last 5 - years and all NLP-based AI tools, starting from ChatGPT rely on that.

Check out my code here: GitHub - abhaskumarsinha/Keras-implementation-of-Transformer-Architecture: This repository presents a Python-based implementation of the Transformer architecture on Keras TensorFlow library, as proposed by Vaswani et al. in their 2017 paper "Attention is all you need."

The modular version of the implementation enables it to adapt to any modern language model easily and can be scaled without any issue.

I’ve plans to implement RLHF (Reinforcement Learning from Human Feedback) from scratch using Keras, TensorFlow library. So, this would complete the whole ChatGPT implementation to OpenSource.

In case you want to connect or have great ideas, feel free to ask for it. I’m all there.

Thank You.

3

Impressive implementation of NLP Transformers and ChatGPT in under 150 lines of Python code. Check out the GitHub repository for a modular and scalable implementation. Looking forward to future plans like RLHF. Open to connecting and discussing ideas. Great work!

4

Hello @Ali_Greer Thank you sir. I’m working on RLHF as well as Adapters (LoRA methods - PEFT) to get all of these into TensorFlow. Feel free to connect to me via the official E-mail provided in the GitHub profile.