genai-weekly-show-case

Generative AI Weekly Update: Last Week's Progress

AbuBakar khan lakhwera's photo
AbuBakar khan lakhwera
·

2 min read

Introduction

Last week, I took significant steps in exploring Generative AI concepts, focusing on tokenization, stemming, lemmatization, and Transformers. This article highlights my learnings and the practical tasks I undertook during the week.

Tokenization

Explored on Colab | 4 days ago

Tokenization is the process of breaking text into smaller units like words, subwords, or characters, preparing it for machine learning models. I explored:

  • Splitting text into tokens and understanding different approaches like Byte Pair Encoding (BPE) and WordPiece.

Key Takeaways:

  • Tokenization helps models handle out-of-vocabulary words effectively.

  • Subword tokenization strikes a balance between vocabulary size and representational power.

Practical Task:

  • Used Hugging Face’s Tokenizer library to implement tokenization.

  • Analyzed its impact on a small language model’s performance.

Stemming and Lemmatization

Explored on Colab | Last Week

Text normalization is key to improving language understanding. I revisited:

  • Stemming: A faster method that removes affixes to reduce words to their root form.

  • Lemmatization: A context-aware method that maps words to their base form.

Key Takeaways:

  • Stemming is computationally efficient but less accurate.

  • Lemmatization, while slower, provides more meaningful transformations for NLP tasks.

Practical Task:

  • Applied stemming and lemmatization to a dataset of tweets.

  • Compared their effects on text classification accuracy.

Transformers

Explored on Colab | 3 days ago

Transformers have revolutionized NLP and Generative AI. Last week, I focused on understanding:

  • The Transformer architecture, including self-attention and multi-head attention mechanisms.

  • The advantages of Transformers over RNNs and LSTMs in handling long-range dependencies.

Key Takeaways:

  • Self-attention allows models to process sequences in parallel, making Transformers highly efficient.

  • Transformers form the foundation of state-of-the-art models like BERT and GPT.

Practical Task:

  • Built a mini-Transformer model using PyTorch.

  • Visualized attention weights to understand how the model processes input text.

Reflections and Next Steps

Last week's progress deepened my understanding of the core techniques powering Generative AI. Next steps include:

  • Fine-tuning pre-trained models for specific tasks.

  • Exploring advanced tokenization methods for handling large-scale datasets.

  • Continuing to document experiments and build projects.

Conclusion

Generative AI is a fascinating field, and each week brings new insights and challenges. By reflecting on my progress, I aim to not only solidify my understanding but also inspire others to embark on their AI journeys.

Let’s keep learning and innovating together!

genai