The Generative Ai Stack.

You surely have heard about generative ai and Large language models. Most of the hype and attention is given to the foundation models, but it is only a small part of any Ai based software solution. Remember that the job of any tool is to solve problems. Let’s go through the entire stack and try to understand all the technologies involved.

Each layer builds upon those below it.

1. Hardware Layer

This layer is what moves the atoms to represent data and perform calculations. Training large neural networks requires large quantity of data and very fast and parallel computing.

Compute:

Think GPUs, TPUs and the surrounding infrastructure. It also includes software that allows us to interact with the hardware itself like CUDA.

Storage:

Database and Object for storing the training and test datasets.

Graph and Vector Databases store learned knowledge and embeddings.
Embeddings are vector representations of words or concepts used in machine learning.

2. Model Layer

Foundation Models

Mostly transformer based model architecture as the base. They are learning to predict the next word given the previous sequence of words.

These models are then aligned to output useful responses using RLHF (Reinforcement Learning with Human Feedback) or other techniques.

Most of these models are trained on massive datasets, usually comprising of a large portion of the (public) internet.

Custom Models

You can train your own models from scratch on your own data. But the larger the models, the more data is needed to learn. So depending on the size of the dataset available, it might not be possible to train a huge model from scratch. Such models will be considered a foundation model for your dataset.

Fine Tuned Models

If you have a specific dataset (possibly proprietary to your organization)
It may be better to fine tune a foundation models.

The foundation models already understand the general relationship between words (ideas and concepts represented by the words). Fine tuning allows the model to learn the word relationships within the specific dataset improving performance on that topic.

<subscribe>

3. Application Layer

This layer is where we can use the models, data, embeddings to create a useful application.

This is mostly about conventional programming and using the generative Ai as another tool to solve problems better, faster, or cheaper.

Support Layer

This layer runs alongside the main AI layers. It consists of the infrastructure and tools that enhance the development and use of Generative AI.

It includes technologies and tools like Graph and Vector Databases, traditional programming, Hugging Face 🤗, etc.

Conclusion

The Generative AI Stack provides a powerful and versatile framework for developing innovative applications powered by generative AI models. By understanding the different layers of the stack and the tools within each layer, developers can leverage generative AI to solve problems in new and creative ways.

In the upcoming posts, I will delve deeper into each of the layers and explore the technologies behind them, build intuition about them and how they can be used to build useful applications.