What is Generative AI ? For the given user input(user query), output like text,image, video etc will be generated. This is called generative AI. How it generates content? A model will be used to generate output. i.e model will receive input and based on that, it will generate a output. What is a model? At its core, model is nothing but a mathematical equation.It will be multidimensional. vast amount of multimodal data (text,audio,video, image etc) would be subjected to training to get the required mathematical equation. To get the desired state, backpropagation will be carried out. 120b model means, equation has, 120 billion parameters. One of the commonly used model type in generative ai is LLM. What is LLM ? LLM stand for large language model. LLM basically predict the next word. If i provide the input query as hello to gpt model, based on the data it was trained, it will predict and returns the next word. In my case i got Hello,How can I help you today? Response will not be generated and sent all at once. It will be generated one by one and sent in a streamed manner(by means of SSE event). How it predicts the next word? In the above example, when i gave hello as input, why "Hi, how can i help you today was returned" ? not hi or world etc . For the given input, model provides some of possibility words like hi, world, howdy, how may i help you etc. For each possible word, it gives a score(most occuring probability). Word which is having highest score will be returned as output. If the scores are hi (0.2), world(0.4), howdy(0.1), how may i help you(0.7), highest score is 0.7, so "how may i help you is returned". Can we tweak the model to control how output should be? This can be achieved by tweaking the following parameters
- Temperature
- Top- K
- Top - P Temperature Temperature controls whether the output generated be factual or imaginative. Temperature value lies between 0 - 1. If it is closer to zero, then it more of a factual and the value is closer to 1, then it is more of a imaginative. Example prompt for low temperature Example prompt for high temperature 2.Top -K K denotes the number of tokens to be returned. For the prompt, The cat sat on the ---- following words are predicted for the varying values of k. 3.Top - P Threshold percentage will be set. From the set of predicted words, those words will be taken whose cumulative probability score approximates to threshold percentage. For the prompt, The cat and top_p = 0.7