How AI Generates Content
Recently, a new term has increasingly emerged in the field of artificial intelligence: Generative AI. It came into focus especially around highly complex applications such as ChatGPT and describes a new level of capabilities for AI. In contrast to AI functionality that makes distinctions (hence discriminative AI), generative AI is able to create its own content such as text, images, audio content, code, and more.
Matching Data to Output Criteria
Looking at previous applications of AI, it is apparent that the new technology is predominantly used to make distinctions: Delivery bill or invoice? Traffic sign or billboard? Spam or no spam? The algorithms have learned this through so-called supervised learning. In this process, they are trained with data that has been prepared by humans, namely assigned to the appropriate output category. From this, the algorithms extract the features that distinguish one object from another and can predict to which category an unknown object most likely belongs.
Expanding the Horizons of AI
An AI that has learned in this way has a thoroughly limited horizon. In a sense, this can be broadened by letting algorithms learn differently: unsupervised or semi-supervised. In unsupervised learning, data is not labeled; in semi-supervised learning, only a small portion of the data is labeled by humans. The algorithm “works out” the patterns according to which it orders the data it is given on its own. In doing so, it can identify patterns other than those that determine assignment to a given output category in supervised learning. The dog and cat pictures could then be sorted, for example, according to the color of the animals’ fur or whether they were photographed on the meadow or the sofa. But again, the algorithm does nothing but make distinctions, albeit in terms of output with no input from the human.
When you pull unsupervised learning bigger, generative AI comes out of it. “Pulling bigger” in this case means that instead of small AI models, predefined rules for processing input and a rather manageable amount of training data, increasingly larger and more complex AI models, learning by means of neural networks and huge amounts of data from a wide variety of digital data sources are used. The AI model is simply given a specification on the desired output, for example “text on the current state of AI”, but acts independently in seeking input and processing it without further human intervention. New architectures in neural networks (for example, GAN and Transformer) increase the performance of the models. In particular, with Transformer, because entire data sequences (sentences and text segments) are used instead of individual data points, the creation of speech-based content is feasible at quite a high level.
Conclusion
Regarding the distinction “generative AI” and “discriminative AI”, some authors consider that any AI is “generative” even if it “only” makes distinctions, because it also generates them and does not take them from anywhere. In other words, “generative AI” technically does nothing more than what an AI can do, namely make distinctions, only on a much larger scale, with much more resource input, and with considerably less human intervention than the simple-minded distinctive AI. But even generative AI is far from being able to think for itself; it only processes what it finds on millions of pages on the web. This is easily enough to answer e-mails in the back office, create offers, contract texts, marketing texts or product descriptions, summarize texts, personalize advertising or answer questions as a chatbot in customer support.
