You might have heard about or used artificial intelligence-based tools like Copilot, ChatGPT, or DALL-E, but how do these tools actually work? This section will cover how tools like these are made, how they generate information, and some considerations to think about as you assess if and how to use them. This guide is designed to broaden one's basic understanding of AI, explore generative AI tools, and critically evaluate the available resources.
This toolkit is currently under construction. Please reach out to Chris Duffy with any additions or comments.
Generative AI
Generative Artificial Intelligence, or Generative AI, is a class of computer algorithms able to create digital content – including text, images, video, music and computer code. They work by deriving patterns from large sets of training data that become encoded into predictive mathematical models, a process commonly referred to as ‘learning’. Generative AI models do not keep a copy of the data they were trained on, but rather generate novel content entirely from the patterns they encode. People can then use interfaces like ChatGPT to input prompts – typically instructions in plain language – to make generative AI models produce new content.
The core of a generative AI is a trained deep-learning model that understands and generates text, image, or other media in a human-like fashion based on a given user input, i.e. prompt. This model is trained on massive amounts of data to learn from patterns in the data. For example, it would learn that certain words tend to follow others, or that certain phrases are more common in certain contexts. The model uses the prompt to produce a completion, which is then presented back to users.
The quality of the generated output depends on several factors, including the amount and quality of the training data, the prompt's complexity, and the model's size. Larger models usually generate better output but require more computing power and resources. Notable examples of generative AI systems include ChatGPT and Microsoft Copilot for Web, which focus on language generation, and and DALL-E which focus on image generation. Large language models and other generative AI models typically run on neural networks. These networks are composed of artificial neurons, which are modelled on how our own neurons in our brains work.
Large Language Models
Large language models (LLMs) are a category of foundation models trained on immense amounts of data making them capable of understanding and generating natural language and other types of content to perform a wide range of tasks.
Researchers have spent years implementing LLMs at different levels to enhance their natural language understanding (NLU) and natural language processing (NLP) capabilities. This has occurred alongside advances in machine learning, machine learning models, algorithms, neural networks and the transformer models that provide the architecture for these AI systems.
LLMs are a class of foundation models, which are trained on enormous amounts of data to provide the foundational capabilities needed to drive multiple use cases and applications, as well as resolve a multitude of tasks.
In a nutshell, LLMs are designed to understand and generate text like a human, in addition to other forms of content, based on the vast amount of data used to train them. They have the ability to infer from context, generate coherent and contextually relevant responses, translate to languages other than English, summarize text, answer questions (general conversation and FAQs) and even assist in creative writing or code generation tasks.
They are able to do this thanks to billions of parameters that enable them to capture intricate patterns in language and perform a wide array of language-related tasks. LLMs are revolutionizing applications in various fields, from chatbots and virtual assistants to content generation, research assistance and language translation.
