1  What is Artificial Intelligence

Central question: What makes an LLM different from previous generations of AI?

1.1 Introduction

Few technical terms have crossed the boundary from the lab to everyday conversation as rapidly as “artificial intelligence.” Before discussing what distinguishes a Large Language Model (an LLM) from the technologies that preceded it, we need to place it within a longer history, one that begins well before today’s conversational assistants. Understanding this genealogy is not an exercise in scholarship: every limitation we will encounter in later chapters has roots in conceptual choices made decades ago, and every recent advance is, to a large extent, a response to an inherited impasse.

This chapter builds the map of the territory. It presents Artificial Intelligence (AI) not as a single technology, but as a broad field that houses very different paradigms, some of them conceptually opposed. We will see how the dream of “machines that think” unfolded into two great traditions, why one of them came to dominate, and where exactly in that tree LLMs are located. By the end, the chapter’s central question, what makes an LLM different from earlier generations of AI, will shed its vague answer (“it’s more advanced”) and acquire precise contours.

The approach here is deliberately conceptual. Our concern at this stage is not how to build an AI system, but what each type of system is, where it came from, and why it works, or fails to work. This framing will accompany the entire book.

1.2 Learning Objectives

Upon completing this chapter, the reader should be able to:

• Distinguish Artificial Intelligence, Machine Learning, Deep Learning, and Generative AI, understanding that they are nested levels, not synonyms.
• Explain the conceptual difference between the symbolic tradition (rule-based) and the connectionist tradition (learning from data), with a practical example of each paradigm.
• Understand why rule-based systems are deterministic and why LLMs introduce non-determinism.
• Locate LLMs within the taxonomy of AI and state, with precision, what distinguishes them from earlier generations.

1.3 Guiding Questions

• If a calculator solves operations a human would find difficult, why don’t we call it “intelligent”?
• What exactly does it mean to say that a machine “learns”?
• Why did explicit rule-based systems, despite being transparent and predictable, fail at tasks such as recognizing a face or translating a text?
• Why can the same question, asked twice to an LLM, receive different answers, while a rule-based system always returns the same answer?

1.4 Conceptual Map of the Chapter

Figure 1.1 organizes the chapter’s central concepts and their relationships. It anticipates the argumentative structure: AI splits into two traditions; one of them, learning from data, branches out until it reaches LLMs.

Figure 1.1 - Conceptual map: AI paradigms and the position of LLMs

Source: prepared by the author (2026).

1.5 AI, Machine Learning and Deep Learning: Who’s Who

1.5.1 What is Artificial Intelligence

Broadly speaking, Artificial Intelligence is the field of study that seeks to build systems capable of performing tasks that, when carried out by human beings, we would associate with intelligence: perceiving, reasoning, learning, deciding, communicating in natural language. This intuitive definition is useful as a starting point, but it has a flaw that has accompanied AI from the outset: it is a moving target. As a task is successfully automated, we tend to stop considering it “intelligent,” a phenomenon so recurrent that it received its own name, the AI effect. Playing chess was once the canonical example of intelligence; today, a program that defeats world champions is seen as mere computation. The boundary of what we call AI shifts over time.

Technically, a less psychological definition is preferred. The field’s reference treatise characterizes AI through four approaches, depending on whether the system seeks to think like humans, act like humans, think rationally, or act rationally, the last one, that of the rational agent that acts to maximize the achievement of its goals, being the most influential in contemporary research (Russell & Norvig, 2021). The importance of this definition for our journey lies in shifting the criterion from “imitating the human mind” to “behaving appropriately in pursuit of an objective,” exactly the kind of criterion under which LLMs are evaluated.

The expression “artificial intelligence” itself was born from a specific document: the proposal, drafted in 1955 by John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon, for a summer workshop at Dartmouth the following year (McCarthy et al., 1955). That 1956 seminar is widely recognized as the founding act of the field. But the question that motivated it had already been formulated, more precisely, a few years earlier.

1.5.2 Turing’s Question

In 1950, Alan Turing published in the journal Mind the article that would open the philosophical and technical discussion about thinking machines (Turing, 1950). Rather than directly confronting the question “can machines think?,” which he considered ill-defined, since it depends on what one means by “think”, Turing proposed substituting it with an operational test, the imitation game: if a human interrogator, conversing in writing with a human and with a machine without knowing which is which, cannot reliably tell them apart, then denying the machine the attribute of “intelligent” becomes a difficult position to sustain. The lasting merit of the proposal was repositioning the question: the criterion ceases to be the internal nature of the system and becomes its observable behavior in language. It is worth holding on to this point, as it will reappear, transformed, when we discuss how we evaluate an LLM.

1.5.3 Two Traditions: Symbolic and Connectionist

The technical history of AI can be read as the tension between two traditions. The first, symbolic, starts from the hypothesis that intelligence consists of manipulating symbols according to explicit logical rules. Building a system, under this view, means encoding the knowledge of human specialists into rules of the type “if condition X, then conclusion Y.” This tradition, retrospectively nicknamed Good Old-Fashioned AI by Haugeland (Haugeland, 1985), dominated the early decades and produced, in the 1970s and 1980s, expert systems: programs that reproduced the reasoning of professionals in narrow domains.

The second tradition, connectionist, starts from the opposite bet: instead of programming knowledge, it lets the system extract it from examples. One does not write the rule that distinguishes a cat from a dog; one shows thousands of labeled images and allows the system to adjust its internal parameters until it gets it right. From this tradition descend Machine Learning, Deep Learning, and, ultimately, LLMs. The subsections that follow traverse each paradigm, from the oldest to the most recent, with an infographic and a practical example for each.

1.6 From the Perceptron to Generative AI

Before surveying the three major approaches (rules, machine learning, and deep learning), it’s worth situating them in time. The history of AI is not a straight line of progress; it is made of waves of enthusiasm followed by disappointment, the so-called AI winters, in which funding and interest cooled when promises went unfulfilled. Knowing these milestones helps read the current moment with perspective, and distrust both hype and easy skepticism alike. Figure 1.2 summarizes the trajectory.

Figure 1.2 - Timeline of AI: from Dartmouth to LLMs, with the winters shaded

Source: prepared by the author (2026), based on the chapter’s references.

Notice the pace: decades of slow, specific advances (one system for chess, another for images), until, from the 2010s onward, deep learning and then Transformers unlocked a new generality. The three approaches below did not succeed one another cleanly; they coexisted and still coexist, but it is learning, not rules, that dominates the current moment.

1.6.1 Rule-Based Learning (Symbolic Systems)

In the symbolic paradigm, all of the system’s knowledge is supplied from the outside, by human beings, in the form of explicit rules. The program discovers nothing: it applies, mechanically, previously written conditions. The virtue of this arrangement is total transparency, since each decision can be traced back to the rule that produced it, making the system predictable and auditable. Its weakness appears when the necessary knowledge is tacit, that is, when we know how to perform the task but cannot articulate the rules we use. Recognizing a face, understanding an ambiguous sentence, or perceiving irony are such cases. Add to this the combinatorial explosion: the number of rules needed to cover real-world exceptions grows uncontrollably, and maintenance becomes prohibitive.

A practical example illuminates the paradigm. The MYCIN system, developed at Stanford University in the early 1970s by Edward Shortliffe, diagnosed bacterial infections from a set of about 600 if-then rules and recommended antibiotics with dosage adjusted to the patient’s weight (Shortliffe, 1976). One of its notable features was the ability to explain the conclusion by retracing the chain of rules that supported it. Figure 1.3 schematizes this operation: an input travels through an engine that applies rules written by specialists, until it produces an output whose origin is entirely traceable.

Figure 1.3 - Rule-based learning: an engine that applies human-written conditions

Source: prepared by the author (2026), based on Shortliffe (1976).

Important

Real case: ELIZA and the illusion of understanding. In 1966, long before LLMs, Joseph Weizenbaum created ELIZA, a program that imitated a psychotherapist with very few rules: it simply rephrased the user’s sentences as questions (“I am sad” became “why are you sad?”). There was no understanding whatsoever, only manipulation of text patterns. Even so, Weizenbaum was disturbed to see people, including his own secretary, growing attached to the program and attributing real empathy to it, to the point of asking for privacy to “talk.” This phenomenon, the human tendency to project understanding and intention where there is only mechanism, became known as the ELIZA effect. It is more relevant than ever: today’s LLMs are infinitely more fluent than ELIZA, and the same projection, now much stronger, is the root of several misconceptions this book combats, from “AI understands” to “AI has an opinion.” Recognizing the ELIZA effect in oneself is the pilot-reader’s first line of defense.

1.6.2 Machine Learning (Learning from Examples)

Machine Learning inverts the previous logic: instead of receiving ready-made rules, the system infers them from data. The term was coined by Arthur Samuel in 1959, in the context of a program that played checkers and improved its performance with experience (Samuel, 1959). The formulation that became standard is Tom Mitchell’s: a program is said to learn from experience \(E\), with respect to a class of tasks \(T\) and a performance measure \(P\), if its performance on \(T\), as measured by \(P\), improves with \(E\) (Mitchell, 1997). The definition is deliberately operational. It invokes neither consciousness nor understanding: learning, here, means exclusively improving a metric on a task as experience accumulates.

Consider, as a practical example, filtering unwanted messages (spam). In the rule-based approach, someone would need to anticipate every suspicious pattern manually. In the learning approach, the system is presented with a large set of messages already labeled as “spam” or “not spam,” and it adjusts the importance of each clue on its own. Figure 1.4 illustrates this mechanism: labeled examples feed the model, which turns its internal “knobs,” the clue weights, until it separates the two classes as best it can. When a new trick emerges, there is no need to rewrite any rule; one simply provides updated examples.

Figure 1.4 - Machine learning: the model adjusts weights from labeled examples

Source: prepared by the author (2026).

1.6.3 Deep Learning

Within Machine Learning, Deep Learning designates the use of neural networks with many layers, capable of learning representations of data at increasing levels of abstraction, from the edges of an image up to the concept of “face,” for example (LeCun et al., 2015). The idea of a neural network is old and had already experienced a winter: in 1969, Minsky and Papert demonstrated that the perceptron, an elementary neural model, could not even represent the “exclusive or” logical function, which cooled enthusiasm and funding for the area for years (Minsky & Papert, 1969). Figure 1.5 opens the “box” of a neural network: each connection carries a weight, a number that regulates the influence of one unit on the next, and it is this set of weights that training adjusts. The deeper layers combine what the earlier ones extracted, moving from simple to complex.

Figure 1.5 - Deep learning: weights on connections and increasing abstraction across layers

Source: prepared by the author (2026). Weight values are illustrative.

The most celebrated practical example of Deep Learning is image recognition. Until 2012, automatically identifying the object present in a photograph was a task in which systems frequently erred. That year, a deep neural network trained on graphics processing units (GPUs) won the ImageNet competition by a wide margin, reducing the baseline error rate from about 26% to 15% (Krizhevsky et al., 2012). The result did not introduce an entirely new idea; it showed that the conjunction of three factors, large volumes of labeled data, parallel computing capacity, and algorithmic adjustments, suddenly made Deep Learning competitive. Figure 1.6 tracks the continuous drop in error in the following years, until it fell below the estimated human-level accuracy of around 5% (Russakovsky et al., 2015).

Figure 1.6 - Drop in ImageNet (ILSVRC) classification error by year

Source: prepared by the author (2026), based on Krizhevsky et al. (2012) and Russakovsky et al. (2015). The dashed line marks the estimated human-level error (around 5%).

1.6.4 Generative AI and the Position of LLMs

The models described so far were, for the most part, discriminative: they classified, that is, assigned labels to inputs. Generative AI designates models that produce new content, images, text, or audio, rather than merely labeling. A milestone of this turn was the generative adversarial networks proposed by Goodfellow and colleagues in 2014, in which two networks compete, one generating samples and the other trying to distinguish them from real ones (Goodfellow et al., 2014). In the language domain, the architectural inflection came in 2017 with the Transformer, introduced in the paper Attention Is All You Need (Vaswani et al., 2017), an architecture that will be the entire subject of Module 4. It was on this foundation that LLMs were built: models trained on enormous text corpora to predict the continuation of a sequence and which, when scaled up, exhibited the ability to solve tasks from just a few examples in the prompt itself, as documented for GPT-3 in 2020 (Brown et al., 2020).

Figure 1.7 shows, schematically, what an LLM does at each step: given the words already present, it computes a probability distribution over what the next word will be and then samples one of them. It is this sampling that explains why the same question can generate different answers, a point we will return to when discussing determinism.

Figure 1.7 - How an LLM generates text: predicting and sampling the next token

Source: prepared by the author (2026). Probabilities are illustrative.

It is here that the chapter’s central question finds its answer. An LLM is, simultaneously, a Machine Learning system (it learns from data), a Deep Learning system (it uses multi-layer neural networks), and a Generative AI system (it produces new content). What distinguishes it from earlier generations is not just scale, but the combination of three traits: generality (the same model handles translation, summarization, programming, and dialogue, without having been programmed for each task), general-purpose learning from unlabeled text, and natural language interaction as an interface. Earlier generations were, as a rule, specialized: one system for chess, another for diagnosis, another for image recognition. The novelty of LLMs is breadth.

It is worth fixing the LLM’s position on this map, because practical decisions depend on it. Models that only classify or predict, such as a spam filter or a fraud detector, are called discriminative; those that produce new content form the generative branch, to which the LLM belongs. But it is not the only member of that branch: diffusion models generate images, video, and audio, and there are models specialized in code. All share the same logic of origin: that of a large-scale pre-trained model that serves as a foundation for many applications, hence the name foundation models. Recognizing that the LLM is a specific case, and not a universal engine, prevents a recurring mistake: treating it as a solution for every problem, when some tasks are better served by a lean discriminative model or by another generative family; a choice that Module 11 revisits from the perspective of cost and suitability.

What made this leap possible was less a single invention and more the convergence of scale: data, parameters, and computational power, without which the same architectures would not exhibit the behavior they now display, as Module 5 details when addressing training. And nothing in this position confers agency upon the LLM: it pursues no goals of its own and “wants” nothing at all, a topic Module 12 develops when discussing human-AI collaboration.

1.6.5 Determinism and Non-Determinism

One concept runs through this entire trajectory and helps us understand LLM behavior: determinism. A system is deterministic when the same input always produces the same output. Rule-based systems are deterministic by construction: faced with the same situation, they apply the same rules and reach the same conclusion, which makes them predictable and auditable. Classical Machine Learning models, once trained, also tend to be deterministic in prediction, since the parameters are fixed and the operation is the direct computation of an output.

LLMs introduce a behavioral difference that is central to understanding them. As Figure 1.7 showed, text generation typically involves sampling from a probability distribution over the next token, with a degree of randomness controlled by a parameter called temperature, detailed in Module 6. For this reason, the same question can receive different answers in different executions. This non-determinism is not a defect but a design choice that favors diversity and fluency; even so, it has important consequences for reproducibility, testability, and evaluation, revisited in Module 7. It is worth noting that even when temperature is reduced to zero to seek the most likely output, small hardware and parallelism variations can prevent perfect reproducibility. The absolute predictability of rule-based systems, therefore, gives way to probabilistic behavior, and this is one of the most important mindset shifts when moving from earlier generations of AI to LLMs.

1.6.6 Narrow AI, General AI, and the Term AGI

A pair of distinctions helps situate everything we’ve seen, and defuses much of the hype. All AI that exists today, without exception, is narrow AI: systems that solve a class of tasks, however broad, without consciousness, will, or understanding in the human sense. A model that plays chess doesn’t know it’s playing; an LLM that converses about any topic doesn’t understand what it says (Module 7). What impresses in LLMs is the breadth of that narrow competence, not a change in nature.

By contrast, there is talk of general AI (AGI, for Artificial General Intelligence): the still-hypothetical idea of a system with cognitive capacity comparable to a human’s in any domain, able to learn and transfer knowledge with a person’s flexibility. It’s important to be honest about this term: AGI has no single, consensual definition, and is used in very different ways, from specific technical goals to marketing promises. There is no agreement on what would count as AGI, nor on whether LLMs are a path toward it or just an impressive technology of another nature. The pilot-reader does well to treat the term with caution: when someone claims “AGI is close,” the first useful question is “what exactly do you mean by AGI?” This book deals with the AI that exists and works today, the narrow kind; the general kind belongs, for now, to the realm of speculation.

1.7 In Practice: Watch Machine Learning Happen

Two free experiments, in the browser, make tangible the difference between programming by rules and learning from examples, the central point of this chapter.

Experiment 1: train your own classifier. Visit Teachable Machine (from Google, at teachablemachine.withgoogle.com). In a few minutes, using your webcam, you “teach” the computer to distinguish, for example, your open hand from your closed fist: just show several examples of each and click train. No one wrote rules about fingers or shadows; the model adjusted its weights from your examples, exactly as Figure 1.4 described. Then test the limits: show it something it didn’t see in training and watch it get it wrong. You’ve just done supervised machine learning without writing a line of code.

Experiment 2: watch a model recognize (and fail at) doodles. Quick, Draw! (also from Google) asks you to draw objects while a model tries to guess what they are, in real time. Besides being fun, it’s instructive: notice when the model gets it quickly and when it gets lost, and notice that it learned from millions of other people’s doodles, which explains its biases (it expects a “cat” to look like the cats most people draw). It’s learning from data, with its strengths and limits, in plain view.

1.8 Analogies: AI as Apprentice, Not as Brain

A recurring analogy describes the difference between the two traditions as the difference between teaching by manual and teaching by examples. The symbolic system is like an employee who receives an exhaustive procedures manual and follows each rule to the letter; the system that learns is like an apprentice who observes thousands of cases and, gradually, develops a “knack” for the task. The analogy works to convey the methodological inversion: from encoding knowledge to extracting it from data. It stops working, however, at two points worth marking from the start. First, the human apprentice’s “knack” relies on world understanding, whereas the model adjusts parameters to minimize error on a metric, without any understanding arising from this (the subject of Module 7). Second, the human apprentice generalizes from very few examples, while classical models required enormous volumes of data. The analogy, therefore, illuminates the superficial how and obscures the deep why, which is why it can never replace the technical explanation.

1.9 The Map of AI Fields

Figure 1.8 represents the relationship among the fields discussed as a containment structure: each level is contained within the one before it. The diagram makes visible what common language tends to conflate: that “AI,” “Machine Learning,” and “Deep Learning” are not interchangeable synonyms, but nested layers. It goes beyond simple containment, however: it distinguishes discriminative models, which classify and predict, from the generative branch, which creates new content; it shows LLMs as one case alongside other generative families; and it indicates, at the base, the axis of scale that made this trajectory possible.

Figure 1.8 - Map of AI fields: containment, discriminative and generative types, and scale

Source: prepared by the author (2026).

The nesting indicates containment, not a temporal flow: all Deep Learning is Machine Learning, but the reverse is false. LLMs inherit the properties of each layer above, which explains why typical failures of statistical models, such as sensitivity to training data and the absence of logical guarantees, reappear in them, however sophisticated they may seem.

1.10 Key Concepts

• Artificial Intelligence (AI): the field that seeks systems capable of tasks associated with human intelligence; under the dominant technical definition, the study of rational agents that act to achieve goals (Russell & Norvig, 2021).
• Symbolic tradition (GOFAI): an approach based on explicit logical rules and knowledge encoded by specialists (Haugeland, 1985).
• Connectionist tradition: an approach in which the system extracts knowledge from examples by adjusting internal parameters.
• Machine Learning: a subfield of AI dedicated to systems that improve their performance with experience (Mitchell, 1997; Samuel, 1959).
• Deep Learning: the use of neural networks with many layers, which learn hierarchical representations of data (LeCun et al., 2015).
• Generative AI: models that produce new content, rather than merely classifying inputs (Goodfellow et al., 2014).
• Determinism: the property of a system in which the same input always produces the same output; present in rule-based systems, it contrasts with the probabilistic behavior of LLMs.
• LLM: a large-scale language model, based on the Transformer architecture, trained to predict text continuations and capable of generalizing to multiple tasks (Brown et al., 2020; Vaswani et al., 2017).
• Narrow AI vs. general AI (AGI): all current AI is narrow (solves classes of tasks, without consciousness); general AI, comparable to a human’s in any domain, is hypothetical and the term AGI has no consensual definition.
• ELIZA effect: the human tendency to project understanding and intention where there is only pattern manipulation, the root of many misconceptions about AI.

1.11 Myths: “AI Thinks,” “It’s Just Statistics”

The first frequent misconception is treating AI, Machine Learning, and Deep Learning as synonyms. As Figure 1.8 makes clear, they are nested levels; using them interchangeably erases distinctions that matter for understanding capabilities and limits.

The second is assuming that all AI learns from data. The symbolic tradition, based on rules, is AI and does not learn in that sense, and it remains in use in domains that require auditability. LLMs belong to the branch that learns, but AI is not reducible to it.

The third misconception is expecting an LLM to have the predictability of a rule-based system. Those who assume that the same question will always generate the same answer confuse two paradigms: LLM generation is probabilistic, not deterministic, as discussed above.

The fourth, and most consequential for the chapters ahead, is interpreting “learning” and “intelligence” in a human sense. Under Mitchell’s definition, learning means improving a metric with experience (Mitchell, 1997); nothing in that definition implies understanding, intention, or consciousness. Much of the misinterpretation about what an LLM “understands” arises from silently carrying over the everyday connotation of these terms into a process that is, in its nature, statistical.

1.12 Connection to the Next Chapters

With LLMs situated on the AI map, the book now proceeds to open them from the inside. Module 2 tackles the question that the connectionist tradition presupposes but does not answer on its own: how to transform text, which is made of symbols without numerical value, into something a neural network can process, through tokenization and embeddings. Module 3 details what a neural network actually is and what it means, mechanically, to adjust parameters. And Module 4 is dedicated to the Transformer architecture, the 2017 innovation that, mentioned here only in passing, was the condition of possibility for everything we today call an LLM.

1.13 Going Further

To do (try it and observe)

• Teachable Machine (teachablemachine.withgoogle.com): train a classifier with your webcam and watch learning by example happen in minutes.
• Quick, Draw! (quickdraw.withgoogle.com): draw and watch a model guess (and fail), learned from millions of doodles.
• Melanie Mitchell, Artificial Intelligence: A Guide for Thinking Humans: an accessible, skeptical book that separates what AI does from what people claim it does.

To read (deepen the fundamentals)

• TURING, A. M. Computing Machinery and Intelligence. Mind, 1950. The founding text of the question, still surprisingly current.
• RUSSELL, S.; NORVIG, P. Artificial Intelligence: A Modern Approach. 4th ed., 2021. The reference treatise; the opening chapters cover the definitions and history of the field.
• LECUN, Y.; BENGIO, Y.; HINTON, G. Deep Learning. Nature, 2015. An accessible synthesis, by the protagonists themselves, of the paradigm that prevailed.