What is symbolic artificial intelligence?

Symbolic AI programs are based on creating explicit structures and behavior rules. Being able to communicate in symbols is one of the main things that make us intelligent. Therefore, symbols have also played a crucial role in the creation of artificial intelligence.

We know how it works out answers to queries, and it doesn’t require energy-intensive training. This aspect also saves time compared with GAI, as without the need for training, models can be up and running in minutes. When deep learning reemerged in 2012, it was with a kind of take-no-prisoners attitude that has characterized most of the last decade. He gave a talk at an AI workshop at Stanford comparing symbols to aether, one of science’s greatest mistakes.

Similarly, LISP machines were built to run LISP, but as the second AI boom turned to bust these companies could not compete with new workstations that could now run LISP or Prolog natively at comparable speeds. “Everywhere we try mixing some of these ideas together, we find that we can create hybrids that are … more than the sum of their parts,” says computational neuroscientist David Cox, IBM’s head of the MIT-IBM Watson AI Lab in Cambridge, Massachusetts. A few years ago, scientists learned something remarkable about mallard ducklings.

How to turn any LLM into an embedding model

And unlike symbolic AI, neural networks have no notion of symbols and hierarchical representation of knowledge. This limitation makes it very hard to apply neural networks to tasks that require logic and reasoning, such as science and high-school math. The advantage of neural networks is that they can deal with messy and unstructured data.

In healthcare, it can integrate and interpret vast datasets, from patient records to medical research, to support diagnosis and treatment decisions. In finance, it can analyze transactions within the context of evolving regulations to detect fraud and ensure compliance. Opposing Chomsky’s views that a human is born with Universal Grammar, a kind of knowledge, John Locke[1632–1704] postulated that mind is a blank slate or tabula rasa. The universe is written in the language of mathematics and its characters are triangles, circles, and other geometric objects. Like Inbenta’s, “our technology is frugal in energy and data, it learns autonomously, and can explain its decisions”, affirms AnotherBrain on its website. And given the startup’s founder, Bruno Maisonnier, previously founded Aldebaran Robotics (creators of the NAO and Pepper robots), AnotherBrain is unlikely to be a flash in the pan.

“It’s one of the most exciting areas in today’s machine learning,” says Brenden Lake, a computer and cognitive scientist at New York University. But the benefits of deep learning and neural networks are not without tradeoffs. Deep learning has several deep challenges and disadvantages in comparison to symbolic AI. Notably, deep learning algorithms are opaque, and figuring out how they work perplexes even their creators.

One solution is to take pictures of your cat from different angles and create new rules for your application to compare each input against all those images. Even if you take a million pictures of your cat, you still won’t account for every possible case. A change in the lighting conditions or the background of the image will change the pixel value and cause the program to fail. Using OOP, you can create extensive and complex symbolic AI programs that perform various tasks. Many of the concepts and tools you find in computer science are the results of these efforts.

Horn clause logic is more restricted than first-order logic and is used in logic programming languages such as Prolog. Extensions to first-order logic include temporal logic, to handle time; epistemic logic, to reason about agent knowledge; modal logic, to handle possibility and necessity; and probabilistic logics to handle logic and probability together. In contrast to the US, in Europe the key AI programming language during that same period was Prolog. Prolog provided a built-in store of facts and clauses that could be queried by a read-eval-print loop. The store could act as a knowledge base and the clauses could act as rules or a restricted form of logic.

The current state of symbolic AI

For example, a symbolic AI built to emulate the ducklings would have symbols such as “sphere,” “cylinder” and “cube” to represent the physical objects, and symbols such as “red,” “blue” and “green” for colors and “small” and “large” for size. The knowledge base would also have a general rule that says that two objects are similar if they are of the same size or color or shape. In addition, the AI needs to know about propositions, which are statements that assert something is true or false, to tell the AI that, in some limited world, there’s a big, red cylinder, a big, blue cube and a small, red sphere. All of this is encoded as a symbolic program in a programming language a computer can understand. To build AI that can do this, some researchers are hybridizing deep nets with what the research community calls “good old-fashioned artificial intelligence,” otherwise known as symbolic AI. The offspring, which they call neurosymbolic AI, are showing duckling-like abilities and then some.

For example, AI developers created many rule systems to characterize the rules people commonly use to make sense of the world. This resulted in AI systems that could help translate a particular symptom into a relevant diagnosis or identify fraud. A certain set of structural rules are innate to humans, independent of sensory experience. With more linguistic stimuli received in the course of psychological development, children then adopt specific syntactic rules that conform to Universal grammar. Equally cutting-edge, France’s AnotherBrain is a fast-growing symbolic AI startup whose vision is to perfect “Industry 4.0” by using their own image recognition technology for quality control in factories. As a consequence, the Botmaster’s job is completely different when using Symbolic AI technology than with Machine Learning-based technology as he focuses on writing new content for the knowledge base rather than utterances of existing content.

Many leading scientists believe that symbolic reasoning will continue to remain a very important component of artificial intelligence. Symbolic techniques were at the heart of the IBM Watson DeepQA system, which beat the best human at answering trivia questions in the game Jeopardy! However, this also required much human effort to organize and link all the facts into a symbolic reasoning system, which did not scale well to new use cases in medicine and other domains.

Popular categories of ANNs include convolutional neural networks (CNNs), recurrent neural networks (RNNs) and transformers. CNNs are good at processing information in parallel, such as the meaning of pixels in an image. New GenAI techniques often use transformer-based neural networks that automate data prep work in training AI systems such as ChatGPT and Google Gemini. In the realm of artificial intelligence, symbolic AI stands as a pivotal concept that has significantly influenced the understanding and development of intelligent systems. This guide aims to provide a comprehensive overview of symbolic AI, covering its definition, historical significance, working principles, real-world applications, pros and cons, related terms, and frequently asked questions.

Disentangling visual attributes with neuro-vector-symbolic architectures, in-memory computing, and device noise

Instead of manually laboring through the rules of detecting cat pixels, you can train a deep learning algorithm on many pictures of cats. When you provide it with a new image, it will return the probability that it contains a cat. Innovations in backpropagation in the late 1980s helped revive interest in neural networks. This helped address some of the limitations in early neural network approaches, but did not scale well. The discovery that graphics processing units could help parallelize the process in the mid-2010s represented a sea change for neural networks.

In this view, deep learning best models the first kind of thinking while symbolic reasoning best models the second kind and both are needed. A key component of the system architecture for all expert systems is the knowledge base, which stores facts and rules for problem-solving.[51]

The simplest approach for an expert system knowledge base is simply a collection or network of production rules. Production rules connect symbols in a relationship similar to an If-Then statement. The expert system processes the rules to make deductions and to determine what additional information it needs, i.e. what questions to ask, using human-readable symbols.

• Constraint logic programming can be used to solve scheduling problems, for example with constraint handling rules (CHR).
• However, neural networks fell out of favor in 1969 after AI pioneers Marvin Minsky and Seymour Papert published a paper criticizing their ability to learn and solve complex problems.
• The researchers trained this neurosymbolic hybrid on a subset of question-answer pairs from the CLEVR dataset, so that the deep nets learned how to recognize the objects and their properties from the images and how to process the questions properly.
• During the first AI summer, many people thought that machine intelligence could be achieved in just a few years.
• It’s taking baby steps toward reasoning like humans and might one day take the wheel in self-driving cars.

One of their projects involves technology that could be used for self-driving cars. Consequently, learning to drive safely requires enormous amounts of training data, and the AI cannot be trained out in the real world. Such causal and counterfactual reasoning about things that are changing with time is extremely difficult for today’s deep neural networks, which mainly excel at discovering static patterns in data, Kohli says. The team solved the first problem by using a number of convolutional neural networks, a type of deep net that’s optimized for image recognition.

In the latter case, vector components are interpretable as concepts named by Wikipedia articles. For the first method, called supervised learning, the team showed the deep nets numerous examples of board positions and the corresponding “good” questions (collected from human players). The deep nets eventually learned to ask good questions on their own, but were rarely creative.

Concerningly, some of the latest GenAI techniques are incredibly confident and predictive, confusing humans who rely on the results. This problem is not just an issue with GenAI or neural networks, but, more broadly, with all statistical AI techniques. Read more about our work in neuro-symbolic AI from the MIT-IBM Watson AI Lab. Our researchers are working to usher in a new era of AI where machines can learn more like the way humans do, by connecting words with images and mastering abstract concepts.

Forward chaining inference engines are the most common, and are seen in CLIPS and OPS5. Backward chaining occurs in Prolog, where a more limited logical representation is used, Horn Clauses. Fulton and colleagues are working on a neurosymbolic AI approach to overcome such limitations. The symbolic part of the AI has a small knowledge base about some limited aspects of the world and the actions that would be dangerous given some state of the world. They use this to constrain the actions of the deep net — preventing it, say, from crashing into an object.

Then they had to turn an English-language question into a symbolic program that could operate on the knowledge base and produce an answer. The difficulties encountered by symbolic AI have, however, been deep, possibly unresolvable ones. One difficult problem encountered by symbolic AI pioneers came to be known as the common sense knowledge problem. In addition, areas that rely on procedural or implicit knowledge such as sensory/motor processes, are much more difficult to handle within the Symbolic AI framework.

Satplan is an approach to planning where a planning problem is reduced to a Boolean satisfiability problem. A more flexible kind of problem-solving occurs when reasoning about what to do next occurs, rather than simply choosing one of the available actions. This kind of meta-level reasoning is used in Soar and in the BB1 blackboard architecture.

Some companies have chosen to ‘boost’ symbolic AI by combining it with other kinds of artificial intelligence. Inbenta works in the initially-symbolic field of Natural Language Processing, but adds a layer of ML to increase the efficiency of this processing. The ML layer processes hundreds of thousands of lexical functions, featured in dictionaries, that allow the system to better ‘understand’ relationships between words.

The team’s solution was about 88 percent accurate in answering descriptive questions, about 83 percent for predictive questions and about 74 percent for counterfactual queries, by one measure of accuracy. The researchers broke the problem into smaller chunks familiar from symbolic AI. In essence, they had to first look at an image and characterize the 3-D shapes and their properties, and generate a knowledge base.

This simple symbolic intervention drastically reduces the amount of data needed to train the AI by excluding certain choices from the get-go. “If the agent doesn’t need to encounter a bunch of bad states, then it needs less data,” says Fulton. While the project still isn’t ready for use outside the lab, Cox envisions a future in which cars with neurosymbolic AI could learn out in the real world, with the symbolic component acting as a bulwark against bad driving. A new approach to artificial intelligence combines the strengths of two leading methods, lessening the need for people to train the systems. We use symbols all the time to define things (cat, car, airplane, etc.) and people (teacher, police, salesperson).

They are also better at explaining and interpreting the AI algorithms responsible for a result. However, this also required much manual effort from experts tasked with deciphering the chain of thought processes that connect various symptoms to diseases or purchasing patterns to fraud. This downside is not a big issue with deciphering the meaning of children’s stories or linking common knowledge, but it becomes more expensive with specialized knowledge.

In 1959, it defeated the best player, This created a fear of AI dominating AI. This lead towards the connectionist paradigm of AI, also called non-symbolic AI which gave rise to learning and neural network-based approaches to solve AI. Symbolic AI has greatly influenced natural language processing by offering formal methods for representing linguistic structures, grammatical rules, and semantic relationships. These symbolic representations have paved the way for the development of language understanding and generation systems. The enduring relevance and impact of symbolic AI in the realm of artificial intelligence are evident in its foundational role in knowledge representation, reasoning, and intelligent system design. You can foun additiona information about ai customer service and artificial intelligence and NLP. As AI continues to evolve and diversify, the principles and insights offered by symbolic AI provide essential perspectives for understanding human cognition and developing robust, explainable AI solutions.

• The practice showed a lot of promise in the early decades of AI research.
• In neural networks, the statistical processing is widely distributed across numerous neurons and interconnections, which increases the effectiveness of correlating and distilling subtle patterns in large data sets.
• In sections to follow we will elaborate on important sub-areas of Symbolic AI as well as difficulties encountered by this approach.
• Critiques from outside of the field were primarily from philosophers, on intellectual grounds, but also from funding agencies, especially during the two AI winters.
• The symbolic aspect points to the rules-based reasoning approach that’s commonly used in logic, mathematics and programming languages.

Crucially, these hybrids need far less training data then standard deep nets and use logic that’s easier to understand, making it possible for humans to track how the AI makes its decisions. Neuro-Symbolic AI aims to create models that can understand and manipulate symbols, which represent entities, relationships, and abstractions, much like the human mind. These models are adept at tasks that require deep understanding and reasoning, such as natural language processing, complex decision-making, and problemsolving. The origins of symbolic AI can be traced back to the early days of AI research, particularly in the 1950s and 1960s, when pioneers such as John McCarthy and Allen Newell laid the foundations for this approach. The concept gained prominence with the development of expert systems, knowledge-based reasoning, and early symbolic language processing techniques.

The researchers also used another form of training called reinforcement learning, in which the neural network is rewarded each time it asks a question that actually helps find the ships. Again, the deep nets eventually learned to ask the right questions, which were both informative and creative. Better yet, the hybrid needed only about 10 percent of the training data required by solutions based purely on deep neural networks.

NSCL uses both rule-based programs and neural networks to solve visual question-answering problems. As opposed to pure neural network–based models, the hybrid AI can learn new tasks with less data and is explainable. And unlike symbolic-only models, NSCL doesn’t struggle to analyze the content of images. Early deep learning systems focused on simple classification tasks like recognizing cats in videos or categorizing animals in images. Now, researchers are looking at how to integrate these two approaches at a more granular level for discovering proteins, discerning business processes and reasoning. Common symbolic AI algorithms include expert systems, logic programming, semantic networks, Bayesian networks and fuzzy logic.

These algorithms are used for knowledge representation, reasoning, planning and decision-making. They work well for applications with well-defined workflows, but struggle when apps are trying to make sense of edge cases. The deep learning hope—seemingly grounded not so much in science, but in a sort of historical grudge—is that intelligent behavior will emerge purely from the confluence of massive data and deep learning.

If machine learning can appear as a revolutionary approach at first, its lack of transparency and a large amount of data that is required in order for the system to learn are its two main flaws. Companies now realize how important it is to have a transparent AI, not only for ethical reasons but also for operational ones, and the deterministic (or symbolic) approach is now becoming popular again. Since its foundation as an academic discipline in 1955, Artificial Intelligence (AI) research field has been divided into different camps, of which symbolic AI and machine learning. While symbolic AI used to dominate in the first decades, machine learning has been very trendy lately, so let’s try to understand each of these approaches and their main differences when applied to Natural Language Processing (NLP). In the realm of mathematics and theoretical reasoning, symbolic AI techniques have been applied to automate the process of proving mathematical theorems and logical propositions. By formulating logical expressions and employing automated reasoning algorithms, AI systems can explore and derive proofs for complex mathematical statements, enhancing the efficiency of formal reasoning processes.

The significance of symbolic AI lies in its role as the traditional framework for modeling intelligent systems and human cognition. It underpins the understanding of formal logic, reasoning, and the symbolic manipulation of knowledge, which are fundamental to various fields within AI, including natural language processing, expert systems, and automated reasoning. Despite the emergence of alternative paradigms such as connectionism and statistical learning, symbolic AI continues to inspire a deep understanding of symbolic representation and reasoning, enriching the broader landscape of AI research and applications. The Symbolic AI paradigm led to seminal ideas in search, symbolic programming languages, agents, multi-agent systems, the semantic web, and the strengths and limitations of formal knowledge and reasoning systems. Not everyone agrees that neurosymbolic AI is the best way to more powerful artificial intelligence. Serre, of Brown, thinks this hybrid approach will be hard pressed to come close to the sophistication of abstract human reasoning.

Expert systems can operate in either a forward chaining – from evidence to conclusions – or backward chaining – from goals to needed data and prerequisites – manner. More advanced knowledge-based systems, such as Soar can also perform meta-level reasoning, that is reasoning about their own reasoning in terms of deciding how to solve problems and monitoring the success of problem-solving strategies. Symbolic artificial intelligence showed early progress at the dawn of AI and computing. You can easily visualize the logic of rule-based programs, communicate them, and troubleshoot them. Another area of innovation will be improving the interpretability and explainability of large language models common in generative AI.

The excitement within the AI community lies in finding better ways to tinker with the integration between symbolic and neural network aspects. For example, DeepMind’s AlphaGo used symbolic techniques to improve the representation of game layouts, process them with neural networks and then analyze the results with symbolic techniques. Other potential use cases of deeper neuro-symbolic integration include improving explainability, labeling data, reducing hallucinations and discerning cause-and-effect relationships. However, virtually all neural models consume symbols, work with them or output them. For example, a neural network for optical character recognition (OCR) translates images into numbers for processing with symbolic approaches. Generative AI apps similarly start with a symbolic text prompt and then process it with neural nets to deliver text or code.

For much of the AI era, symbolic approaches held the upper hand in adding value through apps including expert systems, fraud detection and argument mining. But innovations in deep learning and the infrastructure for training large language models (LLMs) have shifted the focus toward neural networks. The work in AI started by projects like the General Problem Solver and other rule-based reasoning systems like Logic Theorist became the foundation for almost 40 years of research. Symbolic AI (or Classical AI) is the branch of artificial intelligence research that concerns itself with attempting to explicitly represent human knowledge in a declarative form (i.e. facts and rules). If such an approach is to be successful in producing human-like intelligence then it is necessary to translate often implicit or procedural knowledge possessed by humans into an explicit form using symbols and rules for their manipulation.

The video previews the sorts of questions that could be asked, and later parts of the video show how one AI converted the questions into machine-understandable form. Armed with its knowledge base and propositions, symbolic AI employs an inference engine, which uses rules of logic to answer queries. Asked if the sphere and cube are similar, it will answer “No” (because they are not of the same size or color).

Using symbolic AI, everything is visible, understandable and explainable, leading to what is called a ‘transparent box’ as opposed to the ‘black box’ created by machine learning. In a nutshell, symbolic AI involves the explicit embedding of human knowledge and behavior rules into computer programs. As you can easily imagine, this is a very heavy and time-consuming job as there are many many ways of asking or formulating the same question. And if you take into account that a knowledge base usually holds on average 300 intents, you now see how repetitive maintaining a knowledge base can be when using machine learning. Multiple different approaches to represent knowledge and then reason with those representations have been investigated. Below is a quick overview of approaches to knowledge representation and automated reasoning.

For more detail see the section on the origins of Prolog in the PLANNER article. “You can check which module didn’t work properly and needs to be corrected,” says team symbolic ai member Pushmeet Kohli of Google DeepMind in London. For example, debuggers can inspect the knowledge base or processed question and see what the AI is doing.

When a deep net is being trained to solve a problem, it’s effectively searching through a vast space of potential solutions to find the correct one. Adding a symbolic component reduces the space of solutions to search, which speeds up learning. What the ducklings do so effortlessly turns out to be very hard for artificial intelligence. This is especially true of a branch of AI known as deep learning or deep neural networks, the technology powering the AI that defeated the world’s Go champion Lee Sedol in 2016.

Ducklings easily learn the concepts of “same” and “different” — something that artificial intelligence struggles to do. AllegroGraph is a horizontally distributed Knowledge Graph Platform that supports multi-modal Graph (RDF), Vector, and Document (JSON, JSON-LD) storage. It is equipped with capabilities such as SPARQL, Geospatial, Temporal, Social Chat PG Networking, Text Analytics, and Large Language Model (LLM) functionalities. These features enable scalable Knowledge Graphs, which are essential for building Neuro-Symbolic AI applications that require complex data analysis and integration. Once they are built, symbolic methods tend to be faster and more efficient than neural techniques.

Deep learning and neural networks excel at exactly the tasks that symbolic AI struggles with. They have created a revolution in computer vision applications such as facial recognition and cancer detection. Both symbolic and neural network approaches date back to the earliest days of AI in the 1950s. On the symbolic side, the Logic Theorist program in 1956 helped solve simple theorems.

It’s possible to solve this problem using sophisticated deep neural networks. However, Cox’s colleagues at IBM, along with researchers at Google’s DeepMind and MIT, came up with a distinctly different solution that shows the power of neurosymbolic AI. In the context of Neuro-Symbolic AI, AllegroGraph’s W3C standards based graph capabilities allow it to define relationships between entities in a way that can be logically reasoned about. The https://chat.openai.com/ geospatial and temporal features enable the AI to understand and reason about the physical world and the passage of time, which are critical for real-world applications. The inclusion of LLMs allows for the processing and understanding of natural language, turning unstructured text into structured knowledge that can be added to the graph and reasoned about. In fact, rule-based AI systems are still very important in today’s applications.

The unlikely marriage of two major artificial intelligence approaches has given rise to a new hybrid called neurosymbolic AI. It’s taking baby steps toward reasoning like humans and might one day take the wheel in self-driving cars. Maybe in the future, we’ll invent AI technologies that can both reason and learn. But for the moment, symbolic AI is the leading method to deal with problems that require logical thinking and knowledge representation. Neural networks are almost as old as symbolic AI, but they were largely dismissed because they were inefficient and required compute resources that weren’t available at the time.

Integrating Knowledge Graphs into Neuro-Symbolic AI is one of its most significant applications. Knowledge Graphs represent relationships in data, making them an ideal structure for symbolic reasoning. They can store facts about the world, which AI systems can then reason about. Deep neural networks are also very suitable for reinforcement learning, AI models that develop their behavior through numerous trial and error.

The grandfather of AI, Thomas Hobbes said — Thinking is manipulation of symbols and Reasoning is computation. Facial recognition, for example, is impossible, as is content generation. McCarthy’s approach to fix the frame problem was circumscription, a kind of non-monotonic logic where deductions could be made from actions that need only specify what would change while not having to explicitly specify everything that would not change. Other non-monotonic logics provided truth maintenance systems that revised beliefs leading to contradictions.