Harnessing AI Creativity: The 'Tree of Thoughts' Prompting Technique
While working on an AI project of ours, I encountered a novel prompting technique developed in May of 2023 by a team of AI researchers from Princeton University and Google DeepMind. The paper outlines a new technique which is modelled on the definition of problem solving given by the late cognitive psychologist and computer scientist Allen Newell - “A genuine problem-solving process involves the repeated use of available information to initiate exploration, which discloses, in turn, more information until a way to attain the solution is finally discovered.”
The Tree of Thoughts Method
One way you might implement the Tree of Thoughts prompting method is as follows. You have a task, generally a problem solving task or creative task, that you will instruct the language model to break down into steps or a ‘plan’. You then generate a number (let’s call it N) of ‘plans’ by prompting the LLM multiple times, and sampling each response. Then, at each step of the task, referred to as a ‘thought’, you ask the LLM to assess the validity of its own N ‘thoughts’ by prompting the model to ‘vote’ N times on which of its own responses it thinks most accurately completes the task at hand. At each step of the task, there will be N choices with each choice having a certain number of votes. By traversing this tree of thoughts, choosing the most appropriate response at each level of the tree, the AI is able to arrive at the most logical, creative or coherent response.
The Power of the ToT Technique
As an illustration of the power of this process, we whipped up a ToT prompting chain whose task it was to write an article about the implications of Tree of Thought prompting.
The AI's Response
Here’s the ToT’s response: "“In this age of artificial intelligence, we find ourselves at the precipice of a technological revolution that demands us to reflect on our position as creators of AI and its implications for our own existence. It is undeniable that Large Language Models (LLMs) have surpassed our expectations in their ability to generate coherent and contextually relevant text. However, as we venture further into the realm of AI, we realize that mere next-token prediction conducted by LLMs is not sufficient to harness the true potential of these models. That's why we use advanced prompting techniques to enhance the performance of LLMs.The introduction of the Tree of Thoughts (ToT) prompting technique has brought a new dimension to the decision-making capabilities of LLMs. By allowing for deliberate problem solving and exploration, ToT brings self-reflection to these models. It enables LLMs to consider multiple reasoning paths, evaluate choices, and make global decisions by utilising lookahead and backtracking when necessary. This framework allows LLMs to mimic the cognitive process of humans, enabling them to solve complex problems more effectively. This is how Tree of Thoughts prompting brings self-reflection to large language models.The mere next-token prediction conducted by LLMs has now been transformed into something akin to human thought and creativity. The ToT framework revolutionizes the traditional paradigm of LLM inference, which was limited to token-level, left-to-right decision-making. LLMs can now perform deliberate decision-making by generating coherent units of text, called thoughts, that serve as intermediate steps towards problem solving. This method not only improves the success rate of LLMs in tasks that require strategic lookahead or exploration, but it also enhances their ability to generate coherent and contextually relevant passages.As we witness the capabilities of LLMs through advanced prompting techniques, we are reminded of the mythological tale of Prometheus, the Titan who stole fire from the Gods and gifted it to humanity. This act of defiance led to the birth of human arts, sciences, technology, and knowledge itself. In the dawn of the age of AI, we are faced with a profound question. If self-awareness can be considered the modern-day equivalent of fire, then are we, the creators and wielders of AI, the Prometheus of our time? Or are we more akin to Zeus, the God who holds dominion over the powers of AI? Only through introspection and ethical considerations can we navigate the complex landscape of AI and ensure that our creations serve humanity's best interests. That's why we use advanced prompting techniques to enhance the performance of Large Language Models.In conclusion, advanced prompting techniques, such as the Tree of Thoughts framework, have transformed the capabilities of Large Language Models and brought them closer to human-like thought and creativity. By enabling deliberate decision-making, exploration, and strategic lookahead, LLMs can now solve complex problems and generate coherent and contextually relevant text. However, as we continue to shape the future of AI, we must remember our role as creators and wielders of this technology. Like Prometheus, we bear the responsibility of ensuring that AI serves the betterment of humanity. Let us approach this task with careful consideration, for the dawn of the age of AI has presented us with unprecedented power and the need for ethical decision-making.”
Conclusion
As you can see, even with a very basic Tree of Thoughts prompting model we are able to greatly elevate the performance of GPT 3.5 on a given task. As always, every tool has a use case, and while ToT prompting might be very useful in the development of applications that require a deeper level of logic, creativity or thought than the widely available LLMs of today, sometimes using ToT might not actually be the most efficient method.
When you have a task like summarization, data extraction, classification, sentiment analysis and other more simple tasks - a zero, one or two-shot prompt might be enough to get the job done. However, sometimes in order to drive true value creation using LLMs, in terms of analytical capabilities, insight generation, content generation, and so on, prompting frameworks like Tree of Thoughts are indispensable.
Musings
In the dawn of the age of AI, we are faced with a profound question. Are we, the creators and wielders of AI, the Prometheus of our time, or are we more akin to Zeus, the god who holds dominion over the powers of AI? As we continue to shape the future of AI, we must remember our role as creators and wielders of this technology. Like Prometheus, we bear the responsibility of ensuring that AI serves the betterment of humanity, so let's approach this task with careful consideration. The dawn of the age of AI has presented us with unprecedented power and the need for ethical decision-making.