in memory. The original approach was to simply add the phrase “Let’s think step by step” as part of the instructions, but, we’ve found it helpful to make the CoT more specific, where adding specificity via an extra sentence or two often reduces hallucination rates significantly.\n\nFor example, when asking an LLM to summarize a meeting transcript, we can be explicit about the steps:\n\n* First, list out the key decisions, follow-up items, and associated owners in a sketchpad. \n \n* Then, check that the details in the sketchpad are factually consistent with the transcript. \n \n* Finally, synthesize the key points into a concise summary.\n\nNote that in recent times, [some doubt](https://arxiv.org/abs/2405.04776) has been cast on if this technique is as powerful as believed. Additionally, there’s significant debate as to exactly what is going on during inference when Chain-of-Thought is being used. Regardless, this technique is one to experiment with when possible.\n\nProviding relevant resources is a powerful mechanism to expand the model’s knowledge base, reduce hallucinations, and increase the user’s trust. Often accomplished via Retrieval Augmented Generation (RAG), providing the model with snippets of text that it can directly utilize in its response is an essential technique. When providing the relevant resources, it’s not enough to merely include them; don’t forget to tell the model to prioritize their use, refer to them directly, and to mention when none of the resources are sufficient. These help “ground” agent responses to a corpus of resources.\n\n### 1.1.2 Structure your inputs and outputs[](https://applied-llms.org/#structure-your-inputs-and-outputs)\n\nStructured input and output help models better understand the input as well as return output that can reliably integrate with downstream systems. Adding serialization formatting to your inputs can help provide more clues to the model as to the relationships between tokens in the context, additional metadata to specific tokens (like types), or relate the request to similar examples in the model’s training data.\n\nAs an example, many questions on the internet about writing SQL begin by specifying the SQL schema. Thus, you can expect that effective prompting for Text-to-SQL should include [structured schema definitions](https://www.researchgate.net/publication/371223615_SQL-PaLM_Improved_Large_Language_ModelAdaptation_for_Text-to-SQL).\n\nStructured input expresses tasks clearly and resembles how the training data is formatted, increasing the probability of better output. Structured output simplifies integration into downstream components of your system. [Instructor](https://github.com/jxnl/instructor) and [Outlines](https://github.com/outlines-dev/outlines) work well for structured output. (If you’re importing an LLM API SDK, use Instructor; if you’re importing Huggingface for a self-hosted model, use Outlines.)\n\nWhen using structured input, be aware that each LLM family has their own preferences. Claude prefers `` while GPT favors Markdown and JSON. With XML, you can even pre-fill Claude’s responses by providing a `` tag like so.\n\n```\nmessages=[\n {\n \"role\": \"user\",\n \"content\": \"\"\"Extract the , , , and from this product description into your .\n The SmartHome Mini is a compact smart home assistant available in black or white for only $49.99. At just 5 inches wide, it lets you control lights, thermostats, and other connected devices via voice or app—no matter where you place it in your home. This affordable little hub brings convenient hands-free control to your smart devices.\n \"\"\"\n },\n {\n \"role\": \"assistant\",\n \"content\": \"\"\n }\n]\n```\n\n### 1.1.3 Have small prompts that do one thing, and only one thing, well[](https://applied-llms.org/#have-small-prompts-that-do-one-thing-and-only-one-thing-well)\n\nA common anti-pattern / co

What We’ve Learned From A Year of Building with LLMs – Applied LLMs

Podcast Information

Roles

Scripts