Last week we announced the private preview of Duet AI for Google Cloud, an always-on AI collaborator that uses generative AI to provide help to developers and cloud users. This article gives you a detailed look at Duet AI for developers, showing how Duet AI can help provide developers with real-time code suggestions, chat assistance, and enterprise-focused customization. You can sign-up here to join our waitlist for Google Cloud’s AI trusted tester program.  

We believe that addressing these use case​​s with large language models (LLMs) will usher in a major leap in productivity in enterprise development. Duet AI uses Codey, a family of code models built on PaLM 2. 

Developers continuously seek ways to improve their productivity and, over the past few decades, these efforts have resulted in massive productivity leaps due to technological changes. From advanced debuggers and online developer communities, to modern IDEs/notebooks and cloud computing, each advance brought massive changes in productivity. Despite these improvements, developers still face numerous challenges, some of which are unique to cloud development:

• Disruptive context switching and friction when integrating a new tool or service
• Excessive time spent on repetitive tasks
• Time required to understand a new code base or project
• Large cognitive workload when working on large code bases or complex APIs

Duet AI for developers focuses on challenges and tasks across the development lifecycle:

Code/Boilerplate Generation — Developers can describe the tasks they have in mind as a comment or function name, such as creating a Cloud Pub/Sub topic. Duet AI will generate a reference implementation that can be reviewed and modified, so developers don’t need to spend time reading through multiple documentation pages.

Inline Code Completion — To reduce the time spent on repetitive tasks and minimize the cognitive workload of tasks such as writing repetitive code or retrieving variable names, Duet AI provides intelligent, context-aware code completion, helping reduce the time spent on coding and enhancing the quality of the written code.

Enterprise Customization — Organizations frequently have massive code bases and specific recommended frameworks and best practices, which generic code assistance solutions may not be best positioned to support. With Vertex AI, developers will be able to tune and customize the underlying models and connect them to the Duet AI experience, allowing for assistance optimized to the needs of the organization.

Code Explanation — Developers spend significant time and effort reading and understanding code written by their peers or external contributors. To help assist in this process, Duet AI for code assistance provides an “Explain this code” option available whenever a developer selects their code, allowing them to more quickly understand, map, and navigate unfamiliar code bases.

Code security guardrails — Code generated by Duet AI can also be scanned for vulnerable dependencies via Source Protect, helping surface known public vulnerabilities impacting code, along with suggested fixes when available, bringing additional security.

By harnessing the power of AI-driven developer assistance such as the one provided by Duet AI for developers, businesses can unlock unprecedented levels of productivity and efficiency in software development, paving the way for a new era of innovation and growth.

These early features of Duet AI for Google Cloud will be available for limited users and we will be expanding access very soon. Sign up here to join Google Cloud’s AI Trusted Tester Program.

As AI-powered tools become increasingly prevalent, prompt engineering is becoming a skill that developers need to master. Large language models (LLMs) and other generative foundation models require contextual, specific, and tailored natural language instructions to generate the desired output. This means that developers need to write prompts that are clear, concise, and informative. 

In this blog, we will explore six best practices that will make you a more efficient prompt engineer. By following our advice, you can begin creating more personalized, accurate, and contextually aware applications. So let’s get started!

Tip #1: Know the model’s strengths and weaknesses

As AI models evolve and become more complex, it is essential for developers to comprehend their capabilities and limitations. Understanding these strengths and weaknesses can help you, as a developer, avoid making mistakes and create safer, more reliable applications.

For example, an AI model that is trained to recognize images of blueberries may not be able to recognize images of strawberries. Why? Because the model was only trained on a dataset of blueberry images. If a developer uses this model to build an application that is supposed to recognize both blueberries and strawberries, the application would likely make mistakes, leading to an ineffective outcome, and poor user experience.

It’s important to note that AI models have the ability to be biased. This is due to AI models being trained on data that is collected from the real world, and so it can reflect the inequitable power dynamics inherently rooted in our social hierarchy. If the data that is used to train an AI model is biased, then the model will also be biased. This can lead to problems if the model is used to make decisions that affect people by reinforcing societal biases. Addressing these biases is important to ensure that data is fair, promoting equality, and ensuring the responsibility of AI technology. Prompt engineers should be aware of training limitations or biases so they can craft prompts more effectively and understand what kind of prompting is even possible for a given model.

Tip #2: Be as specific as possible

AI models have the ability to comprehend a variety of prompts. For instance Google’s PaLM 2 can understand natural language prompts, multilingual text, and even programming codes like Python and JavaScript. Although AI models can be very knowledgeable, they are still imperfect, and have the ability to misinterpret prompts that are not specific enough. In order for AI models to navigate ambiguity, it is important to tailor your prompts specifically to your desired outcome. 

Let’s say you would like your AI model to generate a recipe for 50 vegan blueberry muffins. If you prompt the model with “what is a recipe for blueberry muffins?”, the model does not know that you need to make 50 muffins. It is thus unlikely to list the larger volume of ingredients you’ll need or include tips to help you more efficiently bake such a large number of muffins. The model can only go off the context that is provided. A more effective prompt would be “I am hosting 50 guests. Generate a recipe for 50 blueberry muffins.” The model is more likely to generate a response that is relevant to your request and meets your specific requirements.

Tip #3: Utilize contextual prompts

Utilize contextual information in your prompts to help the model gain an in-depth understanding of your requests. Contextual prompts can include the specific task you want the model to perform, a replica of the output you’re looking for, or a persona to emulate, from a marketer or engineer to a high school teacher. Defining a tone and perspective for an AI model gives it a blueprint of the tone, style, and focused expertise you’re looking for to improve the quality, relevance, and effectiveness of your output. 

In the case of the blueberry muffins, it is important to prompt the model using the context of the situation. The model might need more context than generating a recipe for 50 people. If it needs to be aware that the recipe must be vegan friendly, you might prompt the model by asking it to answer by emulating a skilled vegan chef. 

By providing contextual prompts, you can help ensure that your AI interactions are as seamless and efficient as possible. The model will be able to more quickly understand your request and it will be able to generate more accurate and relevant responses.

Tip #4: Provide AI models with examples 

When creating prompts for AI models, it is helpful to provide examples. This is because prompts act as instructions for the model, and examples can help the model to understand what you are asking for. Providing a prompt with an example looks something like this: “here are several recipes I like – create a new recipe based on the ones I provided.” The model can now understand the your ability and needs in order to make this pastry,

Tip #5: Experiment with prompts and personas

The way you construct your prompt impacts the model’s output. By creatively exploring different requests, you will soon have an understanding of how the model weighs its answers, and what happens when you interfuse your domain knowledge, expertise, and lived experience with the power of a multi-billion parameter large language model. 

Try experimenting with different keywords, sentence structures, and prompt lengths to discover the perfect formula. Allow yourself to step into the shoes of various personas, from work personas such as “product engineer” or “customer service representatives,” to parental figures or celebrities such as your grandmother, a celebrity chef, and explore everything from cooking to coding!

By crafting unique, and innovative, requests replete with your expertise and experience, you can learn which prompts provide you with your ideal output. Further refining your prompts, known as ‘tuning,’ allows the model to have a greater understanding and framework for your next output.

Tip #6: Try chain-of-thought prompting

Chain of thought prompting is a technique for improving the reasoning capabilities of large language models (LLMs). It works by breaking down a complex problem into smaller steps, and then prompting the LLM to provide intermediate reasoning for each step. This helps the LLM to understand the problem more deeply, and to generate more accurate and informative answers. This will help you to understand the answer better and to make sure that the LLM is actually understanding the problem. 

Conclusion

Prompt engineering is a skill that all workers, across industries and organizations, will need as AI-powered tools are becoming more prevalent. Remember to incorporate these five essential tips the next time you communicate with an AI model, so you can generate the accurate outputs that you desire. AI will forever continue to develop, constantly refining itself as we use it, so I encourage you to remember that learning, for mind and machine, is a never ending journey. Happy Prompting!