Video AI is a powerful way to enable content discovery and engaging video experiences.

Here, try it out right now!

Google Cloud’s easy-to-access video AI solutions can accomplish a bunch of things. Here are a few:

Precise video analysis: Video Intelligence API automatically recognizes more than 20,000 objects, places, and actions in stored and streaming video. It also distinguishes scene changes and extracts rich metadata at the video, shot, or frame level. Use in combination with AutoML Video Intelligence to create your own custom entity labels to categorize content. Imagine being able to categorise hundreds of videos of customer interactions quickly to improve service training!

Recommended content: Build a content recommendation engine with labels generated by Video Intelligence API and a user’s viewing history and preferences. This will simplify content discovery for your users and guide them to the most relevant content that they want.

Simplify media management: Find value in vast archives by making media easily searchable and discoverable. Easily search your video catalog the same way you search text documents. Extract metadata that can be used to index, organize, and search your video content, as well as control and filter content for what’s most relevant.Imagine being able to locate insight in hundreds of enterprise videos to improve productivity and customer experience!

Easily create intelligent video apps: Gain insights from video in near real time using the Video Intelligence Streaming Video APIs, and trigger events based on objects detected. Build engaging customer experiences with highlight reels, recommendations, interactive videos, and more. Marketers, imagine being able to trigger a customer workflow, in real time, based on a live customer interactions.

Automate expensive workflows: Reduce time and costs associated with transcribing videos and generating closed captions, as well as flagging and filtering inappropriate content.

Content moderation: Identify when inappropriate content is being shown in a given video. You can instantly conduct content moderation across petabytes of data and more quickly and efficiently filter your content or user-generated content.

What can your organisation do with video AI?

Cloud developers — you’ve got it all. You can code in your choice of languages, enjoy portability with containers, minimize complexity with serverless, and manage the entire software lifecycle by following DevOps principles. But let’s face it, building and onboarding new cloud applications still requires a lot of manual planning, synthesis, and, yes, hard work. You need to research and plan your deployment, create a workable, secure architecture, and of course, you need to write the actual code,

For the last several decades, the cloud has been primarily a “do it yourself” model with volumes of options that have made development more complicated. The cloud went from overwhelmingly exciting to…. a bit overwhelming.

What if we all could bring that excitement back? What if you had some help that was available whenever and wherever you needed it? 

Say hello to Duet AI for Google Cloud

Powered by Google’s state-of-the-art generative-AI foundation models, Duet AI for Google Cloud is an always-on AI collaborator that provides help to users of all skill levels where they need it. With Duet AI, we’re on a mission to deliver a new cloud experience that’s personalized and intent-driven, and can deeply understand your environment to assist you in building secure, scalable applications, while providing expert guidance. 

As we evolve Google Cloud with Duet AI, we are looking to build a cloud platform that is more human-centric, holistic, and helpful, with responsible AI at the center of the experience:

• Human-centric: With Duet AI, we are making Google Cloud more accessible and personal to any type of user at any skill level by providing them with support whenever they need it, from code recommendations for developers, to prompt-based data insights for data engineers, to chat-based app creation for business users.
• Holistic: With generative AI at the center of the cloud experience, cloud development can be more cohesive, with fewer silos across functions, services, and tech stacks, providing a holistic picture in the format you want, wherever you are in Google Cloud.
• Helpful: To deliver smarter, contextual recommendations for building and operating apps with Google Cloud, we pre-trained Codey, one of the foundation models that powers Duet AI, with Google Cloud-specific content like documentation and sample code, and fine-tuned it based on Google Cloud  user behaviors and patterns. 
• Responsible: Our AI Principles set out our commitment to developing technology responsibly. Your code and recommendations will not be reused for any model learning and development. This helps ensure the privacy of your data and code, and also the integrity of the knowledge space from which our AI models are trained.

New capabilities available in Duet AI for Google Cloud

Here are some of the new capabilities available to get us started on our mission to deliver a new personalized and intent-driven cloud experience:

• Code assistance provides AI-driven code assistance for cloud users such as application developers and data engineers. It gives code recommendations as they type in real time, generates full functions and code blocks, and identifies vulnerabilities and errors in the code, while suggesting fixes.

Code assistance auto-generates code for creating a Google Cloud Storage bucket

Code assistance will be available through multiple products and services across Google Cloud, such as in Cloud Workstations, our fully-managed secure development environment, and other code-editing experiences in the Google Cloud Console. Developers will also find code assistance in Cloud Shell Editor or via our Cloud Code IDE extensions for VSCode and JetBrains IDEs. It supports multiple languages including Go, Java, Javascript, Python, and SQL.

• Chat assistance allows people to use simple natural language to get answers on specific development or cloud-related questions. Users can engage with chat assistance to get real-time guidance on various topics, such as how to use certain cloud services or functions, or get detailed implementation plans for their cloud projects. It can also provide architectural or coding best practices, helping to reduce the need to go searching for relevant documents.

Use chat assistance to get the detailed steps for deploying an app on Cloud Run

Chat assistance will also be available across multiple Google Cloud surface areas, for example IDEs, the Cloud Console, and through products and services. Whether you’re a developer, operator, data engineer, or security professional, you’ll be able to leverage chat assistance to help get more work done faster. 

Looking to optimize these features further for developers specialized in one particular area? With Generative AI support in Vertex AI, enterprises can fine-tune Codey using their own code base. They can consume these customized Codey models directly from Vertex AI today, and later this year, they will be able to connect it to the built-in Duet AI experience. And don’t worry, if you choose to train Codey with your code, your private data is kept private, and not used in the broader foundation model training corpus. You will have transparency and control over where data is stored and how or if it is used. 

• Duet AI for AppSheet will let users create intelligent business applications,  connect their data, and build workflows into Google Workspace via natural language.  With no coding required, users will be able to build apps by describing their needs in a chat guided by AI-powered prompts. This makes app creation accessible to more users, which can allow developer teams to focus their time on other high-impact work.

Create business applications with Duet AI for AppSheet using natural language

Experiment with Duet AI for Google Cloud today

We believe that having an assistant who is constantly evolving by your side will not only reduce an already overwhelmed developer’s workload, but also bring back the excitement of cloud development. With Duet AI, you can navigate the cloud with more confidence, ease, and — dare we say it — fun. 

And this is just the beginning. The future of the cloud experience that we are shaping with Duet AI is full of possibilities. We believe the future of developer productivity is more targeted personalized assistance. Check here to see our vision for Duet AI for Google Cloud – the redefinition of productivity in the workplace through unique end-to-end AI assisted technologies. 

These early features of Duet AI for Google Cloud are available today for limited users and we will be expanding access very soon. Sign up here to join Google Cloud’s AI Trusted Tester Program.

As AI-powered tools become increasingly prevalent, prompt engineering is becoming a skill that developers need to master. Large language models (LLMs) and other generative foundation models require contextual, specific, and tailored natural language instructions to generate the desired output. This means that developers need to write prompts that are clear, concise, and informative. 

In this blog, we will explore six best practices that will make you a more efficient prompt engineer. By following our advice, you can begin creating more personalized, accurate, and contextually aware applications. So let’s get started!

Tip #1: Know the model’s strengths and weaknesses

As AI models evolve and become more complex, it is essential for developers to comprehend their capabilities and limitations. Understanding these strengths and weaknesses can help you, as a developer, avoid making mistakes and create safer, more reliable applications.

For example, an AI model that is trained to recognize images of blueberries may not be able to recognize images of strawberries. Why? Because the model was only trained on a dataset of blueberry images. If a developer uses this model to build an application that is supposed to recognize both blueberries and strawberries, the application would likely make mistakes, leading to an ineffective outcome, and poor user experience.

It’s important to note that AI models have the ability to be biased. This is due to AI models being trained on data that is collected from the real world, and so it can reflect the inequitable power dynamics inherently rooted in our social hierarchy. If the data that is used to train an AI model is biased, then the model will also be biased. This can lead to problems if the model is used to make decisions that affect people by reinforcing societal biases. Addressing these biases is important to ensure that data is fair, promoting equality, and ensuring the responsibility of AI technology. Prompt engineers should be aware of training limitations or biases so they can craft prompts more effectively and understand what kind of prompting is even possible for a given model.

Tip #2: Be as specific as possible

AI models have the ability to comprehend a variety of prompts. For instance Google’s PaLM 2 can understand natural language prompts, multilingual text, and even programming codes like Python and JavaScript. Although AI models can be very knowledgeable, they are still imperfect, and have the ability to misinterpret prompts that are not specific enough. In order for AI models to navigate ambiguity, it is important to tailor your prompts specifically to your desired outcome. 

Let’s say you would like your AI model to generate a recipe for 50 vegan blueberry muffins. If you prompt the model with “what is a recipe for blueberry muffins?”, the model does not know that you need to make 50 muffins. It is thus unlikely to list the larger volume of ingredients you’ll need or include tips to help you more efficiently bake such a large number of muffins. The model can only go off the context that is provided. A more effective prompt would be “I am hosting 50 guests. Generate a recipe for 50 blueberry muffins.” The model is more likely to generate a response that is relevant to your request and meets your specific requirements.

Tip #3: Utilize contextual prompts

Utilize contextual information in your prompts to help the model gain an in-depth understanding of your requests. Contextual prompts can include the specific task you want the model to perform, a replica of the output you’re looking for, or a persona to emulate, from a marketer or engineer to a high school teacher. Defining a tone and perspective for an AI model gives it a blueprint of the tone, style, and focused expertise you’re looking for to improve the quality, relevance, and effectiveness of your output. 

In the case of the blueberry muffins, it is important to prompt the model using the context of the situation. The model might need more context than generating a recipe for 50 people. If it needs to be aware that the recipe must be vegan friendly, you might prompt the model by asking it to answer by emulating a skilled vegan chef. 

By providing contextual prompts, you can help ensure that your AI interactions are as seamless and efficient as possible. The model will be able to more quickly understand your request and it will be able to generate more accurate and relevant responses.

Tip #4: Provide AI models with examples 

When creating prompts for AI models, it is helpful to provide examples. This is because prompts act as instructions for the model, and examples can help the model to understand what you are asking for. Providing a prompt with an example looks something like this: “here are several recipes I like – create a new recipe based on the ones I provided.” The model can now understand the your ability and needs in order to make this pastry,

Tip #5: Experiment with prompts and personas

The way you construct your prompt impacts the model’s output. By creatively exploring different requests, you will soon have an understanding of how the model weighs its answers, and what happens when you interfuse your domain knowledge, expertise, and lived experience with the power of a multi-billion parameter large language model. 

Try experimenting with different keywords, sentence structures, and prompt lengths to discover the perfect formula. Allow yourself to step into the shoes of various personas, from work personas such as “product engineer” or “customer service representatives,” to parental figures or celebrities such as your grandmother, a celebrity chef, and explore everything from cooking to coding!

By crafting unique, and innovative, requests replete with your expertise and experience, you can learn which prompts provide you with your ideal output. Further refining your prompts, known as ‘tuning,’ allows the model to have a greater understanding and framework for your next output.

Tip #6: Try chain-of-thought prompting

Chain of thought prompting is a technique for improving the reasoning capabilities of large language models (LLMs). It works by breaking down a complex problem into smaller steps, and then prompting the LLM to provide intermediate reasoning for each step. This helps the LLM to understand the problem more deeply, and to generate more accurate and informative answers. This will help you to understand the answer better and to make sure that the LLM is actually understanding the problem. 

Conclusion

Prompt engineering is a skill that all workers, across industries and organizations, will need as AI-powered tools are becoming more prevalent. Remember to incorporate these five essential tips the next time you communicate with an AI model, so you can generate the accurate outputs that you desire. AI will forever continue to develop, constantly refining itself as we use it, so I encourage you to remember that learning, for mind and machine, is a never ending journey. Happy Prompting!