As AI-powered tools become increasingly prevalent, prompt engineering is becoming a skill that developers need to master. Large language models (LLMs) and other generative foundation models require contextual, specific, and tailored natural language instructions to generate the desired output. This means that developers need to write prompts that are clear, concise, and informative. 

In this blog, we will explore six best practices that will make you a more efficient prompt engineer. By following our advice, you can begin creating more personalized, accurate, and contextually aware applications. So let’s get started!

Tip #1: Know the model’s strengths and weaknesses

As AI models evolve and become more complex, it is essential for developers to comprehend their capabilities and limitations. Understanding these strengths and weaknesses can help you, as a developer, avoid making mistakes and create safer, more reliable applications.

For example, an AI model that is trained to recognize images of blueberries may not be able to recognize images of strawberries. Why? Because the model was only trained on a dataset of blueberry images. If a developer uses this model to build an application that is supposed to recognize both blueberries and strawberries, the application would likely make mistakes, leading to an ineffective outcome, and poor user experience.

It’s important to note that AI models have the ability to be biased. This is due to AI models being trained on data that is collected from the real world, and so it can reflect the inequitable power dynamics inherently rooted in our social hierarchy. If the data that is used to train an AI model is biased, then the model will also be biased. This can lead to problems if the model is used to make decisions that affect people by reinforcing societal biases. Addressing these biases is important to ensure that data is fair, promoting equality, and ensuring the responsibility of AI technology. Prompt engineers should be aware of training limitations or biases so they can craft prompts more effectively and understand what kind of prompting is even possible for a given model.

Tip #2: Be as specific as possible

AI models have the ability to comprehend a variety of prompts. For instance Google’s PaLM 2 can understand natural language prompts, multilingual text, and even programming codes like Python and JavaScript. Although AI models can be very knowledgeable, they are still imperfect, and have the ability to misinterpret prompts that are not specific enough. In order for AI models to navigate ambiguity, it is important to tailor your prompts specifically to your desired outcome. 

Let’s say you would like your AI model to generate a recipe for 50 vegan blueberry muffins. If you prompt the model with “what is a recipe for blueberry muffins?”, the model does not know that you need to make 50 muffins. It is thus unlikely to list the larger volume of ingredients you’ll need or include tips to help you more efficiently bake such a large number of muffins. The model can only go off the context that is provided. A more effective prompt would be “I am hosting 50 guests. Generate a recipe for 50 blueberry muffins.” The model is more likely to generate a response that is relevant to your request and meets your specific requirements.

Tip #3: Utilize contextual prompts

Utilize contextual information in your prompts to help the model gain an in-depth understanding of your requests. Contextual prompts can include the specific task you want the model to perform, a replica of the output you’re looking for, or a persona to emulate, from a marketer or engineer to a high school teacher. Defining a tone and perspective for an AI model gives it a blueprint of the tone, style, and focused expertise you’re looking for to improve the quality, relevance, and effectiveness of your output. 

In the case of the blueberry muffins, it is important to prompt the model using the context of the situation. The model might need more context than generating a recipe for 50 people. If it needs to be aware that the recipe must be vegan friendly, you might prompt the model by asking it to answer by emulating a skilled vegan chef. 

By providing contextual prompts, you can help ensure that your AI interactions are as seamless and efficient as possible. The model will be able to more quickly understand your request and it will be able to generate more accurate and relevant responses.

Tip #4: Provide AI models with examples 

When creating prompts for AI models, it is helpful to provide examples. This is because prompts act as instructions for the model, and examples can help the model to understand what you are asking for. Providing a prompt with an example looks something like this: “here are several recipes I like – create a new recipe based on the ones I provided.” The model can now understand the your ability and needs in order to make this pastry,

Tip #5: Experiment with prompts and personas

The way you construct your prompt impacts the model’s output. By creatively exploring different requests, you will soon have an understanding of how the model weighs its answers, and what happens when you interfuse your domain knowledge, expertise, and lived experience with the power of a multi-billion parameter large language model. 

Try experimenting with different keywords, sentence structures, and prompt lengths to discover the perfect formula. Allow yourself to step into the shoes of various personas, from work personas such as “product engineer” or “customer service representatives,” to parental figures or celebrities such as your grandmother, a celebrity chef, and explore everything from cooking to coding!

By crafting unique, and innovative, requests replete with your expertise and experience, you can learn which prompts provide you with your ideal output. Further refining your prompts, known as ‘tuning,’ allows the model to have a greater understanding and framework for your next output.

Tip #6: Try chain-of-thought prompting

Chain of thought prompting is a technique for improving the reasoning capabilities of large language models (LLMs). It works by breaking down a complex problem into smaller steps, and then prompting the LLM to provide intermediate reasoning for each step. This helps the LLM to understand the problem more deeply, and to generate more accurate and informative answers. This will help you to understand the answer better and to make sure that the LLM is actually understanding the problem. 

Conclusion

Prompt engineering is a skill that all workers, across industries and organizations, will need as AI-powered tools are becoming more prevalent. Remember to incorporate these five essential tips the next time you communicate with an AI model, so you can generate the accurate outputs that you desire. AI will forever continue to develop, constantly refining itself as we use it, so I encourage you to remember that learning, for mind and machine, is a never ending journey. Happy Prompting!

Many of our customers want to know how to choose a technology stack for solving problems with machine learning (ML). There are many choices for these solutions available, some that you can build and some that you can buy. We’ll be focusing on the build side here, exploring the various options and the problems they solve, along with our recommendations. 

The best ML applications are trained with the largest amount of data

But first, keep in mind an important concept: the quality of your ML model improves with the size of your data. Dramatic ML performance and accuracy are driven by improvements in data size, as shown in the graph below. This is a text model, but the same principles hold for all kinds of ML models.

The X axis represents the size of the data set and the Y axis is the error rate. As the size of the data set increases, the error rate drops. But notice something critical about the size of the data set — the x-axis is2^20, 2^21, 2^ 22, etc. In other words, each new tic here is a doubling of the data set size. To get a linear decrease in your error rate you need to exponentially increase the size of your data set. 

The blue curve in the graph represents a slightly more sophisticated ML model than the orange curve. Suppose you are deciding between two choices: create a better model or double the data set size. Assuming that these two choices cost the same, it’s better to keep gathering more data. It’s only when improvements due to data size increases start to plateau that it becomes necessary to build a better model. 

Secondly, ML systems need to be retrained for new situations. For example, if you have a recommendation system in YouTube and you want to provide recommendations in Google Now, you can’t use the same recommendations model. You have to train it in the second instance on the recommendations you want to make in Google Now. So even though the model, the code, and the principles are the same, you have to retrain the model with new data for new situations. 

Now, let’s combine these two concepts: you get a better ML model when you have more data, and an ML model typically needs to be retrained for a new situation. You have a choice of either spending your time building an ML model or buying a vendor’s off-the-shelf model. 

To answer the question of whether to buy or whether to build, first determine if the buyable model is solving the same problem that you want to solve. Has it been trained on the same input and on similar labels? Let’s say you’re trying to do a product search, and the model has been trained on catalog images as inputs. But you want to do a product search based on users’ mobile phone photographs of the products. The model that was trained on catalog images won’t work on your mobile phone photographs, and you’d have to build a new model. 

But let’s say you’re considering a vendor’s translation model that’s been trained on speeches in the European Parliament. If you want to translate similar speeches, the model works well as it uses the same kind of data. 

The next question to ask: does the vendor have more data than you do? If the vendor has trained their model on speeches in the European Parliament but you have access to more speech data than they have, you should build. If they have more data, then we recommend buying their model. 

Bottom line: buy the vendor’s solution if it’s trained on the same problem and has access to more data than you do. 

Technology stack for common ML use cases

If you need to build, what is the technology stack you need? What are the skills your people need to develop? This depends on the type of problem you are solving.  There are four broad categories of ML applications: predictive analytics, unstructured data, automation, and personalization. The recommended technology stack for each is slightly different. 

Predictive analytics

Predictive analytics includes detecting fraud, predicting click-through rates, and forecasting demand. 

Step one: build an enterprise data warehouse
Here, your data set is primarily structured data, so our recommended first step is to store your data in an enterprise data warehouse (EDW). Your EDW is a source of training examples and product histories tracked over time, and can break down silos and gather data from throughout your organization.

Step two: get good at data analytics
Next, you’d build a data culture, get skilled at data analytics, start to build dashboards, and enable data-driven decisions. At this point, you have all of the data and you know which pieces are trustworthy. 

Step three: build ML
From your EDW, you can build your models using SQL pipelines. We recommend using BigQuery ML when doing ML with the data in your EDW. If you want to build a more sophisticated model, you can train TensorFlow/Keras models on BigQuery data. A third option is AutoML tables for state-of-the-art accuracy and for building online microservices.

Unstructured data

Examples of how our customers use ML to gain insights from unstructured data include annotating videos, identifying eye diseases, and triaging emails. Unstructured data can include videos, images, natural language, and text. Deep learning has revolutionized the way we do ML on unstructured data, whether you’re looking at language understanding, image classification, or speech-to-text. 

For unstructured data, the models you use will  employ deep learning. Here, the ROI heavily favors using AutoML. The amount of time that you’d spend trying to create a new ML model from scratch is almost never worth it. You can spend your money more effectively collecting more data than trying to get a slightly better model. Regardless of the type of unstructured data, our recommendation is to use AutoML for small and medium size data sizes.

But AutoML has a limit to scale. At some point, the size of your data set is going to be so large that architecture search is going to get really expensive. At that point, you may want to go to a best-of-breed model with custom retraining from TensorFlow Hub, for example.  If you have data sets that are in the millions of examples, you can build your own custom neural network (NN) architectures. But determine if your data set size has started to plateau, by plotting a graph similar to the one at the top of this post. Build a custom NN architecture only after you’ve plateaued, where increasing amounts of data won’t give you a better model. 

Automation

Some examples of how customers are using ML for automation include scheduling maintenance, counting retail footfall, and scanning medical forms. The key thing to keep in mind as you pick a technology stack for these problems is that you’re not building just one ML model. If you want to schedule maintenance orwant to reject transactions, for example, you’ll need to train multiple linked models. 

Instead of individual models, think in terms of ML pipelines, which you can orchestrate using all of the technologies already mentioned. Then you have three choices for operationalizing, with three levels of sophistication.

1. Vertex AI has turnkey serverless training and batch/online predictions. This is what is recommended for a team of data scientists. .
2. Deep Learning VM Image, Cloud Run, Cloud Functions or Dataflow feature customized training and batch/online predictions. This is what is recommended if the team consists of  data engineers and  scientists.
3. Vertex AI Pipelines are fully customizable and recommended for organizations with separate ML engineering and data science teams.

When doing automation, the individual models that you chain together into a pipeline will be a mix – some will be prebuilt, some will be customized, and others will be built from scratch. Vertex AI, by providing a unified interface for all these model types, simplifies the operationalization of these models.

Personalization

ML application examples of personalization include customer segmentation, customer targeting, and product recommendations. For personalization, we again recommend using an EDW, because customer segmentation uses structured marketing data. For product recommendations, you will similarly have prior purchases and web logs in your EDW., You can power clustering applications, or recommendation systems like matrix factorization, and create embeddings directly from your EDW for sophisticated recommendation systems.

For specific use cases, choose the technology stack based on your data size and scope. Start with BigQuery ML for its quick, easy matrix factorization approach. Once your application proves viable and you want a slightly better accuracy, then try AutoML recommendations. But once your data set grows beyond the capabilities of AutoML recommendations, consider training your own custom TensorFlow and Keras models. 

To summarize, successful ML starts with the question, “Do I build or do I buy?” If an off-the-shelf solution exists that was trained with similar data and with access to more data than you have, then buy it. Otherwise build it, using the technology stack recommended above for the four categories of ML applications.

Learn more about our artificial intelligence (AI) and ML solutions and check out sessions from our Applied ML Summit on-demand.

Financial markets were among the first to adopt new technologies, and that has certainly been true of the derivatives markets, which were early adopters of electronic trading. Going forward, new capabilities will transform the way industry participants communicate, analyze, and trade.

I sat down with Google Cloud’s Phil Moyer and former SEC Commissioner, Troy Paredes, for a fireside chat at FIA Boca 2022 to discuss the future of markets and policy, the new technologies that are already paving the way for greater speed and transparency, and how cloud can help promote greater resiliency, performance, and security to enable the long-term vision for the market. The following is a summary of our discussion.

The current state of cloud technology

When it comes to technology adoption, we’re seeing the market and participants adopt cloud technologies, and increasingly, machine learning (ML) on a wider scale. Cloud technology allows for easier, faster, and much more secure experimentation with large datasets and ML.

A recent Google sponsored study by Coalition Greenwich (September, 2021) showed that more than 93% of trading systems, exchanges, and data providers are in some way providing services on the cloud. The same study, revealed that about 72% of the financial industry across the buy side and sell side, intend to consume public cloud-data based market data within the next 12 months.

Data-driven decision-making and risk management have always been, and continue to remain, the cornerstones of the financial markets. Over time, technology innovation has facilitated access to better insights from data, and therefore, better decision-making and the ability to manage risk. That expectation is now mainstream, and will continue to grow in sophistication.

The multi-phased technology trajectory

The movement of exchanges to the cloud will occur in a “crawl-walk-run” fashion, with low-hanging fruits the first to be picked in the near term while bigger, paradigmatic changes will occur over the medium and long term. Some organizations are starting all three stages simultaneously, understanding that each will move at an independent cadence.

The “crawl” phase is one in which foundations are built, starting with organizations moving data to the cloud and experimenting with some degree of analytics. It’s one of the most important phases because it’s where the opportunity to increase transparency and risk management takes shape.

In moving to the cloud, the infrastructure – which in the past relied on a combination of people, processes, and some technology – becomes the code that runs applications. This early phase is key to empowering organizations to shift to a cloud-based, agile-first operating model that makes it easier and more seamless to launch new products in the future, including by freeing up people and resources from IT management to more mission-focused work.

Establishing the cloud operating model simplifies the “walk” and “run” phases where compliance is more automated, latency-sensitive applications are more readily available, and the next generation of exchanges, market participants, and regulators is better prepared to meet future challenges.

The “walk” phase is where much of the innovation happens. Exchanges are making significant progress in leveraging foundational data decisions in the “crawl” phase and innovations in the cloud to improve settlement, clearing, risk management, collateral management, and compliance, and launch new products.

And finally, the “run” phase is where organizations will start to move the latency-sensitive markets to the cloud, as the markets increasingly will demand low-latency and high performance along with transparency and analytics to solve historical obstacles to market access.

Opportunities for both regulators and market participants

Any time significant technological change takes place, regulators explore its implications, particularly with respect to their ability to meet their regulatory objectives.

Increasingly, we are seeing technological change driving more opportunities for regulators and market participants alike. Such changes may also allow better protection of the marketplace, with greater integrity and transparency.

Over time, regulatory regimes – rules, regulations, statutes, interpretations, and guidance – will also adjust to new technologies, both benefiting the marketplace and advancing regulatory goals.

As one example, the cloud is increasing the ability to meet compliance obligations by allowing compliance to be built into transactions. Moreover, predicated on the vision of real-time regulatory reporting, and given the pace of technological change in the marketplace over the last several years, various regulators have been using more advanced analytics. This trend will continue to help them more effectively and efficiently meet their objectives, and monitor and meet the expectations they have for the entire market.

Machine learning’s role in the financial markets

Google Cloud’s head of AI and Industry Solutions, Andrew Moore, said that ML will be doing three key things for us in the next 10 years: giving us meaning, providing concierge services, and serving as a guardian. Extracting information that is critical to investor decision-making can be extremely important. With more data than ever, ML can increase the ability to process it while also becoming more accessible in the cloud and better supporting regulatory objectives.

The technology will likely manifest in trading and anti-money laundering activities as they relate market functions, as well as managing a wide variety of risks – supporting the interests of both investors and regulators in terms of decision-making, surveillance, and protections.

Rather than taking individuals out of the equation, the digitization of markets, assets, and guard rails combined with ML will allow people to focus their expertise in different ways to achieve key objectives.

Building the market foundation for the future

The goals of operational resiliency, security, and privacy will continue to be critical for building the market foundation for both participants and regulators. While technology promises to create advantages in concrete, tangible ways, it will be important to scrutinize potential risks and concerns.

Priority one for technology providers is to build an environment of trustless security, including encryption at motion and encryption at rest, ensuring that markets are operationally resilient while instilling confidence for any exchange that runs on top of that infrastructure. Multicloud architectures and approaches are likely also to be part of the solution for operational resilience.

Throughout time, liquidity has been the outcome of improved access, transparency, and security. Technology providers are responding by sharing both the responsibility for, and fate of, the markets of the future to build an efficient, faster, and more transparent and secure financial industry.

You can learn more about our approach in our newest white paper, Building the financial markets foundation for the future.