Prompt Engineering | Iterative optimization and improvement of prompt

When trying to write the first prompt, meet the two principles mentioned in the previous blog: clear and clear, and give the system enough time to think. Then you can run it and see the result. If it doesn’t work the first time, the iterative process is to figure out why the instructions weren’t clear enough or why the algorithm wasn’t given enough time to think, to refine the idea, refine the hint, etc., looping many times until you find the one that works for your application prompt.

? Let’s start with the example of “Generate Marketing Copy from Product Manual” and think about how to iteratively analyze and improve your prompt.

Article directory

  • 1. Example: “Generate Marketing Copy from Product Manual”
  • 2. Problem 1: The generated text is too long!
  • 3. Problem 2: The text focuses on the wrong details!
  • 4. Question 3: A description in tabular form is required
  • 5. Summary:

1. Example: “Generate Marketing Copy from Product Manual”

Here is a product sheet for a chair that describes it as part of a mid-century inspired family, discusses construction, dimensions, chair options, materials, etc., and is made in Italy. Let's say you want to use this cookbook to help your marketing team write marketing-style descriptions for your online retail website.

Example: Product Specifications:

fact_sheet_chair = """
overview

    Part of a beautiful collection of mid-century style office furniture including filing cabinets, desks, bookcases, conference tables and more.
    Various housing colors and base coatings are available.
    Choice of plastic front and rear backrest trim (SWC-100) or full trim in 10 fabrics and 6 leathers (SWC-110).
    Base coating options are: stainless steel, matte black, gloss white or chrome.
    The chair is available with or without armrests.
    Suitable for home or commercial places.
    Eligible for contract use.

structure

    Plastic-coated aluminum base on five wheels.
    Pneumatic chair adjustment for easy lifting.

size

    Width 53 cm | 20.87 inches
    Depth 51 cm | 20.08 inches
    Height 80 cm | 31.50 inches
    Seat height 44 cm | 17.32 inches
    Seat Depth 41 cm | 16.14 in

options

    Soft floor or hard floor roller options.
    Two seat foam densities are available: medium (1.8 lb/ft3) or high (2.8 lb/ft3).
    No armrest or 8 position PU armrest.

Material
housing base slide

    Modified nylon PA6/PA66 coated cast aluminum.
    Shell thickness: 10mm.
    seat
    HD36 foam

country of origin

    Italy
"""

prompt: Create a marketing description based on the specification sheet:

prompt = f"""
Your task is to help the marketing team create a marketing description for the product based on the technical specification.
Write a product description based on the information provided in the technical specification marked with ```.
Technical note: ```{<!-- -->fact_sheet_chair}```
"""

Output:

Product description:

We are proud to present a beautiful collection of mid-century style office furniture that includes filing cabinets, desks, bookcases, conference tables and more. Our products are available in a wide range of housing colors and base coatings to suit your individual requirements. You can choose from plastic front and rear backrest trim (SWC-100) or full trim (SWC-110) in 10 fabrics and 6 leathers to make your office more comfortable and stylish.

Our base finish options include stainless steel, matte black, gloss white or chrome to suit your individual needs. The chair is available with or without armrests, suitable for domestic or commercial settings. Our products are eligible for contract use, providing you with more reliable protection.

Our product features a plastic coated aluminum base on five wheels with pneumatic chair adjustment for easy lifting and lowering. The dimensions are width 53 cm | 20.87 inches, depth 51 cm | 20.08 inches, height 80 cm | 31.50 inches, seat height 44 cm | 17.32 inches, seat depth 41 cm | 16.14 inches, providing you with a comfortable experience.

Our products are also available with soft or hard floor roller options, two seating foam densities: medium (1.8 lb/ft3) or high (2.8 lb/ft3), and no armrests or 8 position PU armrests, To meet your different needs.

Our products adopt modified nylon PA6/PA66 coated cast aluminum shell base sliding parts, the shell thickness is 10 mm, and the seat is made of HD36 foam to provide you with a more comfortable use experience. The country of origin of our products is Italy, to provide you with better quality assurance.

2. Problem 1: The generated text is too long!

The text generated by is too long, you need to modify the prompt to tell the model how long the text should be generated.

Optimized prompt:

# The optimized Prompt requires the generated description to be no more than 50 words
prompt = f"""
Your task is to help the marketing team create a retail website description of a product based on the technical specification.
Write a product description based on the information provided in the technical specification marked with ```.
Use up to 50 words.
Technical description: ```{<!-- -->fact_sheet_chair}```
"""

Output:

Mid-century style office furniture series, including filing cabinets, desks, bookcases, conference tables, etc. Available in various colors and finishes, with or without armrests. Base finish options are stainless steel, matte black, gloss white or chrome. Suitable for home or commercial use and eligible for contract use. made in Italy.

# Since Chinese requires word segmentation, the overall length is directly calculated here
len(response)
11
97

[Note: LLMs are okay, but not that great, at adhering to very precise word limits. Sometimes it outputs 60 or 65 words of content, but that’s more than reasonable. The reason for this is that LLMs interpret text using something called tokenizers, but they tend to be mediocre at counting characters. There are many different ways to try and control the length of the output you get. 】

3. Problem 2: The text focuses on the wrong details!

Assuming that the marketing audience is furniture retailers (they care more about the technical details and materials of the product) rather than consumers, then the text generated above is aimed at consumers. At this time, the prompt needs to be optimized and it is required to focus relevant to the target audience. , to more precisely describe the technical details of the product.

Optimized prompt:

  • Optimized Prompt, explaining object-oriented, what properties it should have and what aspects to focus on
prompt = f"""
Your task is to help the marketing team create a retail website description of a product based on the technical specification.

Write a product description based on the information provided in the technical specification marked with ```.

The description is aimed at furniture retailers and as such should be technical in nature and focus on the material construction of the product.

Use up to 50 words.

Technical note: ```{<!-- -->fact_sheet_chair}```
"""

Output:

This collection of mid-century office furniture includes filing cabinets, desks, credenzas and conference tables for the home or business. Available in a variety of housing colors and base finishes, base finish options are stainless steel, matte black, gloss white or chrome. The chair is available with or without armrests, with rollers for soft or hard floors, and a choice of two seat foam densities. Shell base slides in cast aluminum coated with modified nylon PA6/PA66, seat in HD36 foam. Country of origin is Italy.

If you want to further include the product ID at the end of the description, you can add “At the end of the description, include each 7-character product ID in the technical description” to the prompt.

# go further
prompt = f"""
Your task is to help the marketing team create a retail website description of a product based on the technical specification.

Write a product description based on the information provided in the technical specification marked with ```.

The description is aimed at furniture retailers and as such should be technical in nature and focus on the material construction of the product.

At the end of the description, include each 7-character product ID in the technical description.

Use up to 50 words.

Technical note: ```{<!-- -->fact_sheet_chair}```
"""
This collection of mid-century office furniture includes filing cabinets, desks, credenzas and conference tables for the home or business. Available in a variety of housing colors and base finishes, base finish options are stainless steel, matte black, gloss white or chrome. The chair is available with or without armrests, with a choice of plastic front and rear backrest upholstery or full upholstery in 10 fabrics and 6 leathers. The seat is in HD36 foam, available as medium or high density, with a seat height of 44 cm and a depth of 41 cm. Housing base slides in cast aluminum coated with modified nylon PA6/PA66, housing thickness 10mm. Country of origin is Italy.
Product ID: SWC-100/SWC-110.

4. Question 3: A description in form is required

A more complex prompt is shown below.

Some additional instructions are added below, asking it to extract information and organize it into a table, specifying the table’s columns, table names, and format, and also asking it to format everything into HTML that can be used in a web page.

Optimized prompt:

# Ask it to extract information and organize it into a table, and specify the columns, table name and format of the table
prompt = f"""
Your task is to help the marketing team create a retail website description of a product based on the technical specification.

Write a product description based on the information provided in the technical specification marked with ```.

The description is aimed at furniture retailers and as such should be technical in nature and focus on the material construction of the product.

At the end of the description, include each 7-character product ID in the technical specification.

After the description, include a table giving the dimensions of the product. The table should have two columns. The first column contains the name of the dimension. The second column only includes measurements in inches.

Name the table "Product Size".

Format everything into website-ready HTML. Put the description inside a <div> element.

Specifications: ```{<!-- -->fact_sheet_chair}```
"""

Output:

<div>
<h2>Mid-century style office furniture series chair</h2>
<p>This chair is part of a collection of mid-century style office furniture for home or business. It is available in a variety of housing colors and base finishes including stainless steel, matte black, gloss white or chrome. You can choose the chair with or without armrests, and soft or hard floor roller options. Additionally, you can choose from two seat foam densities: medium (1.8 lb/ft3) or high (2.8 lb/ft3).</p>
<p>The shell base slide of the chair is cast aluminum coated with modified nylon PA6/PA66, the shell thickness is 10 mm. The seat is in HD36 foam and the base is a plastic-coated aluminum base on five wheels, which allow pneumatic chair adjustment for easy lifting. Plus, the chair qualifies for contract use, making it an ideal choice for you.</p>
<p>Product ID: SWC-100</p>
</div>

<table>
  <caption>Product size</caption>
  <tr>
    <th>width</th>
    <td>20.87 inches</td>
  </tr>
  <tr>
    <th>depth</th>
    <td>20.08 inches</td>
  </tr>
  <tr>
    <th>height</th>
    <td>31.50 inches</td>
  </tr>
  <tr>
    <th>Seat height</th>
    <td>17.32 inches</td>
  </tr>
  <tr>
    <th>Seat Depth</th>
    <td>16.14 inches</td>
  </tr>
</table>

The table is rendered in HTML format, load it with ipython to see:

from IPython.display import display, HTML
display(HTML(response))

5. Summary:

Developers need to try to write a prompt first, and then gradually improve it through iterations until the desired result is obtained. The key is to have an efficient process for developing prompts, not to know the perfect prompt. For some more complex applications, multiple samples can be iteratively developed and evaluated. Finally, multiple prompts can be tested for average or worst performance on multiple samples in more mature applications.

?
?
?
?
?
?
Reference link:
[1] OpenAI
[2] Teacher Wu Enda’s: DeepLearning.AI
[3] DataWhale
[4] https://learn.deeplearning.ai/