june 26, 2025
Prompt engineering: experience in optimising LLM behaviour
During the development of a tool for automatic database documentation generation, we encountered several challenges when working with LLMs:
- Excessive, irrelevant information.
- Presence of technical details not relevant to the user.
- Inconsistency with the desired structure and style.
These issues negatively impacted the usefulness of the generated documentation. To improve its quality, we turned to prompt engineering — the technique of crafting precise queries for LLMs and one of the primary methods for guiding text generation outcomes. By applying prompt engineering techniques, we were able to significantly reduce the number of errors, improve the accuracy of the language, and ensure a consistent documentation structure.
«Prompt» is a textual query that is fed to the input of the LLM and determines the nature of its result. In order to obtain an accurate and relevant result, a prompt must clearly and concisely define the problem for the model.
Below is an example of an ineffective prompt:
| Prompt |
|---|
|
Create a description of the object based on this SQL: |
Such a prompt is too vague: it does not specify the expected output format, the style to follow, or which details to include.
A well-crafted prompt is shown below:
| Prompt |
|---|
|
Based on the provided SQL script, create a technical description of the table in the style of official documentation — list the fields along with their data types and purposes, specify primary keys and constraints: |
This prompt sets clear expectations for the output and helps the model generate structured text. The example illustrates the importance of a well-constructed query. In the following sections, we will look at the prompt engineering approaches we applied in practice, and how they helped improve the quality of the generated documentation.
Key rules for constructing prompts
A clearly defined structure helps achieve a formalised result that meets documentation requirements. It’s also recommended to include examples — they help the language model better understand the expected format and content of the output. The most effective approach is to use an output format that can be easily transformed into other formats, such as HTML, Markdown, or JSON.
Below is an example of a prompt we used to define the structure of the description and control the formatting of the output:
| Prompt |
|---|
|
"Example:\n" |
It is also possible to define the role the model should assume when generating text. This helps set the context in which it interprets the task and leads to more accurate and informative output. In our case, when generating documentation from SQL scripts, we specified that the model should act as a database documentation assistant. This helped the model focus on key structural elements, use professional terminology, and maintain a style aligned with technical documentation standards.
To control the degree of diversity and predictability in the model’s responses, generation parameters such as temperature, top-k, and top-p can be adjusted.
Temperature affects the model’s creativity: at low values (e.g. 0.2–0.5), it favours the most likely options, ensuring stability and predictability; at higher values (closer to 1), it allows for more randomness and creativity.
Top-k and top-p parameters control the range of possible tokens from which the model can select the next word:
| Parameters | |
|---|---|
|
Top-k |
Limits the model’s choices to the top k most probable tokens. For example, if k = 50, the model will choose the next token only from the 50 most likely options, ignoring all others. This reduces the risk of rare or inappropriate words appearing but may result in less flexible text generation. |
|
Top-p |
Works differently: instead of a fixed number of tokens, the model considers those whose cumulative probability reaches a specified threshold — for example, 90% when top-p = 0.9. This means the number of token options can vary dynamically depending on the probability distribution, resulting in more flexible and adaptive output. |
In documentation generation tasks, this is especially important: the model must maintain terminological accuracy, structure, and adherence to the required format, while avoiding verbatim repetition or mechanically produced text fragments.
Our results
The use of prompt engineering techniques in our project for automatic database documentation generation yielded tangible results. Well-formulated prompts, a clearly defined structure, the use of examples, and specifying the model’s role all contributed to significantly improving the quality of the generated descriptions. As a result, we achieved:
- A more than threefold reduction in the number of errors;
- Increased consistency in the output;
- Harmonised output style across results.
