The art of prompt engineering for GenAI tools in eDiscovery

So what does each of the inputs do in more detail? You need to think in terms of Search first and then LLM Reasoning second.

The Question box – Search

As mentioned, when enabling Ask on a matter, the system builds a more comprehensive semantic index. It is the key terms and phrases used in the question box which are used by the engine to filter result snippets from the semantic index. It is therefore critical to get that right, to ensure the most relevant snippets of content are selected. When you submit a question to Reveal Ask, the query that is run against the semantic index will use the terms and phrases provided quite literally in the question, to identify the likely most relevant content to pass to the response synthesis stage. As an example, imagine you wanted to detect exchanges of pricing information via email or messages relating to a particular product. Using a generic question such as “Identify examples of information exchange” will prioritise the selection of content which are responsive to the highlighted terms, which is unlikely to yield a good response. By comparison, by being more specific such as the question “ identify emails and messages where pricing for product x is discussed” will again use the highlighted terms to probably yield more relevant snippets to pass to synthesis for a superior response.

The Instructions box – LLM Reasoning

The Instructions box provides the LLM with guidance as to how it should construct its synthesised response. It has no influence on the content which is selected from the semantic index from which the response will be built, as it is applied after those snippets have been returned. Accordingly, getting the question ‘right’ can have significant benefits. As a general rule of thumb, focus on the query design in the first box and then prioritisation and structure of the response in the second. However, within these general guidelines, our advice would be to experiment with variations of the inputs. But some areas to consider would include:

1. Explicitly including the key topics or themes of the query in the prompt rather than being generic. For example:

Rather than “Identify any examples of pricing collusion in this data set”
Try “Detail any instances where discussion on pricing has taken place between Party A and any of Parties X, Y and Z”

2. Consider the complexity of the query. Rather than try to build too many aspects into the desired response, break the query up into smaller tasks. This would appear to reduce the likelihood of hallucination or confusion in the response as the semantic search task that gets run is more focused. For example:

“Detail any instances where discussion on pricing has taken place between Party A and any of Parties X, Y and Z and in each case identify the persons involved and the specifics of the pricing discussion being had” may result in confusion between each of the discussions identified with some conflating of the facts between exchanges.

3. Judicious use of terms like ‘top’, ‘key’ and ‘most’. As described above, the use of terms such as these will be interpreted in the search phase of the process quite literally, and as such, are not necessarily valuable. The engine is not designed to provide exhaustive lists of facts and an understanding of its internal process can help here. The engine will always try to return the ‘top’ results, irrespective of the instruction, as it returns the top 100 (currently) most semantically relevant hits before invoking a further proprietary relevance ranking algorithm to sequence what is sent to the LLM for synthesis. Even when asking for a summary of events, be aware that the engine may truncate its response when there may be further content that it hasn’t considered if the richness of the content set might yield more than 100 snippets. [See comments on granularity later.]


4. Use Additional Instructions wisely. As mentioned above, these provide guidance to the engine in terms of prioritisation and structure of the synthesised response. In our experience, we find that always asking for detailed responses as part of additional instructions, generally yields a more full and better structured response, especially when combined with requesting results in a timeline. [Note: Try also requesting results in tabular format if you haven’t already.]


5. On the topic of timelines, it should be noted that these are derived purely from the body text of the content, not from any associated metadata. If you want to restrict searches and questions to a specific date range based on metadata, you should use the filtering from the dashboard view first to pre-select content and then ask your question of that corpus.