Our experience of what works well with Reveal Ask, along with that of our clients, has shown great potential for the tool but as Reveal themselves would acknowledge, this is just the first iteration of GenAI within their platform.
What works well with Reveal Ask and what does not
We and our clients have had significant success using Ask to surface concepts and themes in support of informing the direction of an investigation. However, we have had mixed results when using Ask to provide transactional data responses. I would consider this as a ‘suitability for purpose’ issue rather than a limitation per se. For example “find all payments to Party A in the dataset” will certainly return results from a corpus of bank statements (say) but it may not return all of them, especially if there are many payments. In machine learning terms, think of this as the engine being tuned for precision, not recall (for a further explanation of these terms see https://salientdiscovery.com/finetuning-generative-ai-for-ediscovery/). So, in essence, Ask is better suited to an early case assessment task than pulling out every single instance of a transaction from a set of documents.
Enhancements
There are a few observations about what works well with Reveal Ask which have come up through initial use, as follows:
- Ask would benefit from the ability to submit follow up questions, where the references and response given, along with the previous prompt, form part of the context of the follow up question. As it is currently implemented, each question you submit stands alone, whereas the likely human thought process would be to assume the tool has remembered the context and response previously given when asking a follow up. We are led to believe that this enhancement is in the development roadmap.
- When reviewing a document from the references list, it would be extremely helpful to be taken to that relevant section of the text.
- Each reference displays a relevance score to the question asked. It would be helpful to filter references to those over a certain percentage of relevance, to further limit noise, or less likely relevant documents.
Other applications
We’re confident that the current Ask implementation will be further enhanced and improved but beyond that, what are the other applications for GenAI that we and our clients have seen a use for within Reveal?
- Document or small corpus summarisation. It would be extremely helpful to be able to summarise the salient points from a given document or set of documents.
- Review support. When performing review, it would be helpful to have a document-level understanding of the relevance of a document to the tagging regime you are applying. To answer the question, “why might this document be considered responsive to test x?”. Particularly useful to support first line reviewers to have that context provided.
- Privilege support. Similar to review, having a rationale for why a document might be considered privileged would be helpful.
Conclusion
In summary, the experience to date with Reveal Ask has been overwhelmingly positive but it should not be considered a replacement for investigators or lawyers. We strongly advise our clients to make themselves aware of the potential, but also of the limitations, which may not always be that obvious.
As such, use it as a tool to assist in the search, but then go and cross reference the results or use alternative and more traditional methods to add more confidence that you haven’t missed anything of value or relevance. Whatever you do, don’t take the verbatim results as the ultimate truth and remember the human in the loop still rules!
GenAI for eDiscovery: Success stories using Reveal Ask
We’ve been testing out RevealAsk with our clients to evaluate the potential for GenAI in their eDiscovery projects. Find out our successes, what we’ve learnt along the way and our views on where the technology will take us next.
1. Introduction, a reminder about how GenAI works in eDiscovery and a practical example about using Reveal Ask.
2. The art of prompt engineering and what we’ve learnt about the importance of getting the prompt ‘right’.
3. How to use Ask to accelerate an investigation: our learnings and strategies for success.
4. Getting the most from the integrated tool set – the power of Ask in combination with the wider Reveal toolset.
5. What works well, which use cases are best suited to the Ask capability and some observations about future enhancements
