As of early 2024, it is estimated that about 65% of businesses utilize generative AI in some capacity, which is almost double the figure from the previous year.

This doesn't come as a surprise, as people are becoming increasingly accustomed to using generative AI and are finding more and more ways to use it as an assistant in their daily tasks.

Plenty of manual work can now be automated. Analyzing data and creating summaries can be completed with just a single prompt, and many of the workers also report that generative AI helps them in their creative process, generating new ideas or even aiding software development.

Aside from the “big things,” AI can also come in handy when answering quick questions, retrieving info, fueling your thought process, and clearing doubts whenever you ask something.

However, when we talk about using generative AI in a corporate setting, we inevitably face the issue of security. 

It's a touchy subject if you find yourself using generative AI to analyze data and give responses that include sensitive information like company sales figures, profit margins, and user data (customer information, patient records, etc.). 

It's natural to think about how safe it is to chat with a generative AI assistant about sensitive company topics and feed it proprietary data from your organization. The concern is that these powerful language models can inadvertently memorize and regurgitate sensitive snippets during their training on corporate data sources. 

Why Does Using Generative AI Pose a Security Threat?

Firstly, let us make one thing clear: Using generative AI directly through well-established LLMs like ChatGPT, Gemini, or Claude doesn’t really pose a significant security threat.

A lot of enterprises find their needs fulfilled simply by subscribing directly to LLMs and granting their employees access to generative AI. The employees are then able to use the gen AI, but whenever they need its assistance with company data, they need to feed the AI with the data in question. This poses no real security threat when handling sensitive data (other than someone else at your work finding your chats, which is entirely preventable), but it doesn’t really provide the level of efficiency achieved when LLMs are thoroughly integrated with your company data.

As we said, some businesses don’t have an extensive need for generative AI. Simply copying some of the company data directly to LLMs and prompting them with specific commands might be enough for some.

However, generative AI can offer so much more. When integrated with company data, the AI gains the ability to dynamically analyze live data, generate insights and automated reports, follow projects in real-time, and assist with a wide array of data-driven tasks. Employees can simply ask any of the business-related questions and get immediate answer summaries and reports.

This level of integration allows businesses to harness the full power of generative AI and enable enterprise-wide automation of various tasks, data-driven decision-making, and full utilization of company data. 

While integrating generative AI with company data unlocks powerful capabilities, it also poses significant security risks. By granting the AI direct access to proprietary data sources, there is a heightened risk of sensitive information like trade secrets, financial data, or personal records being inadvertently exposed or leaked through the AI's outputs if proper data controls and monitoring are not implemented.

How Exactly Can Generative AI Leak Sensitive Data?

As we discussed, integrating generative AI with company data enables powerful capabilities but also introduces security risks. You might be wondering - how exactly could sensitive information get exposed when using these AI models? Let us walk you through some of the potential scenarios in which data leaks can occur if proper safeguards aren't in place.

The root cause of these potential data leaks lies in the way the LLM interacts with the enterprise's data. When an employee prompts the AI for a specific task, the system dynamically retrieves relevant data from the internal repositories to generate an accurate response. 

If these repositories contain sensitive information, such as customer records, financial data, or proprietary business insights, there is a risk of this data being unintentionally included in the AI's output.

Several technical aspects can contribute to these data leaks:

1. Dynamic data integration: Pre-trained LLMs are integrated with enterprise systems, creating a unified data repository that may contain sensitive information.
2. Data access and query handling: When employees prompt the LLM for analysis or summarization, the system fetches relevant data from the repository to provide accurate responses. If this data includes sensitive information, there is a risk of it being inadvertently exposed.
3. Insufficient access controls and monitoring: Without proper access controls and monitoring mechanisms, unauthorized employees might gain access to sensitive data through the LLM. This includes the lack of role-based access control (RBAC) and activity logging.
4. Data anonymization and sanitization issues: If the data fed to the LLM is not adequately anonymized or sanitized, the AI might include sensitive details in its responses, leading to inadvertent data exposure.

To illustrate this further, consider the following scenario:

A retail company integrates a pre-trained LLM with its enterprise search solution to assist employees in generating sales reports, analyzing customer behavior, and summarizing inventory data. The enterprise search solution creates a unified data repository containing sales figures, customer purchase histories, and inventory levels.

However, the customer data in this repository, including personal information and purchase histories, is not fully anonymized. Additionally, the system does not implement strict access controls, allowing all employees to query the LLM for detailed sales and customer information.

In this scenario, an employee, curious about a high-profile customer's purchase history, prompts the LLM with a query like, "What are the recent purchases of our top customers?" The LLM, accessing the unified repository, generates a response that includes specific details about the high-profile customer's purchases, revealing personal and transaction information.

Such leaks are possible, especially if sophisticated users craft prompts that manipulate the AI into revealing sensitive information. These prompts can be designed to exploit specific patterns that trigger AI to respond with sensitive data.

Misconfigurations in how the LLM accesses and processes data present another way that can lead to unintentional data exposure, along with security flaws in the integration software, which can also be exploited to access sensitive information.

How Can Data Leaks Be Prevented?

It is possible to use generative AI safely, but certain methods and systems must be established first to protect data with appropriate security measures:

Storing Sensitive Info Only in Working Memory

The first rule of thumb is to avoid saving raw customer data, especially private details, in any permanent storage. Instead, focus on keeping statistical models or data that's been stripped of personal info. This way, even if someone gets unauthorized access, they won't find any sensitive information.

Respecting Existing App Permissions

Also, make sure the AI system plays by the rules. It should only be allowed to see the same data that the person asking the question is allowed to see in the original applications. This keeps things fair and prevents any accidental leaks of confidential data.

Using Pre-Trained Models

Even though training your own LLM is a pretty niche-down and expensive undertaking, it is the riskiest way to manage sensitive company information, and it will likely lead to errors sooner or later. To keep things secure, use AI models that have already been trained on general data, not on any specific customer information. This way, there's no risk of sensitive details leaking out during the training process.

Processing Data in a Private Cloud

Keep all customer data within a private cloud system, separate from the public internet. This adds an extra layer of security, making it much harder for anyone outside the company to access sensitive information.

Controlling Access to Data

Set up strict rules about who can access what data using a system called "role-based access control." Also, keep a close eye on who's looking at what so that you can catch any suspicious activity right away.

Making Data Anonymous

Before any data goes into the AI system, it must be scrubbed clean of any sensitive details. Special techniques like data sanitization and ensuring that all personal information is removed. This is done to protect extra sensitive info like user data (patient records, transactions) 

Tailoring Responses to User Permissions

Lastly, customize the AI's responses based on who's asking the question. For example, a manager might get a detailed financial report, while a regular employee would only see a summary. This way, everyone gets the information they need, and sensitive data stays confidential.

‍

‍

Using Generative AI Safely with Akooda

We don’t store raw data, which might be sensitive and private, and we include PII in our systems, only statistical modeling. We classify the data so that we know exactly where it’s located in the source app. Therefore, we significantly reduce the risk of data leaks, as we don’t store leakable data.

Akooda will never base its responses or feed the generative AI with the data that is not accessible to the user. For instance, once a user asks a question, the response they get will be based on data that they have access to in the source apps. Hence, no data leak is possible, as any piece of information in the content is already known to the user.

Akooda enterprise search works with pre-trained LLMs, so there is no risk of data leakage based on model training. Users can work only with data that is already accessible to them, and all data is processed in our virtual private cluster, which is owned and managed by Akooda.

For example, if a company employee types in a prompt like "Give me a summary of all financial reports from last year," and some of those reports are only cleared for higher management, Akooda will offer AI summaries that differ based on the user's access level.

• If higher management has access to the financial reports, the response to their query might include this data.
• If another employee who does not have access to this data asks the same question, their answer will be different (based on what it has access to) or empty.

Akooda's approach ensures data security and privacy by only using data accessible to the user, processing it in a secure environment, and tailoring responses based on individual permissions.

‍