Security Considerations for Generative AI Applications

The emerging generative AI solution architecture is shown below:

The control and data in this architecture flow as follows:

1. User actions and prompts are mapped to requests sent by the generative AI application to the semantic broker.
2. The semantic broker enriches the requests with context consisting of enterprise content and data fragments.
3. These fragments are retrieved from the semantic index, which has been pre-populated beforehand by indexing enterprise content and data.
4. The broker forwards the original request and the context to the AI model for processing.
5. The AI model responds with an answer or requests additional data.
6. The semantic broker obtains additional data by invoking the APIs provided by enterprise applications and third-party tools: CRM, ERP, web search, etc.
7. The final answer obtained from the AI model is post-processed by the semantic broker and sent to the generative AI application.
8. The application presents the response to the user.

In this diagram, only the Enterprise box can be considered a safe area. All other elements, including the end users, are sources of security risks.

A comprehensive approach to information security in a generative AI solution must consist of the following elements:

• Authentication. Every arrow in the diagram requires authentication. It may be necessary to propagate user identity throughout the entire call chain to control access to sensitive information. All credentials, such as API keys and passwords, must be reliably protected.
• Authorization. All elements of the solution must enforce access control policies. For instance, HR information stored in the semantic index must only be visible to members of the HR department. Access control must be enforced both for data and APIs.
• Confidentiality. Data must be reliably protected in flight and at rest. Communication must be performed over encrypted channels. Solution elements must store only the minimum amount of information. The information must be encrypted when stored.
• Data integrity. The solution must ensure the accuracy, completeness, and consistency of data. For example, when data changes in an enterprise data source, the semantic index must be updated automatically. One must also ensure the integrity of the underlying infrastructure: Docker images, operating systems, DBMS, etc.
• Non-repudiation. To prevent deniability and ensure accountability, it is necessary to log and retain copies of all requests and responses that pass through the communication links in the solution. Log records may contain sensitive information and must be reliably protected.
• Compliance. Compliance requirements must be mapped to procedures and policies, and the solution must perform and enforce them. For instance, if the index provider cannot guarantee HIPAA compliance, personally identifiable information must not be stored in the index.

All requests and responses in the generative AI solution architecture must pass through the semantic broker. Therefore, a semantic broker, such as AnyQuest, is ideally positioned to enforce information security, data privacy, and regulatory compliance.