Get Started

Client

AI User Manual

What the AI Does (and Does Not Do) 

Intended Purpose 

The AI engine in 3rdComply focuses on benchmarking external vendor security postures against internal organizational baselines. It functions to save research time and provide context-aware drafting aids for vendor risk management. 

Core Boundaries 

Every AI output within 3rdComply is classified strictly as an advisory recommendation. The AI never takes autonomous action on your behalf, nor does it replace professional human expertise and legal counsel. 

The AI IS / DOES... 

The AI IS NOT / DOES NOT... 

Mandatory Rule: Never use AI output as the sole basis for a compliance decision. AI-generated findings, risk scores, and assessments must be reviewed, validated, and approved by a qualified human before any organizational action is taken. 

AI Features per Product 

The 3rdComply platform leverages three primary AI capabilities designed to streamline vendor security alignment and cross-referencing: 

  1. Web Search Query: Queries restricted vendor domains using search grounding to complete unanswered security questions.  
  2. Vendor Standard Benchmark: Extracts expected security standards from internal policies to baseline questions.  
  3. Questionnaire Evaluator: Assesses external vendor answers against best practices to flag weak areas and suggest risk tiers. 

Operational Feature Guides 

  1. Web Search Query 
  • Purpose: Query the web (restricted to the vendor's official domain/subdomains) to answer unanswered vendor questionnaire questions.  
  • Input: Vendor name, URL, and questions.  
  • Output: Grounded answers with cited URLs.  
  • How to Use: 
  1. Navigate to Vendor Portals and open a vendor assessment questionnaire.  
  2. Select any unanswered questions and click Run Web Search.  
  3. Specify the vendor's name and official domain URL (e.g., https://vendor.com).  
  4. The AI executes a targeted search using search grounding, answers the questions based on the vendor's public security pages, and attaches direct citation links (URLs). 
  5. Vendor Standard Benchmark 
  • Purpose: Extract company security standards from internal policies to benchmark vendor questionnaire questions.  
  • Input: Internal policy files + questions.  
  • Output: Expected standards list.  
  • How to Use: 
  1. Navigate to Standards Setup.  
  2. Upload company internal policies (PDF, DOCX, or XLSX).  
  3. Input or select the security questions you wish to evaluate.  
  4. Click Extract Standards Benchmark.  
  5. The AI scans the policies, extracts the company's expected security baselines, and returns the expected standards for each question. 
  6. Questionnaire Evaluator 
  • Purpose: Assess third-party vendor questionnaire responses and suggest risk ratings.  
  • Input: Completed vendor questionnaire.  
  • Output: Risk scores + rationales.  
  • How to Use: 
  1. Navigate to Vendor Portals and open a completed vendor questionnaire.  
  2. Click Analyze Answers.  
  3. The AI reads each vendor response, evaluates it against compliance best practices, flags weak areas, and calculates a suggested risk score (Low, Medium, High). 

Human Oversight & Mandatory Checkpoints 

All AI outputs are advisory recommendations and require explicit human intervention before being finalized or acted upon. 

Checkpoint Matrix 

The following checkpoints must be cleared by qualified personnel before moving forward in the workflow: 

Universal Evaluation Checklist 

When executing any checkpoint review, always assess the output against these parameters: 

  • Factual Accuracy: Are the extracted rules, text quotes, and vendor references correct?  
  • Organizational Context: Does the output account for your specific organization's size, industry, and risk profile?  
  • Recency: Is the information gathered up to date, or does it reference outdated domain policies?  
  • Completeness: Does the output address all aspects of the evaluation prompt, or are there gaps? 

System Override Procedures 

The 3rdComply platform operates on the Override Principle: you always have the final say. The AI never takes autonomous action. 

Overriding Web Search Query Answers 

  • Modify Content: Adjust the reasoning, modify the returned answer text, or correct/replace the cited URLs manually in the questionnaire editing pane. 

Overriding Vendor Standard Benchmark Settings 

  • Edit Extracted Baselines: In the standards editor table, manually edit the extracted expected standard text, or delete a recommended standard and enter a custom requirement. 

Overriding Questionnaire Evaluator Scores 

  • Manual Risk Override: Select a different risk level from the risk dropdown in the vendor profile interface. Enter a mandatory text justification for the manual override.  
  • Edit Rationale: Click the edit icon on the assessment field to rewrite or clear the AI-suggested risk rationale. 

What to Report? 

When you encounter an AI output that is incorrect, unhelpful, or concerning, navigate to Feedback to submit a star rating (1–5) and a text description. Use the following guide to format what to include: 

FAQ (Frequently Asked Questions) 

Q1: Is my data sent to external AI providers? 

A: Yes, document content is sent to a third-party API for processing. However, all PII (names, emails, phone numbers, credit cards, API keys, etc.) is automatically masked before sending. It does not use API data for training; your original data is processed transiently and discarded after the request completes.  

Q2: Can the AI access my company's internal data? 

A: Only if you explicitly upload it. The AI has access strictly to: (1) the curated regulatory knowledge base (public frameworks), and (2) documents you upload during the current session. It cannot access your MySQL database, your other FEHA data, or other companies' data.  

Q3: What happens if I accidentally upload a file with sensitive data? 

A: The PII masking system (Presidio + 11 custom recognizers) automatically detects and masks 22 types of sensitive data before sending it to the LLM. Your original file is processed transiently—it is never stored in the AI system. After processing completes, all uploaded content is discarded.  

Q4: Can someone prompt-inject the AI through my documents? 

A: The system has guardrails that detect 40+ categories of prompt injection patterns in both user input and uploaded document content. Malicious content is rejected with a generic error message. The system also sanitizes retrieved RAG context to prevent injection from the knowledge base.  

Q5: What file formats can I upload? 

A: PDF, DOCX, XLSX, XLS, JSON, PNG, JPG, TIFF. Each file is validated for both file extension and magic byte signature (the system checks the actual file contents, not just the name). The file upload limit is a maximum of 10 MB per file.  

Q6: I got an error saying "inappropriate content detected." What happened? 

A: Your input triggered the guardrail system. This can happen if your text contains patterns that resemble prompt injection, even unintentionally. Try rephrasing your question. If you believe this is a false positive, report it via the Feedback feature.  

Q7: Are my AI interactions logged? 

A: Yes. Each AI request logs: model used, message count, input preview, PII entities detected, response preview, and token usage. These logs are used for quality monitoring and incident investigation. They are not shared externally. Observability is provided through Langfuse. 

Q8: The evaluation generation is taking a very long time to respond. 

A: For file analysis features or background operations like compilation modules, the expected processing time is 30-120 seconds depending on file count and size. Check the job status indicator. If persistent, contact your system administrator to check the AI Services container health. 

Key Reporting & Support Contact 

In accordance with your privacy and data processing frameworks, any suspected model bias, security concerns, or data processing violations can be directed to the Privacy Contact / Data Protection Representatives: 

  • DPO Team: Henry Kevin Marcelino Ratu & Ryan Yosedie  
  • Email: privacy@feha.io