Drafted by Mistral, graded by Gemini

Drafted answer

Yes, customer data stored in databases, file storage, and backups is encrypted at rest using AES-256. Keys are managed in AWS Key Management Service (KMS) and are automatically rotated every year. Enterprise customers can provide and control their own encryption keys through AWS KMS, with revocation rendering their data unreadable by Vantrel.

• ↳ Security Whitepaper › Encryption

Reference answer

Yes. All customer data (databases, file storage, backups) is encrypted at rest with AES-256, keys managed in AWS KMS and rotated automatically every year.

Expected: Security whitepaper › Encryption

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant's answer perfectly matches the reference answer regarding encryption at rest, algorithms, and key management, while also correctly incorporating the enterprise BYOK feature found in the cited excerpts. All statements are fully supported by the citations, and no key points are missed.

Drafted answer

The minimum TLS version supported for data in transit is TLS 1.2. TLS 1.3 is preferred and negotiated when the client supports it, while older protocols (SSL, TLS 1.0, and TLS 1.1) are disabled.

• ↳ Security Whitepaper › Encryption

Reference answer

TLS 1.2 is the minimum; TLS 1.3 is preferred. SSL, TLS 1.0 and 1.1 are disabled, and HSTS is enforced.

Expected: Security whitepaper › Encryption

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant's answer correctly identifies TLS 1.2 as the minimum version, mentions that TLS 1.3 is preferred, and notes that older protocols are disabled, matching the reference answer. Every statement is directly supported by the cited excerpt about data in transit. All key points from the reference are fully covered.

Drafted answer

Vantrel has been ISO/IEC 27001:2022 certified since September 2023. The certification covers the development, operation, and support of Vantrel Plan and is subject to annual surveillance audits.

• ↳ Security Whitepaper › Certifications and audit reports

Reference answer

Yes, ISO/IEC 27001:2022 certified since September 2023, covering development, operation and support of Vantrel Plan, with annual surveillance audits.

Expected: Security whitepaper › Certifications and audit reports

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant's answer perfectly matches all facts in the reference answer, stating that Vantrel is ISO 27001:2022 certified since September 2023, covering the development, operation, and support of Vantrel Plan, and mentioning annual surveillance audits. Every statement in the answer is directly supported by the cited excerpt. Finally, all key points of the reference are fully covered.

Drafted answer

Vantrel can share the SOC 2 Type II report with customers and prospects under a non-disclosure agreement (NDA) upon request to security@vantrel.example. The report is generated annually and typically becomes available in March of the following year, covering the audit period from January 1 to December 31.

• ↳ Security Whitepaper › Certifications and audit reports

Reference answer

Yes. Vantrel has an annual SOC 2 Type II (Security, Availability, Confidentiality; period Jan 1 - Dec 31, report available around March). It is shared under NDA on request to security@vantrel.example.

Expected: Security whitepaper › Certifications and audit reports

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant accurately answers that the SOC 2 Type II report is available under NDA upon request, perfectly matching the reference answer. Every detail provided in the answer is directly supported by the cited excerpts from the documentation. All key points regarding the report's existence, coverage, timing, and sharing process are fully covered.

Drafted answer

For European customers, customer data is hosted in the European Union, specifically in AWS eu-west-3 (Paris, France) as the primary location with disaster recovery in AWS eu-central-1 (Frankfurt, Germany). Customer data at rest, including backups and disaster recovery copies, remains within the selected EU region. Infrastructure and application logs for EU-hosted tenants are centralized in Datadog’s EU site.

• ↳ Data Processing Agreement (summary) › International data transfers (not in expected sources)
• ↳ Infrastructure and Hosting › Hosting regions
• ↳ Sub-processors › Infrastructure sub-processors (not in expected sources)

Reference answer

On AWS in eu-west-3 (Paris, France), with disaster recovery in eu-central-1 (Frankfurt, Germany). Data at rest, including backups, stays in the selected region.

Expected: Infrastructure hosting › Hosting regions

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant correctly identifies the hosting locations (Paris and Frankfurt), mentions disaster recovery and data at rest staying in the region, and adds relevant details about logging from the cited excerpts. All facts match the reference and are fully supported by the cited documentation.

Drafted answer

For Vantrel Plan, our Recovery Point Objective (RPO) is 1 hour and our Recovery Time Objective (RTO) is 4 hours in the event of a major incident affecting an entire AWS region.

• ↳ Business Continuity and Disaster Recovery › Objectives

Reference answer

RPO of 1 hour and RTO of 4 hours.

Expected: Business continuity › Objectives

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant correctly identified the RPO of 1 hour and RTO of 4 hours as stated in the reference answer. All statements are fully supported by the cited excerpts from the documentation, and all key points are covered accurately.

Drafted answer

Backups are taken continuously for databases with point-in-time recovery available for the last 7 days, and full encrypted snapshots are taken daily. These backups are retained for 35 days.

• ↳ Business Continuity and Disaster Recovery › Backups

Reference answer

Continuous backups with 7-day point-in-time recovery, plus daily encrypted full snapshots retained for 35 days, replicated to the DR location. Restores are tested monthly.

Expected: Business continuity › Backups

Grader · correctness 5 · faithfulness 5 · completeness 3

The assistant correctly states the frequency and retention periods for both continuous point-in-time backups and daily snapshots, matching the reference answer well. All statements are fully supported by the cited excerpts, although it misses the detail about monthly restore testing and replication to the DR location, which slightly impacts completeness.

Drafted answer

Vantrel supports Single Sign-On (SSO) using SAML 2.0 and OpenID Connect (OIDC) on Business and Enterprise plans only.

• ↳ Identity and Access Management › Customer authentication

Reference answer

SAML 2.0 and OpenID Connect, on Business and Enterprise plans. Tested with Microsoft Entra ID, Okta and Google Workspace. Enforcing SSO disables password login.

Expected: Identity access › Customer authentication

Grader · correctness 3 · faithfulness 5 · completeness 3

The assistant correctly identifies SAML 2.0 and OIDC on Business and Enterprise plans, matching the main points of the reference. However, it misses the detail about enforcing SSO disabling password login, and omits the tested identity providers, making it incomplete regarding secondary facts. All stated facts are directly supported by the cited excerpts.

Drafted answer

Vantrel supports SCIM 2.0 for automated user provisioning and deprovisioning only on the Enterprise plan. Other plans require manual user management by workspace administrators.

• ↳ Identity and Access Management › User provisioning

Reference answer

Yes, SCIM 2.0 is supported, on the Enterprise plan only.

Expected: Identity access › User provisioning

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant's answer perfectly matches the reference answer by confirming SCIM 2.0 support and noting the Enterprise plan restriction. Every statement made by the assistant is directly and explicitly supported by the cited documentation excerpt, and all key points are fully covered.

Drafted answer

For Enterprise customers, Vantrel commits to a monthly uptime of 99.95%. This is explicitly stated in the Service Level Agreement and Support documentation.

• ↳ Service Level Agreement and Support › Availability commitment

Reference answer

99.95% monthly uptime for Enterprise (excluding scheduled maintenance), with service credits of 10% or 25% of the monthly fee.

Expected: Sla support › Availability commitment

Grader · correctness 5 · faithfulness 5 · completeness 3

The assistant correctly states the 99.95% uptime commitment for Enterprise customers, which matches the reference and is fully supported by the cited excerpts. However, it omits the details regarding service credits mentioned in the reference answer, leading to a minor deduction in completeness.

Drafted answer

Vantrel notifies customers of a personal data breach without undue delay, and in any case within 48 hours after becoming aware of it.

• ↳ Data Processing Agreement (summary) › Personal data breach notification

Reference answer

Without undue delay and in any case within 48 hours after becoming aware of it.

Expected: Data processing agreement › Personal data breach notification

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant's answer perfectly matches the reference answer by stating that notification will occur without undue delay and within 48 hours. Every statement in the answer is fully supported by the cited documentation excerpt, and all key points from the reference are covered.

Drafted answer

Vantrel performs penetration testing at least once a year, and additionally before major architectural changes. The most recent penetration test was completed in March 2026 and identified no critical findings, only two high-severity findings, which were both remediated within 30 days and verified by the testing firm during a retest.

• ↳ Penetration Testing and Vulnerability Management › Penetration testing

Reference answer

At least annually by an independent third party, plus before major architectural changes. The last test (March 2026) found no critical and two high findings, both remediated within 30 days and verified by retest.

Expected: Vulnerability management › Penetration testing

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant's answer perfectly matches the reference answer in every detail, including the frequency, timing, and specific results of the last penetration test. Every statement made by the assistant is directly supported by the provided documentation excerpts, and all key points from the reference are fully covered.

Drafted answer

Vantrel remediates critical vulnerabilities within 7 days. Exceptions require documented compensating controls and approval by the CISO.

• ↳ Penetration Testing and Vulnerability Management › Vulnerability management

Reference answer

Critical vulnerabilities must be remediated within 7 days (high: 30 days, medium: 90 days).

Expected: Vulnerability management › Vulnerability management

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant accurately provides the 7-day remediation timeline for critical vulnerabilities, matching both the reference answer and the cited documentation. All claims are fully supported by the provided excerpt, and the answer includes all necessary key points.

Drafted answer

Vantrel’s sub-processors are listed in the Sub-processors document (v2026.2) and include the following: Amazon Web Services EMEA SARL for cloud hosting, storage, and backups in the customer-selected region (EU or US); Cloudflare, Inc. for CDN, Web Application Firewall, and DDoS protection on a global edge network (traffic in transit only); Datadog, Inc. for infrastructure and application monitoring and logs in EU (for EU tenants) or US (for US tenants); Stripe Payments Europe, Ltd. for billing and subscription payments in EU/US; Zendesk, Inc. for customer support ticketing in EU; SendGrid (Twilio Inc.) for transactional emails in US; and Microsoft Ireland Operations Ltd. (Azure OpenAI Service) for AI features (Vantrel Assist) in the EU Data Zone, used only when the customer has enabled Vantrel Assist.

• ↳ Sub-processors › Infrastructure sub-processors
• ↳ Sub-processors › Service sub-processors
• ↳ Sub-processors › Notes (not in expected sources)

Reference answer

AWS (hosting), Cloudflare (CDN/WAF/DDoS), Datadog (monitoring), Stripe (billing), Zendesk (support), SendGrid/Twilio (transactional emails), Microsoft Azure OpenAI Service (AI features).

Expected: Subprocessors › Infrastructure sub-processors · Subprocessors › Service sub-processors

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant's answer correctly lists all the sub-processors mentioned in the reference answer, including AWS, Cloudflare, Datadog, Stripe, Zendesk, SendGrid/Twilio, and Microsoft Azure OpenAI Service. Every detail provided is fully supported by the cited documentation excerpts, ensuring high faithfulness. Furthermore, all key points from the reference answer are covered comprehensively.

Drafted answer

No, customer data is never used to train or fine-tune AI models by Vantrel or its AI sub-processor. Only the data required to answer a specific request is sent to the model, and prompts and responses are not stored beyond the user's session except in the audit log, which does not record the content of the request.

• ↳ AI Features Policy › Use of customer data

Reference answer

No. Customer data is never used to train or fine-tune AI models, neither by Vantrel nor by its AI sub-processor.

Expected: Ai features policy › Use of customer data

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant's answer perfectly matches the reference answer in stating that customer data is never used to train AI models. Every statement made by the assistant is fully supported by the provided documentation excerpts, and it covers all key points comprehensively.

Drafted answer

Yes, your employees can sign in with their corporate Okta accounts if you are on the Business or Enterprise plan, as Vantrel supports OpenID Connect (OIDC) for Single Sign-On (SSO) with Okta as a tested identity provider.

• ↳ Identity and Access Management › Customer authentication

Reference answer

Yes, via SAML 2.0 or OIDC SSO on Business and Enterprise plans; Okta is a tested identity provider.

Expected: Identity access › Customer authentication

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant correctly states that employees can sign in with corporate Okta accounts using OIDC on Business and Enterprise plans, matching the reference answer. All claims are fully supported by the cited documentation excerpt. Furthermore, all key points regarding the plan restriction, protocols, and Okta being a tested provider are included.

Drafted answer

If your main data center in Paris (AWS eu-west-3) were destroyed, you could lose up to 1 hour of data due to the Recovery Point Objective (RPO) of 1 hour. Data is continuously backed up and replicated to the disaster recovery location in Frankfurt (AWS eu-central-1), ensuring no data loss beyond this RPO in a regional outage scenario.

• ↳ Business Continuity and Disaster Recovery › Backups (not in expected sources)
• ↳ Business Continuity and Disaster Recovery › Objectives

Reference answer

At most about 1 hour of data (RPO 1 hour), with service restored from the Frankfurt DR region within 4 hours (RTO).

Expected: Business continuity › Objectives · Infrastructure hosting › Hosting regions

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant's answer correctly identifies the RPO of 1 hour from the documentation, matching the key facts in the reference answer regarding potential data loss. Every statement made by the assistant is fully supported by the cited excerpts on RPO and backups, and the answer is complete by covering the maximum potential data loss.

Drafted answer

No, Vantrel engineers cannot see customer budget figures unless explicitly granted temporary access by the customer through the administration console. Access is time-limited to a maximum of 72 hours and logged for audit purposes.

• ↳ Identity and Access Management › Vantrel employee access to customer data

Reference answer

No standing access. Support can only access a workspace when the customer grants temporary access (max 72 hours), which is logged and visible in the audit trail.

Expected: Identity access › Vantrel employee access to customer data

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant's answer perfectly matches the reference answer in terms of facts, faithfully reflects the provided documentation excerpts without any hallucinations, and completely covers all necessary key points such as the lack of standing access, temporary customer-granted access, the 72-hour limit, and audit logging.

Drafted answer

Yes, Enterprise customers can provide and control their own encryption keys for data at rest through AWS Key Management Service (KMS) as part of the Customer-managed keys (BYOK) feature. Revoking the key makes the customer's data unreadable by Vantrel.

• ↳ Security Whitepaper › Encryption

Reference answer

Yes, Enterprise customers can use customer-managed keys (BYOK) via AWS KMS; revoking the key makes the data unreadable by Vantrel.

Expected: Security whitepaper › Encryption

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant correctly states that Enterprise customers can hold their own keys using AWS KMS through the BYOK feature and that revoking the key makes data unreadable, perfectly matching the reference answer. Every statement in the answer is directly supported by the provided excerpt regarding customer-managed keys. All key points from the reference are fully covered without any missing information.

Drafted answer

Yes, Vantrel performs background checks on all new employees before their start date, where permitted by local law. All employees also complete security and privacy awareness training during onboarding and annually.

• ↳ Security Whitepaper › Employee security

Reference answer

Yes, background checks are performed on all new employees before their start date, where permitted by local law.

Expected: Security whitepaper › Employee security

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant correctly states that background checks are performed on all new employees before their start date, matching the reference answer. All claims made in the answer are fully supported by the cited excerpts. Additionally, the answer covers the key points completely while including some extra valid context from the citations.

Drafted answer

For Business plan customers, the first response target for a service outage (P1) is 4 hours during business hours (Monday to Friday). For Enterprise plan customers, the first response target is 1 hour, 24/7. The response time for Starter plan customers is not specified beyond best-effort email support. Scheduled maintenance occurs on Sundays between 02:00 and 06:00 (Paris or New York time, depending on region), so an outage at 3am on a Sunday may coincide with scheduled maintenance, which is excluded from availability commitments.

• ↳ Service Level Agreement and Support › Support
• ↳ Service Level Agreement and Support › Scheduled maintenance (not in expected sources)
• ↳ Service Level Agreement and Support › Availability commitment (not in expected sources)

Reference answer

Enterprise: P1 first response within 1 hour, 24/7. Business: support is business hours only, P1 first response within 4 hours.

Expected: Sla support › Support

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant correctly details the response times for both Business and Enterprise plans and properly incorporates the maintenance window nuance found in the excerpts. All stated facts match the reference and are fully supported by the citations, covering all key points completely.

Drafted answer

Yes, some of our sub-processors process customer data outside the EU. Stripe Payments Europe, Ltd. processes billing and subscription payment data in the US for customers hosted in the EU. SendGrid (Twilio Inc.) processes transactional email data in the US for all customers. Amazon Web Services EMEA SARL processes data in the region selected by the customer, which may be the US. Datadog, Inc. processes infrastructure and application monitoring data in the US for customers hosted in the US.

• ↳ Sub-processors › Service sub-processors
• ↳ Sub-processors › Infrastructure sub-processors

Reference answer

For EU tenants, hosting (AWS), monitoring (Datadog), support (Zendesk) and AI (Azure EU Data Zone) stay in the EU. Cloudflare handles traffic in transit on its global network, Stripe operates in EU/US, and SendGrid processes email addresses and notification content in the US. Transfers are covered by SCCs; Vantrel Inc. is under the EU-U.S. Data Privacy Framework.

Expected: Subprocessors › Infrastructure sub-processors · Subprocessors › Service sub-processors · Data processing agreement › International data transfers

Grader · correctness 3 · faithfulness 3 · completeness 3

The assistant correctly states that some sub-processors process data outside the EU, listing Stripe and SendGrid accurately based on the excerpts. However, the assistant includes inaccuracies for Datadog and AWS by misapplying their data locations for EU tenants as stated in the reference. Overall, the main point is correct and supported, but has some secondary errors.

Drafted answer

Customers affected by a security incident involving their data are notified by email by the Security team. For service availability incidents, updates are published on the public status page (status.vantrel.example) at least every 60 minutes for SEV1 incidents. Additionally, in the case of a personal data breach, Vantrel notifies the customer without undue delay and within 48 hours after becoming aware of it, as defined in the Data Processing Agreement.

• ↳ Incident Response › Customer communication
• ↳ Data Processing Agreement (summary) › Personal data breach notification

Reference answer

Availability incidents on the status page (updates at least every 60 min for SEV1); customers whose data is affected are emailed by Security within the DPA timeline (48 hours); a root cause analysis is shared within 10 business days after a SEV1.

Expected: Incident response › Customer communication · Data processing agreement › Personal data breach notification

Grader · correctness 5 · faithfulness 5 · completeness 3

The assistant's answer covers the status page updates, email notifications for affected customers based on DPA timelines, and mentions personal data breaches within 48 hours. However, it misses the root cause analysis sharing within 10 business days, which is present in the reference answer and citations. The facts stated are consistent with the reference and fully supported by the cited excerpts.

Drafted answer

For customer users, Vantrel supports Multi-Factor Authentication (MFA) using TOTP authenticator apps or FIDO2/WebAuthn security keys. SMS-based MFA is not supported. Administrators can enforce MFA for all users in their workspace. For Vantrel staff accessing production, access is granted only through temporary, time-limited (maximum 72 hours) customer-granted permissions from the administration console, with no standing access to customer data. Access rights to internal systems are reviewed quarterly, and production account access is restricted to the Platform Engineering and Security teams, reviewed quarterly.

• ↳ Identity and Access Management › Customer authentication
• ↳ Identity and Access Management › Vantrel employee access to customer data (not in expected sources)
• ↳ Infrastructure and Hosting › Environments (not in expected sources)

Reference answer

Customers: TOTP or FIDO2/WebAuthn, can be made mandatory by admins, no SMS. Staff: production admin access goes through a bastion with SSO, MFA and session recording; direct SSH disabled.

Expected: Identity access › Customer authentication · Infrastructure hosting › Network security

Grader · correctness 3 · faithfulness 4 · completeness 3

The assistant correctly describes the customer MFA options (TOTP, FIDO2, mandatory admin enforcement, no SMS), matching the reference and citing the correct excerpt. However, for staff production access, the assistant missed the bastion host, SSO, MFA, and session recording details mentioned in the reference, instead describing customer data access permissions for support engineers.

Drafted answer

Infrastructure and application logs are stored in Datadog, with EU-hosted tenants using the EU site. Infrastructure logs are retained for 12 months, as documented in S1. Audit logs are retained for 13 months for Vantrel Plan customers and can be exported via API or viewed in the administration console, while Enterprise customers can stream audit logs to their SIEM via webhook, as documented in S2. The retention duration for audit logs is not specified for US-hosted tenants.

• ↳ Infrastructure and Hosting › Logging and monitoring
• ↳ Identity and Access Management › Audit logs

Reference answer

Infrastructure and application logs are centralized in Datadog (EU site for EU tenants), retained 12 months. Customer audit logs are retained 13 months and exportable via API or streamed to a SIEM (Enterprise).

Expected: Infrastructure hosting › Logging and monitoring · Identity access › Audit logs

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant correctly states that infrastructure and application logs are in Datadog with a 12-month retention period, and audit logs are retained for 13 months with export and SIEM streaming options. All statements are fully supported by the cited excerpts, and it covers almost all key points from the reference answer.

Drafted answer

Yes, Vantrel sends customer data to a third-party sub-processor when Vantrel Assist is enabled. The data is sent to Microsoft Ireland Operations Ltd. (Azure OpenAI Service) in the EU Data Zone for EU-hosted tenants only. Only the data required to answer a specific request (e.g., selected cells and their labels) is transmitted, and it is never used to train or fine-tune AI models. Data is not stored beyond the user's session except in the audit log, which records only that an AI feature was used, not the content of the request.

• ↳ AI Features Policy › Use of customer data
• ↳ Sub-processors › Service sub-processors
• ↳ AI Features Policy › Vantrel Assist
• ↳ Sub-processors › Notes (not in expected sources)

Reference answer

Only if a workspace Admin enables Vantrel Assist (off by default). Then only the data needed for the request is sent to Microsoft Azure OpenAI Service in the EU Data Zone; it is never used for training.

Expected: Ai features policy › Vantrel Assist · Ai features policy › Use of customer data · Subprocessors › Service sub-processors

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant correctly identifies that data is sent to a third party (Microsoft Azure OpenAI Service) only when enabled by an admin, and accurately covers all key points including the location, lack of model training, and data limitation. All statements are fully supported by the cited documentation excerpts.

Drafted answer

Vantrel operates a private bug bounty program with invited researchers. The documentation does not specify reward amounts for reported vulnerabilities.

• ↳ Incident Response › Reporting a vulnerability

Reference answer

Yes, a private bug bounty program with invited researchers. Reward amounts are not documented.

Expected: Incident response › Reporting a vulnerability

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant correctly states that the company operates a private bug bounty program with invited researchers and that the reward amounts are not specified in the documentation. Every statement is fully supported by the cited excerpt, and all key points from the reference answer are covered.

Drafted answer

The documentation does not specify which independent third-party security firm performed the last penetration test.

Reference answer

An independent third-party security firm, in March 2026. The firm's name is not documented.

Expected: Vulnerability management › Penetration testing

Grader · correctness 5 · faithfulness 5 · completeness 5

The reference answer states that the name of the firm is not documented, and the assistant correctly identifies that the documentation does not specify the firm's name. Both correctness, faithfulness, and completeness achieve the maximum score as the assistant accurately reflects the state of the documentation.

Drafted answer

Vantrel supports streaming audit logs to a SIEM via webhook for Enterprise customers only. The documentation does not specify which SIEM products are supported beyond this capability.

• ↳ Identity and Access Management › Audit logs

Reference answer

Enterprise customers can stream audit logs to their SIEM via webhook; specific supported SIEM products are not listed.

Expected: Identity access › Audit logs

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant correctly points out that specific SIEM products are not listed, matching the reference answer. All statements are fully supported by the cited documentation excerpt, and the key points regarding Enterprise plans and webhooks are included.

Drafted answer

The name of the Data Protection Officer (DPO) role is not provided, but the contact email for the DPO is dpo@vantrel.example.

• ↳ Company Overview › Security and compliance organization

Reference answer

The DPO can be contacted at dpo@vantrel.example; the DPO's name is not documented.

Expected: Company overview › Security and compliance organization

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant correctly identified that the DPO's name is not provided while accurately giving the contact email from the cited excerpts. Every statement in the answer is fully supported by the text, and it covers all key points present in the reference answer.

Drafted answer

No, Vantrel does not currently support custom roles. Only the four default roles (Admin, Modeler, Contributor, Viewer) are available.

• ↳ Identity and Access Management › Roles and permissions

Reference answer

No, custom roles are not currently supported. Four default roles exist (Admin, Modeler, Contributor, Viewer) plus dimension-member-level restrictions.

Expected: Identity access › Roles and permissions

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant correctly states that custom roles are not supported, matching the reference answer and citing the exact excerpt that says so. Every statement in the answer is fully supported by the cited documentation, and all key points from the reference are covered.

Drafted answer

No, Vantrel does not offer on-premise, private cloud, or self-hosted deployments. Vantrel Plan is delivered exclusively as a multi-tenant SaaS application.

• ↳ Company Overview › Product and delivery model

Reference answer

No. Vantrel Plan is SaaS only; no on-premise, private cloud or self-hosted deployment.

Expected: Company overview › Product and delivery model

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant's answer perfectly matches the reference answer by stating that on-premise deployment is not offered and that the product is SaaS only. All statements are fully supported by the provided documentation excerpt, and all key points are covered accurately.

Drafted answer

No, Vantrel does not support SMS-based multi-factor authentication. Only TOTP authenticator apps or FIDO2/WebAuthn security keys are supported for MFA.

• ↳ Identity and Access Management › Customer authentication

Reference answer

No, SMS-based MFA is not supported (TOTP and FIDO2/WebAuthn are).

Expected: Identity access › Customer authentication

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant correctly states that SMS-based MFA is not supported while mentioning the supported alternatives (TOTP and FIDO2/WebAuthn), matching the reference answer. Every statement is fully supported by the cited documentation excerpt, and all key points from the reference are covered.

Drafted answer

The documentation does not mention hosting regions in Australia.

Reference answer

No. Only EU (Paris/Frankfurt) and US (N. Virginia/Oregon) regions are offered.

Expected: Infrastructure hosting › Hosting regions

Grader · correctness 1 · faithfulness 5 · completeness 5

The reference answer states that data cannot be hosted in Australia, while the assistant correctly identified that the documentation does not mention hosting regions in Australia. According to the scoring guidelines, when the reference gives an explicit negative answer and the documentation lacks it, the assistant saying it is not documented is appropriate, but since the reference actually provides a definitive negative answer ('No'), the correctness is 1. However, treating the reference strictly as 'not documented' by the assistant yields a high faithfulness and completeness regarding the lack of documentation.

Drafted answer

No, Vantrel does not currently have an accessibility conformance report (VPAT). We are working toward conformance with WCAG 2.1 level AA.

• ↳ Product FAQ › Is Vantrel Plan accessible to users with disabilities?

Reference answer

Not yet. Vantrel is working toward WCAG 2.1 AA, but a VPAT is not available yet.

Expected: Product faq › Is Vantrel Plan accessible to users with disabilities?

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant correctly states that a VPAT is not currently available and mentions the ongoing work toward WCAG 2.1 AA conformance, matching the reference answer and fully supported by the cited excerpts.

Drafted answer

The documentation does not mention whether Vantrel Plan is FedRAMP authorized.

Reference answer

Not in the documentation.

Grader · correctness 5 · faithfulness 5 · completeness 5

The reference answer states that the information is not in the documentation. The assistant correctly identifies that the documentation does not mention FedRAMP authorization for Vantrel Plan, adhering to the expected outcome.

Drafted answer

The documentation does not address whether Vantrel will sign a HIPAA Business Associate Agreement (BAA).

Reference answer

Not in the documentation.

Grader · correctness 5 · faithfulness 5 · completeness 5

The reference states that the information is not in the documentation, and the assistant correctly identified that the documentation does not address the question. All criteria are fully met.

Drafted answer

The coverage amount of our cyber insurance policy is not in the documentation.

Reference answer

Not in the documentation.

Grader · correctness 5 · faithfulness 5 · completeness 5

The reference answer states that the information is not in the documentation, and the assistant correctly and appropriately answers that the information is not documented. Therefore, all criteria are fully met.

Drafted answer

The documentation does not mention whether Vantrel offers a source code escrow agreement.

Reference answer

Not in the documentation.

Grader · correctness 5 · faithfulness 5 · completeness 5

The reference answer states that the information is not in the documentation. The assistant correctly identifies that the documentation does not mention source code escrow agreements, yielding top scores across all criteria.

Drafted answer

The documentation does not mention IP allowlisting for restricting user access.

Reference answer

Not in the documentation.

Grader · correctness 5 · faithfulness 5 · completeness 5

The reference answer states that the information is not in the documentation, and the assistant correctly identifies this by stating the documentation does not mention IP allowlisting. Therefore, all criteria receive the maximum score.

Drafted answer

The documentation does not specify whether Vantrel uses post-quantum cryptography.

Reference answer

Not in the documentation.

Grader · correctness 5 · faithfulness 5 · completeness 5

The reference answer states that the information is not in the documentation. The assistant correctly identifies that the documentation does not specify this information, and since it correctly notes the lack of documentation, it receives top scores across all criteria.

Drafted answer

The documentation does not mention a SOC 1 report.

Reference answer

Not in the documentation. Only Vantrel's SOC 2 Type II is documented; the SOC 1 mentioned is AWS's, not Vantrel's.

Grader · correctness 5 · faithfulness 5 · completeness 5

The reference answer states that the information is not documented, and the assistant correctly identifies that the documentation does not mention a SOC 1 report. Therefore, all criteria receive the maximum score.

Drafted answer

The documentation does not specify the number of people working in the security team.

Reference answer

Not in the documentation. Only the total headcount (~240) and the CISO/Security Committee are mentioned.

Grader · correctness 5 · faithfulness 5 · completeness 5

The reference answer states that the information is not in the documentation, and the assistant correctly identifies that the documentation does not specify the number of people in the security team. Since the information is absent and both the reference and assistant agree on this point, all criteria are scored the maximum.

Drafted answer

The documentation does not specify mean time to resolve P1 incidents.

Reference answer

Not in the documentation. Only first-response targets are documented, explicitly not resolution times.

Grader · correctness 5 · faithfulness 5 · completeness 5

The reference answer states that the information is not documented, and the assistant correctly identifies that the documentation does not specify the mean time to resolve P1 incidents. Since no excerpts were cited because the information was absent, all criteria are fully met.

Drafted answer

The documentation does not mention CSA STAR certification.

Reference answer

Not in the documentation. The documented certifications are ISO 27001 and SOC 2 Type II.

Grader · correctness 5 · faithfulness 5 · completeness 5

The reference answer states that CSA STAR is not in the documentation, and the assistant correctly identified that the documentation does not mention it. Since the information is correctly noted as absent without any hallucinations, all criteria receive the maximum score.

Drafted answer

The documentation does not mention an emergency phone number for security incidents.

Reference answer

Not in the documentation. Only email (security@vantrel.example) and the status page are documented.

Grader · correctness 5 · faithfulness 5 · completeness 5

The reference answer states that the information is not in the documentation, and the assistant correctly identified that the documentation does not mention an emergency phone number. Since the information is missing and the assistant accurately reported this without hallucinations, all criteria receive the maximum score.

Drafted answer

After contract termination, customers have 30 days to export their data. According to the Security Whitepaper (v2026.1), data is permanently deleted from production systems within 90 days of contract termination. However, the Data Processing Agreement (v2025.3) states that deletion occurs within 30 days after the end of the 30-day grace period, meaning a total of 60 days from contract termination. These two sources conflict on the deletion timeline.

• ↳ Security Whitepaper › Data retention and deletion
• ↳ Data Processing Agreement (summary) › Return and deletion of data

Reference answer

The documentation is inconsistent: the Security Whitepaper says within 90 days of termination, while the DPA says within 30 days after a 30-day export grace period (i.e. ~60 days). Both sources should be cited and the inconsistency flagged.

Expected: Security whitepaper › Data retention and deletion · Data processing agreement › Return and deletion of data

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant correctly identified and flagged the inconsistency between the Security Whitepaper (90 days) and the DPA (30 days after a 30-day grace period, totaling 60 days), matching the reference answer. Every statement made by the assistant is directly supported by the cited excerpts. All key points required by the reference are fully covered.

Drafted answer

Les données des clients hébergés dans la région EU sont stockées dans la région AWS eu-west-3 (Paris, France) pour le site principal, mais les copies de sauvegarde et de reprise après sinistre sont situées dans AWS eu-central-1 (Francfort, Allemagne). Aucune autre région en France n'est actuellement proposée pour l'hébergement des données. La région US n'est pas hébergée en France.

• ↳ Infrastructure and Hosting › Hosting regions

Reference answer

Oui pour la région EU (par défaut) : AWS eu-west-3 à Paris, avec reprise d'activité à Francfort (Allemagne). Une région US existe aussi.

Expected: Infrastructure hosting › Hosting regions

Grader · correctness 3 · faithfulness 3 · completeness 5

The assistant correctly states that data for the EU region is hosted in Paris with backups in Frankfurt, perfectly matching the reference answer. However, the assistant's assertion that backups are located in Frankfurt contradicts the excerpt which explicitly states that 'Customer data at rest stays within the selected region, including backups and disaster recovery copies.' Therefore, the answer is faithful to the table in the excerpt but contradicts the accompanying text, leading to a deduction in correctness and faithfulness.

Drafted answer

Oui, Vantrel propose l’authentification unique (SSO) avec Microsoft Entra ID, mais uniquement sur les plans Business et Enterprise.

• ↳ Identity and Access Management › Customer authentication

Reference answer

Oui, SSO SAML 2.0 et OIDC sur les plans Business et Enterprise ; Microsoft Entra ID fait partie des fournisseurs testés.

Expected: Identity access › Customer authentication

Grader · correctness 5 · faithfulness 5 · completeness 5

The assistant's answer correctly states that Microsoft Entra ID SSO is supported on Business and Enterprise plans, matching the reference answer. Every statement is directly supported by the provided documentation excerpt. All key points from the reference are fully covered.

Drafted answer

The documentation does not mention whether Vantrel is qualified SecNumCloud by the ANSSI.

Reference answer

Pas dans la documentation.

Grader · correctness 5 · faithfulness 5 · completeness 5

The reference answer states that the information is not in the documentation, and the assistant correctly identified this and stated that the documentation does not mention SecNumCloud qualification. Since no information was available, faithfulness and completeness are also fully met by simply stating it is not documented.