Entity Service

This documentation covers the design, implementation, and best practices for handling Entities and Entity Permissions within the Vyndara Entity Service. This guide is intended for developers who need to integrate, troubleshoot, or extend the entity and permission models.

Entities Overview

What is an Entity?

In Vyndara, an Entity represents a dynamic data object whose structure is defined by an Entity Definition. Entities allow you to store flexible, schema-driven data within a relational database (using PostgreSQL's jsonb for dynamic fields) while still benefiting from SQL's reliability.

Key Characteristics:

• Dynamic Schema: Entities are not restricted to a fixed schema and can evolve over time.
• Storage: Each entity is stored with a reference to its definition and contains all relevant data in a flexible JSON structure.

Entity Structure

An entity typically includes the following properties:

• id: Unique identifier for the entity.
• entity_definition_id: A reference to the corresponding Entity Definition that outlines the schema.
• data: A JSON object containing the actual data fields.
• created_at: Create date
• updated_at: Update date

Sample Entity JSON

{
  "id": "019528e0-60b1-76b1-85a9-85e4046db72a",
  "entity_definition_id": "219545e0-34b1-86b1-85a9-85t6046db72b",
  "data": {
    "name": "Jane Doe",
    "email": "jane@example.com",
    "status": "active"
  },
  "created_at": "2025-02-28T12:00:00Z",
  "updated_at": "2025-02-28T12:00:00Z"
}

Entity Definition

The EntityDefinition is the validation layer for the json data field of an entity. When data is uploaded it will be checked against the Entity Definition for the entity endpoint where data should be saved on and blocks requests with wrong request bodies.

Also the Entity Definition is used with the permissions to be aware of the fields that exist on a entity.

Entity Definition Structure

The following diagram shows how Entity Definitions work

Core Components

1.EntityDefinition

Describes a custom data model or schema that can be versioned over time. It references one or more EntityDefinitionFieldAssignment records to define which fields are included.

2.EntityDefinitionField

A reusable definition of a single data field (for example, email, phone, address). Each field is linked to a TypeDefinition for advanced validation rules (e.g., regex patterns).

3.EntityDefinitionFieldAssignment

A versioned entity that serves as the many-to-many join between EntityDefinition and EntityDefinitionField. It can store additional properties such as IsRequired. By making this a real entity (instead of a raw join table), changes can be tracked and audited.

4.TypeDefinition

Encapsulates the underlying data type or validation logic for a field. By using a RegexPattern or other constraints, you can create robust validation rules that multiple fields or definitions can share.

Data Model Explanation

• EntityDefinition can have multiple EntityDefinitionFieldAssignment records. Each assignment points to an EntityDefinitionField.
• EntityDefinitionField references a TypeDefinition, which contains validation details.
• EntityDefinitionFieldAssignment is fully versioned, so you can track each assignment's lifecycle. It also allows for flags like IsRequired.
• This structure makes it easy to:
• Reuse fields across multiple definitions.
• Enforce consistent validation via TypeDefinition.
• Maintain audit trails when fields are added or removed from a definition.

Entity Permissions

The system should implement a fine-granted Entity Permission logic. This logic should restrict Read, Write, Update, Delete, Execute permissions. The execute permission in this context is required for pipelines.

Data

An EntityFieldPermission is always binded to a EntityDefinition and a EntityDefinitionField. With this logic its possible to have different permission levels for different Entity Definitions but the same field. Each EntityFieldPermission is binded to a Role, these roles only sync with the roles of the Identity Provider and don't need to be applied to the users directly.

The field permission are int values, so each permission level, requires the levels below

Name	Value
Read	0
Write	1
Update	2
Delete	3
Execute	4

Deletion

Entity field and entity-level deletion require robust handling to ensure compliance (e.g., GDPR), tenant-level configurability, data integrity, and complete audit trails. This section now covers:

• Soft Deletion (field/entity) with data retention for audits/restore
• Hard Deletion (field/entity) compliant with GDPR (irreversible erase)
• Audit Logging (always-on, multi-tenant aware)
• Configurable Deletion Policies per Tenant
• History, Deprecation, and Data Access Flows

---

1. Deletion Strategy Matrix

Level	Soft Delete	Hard Delete
Entity	Set deleted_at TIMESTAMP,	Physical removal (only per-tenant
	remain in queries (filtered out),	hard-delete policy or explicit
	recoverable. Audit logged.	GDPR/PII erase). Audit logged.
Field	Mark as deleted: true,	Remove field from DB schema/JOSNB
	deleted_at, deleted_by}` in schema	+ audit, and from all versions.
	or JSONB. Excluded in API.	Non-recoverable.

---

2. PostgreSQL & Data Architecture

Entity Table Changes

ALTER TABLE entity ADD COLUMN deleted_at TIMESTAMP NULL;

Field Soft Deletion in JSONB/Schema

Store deletion metadata for fields (if flexible schema, e.g., JSONB):

{
  "fields": {
    "email": {
      "type": "string", "deleted":true, "deleted_at": "2024-06-21T08:41:00Z", "deleted_by": "admin:john"
    }
  }
}

Audit Log Table (per event, per tenant)

CREATE TABLE audit_log (
  tenant_id TEXT,
  entity_id UUID,
  event_type TEXT,  -- e.g. field.soft_delete, entity.hard_delete
  target_level TEXT,  -- entity, field
  target_key TEXT,  -- entity_id or field name
  performed_by TEXT,  -- user or service
  old_value JSONB,
  new_value JSONB,
  occurred_at TIMESTAMP DEFAULT now()
);

Tenant Policy Table Example

CREATE TABLE tenant_deletion_policy (
  tenant_id TEXT PRIMARY KEY,
  soft_delete_enabled BOOLEAN DEFAULT TRUE,
  hard_delete_enabled BOOLEAN DEFAULT FALSE,
  gdpr_enforced BOOLEAN DEFAULT TRUE
);

---

3. Deletion Workflows

3.1 Entity Deletion

3.2 Field Deletion (Deprecation/Removal)

---

4. API Patterns (Go-Service, REST)

• DELETE /entities/{id}
  • By default, soft delete (sets deleted_at), unless tenant or GDPR policy triggers hard delete.
  • Always logs audit.
• DELETE /entities/{id}/fields/{field}
  • Soft: Field deprecation via deletion metadata; excluded from reads and writes.
  • Hard: Deletes from storage/schema when enforced.
• Reading:
  • Filters out soft-deleted entities/fields as default, unless forced for migration/audits.
• Every change logs full old/new in audit tables, scoped per tenant.

---

5. Multi-Tenancy & Compliance

• Every deletion, audit, and history row includes tenant_id.
• Each operation cross-checks tenant’s deletion policy before finalizing.
• "Right to be forgotten" is only hard-delete and triggers all direct and indirect data erase.
• All audit logs are never deleted unless a GDPR-driven erase is triggered.
• Field-level visibility can be deprecated (soft) instead of immediate erase for compatibility.

---

6. Example: Field Deletion Event (Audit)

{
  "tenant_id": "acme-corp",
  "entity_id": "user-123",
  "field": "nickname",
  "event": "field.soft_delete",
  "actor": "admin:john",
  "timestamp": "2024-06-21T08:41:00Z",
  "prior": { "deleted": false, "value": "Johnny" },
  "new":   { "deleted": true, "deleted_at": "2024-06-21T08:41:00Z" }
}

---

7. Best Practices and Notes

• Default to soft deletion whenever possible for recovery/compliance.
• Tenant config is SYSTEM critical—always check before executing.
• Never leave data visible between "soft delete" and actual API removal.
• Always test all flows (soft/hard, rollback, migration) per policy/tenant scenario.

---