Temporal Data
Lumenize automatically maintains a complete history of all data changes using valid time semantics, enabling powerful time-travel queries and audit capabilities.
Understanding Temporal Data
Valid Time Semantics
Every entity in Lumenize has temporal properties:
// Every record has temporal metadataconst user = await client.users.findById('user_123');console.log(user._temporal);// {// validFrom: '2024-01-01T10:00:00.000Z',// validTo: '9999-01-01T00:00:00.000Z',// version: 3,// createdAt: '2024-01-01T10:00:00.000Z',// updatedAt: '2024-01-15T14:30:00.000Z'// }
Snapshots
Each change creates a new immutable snapshot:
// Initial creationconst user = await client.users.create({ name: 'John Doe', email: 'john@example.com'});
// Update creates new snapshotawait client.users.update('user_123', { name: 'John Smith'});
// Previous snapshot is preserved with validTo timestamp
Time Travel Queries
Query at Specific Time
// Get user state as of January 1st, 2024const userAtTime = await client.users.findById('user_123', { asOf: new Date('2024-01-01')});
// Get all users as they were at a specific timeconst usersAtTime = await client.users.findMany({ where: { active: true }, asOf: new Date('2024-01-01T12:00:00Z')});
Query Between Time Ranges
// Get all versions of a user between datesconst userHistory = await client.users.findVersions('user_123', { from: new Date('2024-01-01'), to: new Date('2024-01-31')});
// Find users who were active during Q1 2024const activeUsersQ1 = await client.users.findMany({ where: { status: 'active' }, during: { from: new Date('2024-01-01'), to: new Date('2024-03-31') }});
Current vs Historical Data
// Current data (default)const currentUser = await client.users.findById('user_123');
// All historical versionsconst allVersions = await client.users.findVersions('user_123');
// Specific versionconst version2 = await client.users.findVersion('user_123', { version: 2 });
Change Tracking
Automatic Change Detection
// Lumenize automatically tracks what changedconst update = await client.users.update('user_123', { name: 'Jane Doe', email: 'jane@example.com'});
console.log(update._changes);// {// name: { from: 'John Smith', to: 'Jane Doe' },// email: { from: 'john@example.com', to: 'jane@example.com' }// }
Change Events
// Subscribe to change eventsclient.users.subscribe('user_123', (event) => { if (event.type === 'updated') { console.log('Changes:', event.changes); console.log('Previous version:', event.previousVersion); console.log('New version:', event.data); }});
Audit Queries
// Find all changes to a userconst changes = await client.audit.findChanges({ entityType: 'User', entityId: 'user_123', from: new Date('2024-01-01')});
// Find who made changesconst auditLog = await client.audit.findMany({ entityType: 'User', entityId: 'user_123', include: ['user', 'changes']});
// Find all changes by a specific userconst userChanges = await client.audit.findMany({ changedBy: 'admin_456', from: new Date('2024-01-01')});
Temporal Relationships
Relationship History
// Track when relationships changeconst projectHistory = await client.projects.findVersions('proj_123', { include: { members: { temporal: true // Include historical membership } }});
// Find who was on the team at a specific timeconst teamAtTime = await client.projects.findById('proj_123', { asOf: new Date('2024-06-01'), include: { members: true }});
Effective Dating
// Model entities with effective datesconst salary = await client.salaries.create({ employeeId: 'emp_123', amount: 75000, effectiveFrom: new Date('2024-01-01'), effectiveTo: new Date('2024-12-31')});
// Query effective salariesconst currentSalary = await client.salaries.findEffective('emp_123', { asOf: new Date() // Current effective salary});
const salaryAt = await client.salaries.findEffective('emp_123', { asOf: new Date('2024-06-01') // Salary effective on June 1st});
Temporal Analytics
Change Frequency Analysis
// Analyze how frequently entities changeconst changeStats = await client.analytics.changeFrequency({ entityType: 'Project', from: new Date('2024-01-01'), to: new Date('2024-12-31'), groupBy: 'month'});
console.log(changeStats);// [// { month: '2024-01', changes: 45 },// { month: '2024-02', changes: 62 },// ...// ]
Data Lifecycle Analysis
// Track entity lifecycleconst lifecycle = await client.analytics.lifecycle({ entityType: 'Task', from: new Date('2024-01-01'), metrics: ['created', 'updated', 'completed', 'deleted']});
// Analyze data retentionconst retention = await client.analytics.retention({ entityType: 'User', cohortBy: 'month', retentionPeriod: 'week'});
Temporal Aggregations
// Revenue over timeconst revenue = await client.analytics.timeSeries({ entity: 'Order', metric: 'total', aggregation: 'sum', groupBy: 'day', from: new Date('2024-01-01'), to: new Date('2024-01-31')});
// Active users at different points in timeconst activeUsers = await client.analytics.temporalSnapshot({ entity: 'User', field: 'lastLoginAt', threshold: '30 days', snapshots: [ new Date('2024-01-01'), new Date('2024-01-15'), new Date('2024-01-31') ]});
Restoration and Recovery
Soft Deletes
// Soft delete preserves historyawait client.users.delete('user_123', { soft: true });
// Query includes deleted by default in temporal queriesconst userHistory = await client.users.findVersions('user_123');
// Restore from soft deleteawait client.users.restore('user_123');
Point-in-Time Recovery
// Restore entity to previous stateawait client.users.restoreTo('user_123', { asOf: new Date('2024-01-15T10:00:00Z')});
// Restore specific versionawait client.users.restoreVersion('user_123', { version: 2 });
// Bulk restoreawait client.users.restoreMany({ where: { deletedAt: { gte: new Date('2024-01-01') } }, asOf: new Date('2024-01-01')});
Data Archival
// Configure automatic archivalawait client.config.setArchivalPolicy({ entityType: 'AuditLog', archiveAfter: '2 years', storage: 'cold', compression: true});
// Manual archivalawait client.archive.entities({ entityType: 'Order', where: { createdAt: { lt: new Date('2023-01-01') }, status: 'completed' }});
Performance Considerations
Temporal Indexes
{ "name": "User", "fields": { "name": {"type": "string"}, "email": {"type": "email"} }, "temporalIndexes": [ "email", ["status", "_temporal.validFrom"], { "fields": ["department", "_temporal.validFrom", "_temporal.validTo"], "name": "department_timeline" } ]}
Efficient Queries
// Use specific time ranges for better performanceconst recentChanges = await client.users.findVersions('user_123', { from: new Date('2024-01-01'), limit: 10});
// Index on temporal fields for faster queriesconst activeAtTime = await client.users.findMany({ where: { status: 'active', '_temporal.validFrom': { lte: specificDate }, '_temporal.validTo': { gt: specificDate } }});
Data Retention Policies
// Configure retention policiesawait client.config.setRetentionPolicy({ entityType: 'AuditLog', retainFor: '7 years', summarizeAfter: '1 year', purgeAfter: '10 years'});
// Custom retention rulesawait client.config.setRetentionPolicy({ entityType: 'UserActivity', rules: [ { condition: { eventType: 'login' }, retainFor: '2 years' }, { condition: { eventType: 'page_view' }, retainFor: '90 days' } ]});
Integration with JavaScript Temporal API
Using Temporal Types
import { Temporal } from '@js-temporal/polyfill';
// Create with Temporal.Instantconst user = await client.users.create({ name: 'John Doe', hiredAt: Temporal.Now.instant()});
// Query with Temporal.Durationconst recentUsers = await client.users.findMany({ where: { createdAt: { gte: Temporal.Now.instant().subtract(Temporal.Duration.from('P30D')) } }});
// Time zone aware queriesconst localTime = Temporal.Now.zonedDateTimeISO('America/New_York');const businessHoursData = await client.analytics.findMany({ where: { createdAt: { gte: localTime.with({ hour: 9, minute: 0 }), lt: localTime.with({ hour: 17, minute: 0 }) } }});
Duration Calculations
// Calculate how long something existedconst project = await client.projects.findById('proj_123');const duration = project._temporal.validTo === '9999-01-01T00:00:00.000Z' ? Temporal.Duration.between( Temporal.Instant.from(project._temporal.validFrom), Temporal.Now.instant() ) : Temporal.Duration.between( Temporal.Instant.from(project._temporal.validFrom), Temporal.Instant.from(project._temporal.validTo) );
console.log(`Project existed for: ${duration.total('days')} days`);
Best Practices
Query Optimization
// ✅ Good: Specific time rangesconst changes = await client.audit.findChanges({ entityId: 'user_123', from: new Date('2024-01-01'), to: new Date('2024-01-31')});
// ❌ Avoid: Open-ended temporal queriesconst allChanges = await client.audit.findChanges({ entityId: 'user_123' // No time bounds - could be expensive});
Data Modeling
// ✅ Good: Use effective dating for planned changesconst salaryIncrease = await client.salaries.create({ employeeId: 'emp_123', amount: 80000, effectiveFrom: new Date('2024-07-01'), // Future date reason: 'Annual review increase'});
// ✅ Good: Track business-relevant temporal eventsconst projectMilestone = await client.milestones.create({ projectId: 'proj_123', name: 'Beta Release', targetDate: new Date('2024-06-01'), actualDate: new Date('2024-06-03')});
Compliance and Auditing
// Configure for compliance requirementsawait client.config.setComplianceSettings({ retentionPeriod: '7 years', // SOX compliance immutableAuditLog: true, // Prevent tampering encryptHistorical: true, // Encrypt old data auditLogIntegrity: 'blockchain' // Blockchain verification});
Troubleshooting
Performance Issues
// Check temporal query performanceconst performance = await client.debug.analyzeQuery({ entity: 'User', query: { asOf: new Date('2024-01-01'), where: { department: 'engineering' } }});
console.log('Query execution plan:', performance.plan);console.log('Index usage:', performance.indexes);
Data Consistency
// Verify temporal data integrityconst integrity = await client.debug.checkTemporalIntegrity({ entityType: 'User', entityId: 'user_123'});
if (integrity.errors.length > 0) { console.error('Temporal integrity issues:', integrity.errors);}
Next Steps
- Analytics - Build reports with temporal data
- Real-time Features - Live temporal updates
- Examples - See temporal features in action
- API Reference - Complete temporal API documentation