NeahStable/CALENDAR_SYNC_ARCHITECTURE_ANALYSIS.md
2026-01-15 12:11:26 +01:00

729 lines
23 KiB
Markdown

# Calendar Synchronization Architecture - Deep Analysis
## Executive Summary
This document provides a comprehensive architectural analysis of the calendar synchronization system, focusing on:
- **Agenda Widget** (Dashboard widget)
- **Agenda Page** (Full calendar view)
- **Courrier Page** (Email integration)
- **Calendar Synchronization Services** (Infomaniak CalDAV & Microsoft Graph API)
---
## 1. System Architecture Overview
### 1.1 Component Hierarchy
```
┌─────────────────────────────────────────────────────────────┐
│ Dashboard (app/page.tsx) │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Calendar │ │ News │ │ Email │ │
│ │ Widget │ │ Widget │ │ Widget │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Agenda Page (app/agenda/page.tsx) │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ CalendarClient Component │ │
│ │ ┌──────────────┐ ┌─────────────────────────────┐ │ │
│ │ │ Calendar │ │ FullCalendar (FullCalendar)│ │ │
│ │ │ Selector │ │ - Month/Week/Day Views │ │ │
│ │ │ (Sidebar) │ │ - Event Creation/Edit │ │ │
│ │ └──────────────┘ └─────────────────────────────┘ │ │
│ └──────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Courrier Page (app/courrier/page.tsx) │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Email │ │ Email │ │ Email │ │
│ │ Sidebar │ │ List │ │ Detail │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Calendar Sync Services │
│ ┌──────────────────────┐ ┌──────────────────────┐ │
│ │ Infomaniak CalDAV │ │ Microsoft Graph API │ │
│ │ Sync Service │ │ Sync Service │ │
│ └──────────────────────┘ └──────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
```
---
## 2. Agenda Widget Analysis
### 2.1 Component Location
- **File**: `components/calendar.tsx`
- **Usage**: Dashboard widget showing upcoming 7 events
- **Type**: Client Component (`"use client"`)
### 2.2 Key Features
#### Data Fetching
```typescript
// Fetches from /api/calendars?refresh=true
// Bypasses cache with refresh=true parameter
const response = await fetch('/api/calendars?refresh=true');
```
#### Event Processing
1. **Extracts events** from all calendars
2. **Filters** for upcoming events (from today onwards)
3. **Sorts** by start date
4. **Limits** to 7 events
5. **Maps** calendar color and name to each event
#### Refresh Mechanism
- **Manual refresh** only via button click
- **No automatic polling** or unified refresh integration
- **Fetches on mount** only
### 2.3 Issues Identified
1. **❌ No Unified Refresh Integration**
- Not using `useUnifiedRefresh` hook
- Manual refresh only
- No automatic background updates
2. **❌ Cache Bypass**
- Always uses `?refresh=true` parameter
- Bypasses Redis cache every time
- May cause unnecessary database load
3. **❌ No Error Recovery**
- Basic error handling
- No retry mechanism
- No offline state handling
4. **⚠️ Date Filtering Logic**
- Filters events from "start of day" (00:00:00)
- May miss events happening later today if fetched early morning
### 2.4 Data Flow
```
User Dashboard
├─> Calendar Widget (components/calendar.tsx)
│ │
│ ├─> useEffect() triggers on mount
│ │
│ ├─> fetch('/api/calendars?refresh=true')
│ │ │
│ │ ├─> API Route (app/api/calendars/route.ts)
│ │ │ │
│ │ │ ├─> Checks Redis cache (skipped if refresh=true)
│ │ │ │
│ │ │ ├─> Prisma: Calendar.findMany()
│ │ │ │ └─> Include: events (ordered by start)
│ │ │ │
│ │ │ └─> Cache result in Redis
│ │ │
│ │ └─> Returns: Calendar[] with events[]
│ │
│ ├─> Process events:
│ │ ├─> flatMap all calendars.events
│ │ ├─> Filter: start >= today (00:00:00)
│ │ ├─> Sort by start date
│ │ └─> Slice(0, 7)
│ │
│ └─> Render: Card with event list
```
---
## 3. Agenda Page Analysis
### 3.1 Component Location
- **File**: `app/agenda/page.tsx` (Server Component)
- **Client Component**: `components/calendar/calendar-client.tsx`
- **Route**: `/agenda`
### 3.2 Server-Side Logic (page.tsx)
#### Auto-Setup Calendar Sync
The page automatically sets up calendar synchronization for email accounts:
1. **Infomaniak Accounts** (CalDAV)
- Discovers calendars using `discoverInfomaniakCalendars()`
- Creates "Privée" calendar if not exists
- Creates `CalendarSync` record with:
- `provider: 'infomaniak'`
- `syncFrequency: 15` minutes
- `externalCalendarUrl` from discovery
- Triggers initial sync
2. **Microsoft Accounts** (OAuth)
- Discovers calendars using `discoverMicrosoftCalendars()`
- Creates "Privée" calendar if not exists
- Creates `CalendarSync` record with:
- `provider: 'microsoft'`
- `syncFrequency: 5` minutes (more reactive)
- `externalCalendarId` from Graph API
- Triggers initial sync
#### Calendar Filtering Logic
```typescript
// Excludes "Privée" and "Default" calendars that are NOT synced
calendars = calendars.filter(cal => {
const isPrivateOrDefault = cal.name === "Privée" || cal.name === "Default";
const hasActiveSync = cal.syncConfig?.syncEnabled === true &&
cal.syncConfig?.mailCredential;
// Exclude "Privée"/"Default" calendars that are not actively synced
if (isPrivateOrDefault && !hasActiveSync) {
return false;
}
return true;
});
```
#### Background Sync Trigger
For Microsoft calendars, the page triggers background sync if needed:
- Checks if sync is needed (2 minutes minimum interval)
- Triggers async sync (doesn't block page load)
- Sync runs in background, updates database
- Next page load will show synced events
### 3.3 Client-Side Component (calendar-client.tsx)
#### Key Features
1. **FullCalendar Integration**
- Uses `@fullcalendar/react` library
- Views: Month, Week, Day
- Plugins: dayGrid, timeGrid, interaction
2. **Calendar Management**
- Calendar selector sidebar (left column)
- Visibility toggle per calendar
- Calendar creation/editing dialog
- Sync configuration UI
3. **Event Management**
- Create events by clicking/selecting dates
- Edit events by clicking on them
- Delete events
- Event form with:
- Title, description, location
- Start/end date and time
- All-day toggle
- Calendar selection
4. **Data Refresh**
- `fetchCalendars()` function
- Calls `/api/calendars` (no refresh parameter)
- Uses Redis cache by default
- Updates FullCalendar via `calendarApi.refetchEvents()`
#### Calendar Display Logic
```typescript
// Sorts calendars: synced first, then groups, then missions
const sortCalendars = (cals) => {
return [...filtered].sort((a, b) => {
const aIsSynced = a.syncConfig?.syncEnabled && a.syncConfig?.mailCredential;
const bIsSynced = b.syncConfig?.syncEnabled && b.syncConfig?.mailCredential;
// Synced calendars first
if (aIsSynced && !bIsSynced) return -1;
if (!aIsSynced && bIsSynced) return 1;
// Then groups, then missions, then by name
// ...
});
};
```
#### Event Rendering
Events are mapped to FullCalendar format:
```typescript
events={calendars.flatMap(cal =>
cal.events.map(event => ({
id: event.id,
title: event.title,
start: new Date(event.start),
end: new Date(event.end),
allDay: event.isAllDay,
backgroundColor: `${cal.color}dd`,
borderColor: cal.color,
extendedProps: {
calendarName: cal.name,
location: event.location,
description: cleanDescription(event.description),
calendarId: event.calendarId,
originalEvent: event,
color: cal.color
}
}))
)}
```
### 3.4 Issues Identified
1. **⚠️ Sync Timing**
- Microsoft sync triggers on page load (background)
- May cause delay before events appear
- No loading indicator for background sync
2. **⚠️ Calendar Filtering Complexity**
- Complex logic for "Privée"/"Default" calendars
- May hide calendars that should be visible
- Logging is extensive but may be confusing
3. **❌ No Real-Time Updates**
- Events only update on manual refresh
- No WebSocket or polling for new events
- User must refresh to see synced events
4. **⚠️ Event Matching Logic**
- Infomaniak: Matches by title + start date (within 1 minute)
- Microsoft: Matches by `[MS_ID:xxx]` in description
- May create duplicates if matching fails
---
## 4. Courrier Page Analysis
### 4.1 Component Location
- **File**: `app/courrier/page.tsx`
- **Route**: `/courrier`
- **Type**: Client Component
### 4.2 Key Features
#### Email Account Management
- Multiple email account support
- Account colors for visual distinction
- Account settings (display name, password, color)
#### Email Operations
- Folder navigation (INBOX, Sent, Drafts, etc.)
- Email reading, composing, replying
- Email search
- Bulk operations (delete, mark read/unread)
#### Integration with Calendar Sync
The Courrier page is **indirectly** related to calendar sync:
1. **Email Accounts** are stored in `MailCredentials` table
2. **Calendar Sync** uses `MailCredentials` for authentication
3. **Auto-Setup** in Agenda page discovers accounts from Courrier
### 4.3 Connection to Calendar Sync
```typescript
// In agenda/page.tsx:
const infomaniakAccounts = await prisma.mailCredentials.findMany({
where: {
userId: session?.user?.id || '',
host: { contains: 'infomaniak' },
password: { not: null }
}
});
// For each account, create calendar sync
for (const account of infomaniakAccounts) {
// Discover calendars
const calendars = await discoverInfomaniakCalendars(account.email, account.password!);
// Create calendar and sync config
// ...
}
```
### 4.4 Issues Identified
1. **⚠️ No Direct Calendar Integration**
- Courrier page doesn't show calendar events
- No email-to-calendar event creation
- Separate systems (email vs calendar)
2. **⚠️ Account Deletion Impact**
- Deleting email account may orphan calendar sync
- Agenda page has cleanup logic, but may miss edge cases
---
## 5. Calendar Synchronization Services
### 5.1 Infomaniak CalDAV Sync
#### Service Location
- **File**: `lib/services/caldav-sync.ts`
#### Key Functions
1. **`discoverInfomaniakCalendars(email, password)`**
- Uses WebDAV client (`webdav` library)
- Connects to `https://sync.infomaniak.com`
- Lists directories using `PROPFIND`
- Extracts calendar name and color from XML
2. **`fetchCalDAVEvents(email, password, calendarUrl, startDate, endDate)`**
- Uses `REPORT` method with `calendar-query`
- Filters events by date range
- Parses iCalendar format (`.ics`)
- Returns `CalDAVEvent[]`
3. **`syncInfomaniakCalendar(calendarSyncId, forceSync)`**
- Fetches events from CalDAV (1 month ago to 3 months ahead)
- Gets existing events from database
- Matches events by title + start date (within 1 minute)
- Creates new events or updates existing
- Updates `lastSyncAt` timestamp
#### Event Matching Logic
```typescript
// Tries to find existing event by:
const existingEvent = existingEvents.find(
(e) =>
e.title === caldavEvent.summary &&
Math.abs(new Date(e.start).getTime() - caldavEvent.start.getTime()) < 60000
);
```
**Issue**: This matching is fragile:
- May create duplicates if title changes
- May miss events if timezone conversion causes >1 minute difference
- No UID-based matching (iCalendar has UID field)
### 5.2 Microsoft Graph API Sync
#### Service Location
- **File**: `lib/services/microsoft-calendar-sync.ts`
#### Key Functions
1. **`discoverMicrosoftCalendars(userId, email)`**
- Uses Microsoft Graph API
- Endpoint: `https://graph.microsoft.com/v1.0/me/calendars`
- Requires OAuth token with calendar scope
- Returns `MicrosoftCalendar[]`
2. **`fetchMicrosoftEvents(userId, email, calendarId, startDate, endDate)`**
- Endpoint: `https://graph.microsoft.com/v1.0/me/calendars/{calendarId}/events`
- Filters by date range using `$filter` parameter
- Returns `MicrosoftEvent[]`
3. **`syncMicrosoftCalendar(calendarSyncId, forceSync)`**
- Fetches events (1 month ago to 6 months ahead)
- Converts Microsoft events to CalDAV-like format
- **Stores Microsoft ID in description**: `[MS_ID:xxx]`
- Matches events by Microsoft ID first, then by title+date
- Creates/updates events in database
#### Event Matching Logic
```typescript
// First: Match by Microsoft ID in description
let existingEvent = existingEvents.find((e) => {
if (e.description && e.description.includes(`[MS_ID:${microsoftId}]`)) {
return true;
}
return false;
});
// Fallback: Match by title + start date
if (!existingEvent) {
existingEvent = existingEvents.find(
(e) =>
e.title === caldavEvent.summary &&
Math.abs(new Date(e.start).getTime() - caldavEvent.start.getTime()) < 60000
);
}
```
**Better than Infomaniak**: Uses ID-based matching, but stores ID in description (hacky)
### 5.3 Sync Job Service
#### Service Location
- **File**: `lib/services/calendar-sync-job.ts`
#### Function: `runCalendarSyncJob()`
- Gets all enabled sync configurations
- Checks if sync is needed (based on `syncFrequency`)
- Calls appropriate sync function based on provider
- Logs results (successful, failed, skipped)
**Usage**: Should be called by cron job or scheduled task
### 5.4 Issues Identified
1. **❌ No UID-Based Matching for Infomaniak**
- Should use iCalendar UID field
- Current matching is fragile
2. **⚠️ Microsoft ID Storage**
- Stores ID in description field (hacky)
- Should use dedicated `externalEventId` field in Event model
3. **❌ No Event Deletion**
- Sync only creates/updates events
- Doesn't delete events removed from external calendar
- May show stale events
4. **⚠️ Error Handling**
- Errors are logged but sync continues
- May leave calendar in inconsistent state
- No retry mechanism for transient failures
5. **⚠️ Sync Frequency**
- Infomaniak: 15 minutes (may be too slow)
- Microsoft: 5 minutes (better, but still not real-time)
- No user-configurable frequency
6. **❌ No Incremental Sync**
- Always fetches full date range
- May be slow for calendars with many events
- Should use `lastSyncAt` to fetch only changes
---
## 6. API Routes Analysis
### 6.1 `/api/calendars` (GET)
**File**: `app/api/calendars/route.ts`
#### Features
- Redis caching (unless `?refresh=true`)
- Returns calendars with events
- Events ordered by start date
#### Issues
- Cache TTL not specified (uses default)
- No cache invalidation on event creation/update
- `refresh=true` bypasses cache (used by widget)
### 6.2 `/api/calendars/sync` (POST, PUT, GET, DELETE)
**File**: `app/api/calendars/sync/route.ts`
#### Features
- **POST**: Create sync configuration
- **PUT**: Trigger manual sync
- **GET**: Get sync status
- **DELETE**: Remove sync configuration
#### Issues
- Manual sync triggers full sync (not incremental)
- No webhook support for real-time updates
---
## 7. Database Schema
### 7.1 Calendar Model
```prisma
model Calendar {
id String @id @default(uuid())
name String
color String
description String?
userId String
missionId String?
events Event[]
syncConfig CalendarSync?
// ...
}
```
### 7.2 CalendarSync Model
```prisma
model CalendarSync {
id String @id @default(uuid())
calendarId String @unique
mailCredentialId String?
provider String // "infomaniak" | "microsoft"
externalCalendarId String?
externalCalendarUrl String?
syncEnabled Boolean @default(true)
lastSyncAt DateTime?
syncFrequency Int @default(15) // minutes
lastSyncError String?
// ...
}
```
### 7.3 Event Model
```prisma
model Event {
id String @id @default(uuid())
title String
description String?
start DateTime
end DateTime
location String?
isAllDay Boolean @default(false)
calendarId String
userId String
// ...
}
```
**Missing Fields**:
- `externalEventId` (for reliable matching)
- `externalEventUrl` (for linking to external calendar)
- `lastSyncedAt` (for incremental sync)
---
## 8. Critical Issues Summary
### 8.1 High Priority
1. **Event Matching Fragility**
- Infomaniak: No UID-based matching
- Microsoft: ID stored in description (hacky)
- **Impact**: Duplicate events, missed updates
2. **No Event Deletion**
- Removed events stay in database
- **Impact**: Stale events shown to users
3. **No Real-Time Updates**
- Widget and page don't auto-refresh
- **Impact**: Users must manually refresh to see new events
### 8.2 Medium Priority
4. **Cache Invalidation**
- Events created/updated don't invalidate cache
- **Impact**: Stale data shown until cache expires
5. **Sync Frequency**
- Infomaniak: 15 minutes (too slow)
- **Impact**: Delayed event appearance
6. **No Incremental Sync**
- Always fetches full date range
- **Impact**: Slow sync, unnecessary API calls
### 8.3 Low Priority
7. **Widget Refresh Integration**
- Not using unified refresh system
- **Impact**: Inconsistent refresh behavior
8. **Error Recovery**
- No retry mechanism
- **Impact**: Sync failures require manual intervention
---
## 9. Recommendations
### 9.1 Immediate Fixes
1. **Add `externalEventId` field to Event model**
```prisma
model Event {
// ... existing fields
externalEventId String? // UID from iCalendar or Microsoft ID
externalEventUrl String? // Link to external calendar event
}
```
2. **Implement UID-based matching for Infomaniak**
- Use iCalendar UID field for matching
- Store in `externalEventId`
3. **Implement event deletion**
- Compare external events with database events
- Delete events not in external calendar
4. **Add cache invalidation**
- Invalidate Redis cache on event create/update/delete
- Invalidate on sync completion
### 9.2 Short-Term Improvements
5. **Implement incremental sync**
- Use `lastSyncAt` to fetch only changes
- Use CalDAV `sync-token` for Infomaniak
- Use Microsoft Graph delta queries
6. **Add real-time updates**
- WebSocket or Server-Sent Events
- Notify clients when sync completes
- Auto-refresh widget and page
7. **Improve error handling**
- Retry mechanism for transient failures
- Better error messages for users
- Sync status indicator in UI
### 9.3 Long-Term Enhancements
8. **Unified refresh system integration**
- Use `useUnifiedRefresh` hook in widget
- Consistent refresh behavior across app
9. **User-configurable sync frequency**
- Allow users to set sync interval
- Different frequencies per calendar
10. **Two-way sync**
- Sync local events to external calendars
- Handle conflicts (last-write-wins or user choice)
---
## 10. Testing Recommendations
### 10.1 Unit Tests
- Event matching logic (UID-based, title+date fallback)
- Date range filtering
- iCalendar parsing
- Microsoft event conversion
### 10.2 Integration Tests
- Full sync flow (discover → sync → verify)
- Event creation/update/deletion
- Cache invalidation
- Error recovery
### 10.3 E2E Tests
- Widget displays events correctly
- Page shows synced events
- Manual sync triggers correctly
- Calendar creation/editing
---
## 11. Conclusion
The calendar synchronization system is **functional but has several architectural issues** that impact reliability and user experience:
1. **Fragile event matching** may cause duplicates or missed updates
2. **No event deletion** leaves stale events in the database
3. **No real-time updates** requires manual refresh
4. **Cache invalidation** is missing, causing stale data
**Priority**: Focus on fixing event matching and deletion first, as these directly impact data integrity. Then implement cache invalidation and real-time updates for better UX.
---
**Document Version**: 1.0
**Last Updated**: 2025-01-XX
**Author**: Senior Software Architect Analysis