NeahStable/CALENDAR_SYNC_ARCHITECTURE_ANALYSIS.md

Calendar Synchronization Architecture - Deep Analysis

Executive Summary

This document provides a comprehensive architectural analysis of the calendar synchronization system, focusing on:

• Agenda Widget (Dashboard widget)
• Agenda Page (Full calendar view)
• Courrier Page (Email integration)
• Calendar Synchronization Services (Infomaniak CalDAV & Microsoft Graph API)

---

1. System Architecture Overview

1.1 Component Hierarchy

┌─────────────────────────────────────────────────────────────┐
│                    Dashboard (app/page.tsx)                  │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐      │
│  │ Calendar     │  │ News         │  │ Email        │      │
│  │ Widget       │  │ Widget       │  │ Widget       │      │
│  └──────────────┘  └──────────────┘  └──────────────┘      │
└─────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│              Agenda Page (app/agenda/page.tsx)              │
│  ┌──────────────────────────────────────────────────────┐   │
│  │         CalendarClient Component                     │   │
│  │  ┌──────────────┐  ┌─────────────────────────────┐  │   │
│  │  │ Calendar     │  │   FullCalendar (FullCalendar)│  │   │
│  │  │ Selector     │  │   - Month/Week/Day Views     │  │   │
│  │  │ (Sidebar)    │  │   - Event Creation/Edit     │  │   │
│  │  └──────────────┘  └─────────────────────────────┘  │   │
│  └──────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│         Courrier Page (app/courrier/page.tsx)               │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐      │
│  │ Email        │  │ Email        │  │ Email        │      │
│  │ Sidebar      │  │ List         │  │ Detail       │      │
│  └──────────────┘  └──────────────┘  └──────────────┘      │
└─────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│         Calendar Sync Services                              │
│  ┌──────────────────────┐  ┌──────────────────────┐       │
│  │ Infomaniak CalDAV    │  │ Microsoft Graph API   │       │
│  │ Sync Service         │  │ Sync Service          │       │
│  └──────────────────────┘  └──────────────────────┘       │
└─────────────────────────────────────────────────────────────┘

---

2. Agenda Widget Analysis

2.1 Component Location

• File: components/calendar.tsx
• Usage: Dashboard widget showing upcoming 7 events
• Type: Client Component ("use client")

2.2 Key Features

Data Fetching

// Fetches from /api/calendars?refresh=true
// Bypasses cache with refresh=true parameter
const response = await fetch('/api/calendars?refresh=true');

Event Processing

1. Extracts events from all calendars
2. Filters for upcoming events (from today onwards)
3. Sorts by start date
4. Limits to 7 events
5. Maps calendar color and name to each event

Refresh Mechanism

• Manual refresh only via button click
• No automatic polling or unified refresh integration
• Fetches on mount only

2.3 Issues Identified

1. ❌ No Unified Refresh Integration

   • Not using useUnifiedRefresh hook
   • Manual refresh only
   • No automatic background updates

2. ❌ Cache Bypass

   • Always uses ?refresh=true parameter
   • Bypasses Redis cache every time
   • May cause unnecessary database load

3. ❌ No Error Recovery

   • Basic error handling
   • No retry mechanism
   • No offline state handling

4. ⚠️ Date Filtering Logic

   • Filters events from "start of day" (00:00:00)
   • May miss events happening later today if fetched early morning

2.4 Data Flow

User Dashboard
    │
    ├─> Calendar Widget (components/calendar.tsx)
    │       │
    │       ├─> useEffect() triggers on mount
    │       │
    │       ├─> fetch('/api/calendars?refresh=true')
    │       │       │
    │       │       ├─> API Route (app/api/calendars/route.ts)
    │       │       │       │
    │       │       │       ├─> Checks Redis cache (skipped if refresh=true)
    │       │       │       │
    │       │       │       ├─> Prisma: Calendar.findMany()
    │       │       │       │       └─> Include: events (ordered by start)
    │       │       │       │
    │       │       │       └─> Cache result in Redis
    │       │       │
    │       │       └─> Returns: Calendar[] with events[]
    │       │
    │       ├─> Process events:
    │       │       ├─> flatMap all calendars.events
    │       │       ├─> Filter: start >= today (00:00:00)
    │       │       ├─> Sort by start date
    │       │       └─> Slice(0, 7)
    │       │
    │       └─> Render: Card with event list

---

3. Agenda Page Analysis

3.1 Component Location

• File: app/agenda/page.tsx (Server Component)
• Client Component: components/calendar/calendar-client.tsx
• Route: /agenda

3.2 Server-Side Logic (page.tsx)

Auto-Setup Calendar Sync

The page automatically sets up calendar synchronization for email accounts:

1. Infomaniak Accounts (CalDAV)

   • Discovers calendars using discoverInfomaniakCalendars()
   • Creates "Privée" calendar if not exists
   • Creates CalendarSync record with:
     • provider: 'infomaniak'
     • syncFrequency: 15 minutes
     • externalCalendarUrl from discovery
   • Triggers initial sync

2. Microsoft Accounts (OAuth)

   • Discovers calendars using discoverMicrosoftCalendars()
   • Creates "Privée" calendar if not exists
   • Creates CalendarSync record with:
     • provider: 'microsoft'
     • syncFrequency: 5 minutes (more reactive)
     • externalCalendarId from Graph API
   • Triggers initial sync

Calendar Filtering Logic

// Excludes "Privée" and "Default" calendars that are NOT synced
calendars = calendars.filter(cal => {
  const isPrivateOrDefault = cal.name === "Privée" || cal.name === "Default";
  const hasActiveSync = cal.syncConfig?.syncEnabled === true && 
                        cal.syncConfig?.mailCredential;
  
  // Exclude "Privée"/"Default" calendars that are not actively synced
  if (isPrivateOrDefault && !hasActiveSync) {
    return false;
  }
  return true;
});

Background Sync Trigger

For Microsoft calendars, the page triggers background sync if needed:

• Checks if sync is needed (2 minutes minimum interval)
• Triggers async sync (doesn't block page load)
• Sync runs in background, updates database
• Next page load will show synced events

3.3 Client-Side Component (calendar-client.tsx)

Key Features

1. FullCalendar Integration

   • Uses @fullcalendar/react library
   • Views: Month, Week, Day
   • Plugins: dayGrid, timeGrid, interaction

2. Calendar Management

   • Calendar selector sidebar (left column)
   • Visibility toggle per calendar
   • Calendar creation/editing dialog
   • Sync configuration UI

3. Event Management

   • Create events by clicking/selecting dates
   • Edit events by clicking on them
   • Delete events
   • Event form with:
     • Title, description, location
     • Start/end date and time
     • All-day toggle
     • Calendar selection

4. Data Refresh

   • fetchCalendars() function
   • Calls /api/calendars (no refresh parameter)
   • Uses Redis cache by default
   • Updates FullCalendar via calendarApi.refetchEvents()

Calendar Display Logic

// Sorts calendars: synced first, then groups, then missions
const sortCalendars = (cals) => {
  return [...filtered].sort((a, b) => {
    const aIsSynced = a.syncConfig?.syncEnabled && a.syncConfig?.mailCredential;
    const bIsSynced = b.syncConfig?.syncEnabled && b.syncConfig?.mailCredential;
    
    // Synced calendars first
    if (aIsSynced && !bIsSynced) return -1;
    if (!aIsSynced && bIsSynced) return 1;
    
    // Then groups, then missions, then by name
    // ...
  });
};

Event Rendering

Events are mapped to FullCalendar format:

events={calendars.flatMap(cal => 
  cal.events.map(event => ({
    id: event.id,
    title: event.title,
    start: new Date(event.start),
    end: new Date(event.end),
    allDay: event.isAllDay,
    backgroundColor: `${cal.color}dd`,
    borderColor: cal.color,
    extendedProps: {
      calendarName: cal.name,
      location: event.location,
      description: cleanDescription(event.description),
      calendarId: event.calendarId,
      originalEvent: event,
      color: cal.color
    }
  }))
)}

3.4 Issues Identified

1. ⚠️ Sync Timing

   • Microsoft sync triggers on page load (background)
   • May cause delay before events appear
   • No loading indicator for background sync

2. ⚠️ Calendar Filtering Complexity

   • Complex logic for "Privée"/"Default" calendars
   • May hide calendars that should be visible
   • Logging is extensive but may be confusing

3. ❌ No Real-Time Updates

   • Events only update on manual refresh
   • No WebSocket or polling for new events
   • User must refresh to see synced events

4. ⚠️ Event Matching Logic

   • Infomaniak: Matches by title + start date (within 1 minute)
   • Microsoft: Matches by [MS_ID:xxx] in description
   • May create duplicates if matching fails

---

4. Courrier Page Analysis

4.1 Component Location

• File: app/courrier/page.tsx
• Route: /courrier
• Type: Client Component

4.2 Key Features

Email Account Management

• Multiple email account support
• Account colors for visual distinction
• Account settings (display name, password, color)

Email Operations

• Folder navigation (INBOX, Sent, Drafts, etc.)
• Email reading, composing, replying
• Email search
• Bulk operations (delete, mark read/unread)

Integration with Calendar Sync

The Courrier page is indirectly related to calendar sync:

1. Email Accounts are stored in MailCredentials table
2. Calendar Sync uses MailCredentials for authentication
3. Auto-Setup in Agenda page discovers accounts from Courrier

4.3 Connection to Calendar Sync

// In agenda/page.tsx:
const infomaniakAccounts = await prisma.mailCredentials.findMany({
  where: {
    userId: session?.user?.id || '',
    host: { contains: 'infomaniak' },
    password: { not: null }
  }
});

// For each account, create calendar sync
for (const account of infomaniakAccounts) {
  // Discover calendars
  const calendars = await discoverInfomaniakCalendars(account.email, account.password!);
  
  // Create calendar and sync config
  // ...
}

4.4 Issues Identified

1. ⚠️ No Direct Calendar Integration

   • Courrier page doesn't show calendar events
   • No email-to-calendar event creation
   • Separate systems (email vs calendar)

2. ⚠️ Account Deletion Impact

   • Deleting email account may orphan calendar sync
   • Agenda page has cleanup logic, but may miss edge cases

---

5. Calendar Synchronization Services

5.1 Infomaniak CalDAV Sync

Service Location

• File: lib/services/caldav-sync.ts

Key Functions

1. discoverInfomaniakCalendars(email, password)

   • Uses WebDAV client (webdav library)
   • Connects to https://sync.infomaniak.com
   • Lists directories using PROPFIND
   • Extracts calendar name and color from XML

2. fetchCalDAVEvents(email, password, calendarUrl, startDate, endDate)

   • Uses REPORT method with calendar-query
   • Filters events by date range
   • Parses iCalendar format (.ics)
   • Returns CalDAVEvent[]

3. syncInfomaniakCalendar(calendarSyncId, forceSync)

   • Fetches events from CalDAV (1 month ago to 3 months ahead)
   • Gets existing events from database
   • Matches events by title + start date (within 1 minute)
   • Creates new events or updates existing
   • Updates lastSyncAt timestamp

Event Matching Logic

// Tries to find existing event by:
const existingEvent = existingEvents.find(
  (e) =>
    e.title === caldavEvent.summary &&
    Math.abs(new Date(e.start).getTime() - caldavEvent.start.getTime()) < 60000
);

Issue: This matching is fragile:

• May create duplicates if title changes
• May miss events if timezone conversion causes >1 minute difference
• No UID-based matching (iCalendar has UID field)

5.2 Microsoft Graph API Sync

Service Location

• File: lib/services/microsoft-calendar-sync.ts

Key Functions

1. discoverMicrosoftCalendars(userId, email)

   • Uses Microsoft Graph API
   • Endpoint: https://graph.microsoft.com/v1.0/me/calendars
   • Requires OAuth token with calendar scope
   • Returns MicrosoftCalendar[]

2. fetchMicrosoftEvents(userId, email, calendarId, startDate, endDate)

   • Endpoint: https://graph.microsoft.com/v1.0/me/calendars/{calendarId}/events
   • Filters by date range using $filter parameter
   • Returns MicrosoftEvent[]

3. syncMicrosoftCalendar(calendarSyncId, forceSync)

   • Fetches events (1 month ago to 6 months ahead)
   • Converts Microsoft events to CalDAV-like format
   • Stores Microsoft ID in description: [MS_ID:xxx]
   • Matches events by Microsoft ID first, then by title+date
   • Creates/updates events in database

Event Matching Logic

// First: Match by Microsoft ID in description
let existingEvent = existingEvents.find((e) => {
  if (e.description && e.description.includes(`[MS_ID:${microsoftId}]`)) {
    return true;
  }
  return false;
});

// Fallback: Match by title + start date
if (!existingEvent) {
  existingEvent = existingEvents.find(
    (e) =>
      e.title === caldavEvent.summary &&
      Math.abs(new Date(e.start).getTime() - caldavEvent.start.getTime()) < 60000
  );
}

Better than Infomaniak: Uses ID-based matching, but stores ID in description (hacky)

5.3 Sync Job Service

Service Location

• File: lib/services/calendar-sync-job.ts

Function: runCalendarSyncJob()

• Gets all enabled sync configurations
• Checks if sync is needed (based on syncFrequency)
• Calls appropriate sync function based on provider
• Logs results (successful, failed, skipped)

Usage: Should be called by cron job or scheduled task

5.4 Issues Identified

1. ❌ No UID-Based Matching for Infomaniak

   • Should use iCalendar UID field
   • Current matching is fragile

2. ⚠️ Microsoft ID Storage

   • Stores ID in description field (hacky)
   • Should use dedicated externalEventId field in Event model

3. ❌ No Event Deletion

   • Sync only creates/updates events
   • Doesn't delete events removed from external calendar
   • May show stale events

4. ⚠️ Error Handling

   • Errors are logged but sync continues
   • May leave calendar in inconsistent state
   • No retry mechanism for transient failures

5. ⚠️ Sync Frequency

   • Infomaniak: 15 minutes (may be too slow)
   • Microsoft: 5 minutes (better, but still not real-time)
   • No user-configurable frequency

6. ❌ No Incremental Sync

   • Always fetches full date range
   • May be slow for calendars with many events
   • Should use lastSyncAt to fetch only changes

---

6. API Routes Analysis

6.1 /api/calendars (GET)

File: app/api/calendars/route.ts

Features

• Redis caching (unless ?refresh=true)
• Returns calendars with events
• Events ordered by start date

Issues

• Cache TTL not specified (uses default)
• No cache invalidation on event creation/update
• refresh=true bypasses cache (used by widget)

6.2 /api/calendars/sync (POST, PUT, GET, DELETE)

File: app/api/calendars/sync/route.ts

Features

• POST: Create sync configuration
• PUT: Trigger manual sync
• GET: Get sync status
• DELETE: Remove sync configuration

Issues

• Manual sync triggers full sync (not incremental)
• No webhook support for real-time updates

---

7. Database Schema

7.1 Calendar Model

model Calendar {
  id          String   @id @default(uuid())
  name        String
  color       String
  description String?
  userId      String
  missionId   String?
  events      Event[]
  syncConfig  CalendarSync?
  // ...
}

7.2 CalendarSync Model

model CalendarSync {
  id                  String   @id @default(uuid())
  calendarId         String   @unique
  mailCredentialId   String?
  provider           String   // "infomaniak" | "microsoft"
  externalCalendarId String?
  externalCalendarUrl String?
  syncEnabled        Boolean  @default(true)
  lastSyncAt         DateTime?
  syncFrequency      Int      @default(15) // minutes
  lastSyncError      String?
  // ...
}

7.3 Event Model

model Event {
  id          String   @id @default(uuid())
  title       String
  description String?
  start       DateTime
  end         DateTime
  location    String?
  isAllDay    Boolean  @default(false)
  calendarId  String
  userId      String
  // ...
}

Missing Fields:

• externalEventId (for reliable matching)
• externalEventUrl (for linking to external calendar)
• lastSyncedAt (for incremental sync)

---

8. Critical Issues Summary

8.1 High Priority

1. Event Matching Fragility

   • Infomaniak: No UID-based matching
   • Microsoft: ID stored in description (hacky)
   • Impact: Duplicate events, missed updates

2. No Event Deletion

   • Removed events stay in database
   • Impact: Stale events shown to users

3. No Real-Time Updates

   • Widget and page don't auto-refresh
   • Impact: Users must manually refresh to see new events

8.2 Medium Priority

4. Cache Invalidation

   • Events created/updated don't invalidate cache
   • Impact: Stale data shown until cache expires

5. Sync Frequency

   • Infomaniak: 15 minutes (too slow)
   • Impact: Delayed event appearance

6. No Incremental Sync

   • Always fetches full date range
   • Impact: Slow sync, unnecessary API calls

8.3 Low Priority

7. Widget Refresh Integration

   • Not using unified refresh system
   • Impact: Inconsistent refresh behavior

8. Error Recovery

   • No retry mechanism
   • Impact: Sync failures require manual intervention

---

9. Recommendations

9.1 Immediate Fixes

1. Add externalEventId field to Event model

   model Event {
     // ... existing fields
     externalEventId  String?  // UID from iCalendar or Microsoft ID
     externalEventUrl String?  // Link to external calendar event
   }
   

2. Implement UID-based matching for Infomaniak

   • Use iCalendar UID field for matching
   • Store in externalEventId

3. Implement event deletion

   • Compare external events with database events
   • Delete events not in external calendar

4. Add cache invalidation

   • Invalidate Redis cache on event create/update/delete
   • Invalidate on sync completion

9.2 Short-Term Improvements

5. Implement incremental sync

   • Use lastSyncAt to fetch only changes
   • Use CalDAV sync-token for Infomaniak
   • Use Microsoft Graph delta queries

6. Add real-time updates

   • WebSocket or Server-Sent Events
   • Notify clients when sync completes
   • Auto-refresh widget and page

7. Improve error handling

   • Retry mechanism for transient failures
   • Better error messages for users
   • Sync status indicator in UI

9.3 Long-Term Enhancements

8. Unified refresh system integration

   • Use useUnifiedRefresh hook in widget
   • Consistent refresh behavior across app

9. User-configurable sync frequency

   • Allow users to set sync interval
   • Different frequencies per calendar

10. Two-way sync

    • Sync local events to external calendars
    • Handle conflicts (last-write-wins or user choice)

---

10. Testing Recommendations

10.1 Unit Tests

• Event matching logic (UID-based, title+date fallback)
• Date range filtering
• iCalendar parsing
• Microsoft event conversion

10.2 Integration Tests

• Full sync flow (discover → sync → verify)
• Event creation/update/deletion
• Cache invalidation
• Error recovery

10.3 E2E Tests

• Widget displays events correctly
• Page shows synced events
• Manual sync triggers correctly
• Calendar creation/editing

---

11. Conclusion

The calendar synchronization system is functional but has several architectural issues that impact reliability and user experience:

1. Fragile event matching may cause duplicates or missed updates
2. No event deletion leaves stale events in the database
3. No real-time updates requires manual refresh
4. Cache invalidation is missing, causing stale data

Priority: Focus on fixing event matching and deletion first, as these directly impact data integrity. Then implement cache invalidation and real-time updates for better UX.

---

Document Version: 1.0
Last Updated: 2025-01-XX
Author: Senior Software Architect Analysis