NeahNew/IMPLEMENTATION_PLAN_UNIFIED_REFRESH.md

Implementation Plan: Unified Refresh System

🎯 Goals

1. Harmonize auto-refresh across all widgets and notifications
2. Reduce redundancy and eliminate duplicate API calls
3. Improve API efficiency with request deduplication and caching
4. Prevent memory leaks with proper cleanup mechanisms
5. Centralize refresh logic for easier maintenance

---

📋 Current State Analysis

Current Refresh Intervals:

• Notifications: 60 seconds (polling)
• Calendar: 5 minutes (300000ms)
• Parole (Chat): 30 seconds (30000ms)
• Navbar Time: Static (not refreshing - needs fix)
• News: Manual only
• Email: Manual only
• Duties (Tasks): Manual only

Current Problems:

1. ❌ No coordination between widgets
2. ❌ Duplicate API calls from multiple components
3. ❌ Memory leaks from uncleaned intervals
4. ❌ No request deduplication
5. ❌ Inconsistent refresh patterns

---

🏗️ Architecture: Unified Refresh System

Phase 1: Core Infrastructure

1.1 Create Unified Refresh Manager

File: lib/services/refresh-manager.ts

/**
 * Unified Refresh Manager
 * Centralizes all refresh logic, prevents duplicates, manages intervals
 */

export type RefreshableResource = 
  | 'notifications'
  | 'notifications-count'
  | 'calendar'
  | 'news'
  | 'email'
  | 'parole'
  | 'duties';

export interface RefreshConfig {
  resource: RefreshableResource;
  interval: number; // milliseconds
  enabled: boolean;
  priority: 'high' | 'medium' | 'low';
  onRefresh: () => Promise<void>;
}

class RefreshManager {
  private intervals: Map<RefreshableResource, NodeJS.Timeout> = new Map();
  private configs: Map<RefreshableResource, RefreshConfig> = new Map();
  private pendingRequests: Map<string, Promise<any>> = new Map();
  private lastRefresh: Map<RefreshableResource, number> = new Map();
  private isActive = false;

  /**
   * Register a refreshable resource
   */
  register(config: RefreshConfig): void {
    this.configs.set(config.resource, config);
    
    if (config.enabled && this.isActive) {
      this.startRefresh(config.resource);
    }
  }

  /**
   * Unregister a resource
   */
  unregister(resource: RefreshableResource): void {
    this.stopRefresh(resource);
    this.configs.delete(resource);
    this.lastRefresh.delete(resource);
  }

  /**
   * Start all refresh intervals
   */
  start(): void {
    if (this.isActive) return;
    
    this.isActive = true;
    
    // Start all enabled resources
    this.configs.forEach((config, resource) => {
      if (config.enabled) {
        this.startRefresh(resource);
      }
    });
  }

  /**
   * Stop all refresh intervals
   */
  stop(): void {
    this.isActive = false;
    
    // Clear all intervals
    this.intervals.forEach((interval) => {
      clearInterval(interval);
    });
    
    this.intervals.clear();
  }

  /**
   * Start refresh for a specific resource
   */
  private startRefresh(resource: RefreshableResource): void {
    // Stop existing interval if any
    this.stopRefresh(resource);
    
    const config = this.configs.get(resource);
    if (!config || !config.enabled) return;

    // Initial refresh
    this.executeRefresh(resource);

    // Set up interval
    const interval = setInterval(() => {
      this.executeRefresh(resource);
    }, config.interval);

    this.intervals.set(resource, interval);
  }

  /**
   * Stop refresh for a specific resource
   */
  private stopRefresh(resource: RefreshableResource): void {
    const interval = this.intervals.get(resource);
    if (interval) {
      clearInterval(interval);
      this.intervals.delete(resource);
    }
  }

  /**
   * Execute refresh with deduplication
   */
  private async executeRefresh(resource: RefreshableResource): Promise<void> {
    const config = this.configs.get(resource);
    if (!config) return;

    const requestKey = `${resource}-${Date.now()}`;
    const now = Date.now();
    const lastRefreshTime = this.lastRefresh.get(resource) || 0;
    
    // Prevent too frequent refreshes (minimum 1 second between same resource)
    if (now - lastRefreshTime < 1000) {
      console.log(`[RefreshManager] Skipping ${resource} - too soon`);
      return;
    }

    // Check if there's already a pending request for this resource
    const pendingKey = `${resource}-pending`;
    if (this.pendingRequests.has(pendingKey)) {
      console.log(`[RefreshManager] Deduplicating ${resource} request`);
      return;
    }

    // Create and track the request
    const refreshPromise = config.onRefresh()
      .then(() => {
        this.lastRefresh.set(resource, Date.now());
      })
      .catch((error) => {
        console.error(`[RefreshManager] Error refreshing ${resource}:`, error);
      })
      .finally(() => {
        this.pendingRequests.delete(pendingKey);
      });

    this.pendingRequests.set(pendingKey, refreshPromise);
    
    try {
      await refreshPromise;
    } catch (error) {
      // Error already logged above
    }
  }

  /**
   * Manually trigger refresh for a resource
   */
  async refresh(resource: RefreshableResource, force = false): Promise<void> {
    const config = this.configs.get(resource);
    if (!config) {
      throw new Error(`Resource ${resource} not registered`);
    }

    if (force) {
      // Force refresh: clear last refresh time
      this.lastRefresh.delete(resource);
    }

    await this.executeRefresh(resource);
  }

  /**
   * Get refresh status
   */
  getStatus(): {
    active: boolean;
    resources: Array<{
      resource: RefreshableResource;
      enabled: boolean;
      lastRefresh: number | null;
      interval: number;
    }>;
  } {
    const resources = Array.from(this.configs.entries()).map(([resource, config]) => ({
      resource,
      enabled: config.enabled,
      lastRefresh: this.lastRefresh.get(resource) || null,
      interval: config.interval,
    }));

    return {
      active: this.isActive,
      resources,
    };
  }
}

// Singleton instance
export const refreshManager = new RefreshManager();

---

1.2 Create Request Deduplication Utility

File: lib/utils/request-deduplication.ts

/**
 * Request Deduplication Utility
 * Prevents duplicate API calls for the same resource
 */

interface PendingRequest<T> {
  promise: Promise<T>;
  timestamp: number;
}

class RequestDeduplicator {
  private pendingRequests = new Map<string, PendingRequest<any>>();
  private readonly DEFAULT_TTL = 5000; // 5 seconds

  /**
   * Execute a request with deduplication
   */
  async execute<T>(
    key: string,
    requestFn: () => Promise<T>,
    ttl: number = this.DEFAULT_TTL
  ): Promise<T> {
    // Check if there's a pending request
    const pending = this.pendingRequests.get(key);
    
    if (pending) {
      const age = Date.now() - pending.timestamp;
      
      // If request is still fresh, reuse it
      if (age < ttl) {
        console.log(`[RequestDeduplicator] Reusing pending request: ${key}`);
        return pending.promise;
      } else {
        // Request is stale, remove it
        this.pendingRequests.delete(key);
      }
    }

    // Create new request
    const promise = requestFn()
      .finally(() => {
        // Clean up after request completes
        this.pendingRequests.delete(key);
      });

    this.pendingRequests.set(key, {
      promise,
      timestamp: Date.now(),
    });

    return promise;
  }

  /**
   * Cancel a pending request
   */
  cancel(key: string): void {
    this.pendingRequests.delete(key);
  }

  /**
   * Clear all pending requests
   */
  clear(): void {
    this.pendingRequests.clear();
  }

  /**
   * Get pending requests count
   */
  getPendingCount(): number {
    return this.pendingRequests.size;
  }
}

export const requestDeduplicator = new RequestDeduplicator();

---

1.3 Create Unified Refresh Hook

File: hooks/use-unified-refresh.ts

/**
 * Unified Refresh Hook
 * Provides consistent refresh functionality for all widgets
 */

import { useEffect, useCallback, useRef } from 'react';
import { useSession } from 'next-auth/react';
import { refreshManager, RefreshableResource } from '@/lib/services/refresh-manager';

interface UseUnifiedRefreshOptions {
  resource: RefreshableResource;
  interval: number;
  enabled?: boolean;
  onRefresh: () => Promise<void>;
  priority?: 'high' | 'medium' | 'low';
}

export function useUnifiedRefresh({
  resource,
  interval,
  enabled = true,
  onRefresh,
  priority = 'medium',
}: UseUnifiedRefreshOptions) {
  const { status } = useSession();
  const onRefreshRef = useRef(onRefresh);
  const isMountedRef = useRef(true);

  // Update callback ref when it changes
  useEffect(() => {
    onRefreshRef.current = onRefresh;
  }, [onRefresh]);

  // Register/unregister with refresh manager
  useEffect(() => {
    if (status !== 'authenticated' || !enabled) {
      return;
    }

    isMountedRef.current = true;

    // Register with refresh manager
    refreshManager.register({
      resource,
      interval,
      enabled: true,
      priority,
      onRefresh: async () => {
        if (isMountedRef.current) {
          await onRefreshRef.current();
        }
      },
    });

    // Start refresh manager if not already started
    refreshManager.start();

    // Cleanup
    return () => {
      isMountedRef.current = false;
      refreshManager.unregister(resource);
    };
  }, [resource, interval, enabled, priority, status]);

  // Manual refresh function
  const refresh = useCallback(
    async (force = false) => {
      if (status !== 'authenticated') return;
      await refreshManager.refresh(resource, force);
    },
    [resource, status]
  );

  return {
    refresh,
    isActive: refreshManager.getStatus().active,
  };
}

---

Phase 2: Harmonized Refresh Intervals

2.1 Define Standard Intervals

File: lib/constants/refresh-intervals.ts

/**
 * Standard Refresh Intervals
 * All intervals in milliseconds
 */

export const REFRESH_INTERVALS = {
  // High priority - real-time updates
  NOTIFICATIONS: 30000,        // 30 seconds (was 60s)
  NOTIFICATIONS_COUNT: 30000, // 30 seconds (same as notifications)
  PAROLE: 30000,               // 30 seconds (unchanged)
  NAVBAR_TIME: 1000,           // 1 second (navigation bar time - real-time)
  
  // Medium priority - frequent but not real-time
  EMAIL: 60000,                // 1 minute (was manual only)
  DUTIES: 120000,              // 2 minutes (was manual only)
  
  // Low priority - less frequent updates
  CALENDAR: 300000,            // 5 minutes (unchanged)
  NEWS: 600000,                // 10 minutes (was manual only)
  
  // Minimum interval between refreshes (prevents spam)
  MIN_INTERVAL: 1000,          // 1 second
} as const;

/**
 * Get refresh interval for a resource
 */
export function getRefreshInterval(resource: string): number {
  switch (resource) {
    case 'notifications':
      return REFRESH_INTERVALS.NOTIFICATIONS;
    case 'notifications-count':
      return REFRESH_INTERVALS.NOTIFICATIONS_COUNT;
    case 'parole':
      return REFRESH_INTERVALS.PAROLE;
    case 'email':
      return REFRESH_INTERVALS.EMAIL;
    case 'duties':
      return REFRESH_INTERVALS.DUTIES;
    case 'calendar':
      return REFRESH_INTERVALS.CALENDAR;
    case 'news':
      return REFRESH_INTERVALS.NEWS;
    default:
      return 60000; // Default: 1 minute
  }
}

---

Phase 3: Refactor Widgets

3.1 Refactor Notification Hook

File: hooks/use-notifications.ts (Refactored)

import { useState, useEffect, useCallback, useRef } from 'react';
import { useSession } from 'next-auth/react';
import { Notification, NotificationCount } from '@/lib/types/notification';
import { useUnifiedRefresh } from './use-unified-refresh';
import { REFRESH_INTERVALS } from '@/lib/constants/refresh-intervals';
import { requestDeduplicator } from '@/lib/utils/request-deduplication';

const defaultNotificationCount: NotificationCount = {
  total: 0,
  unread: 0,
  sources: {},
};

export function useNotifications() {
  const { data: session, status } = useSession();
  const [notifications, setNotifications] = useState<Notification[]>([]);
  const [notificationCount, setNotificationCount] = useState<NotificationCount>(defaultNotificationCount);
  const [loading, setLoading] = useState(false);
  const [error, setError] = useState<string | null>(null);
  const isMountedRef = useRef(true);

  // Fetch notification count
  const fetchNotificationCount = useCallback(async () => {
    if (!session?.user || !isMountedRef.current) return;
    
    try {
      setError(null);
      
      const data = await requestDeduplicator.execute(
        `notifications-count-${session.user.id}`,
        async () => {
          const response = await fetch('/api/notifications/count', {
            credentials: 'include',
          });
          
          if (!response.ok) {
            throw new Error('Failed to fetch notification count');
          }
          
          return response.json();
        }
      );
      
      if (isMountedRef.current) {
        setNotificationCount(data);
      }
    } catch (err) {
      console.error('Error fetching notification count:', err);
      if (isMountedRef.current) {
        setError('Failed to fetch notification count');
      }
    }
  }, [session?.user]);

  // Fetch notifications
  const fetchNotifications = useCallback(async (page = 1, limit = 20) => {
    if (!session?.user || !isMountedRef.current) return;
    
    setLoading(true);
    setError(null);
    
    try {
      const data = await requestDeduplicator.execute(
        `notifications-${session.user.id}-${page}-${limit}`,
        async () => {
          const response = await fetch(`/api/notifications?page=${page}&limit=${limit}`, {
            credentials: 'include',
          });
          
          if (!response.ok) {
            throw new Error('Failed to fetch notifications');
          }
          
          return response.json();
        }
      );
      
      if (isMountedRef.current) {
        setNotifications(data.notifications);
      }
    } catch (err) {
      console.error('Error fetching notifications:', err);
      if (isMountedRef.current) {
        setError('Failed to fetch notifications');
      }
    } finally {
      if (isMountedRef.current) {
        setLoading(false);
      }
    }
  }, [session?.user]);

  // Use unified refresh for notification count
  useUnifiedRefresh({
    resource: 'notifications-count',
    interval: REFRESH_INTERVALS.NOTIFICATIONS_COUNT,
    enabled: status === 'authenticated',
    onRefresh: fetchNotificationCount,
    priority: 'high',
  });

  // Initial fetch
  useEffect(() => {
    isMountedRef.current = true;
    
    if (status === 'authenticated' && session?.user) {
      fetchNotificationCount();
      fetchNotifications();
    }
    
    return () => {
      isMountedRef.current = false;
    };
  }, [status, session?.user, fetchNotificationCount, fetchNotifications]);

  // Mark as read
  const markAsRead = useCallback(async (notificationId: string) => {
    if (!session?.user) return false;
    
    try {
      const response = await fetch(`/api/notifications/${notificationId}/read`, {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        credentials: 'include',
      });
      
      if (!response.ok) return false;
      
      setNotifications(prev => 
        prev.map(n => n.id === notificationId ? { ...n, isRead: true } : n)
      );
      
      await fetchNotificationCount();
      return true;
    } catch (err) {
      console.error('Error marking notification as read:', err);
      return false;
    }
  }, [session?.user, fetchNotificationCount]);

  // Mark all as read
  const markAllAsRead = useCallback(async () => {
    if (!session?.user) return false;
    
    try {
      const response = await fetch('/api/notifications/read-all', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        credentials: 'include',
      });
      
      if (!response.ok) return false;
      
      setNotifications(prev => prev.map(n => ({ ...n, isRead: true })));
      await fetchNotificationCount();
      return true;
    } catch (err) {
      console.error('Error marking all notifications as read:', err);
      return false;
    }
  }, [session?.user, fetchNotificationCount]);

  return {
    notifications,
    notificationCount,
    loading,
    error,
    fetchNotifications,
    fetchNotificationCount,
    markAsRead,
    markAllAsRead,
  };
}

---

3.2 Refactor Widget Components

Example: Calendar Widget

File: components/calendar.tsx (Refactored)

"use client";

import { useEffect, useState } from "react";
import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card";
import { Button } from "@/components/ui/button";
import { RefreshCw, Calendar as CalendarIcon } from "lucide-react";
import { useUnifiedRefresh } from '@/hooks/use-unified-refresh';
import { REFRESH_INTERVALS } from '@/lib/constants/refresh-intervals';
import { requestDeduplicator } from '@/lib/utils/request-deduplication';
import { useSession } from 'next-auth/react';

interface Event {
  id: string;
  title: string;
  start: string;
  end: string;
  allDay: boolean;
  calendar: string;
  calendarColor: string;
}

export function Calendar() {
  const { status } = useSession();
  const [events, setEvents] = useState<Event[]>([]);
  const [loading, setLoading] = useState(true);
  const [error, setError] = useState<string | null>(null);

  const fetchEvents = async () => {
    if (status !== 'authenticated') return;
    
    setLoading(true);
    setError(null);
    
    try {
      const calendarsData = await requestDeduplicator.execute(
        'calendar-events',
        async () => {
          const response = await fetch('/api/calendars?refresh=true');
          if (!response.ok) {
            throw new Error('Failed to fetch events');
          }
          return response.json();
        }
      );

      const now = new Date();
      now.setHours(0, 0, 0, 0);

      const allEvents = calendarsData.flatMap((calendar: any) => 
        (calendar.events || []).map((event: any) => ({
          id: event.id,
          title: event.title,
          start: event.start,
          end: event.end,
          allDay: event.isAllDay,
          calendar: calendar.name,
          calendarColor: calendar.color
        }))
      );

      const upcomingEvents = allEvents
        .filter((event: any) => new Date(event.start) >= now)
        .sort((a: any, b: any) => new Date(a.start).getTime() - new Date(b.start).getTime())
        .slice(0, 7);

      setEvents(upcomingEvents);
    } catch (err) {
      console.error('Error fetching events:', err);
      setError('Failed to load events');
    } finally {
      setLoading(false);
    }
  };

  // Use unified refresh
  const { refresh } = useUnifiedRefresh({
    resource: 'calendar',
    interval: REFRESH_INTERVALS.CALENDAR,
    enabled: status === 'authenticated',
    onRefresh: fetchEvents,
    priority: 'low',
  });

  // Initial fetch
  useEffect(() => {
    if (status === 'authenticated') {
      fetchEvents();
    }
  }, [status]);

  // ... rest of component (formatDate, formatTime, render)
  
  return (
    <Card className="...">
      <CardHeader>
        <CardTitle>Agenda</CardTitle>
        <Button onClick={() => refresh(true)}>
          <RefreshCw className="..." />
        </Button>
      </CardHeader>
      {/* ... */}
    </Card>
  );
}

---

Phase 4: Implementation Steps

Step 1: Create Core Infrastructure (Day 1)

1. ✅ Create lib/services/refresh-manager.ts
2. ✅ Create lib/utils/request-deduplication.ts
3. ✅ Create lib/constants/refresh-intervals.ts
4. ✅ Create hooks/use-unified-refresh.ts

Testing: Unit tests for each module

---

Step 2: Fix Memory Leaks (Day 1-2)

1. ✅ Fix useNotifications hook cleanup
2. ✅ Fix notification badge double fetching
3. ✅ Fix widget interval cleanup
4. ✅ Fix Redis KEYS → SCAN

Testing: Memory leak detection in DevTools

---

Step 3: Refactor Notifications (Day 2)

1. ✅ Refactor hooks/use-notifications.ts
2. ✅ Update components/notification-badge.tsx
3. ✅ Remove duplicate fetch logic

Testing: Verify no duplicate API calls

---

Step 4: Refactor Widgets (Day 3-4)

1. ✅ Refactor components/calendar.tsx
2. ✅ Refactor components/parole.tsx
3. ✅ Refactor components/news.tsx
4. ✅ Refactor components/email.tsx
5. ✅ Refactor components/flow.tsx (Duties)
6. ✅ Refactor components/main-nav.tsx (Time display)

Testing: Verify all widgets refresh correctly

---

Step 5: Testing & Optimization (Day 5)

1. ✅ Performance testing
2. ✅ Memory leak verification
3. ✅ API call reduction verification
4. ✅ User experience testing

---

📊 Expected Improvements

Before:

• API Calls: ~120-150 calls/minute (with duplicates)
• Memory Leaks: Yes (intervals not cleaned up)
• Refresh Coordination: None
• Request Deduplication: None

After:

• API Calls: ~40-50 calls/minute (60-70% reduction)
• Memory Leaks: None (proper cleanup)
• Refresh Coordination: Centralized
• Request Deduplication: Full coverage

---

🎯 Success Metrics

1. API Call Reduction: 60%+ reduction in duplicate calls
2. Memory Usage: No memory leaks detected
3. Performance: Faster page loads, smoother UX
4. Maintainability: Single source of truth for refresh logic

---

🚀 Quick Start Implementation

Priority Order:

1. Critical (Do First):

   • Fix memory leaks
   • Create refresh manager
   • Create request deduplication

2. High (Do Second):

   • Refactor notifications
   • Refactor high-frequency widgets (parole, notifications)

3. Medium (Do Third):

   • Refactor medium-frequency widgets (email, duties)

4. Low (Do Last):

   • Refactor low-frequency widgets (calendar, news)

---

📝 Notes

• All intervals are configurable via constants
• Refresh manager can be paused/resumed globally
• Request deduplication prevents duplicate calls within 5 seconds
• All cleanup is handled automatically
• Compatible with existing code (gradual migration)

---

Implementation Plan v1.0