L
Leo Sivak
The current retry mechanism in Port does not handle well synchronization failures caused by race conditions and temporary issues. This feature request proposes an enhanced retry system to improve data consistency and reliability with eventual consistency.
  1. Automated Re-sync for Specific Blueprints (based on missing realations)
Add support for automated re-sync based on missing relations in blueprint entities.
  • Improve the current periodic re-sync feature to support specific blueprints
  • Add support for configuring conditional re-sync only if there are errors caused by missing relations
This will ensure eventual consistency for objects with infrequent updates (e.g., re-sync only for k8s namespaces when k8s pod has missing namespace as a relation)
  1. Implement Exponential Backoff
  • Replace the current retry system with an exponential backoff mechanism
  • Gradually increase the time between retry attempts to reduce system load and improve success rates
  1. Retry for Relation Sync Issues
  • Enable retry attempts specifically for entities created with relation sync issues
  • Target scenarios where related entities may not have been imported yet due to timing
Benefits
  • More efficient use of system resources during retries
  • Higher success rate for entity synchronization
  • Automatic resolution of temporary inconsistencies
  • Improved data integrity and completeness over time
This feature will enhance Port's ability to maintain consistent and complete data, especially in complex scenarios involving multiple related entities and large-scale imports.