Expose improved sync status & metrics on integrations
complete
Matan Grady
Provide users with an at-a-glance view of what happened during the last sync, including high-level statistics (raw objects, transformed objects, success/failure count), sync duration, and time of last run. This allows users to assess sync behavior without diving into raw logs, improving usability and reducing support load.
See prototype here - https://effulgent-nasturtium-76fd8f.netlify.app/
Similar FRs
M
Maya Margalit
Merged in a post:
Expose metrics from ocean integrations
T
Tero Ahonen
Would it be possible to expose "standard" /metrics endpoint for ocean exporters. Minimal metrics would be that is the integration running fine or not. Really common use case would be that credentials have expired and integration is not working. Really nice would be to have number of mapped entities. This would also help to figure out is the integration working ok and number of entities is increasing. Who ever runs exporters in k8s most certainly have Grafana Alloy or Prometheus running also. And adding ServiceMonitor object to the helm charts makes the integration even easier.
Current status of the integration could be ready using API and checked the .integrations[].resyncState block, but having /metrics endpoint would be more "cloud-native". Using API endpoint would required some extra components to run to check and calculate the status.
K
Keren Ben Zion
Also will be good to add it to k8s-exporter and port-agents components
M
Maya Margalit
updated the status to
complete
M
Maya Margalit
This is now available in Port!
*Available for Ocean integrations.
Photo Viewer
View photos in a modal
G
Glenn Moen
The prototype looks great. Can you explain what the transformed number represents? I am currently working on an issue and want to understand how many of the raw objects resulted in entity api calls to port. Wondering if this sort of dashboard would help.
Matan Grady
Glenn Moen: sure! The ingestion flow:
- Gets raw objects
- Apply transformation (in here JQ mapping errors can occur)
- Send to Port
So to your case you'll be able to understand how many objects were identified, how many were successfully transformed, and finally ingested to Port
Matan Grady
We are exploring a similar idea, please check out the prototype and upvote!
Matan Grady
Merged in a post:
Improve visibility of data sync and errors
Matan Grady
As a builder, I want to understand my integration status so I will be able to fix errors and keep my Portal up-to-date
===
We want to show a more apparent status and surface outstanding errors with the integration
Matan Grady
updated the status to
exploring