Feature ideas

Data sources can create, update and delete entities - to align with the remote data. However, I'm investigating an issue where we are seeing entities being deleted from our catalogue - and it's unclear why the data source refresh has done this. The entity is sourced from a file held in a github repo The file has not been changed since the entity was first created Manually "re-synching" the datasource re-creates the missing entity Then, some "random" time later, the entity is deleted again We can see that the entity is being created/deleted. And we can see "events" related to the data source. But none of this explains why the entity is being deleted - i.e. what data, from the source, has indicated that an entity deletion is appropriate.

The Port GCP BigQuery data source only contains a subset of the fields available in https://cloud.google.com/bigquery/docs/reference/rest/v2/tables . Some useful fields that are currently missing in Port are: - column metadata (from https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#TableSchema - including name, description, type and any policy tags) - numRows - lastModifiedTime Why: I'd like to use Port as a Data Dictionary, and so it will be useful to have as much information about tables as possible.

Add option to specifically resync a predefined list of resources & do a scheduled resync only for a selected list of resources

Currently, self-hosted Ocean integrations require organizational credentials. Ideally, service account credentials should be used to improve security and create different tokens for different integrations.

Enum properties are great. Both visually, better UX when selecting a value, better and clearer validation However, it's really hard to know what your unique values are when designing a blueprint. (e.g. JIRA Issue Types, Azure DevOps Work Item types) This is one of two features that would be amazing around enums: convert an existing string property to Enum (opened this separately) enable a data source to add a new enum value to the set of options (if it is valid upstream, there should be a way to optionally make it valid in port with no extra config)

I want to be able to insert entities by batch or bulk through webhook/REST api to prevent calling the same endpoint many times. I also want to be able to do the same when deleting entities. Having checkboxes in catalogs to select and do a specific action for selected entities can also help :)

Support pulling from Github's dependency graph https://docs.github.com/en/code-security/supply-chain-security/understanding-your-software-supply-chain/about-the-dependency-graph which would enable us to do in-depth analysis of all the software dependencies for the repositories we have in Port. There is already a REST api for it: https://docs.github.com/en/rest/dependency-graph/sboms?apiVersion=2022-11-28#export-a-software-bill-of-materials-sbom-for-a-repository Use cases for having dependencies catalogged in Port Track usage of internal packages and how up-to-date consumers are, either through dasboards or scorecards Build comprehensive documentation around service dependencies Build automations/self-service actions around notification for migrations, deprecations, releases with breaking changes, targeted towards the proper teams instead of everyone Detect inefficiencies/antipatterns related to dependencies Enforce standards/protection measures related to dependencies (ensure no-one uses package X, ensure everyone uses package Y at least at version A.B.C, etc.)

When visiting a specific page in Port I would like to automatically trigger the (re)sync of a specific data source. Currently my option is to manually resync it from the Builder page, or start a Self service action to trigger it.

When installing an integration from the data sources modal, it would help to have a search box to make it easy to find the desired integration.

Currently, each data source requires specific mapping configurations tied to the exporter / integration deployment phase. This approach is inefficient when managing multiple data sources. We propose implementing a centralized and versioned configuration in Port to improve this process. Proposal: Port will store a list of mapping configurations for each integration type. Each data source will have an option to specify which configuration version to use and whether to stream data into port or not. This will allow sharing configurations between different exporters of same integration. The process will decouple data source management from exporter / integration rollout. Benefits: Reduced configuration duplication Easier updates and maintenance Improved scalability for managing many data sources Consistent configuration across environments This feature aims to streamline data source management, making it more efficient for teams working with multiple data sources. --- ## Additional explanation based on k8s integration Today, each k8s cluster is represented as a separate data source without any control over shared configuration for all clusters, in addition k8s data source can't be created on demand and depends on deployment of k8s exporter. Each configuration change needs to be applied to all k8s data sources explicitly either via API/UI/Terraform or based on config-maps deployed on each cluster. After applying the proposed approach, port builders will be able define and share configuration among multiple k8s clusters and apply experimental changes gradually. How this could work in practice: every k8s exporter is deployed on each k8s cluster when it is supplied with appropriate credentials to access port and publishes only the cluster name. In addition the exporter subscribes to receive new instructions from port (which data to stream). port builders (developers) discover all k8s clusters associated to their account and choose which clusters should be enabled as data sources and which mapping configuration version should be applied. This way k8s exporters are deployed once and upgraded only when there is a new version of exporter. The control over data input is managed centrally from port and configurations can be shared among multiple data sources of same type.