A
Anthony Clever
For some kinds available in integration, the API responses can vary greatly depending on certain conditions. As a concrete example, in the github integration, the
branch
kind only contains properties like
.branch.commit.commit.author
when the branch is the default branch of a given repository, and in my mapping I set the query to
.branch.name == .repository.default_branch
to ensure I have access to that value.
What I'm currently finding is that the sendRawDataExamples option in integrations does not respect this query setting, so the ~5 examples retained from the api call might not even contain a valid api response (as in, one that passes the check in
query
).
I have a few varied suggestions here:
  • Allow
    sendRawDataExamples
    to respect the given query in a mapping.
  • Allow
    sendRawDataExamples
    to have it's own query for stricter or looser example api responses.
  • Allow independent resync of
    sendRawDataExamples
    separate from a resync of the data source.
Giving builders the ability to have stronger control over the examples defined in the data source view would significantly improve iteration time when dealing with complex mappings across multiple data sources.