Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(eap): add a migration to namespace sentry tags #6326

Merged
merged 1 commit into from
Sep 23, 2024

Conversation

colin-sentry
Copy link
Member

We need to be able to disambiguate things sentry sets (like name) from things user set (like setAttribute("name")) - this facilitates that in the materialized view

Copy link

This PR has a migration; here is the generated SQL

-- start migrations

-- forward migration events_analytics_platform : 0015_span_attribute_table_namespaced
Local op: DROP TABLE IF EXISTS spans_str_attrs_2_mv;
Local op: DROP TABLE IF EXISTS spans_num_attrs_2_mv;
Local op: CREATE MATERIALIZED VIEW IF NOT EXISTS spans_str_attrs_2_mv TO spans_str_attrs_2_local (organization_id UInt64, project_id UInt64, attr_key String CODEC (ZSTD(1)), attr_value String CODEC (ZSTD(1)), timestamp DateTime CODEC (ZSTD(1)), retention_days UInt16, count SimpleAggregateFunction(sum, UInt64)) AS 
SELECT
    organization_id,
    project_id,
    attrs.1 as attr_key,
    attrs.2 as attr_value,
    toStartOfDay(_sort_timestamp) AS timestamp,
    retention_days,
    1 AS count
FROM eap_spans_local
LEFT ARRAY JOIN
    arrayConcat(
        CAST(attr_str_0, 'Array(Tuple(String, String))'), CAST(attr_str_1, 'Array(Tuple(String, String))'), CAST(attr_str_2, 'Array(Tuple(String, String))'), CAST(attr_str_3, 'Array(Tuple(String, String))'), CAST(attr_str_4, 'Array(Tuple(String, String))'), CAST(attr_str_5, 'Array(Tuple(String, String))'), CAST(attr_str_6, 'Array(Tuple(String, String))'), CAST(attr_str_7, 'Array(Tuple(String, String))'), CAST(attr_str_8, 'Array(Tuple(String, String))'), CAST(attr_str_9, 'Array(Tuple(String, String))'), CAST(attr_str_10, 'Array(Tuple(String, String))'), CAST(attr_str_11, 'Array(Tuple(String, String))'), CAST(attr_str_12, 'Array(Tuple(String, String))'), CAST(attr_str_13, 'Array(Tuple(String, String))'), CAST(attr_str_14, 'Array(Tuple(String, String))'), CAST(attr_str_15, 'Array(Tuple(String, String))'), CAST(attr_str_16, 'Array(Tuple(String, String))'), CAST(attr_str_17, 'Array(Tuple(String, String))'), CAST(attr_str_18, 'Array(Tuple(String, String))'), CAST(attr_str_19, 'Array(Tuple(String, String))'),
        array(
            tuple('sentry.service', `service`),
            tuple('sentry.segment_name', `segment_name`),
            tuple('sentry.name', `name`)
        )
    ) AS attrs
GROUP BY
    organization_id,
    project_id,
    attr_key,
    attr_value,
    timestamp,
    retention_days
;
Local op: CREATE MATERIALIZED VIEW IF NOT EXISTS spans_num_attrs_2_mv TO spans_num_attrs_2_local (organization_id UInt64, project_id UInt64, attr_key String, attr_min_value SimpleAggregateFunction(min, Float64), attr_max_value SimpleAggregateFunction(max, Float64), timestamp DateTime CODEC (ZSTD(1)), retention_days UInt16, count SimpleAggregateFunction(sum, UInt64)) AS 
SELECT
    organization_id,
    project_id,
    attrs.1 as attr_key,
    attrs.2 as attr_min_value,
    attrs.2 as attr_max_value,
    toStartOfDay(_sort_timestamp) AS timestamp,
    retention_days,
    1 AS count
FROM eap_spans_local
LEFT ARRAY JOIN
    arrayConcat(
        CAST(attr_num_0, 'Array(Tuple(String, Float64))'),CAST(attr_num_1, 'Array(Tuple(String, Float64))'),CAST(attr_num_2, 'Array(Tuple(String, Float64))'),CAST(attr_num_3, 'Array(Tuple(String, Float64))'),CAST(attr_num_4, 'Array(Tuple(String, Float64))'),CAST(attr_num_5, 'Array(Tuple(String, Float64))'),CAST(attr_num_6, 'Array(Tuple(String, Float64))'),CAST(attr_num_7, 'Array(Tuple(String, Float64))'),CAST(attr_num_8, 'Array(Tuple(String, Float64))'),CAST(attr_num_9, 'Array(Tuple(String, Float64))'),CAST(attr_num_10, 'Array(Tuple(String, Float64))'),CAST(attr_num_11, 'Array(Tuple(String, Float64))'),CAST(attr_num_12, 'Array(Tuple(String, Float64))'),CAST(attr_num_13, 'Array(Tuple(String, Float64))'),CAST(attr_num_14, 'Array(Tuple(String, Float64))'),CAST(attr_num_15, 'Array(Tuple(String, Float64))'),CAST(attr_num_16, 'Array(Tuple(String, Float64))'),CAST(attr_num_17, 'Array(Tuple(String, Float64))'),CAST(attr_num_18, 'Array(Tuple(String, Float64))'),CAST(attr_num_19, 'Array(Tuple(String, Float64))'),
        array(
            tuple('sentry.duration_ms', `duration_ms`::Float64)
        )
    ) AS attrs
GROUP BY
    organization_id,
    project_id,
    attrs.1,
    attrs.2,
    timestamp,
    retention_days
;
-- end forward migration events_analytics_platform : 0015_span_attribute_table_namespaced




-- backward migration events_analytics_platform : 0015_span_attribute_table_namespaced
Local op: DROP TABLE IF EXISTS spans_str_attrs_2_mv;
Local op: DROP TABLE IF EXISTS spans_num_attrs_2_mv;
-- end backward migration events_analytics_platform : 0015_span_attribute_table_namespaced

wmak added a commit to getsentry/sentry that referenced this pull request Sep 19, 2024
wmak added a commit to getsentry/sentry that referenced this pull request Sep 19, 2024
),
]

def backwards_ops(self) -> Sequence[SqlOperation]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Backwards ops is not exactly restoring anything. Should it just be a no-op?

@colin-sentry colin-sentry merged commit 9c54251 into master Sep 23, 2024
30 checks passed
@colin-sentry colin-sentry deleted the eap_namespaced_sentry_items branch September 23, 2024 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants