Skip to content

Fix N+1 translation cache lookups in celery translation tasks#2731

Open
szabozoltan69 wants to merge 1 commit intodevelopfrom
fix/translate-model-n+1-fix
Open

Fix N+1 translation cache lookups in celery translation tasks#2731
szabozoltan69 wants to merge 1 commit intodevelopfrom
fix/translate-model-n+1-fix

Conversation

@szabozoltan69
Copy link
Copy Markdown
Contributor

@szabozoltan69 szabozoltan69 commented Apr 30, 2026

For Sentry issue /organizations/ifrc-go/issues/5764

Copy link
Copy Markdown
Member

@thenav56 thenav56 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @szabozoltan69 for integrating with cache table.

Comment thread lang/tasks.py
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can refactor the whole function with the new cache logic.
Something like this

def translate_fields_object(self, obj, field):
    initial_lang = getattr(obj, TRANSLATOR_ORIGINAL_LANGUAGE_FIELD_NAME)
    initial_value = getattr(obj, build_localized_fieldname(field, initial_lang), None)
    if not initial_value or not initial_lang:
        return

    model = type(obj)
    table_field = f"{model._meta.app_label}:{model._meta.model_name}:{field}"
    field_max_length = model._meta.get_field(field).max_length

    # Determine which languages still need translation
    pending_langs = {
        lang: build_localized_fieldname(field, lang)
        for lang in AVAILABLE_LANGUAGES
        if not getattr(obj, build_localized_fieldname(field, lang), None)
    }
    if not pending_langs:
        return

    # Batch-fetch cached translations for all pending languages at once
    cached = self.translator.get_cached_translations(
        initial_value,
        list(pending_langs.keys()),
        source_language=initial_lang,
        table_field=table_field,
    )

    for lang, lang_field in pending_langs.items():
        translated = cached.get(lang) or self.translator.translate_text(
            initial_value,
            lang,
            source_language=initial_lang,
            table_field=table_field,
        )

        if translated is None:
            logger.warning(
                "Translation failed for %s.%s pk=%s",
                model.__name__,
                lang_field,
                obj.pk,
            )
            continue

        if field_max_length and len(translated) > field_max_length:
            logger.warning(
                "Translation exceeds max_length (%d) for %s.%s pk=%s",
                field_max_length, model.__name__, lang_field, obj.pk,
            )
            translated = translated[:field_max_length]

        setattr(obj, lang_field, translated)
        yield lang_field

I haven't tested this.

Comment thread lang/translation.py
Comment on lines +217 to +220
if not caches:
return {}

cache_by_lang = {cache.dest_language: cache for cache in caches}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if not caches will be always be true as caches is a queryset object.

We can instead use cache_by_lang for if

Suggested change
if not caches:
return {}
cache_by_lang = {cache.dest_language: cache for cache in caches}
cache_by_lang = {cache.dest_language: cache for cache in caches}
if not cache_by_lang:
return {}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants