Skip to content

Exports to use ScheduleMixin#177

Open
kaapstorm wants to merge 19 commits intomasterfrom
nh/exp_sched_3
Open

Exports to use ScheduleMixin#177
kaapstorm wants to merge 19 commits intomasterfrom
nh/exp_sched_3

Conversation

@kaapstorm
Copy link
Copy Markdown
Contributor

@kaapstorm kaapstorm commented Apr 10, 2026

Exports allow users to schedule export runs every N minutes.

Data forwarding and scheduled refreshes use ScheduleMixin to schedule runs. They can do what export schedules can, and a lot more.

This PR migrates exports to use the same scheduling as data forwarding and scheduled refreshes.

@kaapstorm kaapstorm marked this pull request as draft April 10, 2026 23:30
@kaapstorm kaapstorm force-pushed the nh/exp_sched_3 branch 2 times, most recently from db283e9 to 9f87568 Compare April 13, 2026 23:02
class Migration(migrations.Migration):
dependencies = [
('exports', '0024_exportconfig_days_of_week_and_more'),
('django_celery_beat', '__latest__'),
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line ensures that all django-celery-beat migrations have run. This migration creates IntervalSchedule and PeriodicTask objects, so django-celery-beat's tables need to exist already.

@kaapstorm kaapstorm marked this pull request as ready for review April 13, 2026 23:03
Comment thread apps/exports/migrations/0025_migrate_export_schedules.py Outdated
Copy link
Copy Markdown

@jingcheng16 jingcheng16 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed until 91df2ef

Comment thread apps/schedules/signals.py Outdated
Comment thread apps/schedules/mixin.py
Comment thread apps/schedules/mixin.py Outdated
Comment thread apps/exports/migrations/0025_migrate_export_schedules.py Outdated
Comment thread apps/exports/migrations/0025_migrate_export_schedules.py Outdated
kaapstorm and others added 17 commits April 16, 2026 12:40
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
exports.0025 is a data migration that calls apps.get_model(), which
triggers StateApps.__init__ -> _check_lazy_references(). Without explicit
dependencies on forwarding.0002_update_database_fk and
refreshes.0002_update_database_fk, the migration plan could apply
exports.0025 before those fixup migrations. At that point the state still
has the old 'exports.exportdatabase' FK targets (from the 0001_initial
migrations) while exports.0021 has already removed ExportDatabase,
causing a lazy-reference validation error in CI.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Also removes these fields from export forms and admin list_filter
since they are no longer database columns (is_paused is now a
property from ScheduleMixin).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove the run_all_exports polling pattern from celery.py and
scheduling.py. Each ExportConfig and MultiProjectExportConfig now gets
its own celery-beat PeriodicTask via ScheduleMixin signals. The
run_export_task and run_multi_project_export_task no longer accept
ignore_schedule_checks since scheduling is handled by celery-beat.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add tests verifying:
- Interval schedule creates PeriodicTask on save (via signals)
- Export without schedule has is_paused == True
- has_queued_runs() detects QUEUED status correctly
- Deleting export cleans up its PeriodicTask

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove bind=True from run_scheduled_export_task and
run_scheduled_multi_export_task — self was never used.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
If an export is deleted between beat scheduling the task and a worker
picking it up, the task now logs a warning and returns cleanly rather
than crashing with an unhandled exception.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Eliminate the duplicated has_queued_runs / ExportRun.create /
next_task.delay logic shared by both celery-beat entry points.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ExportConfigBase.details_url used isinstance() to dispatch on its own
concrete subclasses, coupling the base class to all its descendants.
Replace with an abstract property on the base and concrete overrides
on ExportConfig and MultiProjectExportConfig.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
kaapstorm and others added 2 commits April 16, 2026 12:49
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@kaapstorm
Copy link
Copy Markdown
Contributor Author

Sorry for the rebase. Hopefully it has resulted in a PR that is easier to review going forward.

Reviewed until 91df2ef

That commit is now 5e0a4aa.

@kaapstorm kaapstorm marked this pull request as ready for review April 16, 2026 16:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants