fix: ignore docs, tests, and tutorials in Ray uploads

[nightcityblade]

Description

closes #1337

Add a .rayignore file so Ray uploads skip the docs, tests, and tutorials directories that do not need to be shipped with pipeline code.

Usage

# No code changes required. Ray now respects .rayignore when packaging the working directory.

Checklist

• The title of the PR starts with fix or feat, if this applies to your PR.
• Changes are manually tested and documented in the PR if automated coverage is not practical.
• New or Existing tests cover these changes.
• The documentation is up to date with these changes.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[greptile-apps]

Greptile Summary

This PR adds a .rayignore file to the repository root to prevent Ray from uploading docs/, tests/, and tutorials/ when packaging the working directory for cluster deployment. This is a clean, low-risk config-only change.

.rayignore is officially supported in Ray 2.54.1+ (per current Ray docs: "if the local directory contains a .gitignore and/or .rayignore file, the specified files are not uploaded to the cluster"), so the file will be respected as intended. The patterns follow valid gitignore syntax.

Confidence Score: 5/5

Safe to merge — config-only change with no runtime risk.

This is a single, config-only file addition. The .rayignore format is confirmed as officially supported in Ray 2.54.1+, the gitignore-style patterns are syntactically correct, and the excluded directories (docs, tests, tutorials) are clearly non-runtime artifacts. The only open item is a P2 suggestion to also exclude benchmarking/.

No files require special attention.

Important Files Changed

Filename	Overview
.rayignore	New file that excludes docs/, tests/, and tutorials/ from Ray working directory uploads; syntax is valid gitignore-style and the file is officially supported in Ray 2.54.1+.

Sequence Diagram

sequenceDiagram
    participant Dev as Developer
    participant Ray as Ray CLI / SDK
    participant Cluster as Ray Cluster

    Dev->>Ray: ray job submit --working-dir .
    Ray->>Ray: Read .rayignore
    Ray->>Ray: Exclude docs/, tests/, tutorials/
    Ray->>Cluster: Upload nemo_curator/ + remaining files
    Cluster->>Cluster: Run pipeline

Loading

_{Reviews (1): Last reviewed commit: "fix: ignore docs, tests, and tutorials i..." | Re-trigger Greptile}

[greptile-apps]

Consider also excluding benchmarking/

The top-level benchmarking/ directory is not excluded. Like docs/ and tutorials/, it is unlikely to be needed at runtime on a Ray cluster and could be excluded to further reduce upload size.

Suggested change

	docs/
	tests/
	tutorials/
	benchmarking/
	docs/
	tests/
	tutorials/